Kunal Mishra

2 followers · 5 following · 342 views

on the atlas — 30

highlights — 83

A modern INS will typically use an inertial measurement unit (IMU) that contains accelerometers that can detect small changes in acceleration, a gyroscope to detect rotation, a magnetometer to determine direction based on sensing the Earth’s magnetic field strength for finding North, and a barometric pressure sensor to determine altitude.
Pedestrian Dead Reckoning (PDR) – An Introduction
Dead-reckoning is the term given to approximate one’s current position based on estimated movements from a previously known location. That is, having your starting point coordinates, then estimating heading (direction) and distance travelled between each change in direction, using velocity and time, to calculate where you are now.
Pedestrian Dead Reckoning (PDR) – An Introduction
FLOPs/s-to-memory-bandwidth-ratio (FLOPs/byte)
⭐️ Fast LLM Inference From Scratch
Almost2 every major open-weights LLM uses the same architecture3 (sequential transformer blocks), with some minor variations/innovations since GPT-2: Grouped query attention (and multi-query attention) Mixture-of-experts-based feedforward networks GLU-based instead of MLP-based feedforward networks Different activation functions for feedforward networks Different layer normalizations Rotary position embeddings
⭐️ Fast LLM Inference From Scratch
In the future, the amount of intelligence you get will just be based on how much compute you throw at it.
Grok 4 Various Things — LessWrong
achieving the end-to-end deployment of SLMs on the microcontroller (MCU)-class chips without high-bandwidth off-chip main memory access is still an open challenge
Deeploy: Enabling Energy-Efficient Deployment of Small Language Models on Heterogeneous Microcontrollers | IEEE Journals & Magazine | IEEE Xplore
( 𝑖 ) minimize overall interference (shift neighboring APs to alternative channels), ( 𝑖𝑖 ) enhance throughput (shift clients to alternative APs), and ( 𝑖𝑖𝑖 ) reduce energy consumption (shut down redundant APs during periods of low load)
arxiv.org/pdf/2101.09515
two unique challenges associated with this approach, namely Cardinality Mismatch and High Client Scan Latency . The “Mismatch” challenge results in a significant mismatch between the set of access points (APs) reporting a client in the offline and online phases, while the “Latency” challenge results in a low number of APs reporting data for any particular client.
arxiv.org/pdf/2101.09515
a perfect 10cm localization accuracy within a 0.9m×1m area-of-interest in a NLOS environment
High accuracy indoor localization: A WiFi-based approach | IEEE Conference Publication | IEEE Xplore
In [6], the authors proposed FIFS using the summation of power across subcarriers in CSI to achieve 0.60m median accuracy.
High accuracy indoor localization: A WiFi-based approach | IEEE Conference Publication | IEEE Xplore
CSIs are estimated from the long training preambles (LTP) on subcarrier level in frequency domain
High accuracy indoor localization: A WiFi-based approach | IEEE Conference Publication | IEEE Xplore
channel state information (CSI), a physical layer, fine-grained information at the receiver
High accuracy indoor localization: A WiFi-based approach | IEEE Conference Publication | IEEE Xplore
Received Signal Strength Indicator (RSSI) is a MAC layer, coarse-grained information available in mainstream wireless network interface controllers
High accuracy indoor localization: A WiFi-based approach | IEEE Conference Publication | IEEE Xplore
The last scenario is that there is no previous Scan Data Frame, which commonly happens to the first scan result of data collecting procedures. Our analysis for WiFi scan procedures suggests that APs in channel 1 tend to have the smallest last-seen timestamps among all fresh records. In light of this, our filter adopts the maximum last-seen timestamp from the records in channel 1 as a reference baseline and removes any records with last-seen timestamps precede this baseline.
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
a new WiFi scan process is initiated immediately following the reception of an availableAction. Therefore, we can utilize the local timestamp of the preceding Scan Data Frame as a proxy for the start time of the WiFi scan process
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
WiFi operates on multiple channels within the 2 . 4 GHz (e.g., up to 13 in the EU and China) and 5 GHz (e.g., up to 19 in the EU, up to 13 in China) bands
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
WiFi operates on multiple channels within the 2 . 4 GHz (e.g., up to 13 in the EU and China) and 5 GHz (e.g., up to 19 in the EU, up to 13 in China) bands
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
APs operating on the same carrier frequency typically exhibit similar last-seen timestamps
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
a new pipeline for the offline phase, strategically filtering stale RSSI data, and rectifying time drift resulting from misinterpretations of collection time and local time. For the online phase, we devise an AP filling approach to contend with scenarios where selected APs may be unavailable in the scan results. Furthermore, we propose a new AP selection strategy based on the Wasserstein distance in signal space.
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
Since the scanning results for APs are recorded but not returned to the user until the entire process is completed, using the local timestamp as the actual collection time could result in inaccurate RSSI maps.
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
Specifically, the prevailing method for collecting RSSI data involves instructing the device to conduct WiFi scanning and extracting RSSI data from the scan results. The device initiates a scan across all available APs on all channels. Upon receiving the scan results, the local clock time, referred to as the local timestamp, is appended to the RSSI data obtained from the scan. Apart from the local timestamp appended for each scan result, the scanning result also includes many timestamps for every RSSI records indicating when the corresponding AP was last observed, termed the last-seen timestam…
Last-seen time is critical: Revisiting RSSI-based WiFi indoor localization - ScienceDirect
Unfortunately, if I were to start an Alternative Investment Fund (AIF), which is what domestic VC funds are classified as, focused on startups doing R&D in India, I would only be able to invest 25% of my corpus in foreign-headquartered startups.
Overcoming India's technological cowardice
Far from decrying and attempting to limit foreign investment in these companies, the government should lay out the red carpet for the foreign VCs willing to take these risks, and demolish the many roadblocks that currently make investing in Indian Private Limited Companies extremely unattractive for most VCs.
Overcoming India's technological cowardice
The goal of this R&D cannot be import substitution, which has been a failure every time it has been tried – truly best-in-class products will only be forged in the white-hot fire of global competition. We must look to export these products from day 1, and if we cannot, it is because they aren’t good enough, and we must make them better, not throw up trade barriers to sell something substandard to our own citizens.
Overcoming India's technological cowardice
the United States, long the world’s largest funder of R&D, is executing the sharpest pullback in funding to the sciences in the last half century, in particular biotech funding via the National Institutes of Health (NIH), and tightening immigration policies, including cancelling a large number of student visas. It is their sovereign right to make those decisions in their wisdom about what will be best for the US, but I believe it is our duty in India to seize this golden opportunity to massively scale up our R&D across many sectors, from materials science to matrix multiplication, leveraging n…
Overcoming India's technological cowardice
indeed, I don’t think it would have been feasible for me to start the company in the US even if I’d wanted to, because the cost of doing even minimal wet lab R&D in that country is prohibitively high. I ran PopVax off $30k of my own money for our first 6 months, which would have bought me the ability to produce essentially no usable wet lab data in Cambridge, MA, or even Cambridge, UK, but in Hyderabad, working out a shared lab space we rented from the Centre for Cellular and Molecular Biology (CCMB), we were able to generate data that led to hundreds of thousands and then millions of dollars …
Overcoming India's technological cowardice
Checksum is the 16-bit one's complement of the one's complement sum of a pseudo header of information from the IP header, the UDP header, and the data, padded with zero octets at the end (if necessary) to make a multiple of two octets.
ietf.org/rfc/rfc768.txt
Source Port is an optional field, when meaningful, it indicates the port of the sending process, and may be assumed to be the port to which a reply should be addressed in the absence of any other information. If not used, a value of zero is inserted.
ietf.org/rfc/rfc768.txt
only use violence against entities that use violence more loosely than you; the equilibrium of adopting this rule is no violence, as all more violent entities are gradually eliminated by less violent ones
What I'm thinking about these days - Alexey Guzey
liquid time constant neural networks (LTCs)
Research - Intempus
At the last NeurIPS, Ilya Sutskever notes that data is “the fossil fuel of AI”.
Research - Intempus
While we can instruct an LLM to output current timestamps or locations, it lacks the ability to truly associate actions and experiences within a relative dimension of time and space as humans do.
Research - Intempus
To measure the efficacy of a given segmenter, we evaluate the quality of the reconstructed sentences with AutoBLEU . It is defined as a BLEU score (Papineni et al., 2002) comparing the decoded text from a SONAR vector after encoding a segment, to the the reference segment. A good segmentation will yield segments that can be encoded and then decoded without loss of signal, and thus score a higher AutoBLEU
470149925_936340665123313_5359535905316748287_n.pdf
the LCM neither has information on the input language or modality nor generates output in a particular language or modality
470149925_936340665123313_5359535905316748287_n.pdf
e unchanged sequence of concepts at the output of the LCM can be decoded into other languages or modalities without performing again the whole reasoning proces
470149925_936340665123313_5359535905316748287_n.pdf
We posit that a sentence is an appropriate unit to achieve language independence, in opposition to single word
470149925_936340665123313_5359535905316748287_n.pdf
We define a concept as an abstract atomic idea
470149925_936340665123313_5359535905316748287_n.pdf
In order to verify our approach, we limit our study to two levels of abstraction: subword tokens and
470149925_936340665123313_5359535905316748287_n.pdf
we aim to model the underlying reasoning process at a purely semantic level, not its instantiation in a specific language
470149925_936340665123313_5359535905316748287_n.pdf
the same underlying architecture: a transformer-based, decoder-only language model, which is pretrained to predict the next token, given a long context of preceding tokens
470149925_936340665123313_5359535905316748287_n.pdf
Knowledge acquisition in LLMs is heavily data-driven and extending them to more languages or modalities usually requires injecting additional (synthetic) data to cover them
470149925_936340665123313_5359535905316748287_n.pdf
The Large Concept Model is trained to perform autoregressive sentence prediction in an embedding space
470149925_936340665123313_5359535905316748287_n.pdf
Concepts are language- and modality-agnostic and represent a higher level idea or action in a flow.
470149925_936340665123313_5359535905316748287_n.pdf
Marginal Revolution University Principles of Micro Marginal Revolution University Principles of Macro
Online courses and textbooks I recommend - Alexey Guzey
The Correct™ way to self-study books like this is to email a professor at a local college and ask them if they could help you with stuff you don’t understand and problems
Online courses and textbooks I recommend - Alexey Guzey
In general, I think it’s good to overcommit a little bit. I find that I generally get done what I take on, and if I have a little bit too much to do it makes me more efficient at everything, which is a way to train to avoid distractions (a great habit to build!). However, overcommitting a lot is disastrous.
Productivity - Sam Altman
I take a low dose of sleeping pills (like a third of a normal dose) or a very low dose of cannabis whenever I can’t sleep. I am a bad sleeper in general, and a particularly bad sleeper when I travel. It likely has tradeoffs, but so does not sleeping well.
Productivity - Sam Altman
probably 90% of the random meetings I take are a waste of time, the other 10% really make up for it.
Productivity - Sam Altman
The more I get done, the better I feel, and then the more I get done.
Productivity - Sam Altman
The more I get done, the better I feel, and then the more I get done.
Productivity - Sam Altman