Alex Shaw

1 followers · 1 following · 851 views

on the atlas · 18

highlights · 47

There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
The End of Theory: The Data Deluge Makes the Scientific Method Obsolete | WIRED
For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn't pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right.
The End of Theory: The Data Deluge Makes the Scientific Method Obsolete | WIRED
Keys, queries and value vectors are, in some sense, superficial.
A Mathematical Framework for Transformer Circuits
We generally think of the residual stream as a communication channel, since it doesn't do any processing itself and all layers communicate through it.
A Mathematical Framework for Transformer Circuits
Our aim is to discover simple algorithmic patterns, motifs, or frameworks that can subsequently be applied to larger and more complex models.
A Mathematical Framework for Transformer Circuits
How can we control whether superposition and polysemanticity occur?
Toy Models of Superposition
Additional Considerations
Toy Models of Superposition
For any feature, whether or not it is represented is determined by , the norm of its embedding vector.
Toy Models of Superposition
roughly, it allows models to discard small amounts of noise.
Toy Models of Superposition
Concretely, our synthetic data is defined as follows: The input vectors are synthetic data intended to simulate the properties we believe the true underlying features of our task have. We consider each dimension to be a "feature". Each one has an associated sparsity and importance . We let with probability , but is otherwise uniformly distributed between .In practice, we focus on the case where all features have the same sparsity, .
Toy Models of Superposition
The intuition from linear models would be that this isn't possible: the best a linear model can do is to store the principal components. But we'll see that adding just a slight nonlinearity can make models behave in a radically different way!
Toy Models of Superposition
Linear representations make features "linearly accessible."
Toy Models of Superposition
Let's call a neural network representation linear if features correspond to directions in activation space. In a linear representation, each feature has a corresponding representation direction . The presence of multiple features activating with values is represented by . To be clear, the features being represented are almost certainly nonlinear functions of the input. It's only the map from features to activation vectors which is linear. Note that whether something is a linear representation depends on what you consider to be the features.
Toy Models of Superposition
Privileged Basis: Only some representations have a privileged basis which encourages features to align with basis directions (i.e. to correspond to neurons). Superposition: Linear representations can represent more features than dimensions, using a strategy we call superposition. This can be seen as neural networks simulating larger networks. This pushes features away from corresponding to neurons.
Toy Models of Superposition
More broadly, the toy model we investigate has unexpectedly rich structure, exhibiting phase changes, a geometric structure based on uniform polytopes, "energy level"-like jumps during training, and a phenomenon which is qualitatively similar to the fractional quantum Hall effect in physics.
Toy Models of Superposition
models represent more features than they have dimensions. We call this phenomenon superposition.
Toy Models of Superposition
A key difference between ESMFold and AlphaFold2 is the use of language model representations to remove the need for explicit homologous sequences
Language models of protein sequences at the scale of evolution enable accurate structure prediction | bioRxiv
In linguistics, the distributional hypothesis proposes that meaning can be inferred from text by the way it constrains the patterns of words (1).
Language models of protein sequences at the scale of evolution enable accurate structure prediction | bioRxiv
There is increasing evidence that large pre-trained language models learn representations that compress NLP tasks well
Recent Advances in Language Model Fine-tuning
set θ task = θ low M θtask=θlowM where θ low θlow is a low-dimensional vector and M M is a random linear projection (in their case, the FastFood transform
Recent Advances in Language Model Fine-tuning
Each position outputs a vector of size hidden_size
The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) – Jay Alammar – Visualizing machine learning one concept at a time.
We call this activation function a "softmax linear unit" or SoLU:
Softmax Linear Units
The first approaches in this line of work are based on adapters
Recent Advances in Language Model Fine-tuning
Adaptive fine-tuning is thus most useful when high performance on (potentially multiple) tasks of a single domain is important
Recent Advances in Language Model Fine-tuning
Knowledge must be of what is real The world experienced via the senses is what is real Knowledge must be of what is fixed and unchanging The world experienced via the senses is not fixed and unchanging
Comparing the Similarities and Differences Between Plato and Aristotle - Owlcation
You can try to phrase the instruction differently.
Build a CV profile generator using AI21 Studio
Find a way to understand representations with superposition.
Softmax Linear Units
Neural networks represent features as directions in activation space.
Softmax Linear Units
Cammarata et al. spend two papers investigating a handful of curve detector neurons and the circuits that implement them, using seven different lines of evidence to corroborate that the neurons really are curve detectors, with the goal of dispositively establishing that at least some neurons really are interpretable.
Softmax Linear Units
In transformers, the token embeddings, residual stream, and attention vectors are non-privileged, while MLP layer activations are privileged.
Softmax Linear Units
For example, if the layer has a coordinate-wise non-linear activation function (eg. ReLU), this “breaks the symmetry," distinguishing the specific basis of the activations as the unique basis in which the nonlinearity is applied.
Softmax Linear Units
For example, in a word embedding, one might define a gender direction by subtracting "man" and "woman"
Softmax Linear Units
Concretely, there are many more parameters than activations, so the activations seem like a more likely “key” to what’s going on.
Softmax Linear Units
a human can quickly find a clear hypothesis explaining its activations from 35% to 60%
Softmax Linear Units
the superposition hypothesis , which suggests that neural network layers have more features than neurons as part of a “sparse coding” strategy to simulate a much larger layer.
Softmax Linear Units
The underlying issue is that many neurons appear to be polysemantic
Softmax Linear Units
Mechanistic interpretability – the project of attempting to reverse engineer neural networks into understandable computer programs – offers one possible avenue for addressing these safety issues
Softmax Linear Units
Another way to do it would be to hold on to, say, the top two words (say, ‘I’ and ‘a’ for example), then in the next step, run the model twice: once assuming the first output position was the word ‘I’, and another time assuming the first output position was the word ‘a’, and whichever version produced less error considering both positions #1 and #2 is kept. We repeat this for positions #2 and #3…etc. This method is called “beam search”, where in our example, beam_size was two (meaning that at all times, two partial hypotheses (unfinished translations) are kept in memory), and top_beams is also…
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
So for each word, we create a Query vector, a Key vector, and a Value vector.
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
the exact same network with each vector flowing through it separately.
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
basically it would be the length of the longest sentence in our training dataset
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
yet they do not share weights
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
The exact same feed-forward network is independently applied to each position.
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
we begin by turning each input word into a vector using an embedding algorithm.
The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.
When added to RNNs, attention mechanisms increase performance. The development of the Transformer architecture revealed that attention mechanisms were powerful in themselves and that sequential recurrent processing of data was not necessary to achieve the quality gains of RNNs with attention.
Transformer (machine learning model)
This model hints at why investors often say ‘no’ to companies quickly without doing much diligence.
Three Mathematical Models for Building a More Valuable Company
Product/Market Fit Risk (is your product something that people want?) Product Quality Risk (can you build a great product?) Team Risk (do you have a great team for achieving your vision?) Recruiting Risk (are you able to grow your team effectively with strong talent?) Sales Risk (can you and your team sell your product effectively?) Market Risk (is your target market large enough to support a huge company?) Funding Risk (do you have enough capital to hit milestones that will either let you raise more capital on better terms, or get you to a place where you no longer need to raise capital?) Sho…
Three Mathematical Models for Building a More Valuable Company