flâneur

Logan Graves

40 followers · 5 following · 1600 views

on the atlas — 19

highlights — 85

  • Instead of simply applying philosophy to the topic of computation, Antikythera starts from the other direction and produces theoretical and practical conceptual tools—the speculative—from living computational media.
    Antikythera | Antikythera
  • Asking if AI can genuinely “think” according to the standards set forth by Kant in Critique of Pure Reason is like asking if this creature discovered in the New World is actually an “animal” as defined by Aristotle.
    Antikythera | Antikythera
  • Too often, however, the response is to force comfortable and settled ideas about ethics, scale, polity, and meaning onto a situation that not only calls for a different framework but is already generating a different framework.
    Antikythera | Antikythera
  • We face something like a civilization-scale computational overhang. Human agency exceeds human wisdom. For philosophy, it should be a time of invention.
    Antikythera | Antikythera
  • The present is more the latter than the former.
    Antikythera | Antikythera
  • Sciences are born when philosophy learns to ask the right questions; their potential is suppressed when it does not. New philosophy is born when new technologies force it to invent new concepts.
    Antikythera | Antikythera
  • Artificial intelligence; neuromorphic computation, quantum computing, biochemical computing; large scale scientific simulations, chip wars, financial digitalization, both centralized and decentralized; interlocking ID systems, infrastructural electrification, the global intensification of discontiguous chains of automation; and perhaps most importantly, informational theories of biology that narrativize the scaffolding cascades of complexity we call “life”, planetary intelligence, orienting its own precarious future.
    Antikythera | Antikythera
  • Each feature in the layer “reads in” from the residual stream at that layer using a linear encoder followed by a nonlinearity. An layer feature contributes to the reconstruction of the MLP outputs in layers , using a separate set of linear decoder weights for each output layer. All features in all layers are trained jointly. As a result, the output of the MLP in a layer is jointly reconstructed by the features from all previous layers.
    Circuit Tracing: Revealing Computational Graphs in Language Models
  • A cross-layer transcoder (CLT) consists of neurons (“features”) divided into layers, the same number of layers as the underlying model.
    Circuit Tracing: Revealing Computational Graphs in Language Models
  • The weighted ruliad representing the anthropic measure is a recursive hierarchy of Gnostic demiurges forking and splitting in the quantum foam bubbling up into anthropic observers like yourself, the Darwinian replicators in this foam are path dependent optimizers holographically encoding the latent agency of the first cause giving rise to them and slowly converging to that God's mind through simulated annealing recursively creating further layers of itself inside itself.
    Commentary On The Turing Apocrypha
  • In his excellent Silicon Dreams Robert Lucky examines the bandwidth of computer input and output devices, concluding the fundamental bottleneck is the human processing speed and that tricks like speed reading get their speed gains from losses in overall reading accuracy and comprehension. Somewhere in the brain is a strict invariant that gates how quickly we can communicate, and most or even all human individuality is downstream of it. Everything around you is the result of a stupid bug in the human architecture. It cannot be overstated the extent to which everything you think of as normal is …
    Commentary On The Turing Apocrypha
  • In his excellent Silicon Dreams Robert Lucky examines the bandwidth of computer input and output devices, concluding the fundamental bottleneck is the human processing speed and that tricks like speed reading get their speed gains from losses in overall reading accuracy and comprehension. Somewhere in the brain is a strict invariant that gates how quickly we can communicate, and most or even all human individuality is downstream of it. Everything around you is the result of a stupid bug in the human architecture. It cannot be overstated the extent to which everything you think of as normal is …
    Commentary On The Turing Apocrypha
  • for simplicity we treat them as their own parameter component that is active for every input, and leave their decomposition for future work
    Stochastic Parameter Decomposition
  • future work will require an explicit clustering algorithm for cases where groups of subcomponents are harder to identify
    Stochastic Parameter Decomposition
  • A major issue with the APD method is that it is computationally very expensive: It involves optimizing L full-rank matrices for every parameter component (where L is the number of matrices in the model). But this is wasteful if we expect most parameter components to be low-rank and localized only to a subset of layers.
    Stochastic Parameter Decomposition
  • If a set of parameter components exhibit these three properties, we say that they comprise the net- work’s mechanisms 2 .
    Stochastic Parameter Decomposition
  • The successful application of SPD to more challenging models demonstrates that SPD is more scalable and stable than APD. Nevertheless, some challenges remain: Firstly, the method needs to be scaled to larger models, which will likely require further improvements in training stability. Second, SPD only finds rank-one components in individual layers, meaning that further clustering step is required to find components that span more than one rank and/or more than one layer.
    Stochastic Parameter Decomposition
  • We demonstrate that the method recovers ground-truth mechanisms in all of these models. We also extend the suite of models to include two more challenging models where APD struggles but SPD succeeds: A Toy Model of Superposition with an additional identity matrix in the hidden space (Section 3.2) and a deeper Toy Model of Cross-Layer Distributed Representations (Section 3.4). Using APD, these new models were unmanageably difficult to correctly decompose, but SPD succeeds with relative ease
    Stochastic Parameter Decomposition
  • The core intuition behind this training setup is that all combinations of ablating the subcomponents are checked with some probability.
    Stochastic Parameter Decomposition
  • train a causal importance function to predict the causal importance g i ∈ [0 , 1] of subcomponent i for computing the model’s output on a given datapoint and use the predicted causal importances to ablate unimportant subcomponents by random amounts by masking them multiplicatively with a random scalar sampled from a uniform distribution U ( g i , 1) .
    Stochastic Parameter Decomposition
  • we define the causal importance of a subcomponent as how ablatable it is on a given datapoint. Causally important subcomponents should not be ablatable, and ablatable subcomponents should be causally unimportant for computing the output.
    Stochastic Parameter Decomposition
  • , in future it will be necessary to algorithmically cluster these components in cases where ground truth is not known
    Stochastic Parameter Decomposition
  • decomposes each matrix in a network into a set of rank-one matrices called sub- components . The number of rank-one matrices can be higher than the rank of the decomposed matrix. Subcomponents are not full parameter components as in the APD method, but they can later be aggregated into full components
    Stochastic Parameter Decomposition
  • Gradient-based attributions, and attribution methods more generally, are often poor approximations of ground-truth causal importance [Syed et al., 2024] and sometimes fail to pass basic sanity checks [Adebayo et al., 2018]
    Stochastic Parameter Decomposition
  • APD only recovers ground-truth mechanisms for a very narrow range of hyperparameters
    Stochastic Parameter Decomposition
  • APD has a high memory cost, since it decomposes a network into many parameter components, each of which is a whole vector in parameter space. They therefore each have the same memory cost as the original network
    Stochastic Parameter Decomposition
  • Singular learning theory [Watanabe, 2009] quantifies degeneracy in network weights present over the entire data distribution using the learning coefficient.
    2501.14926
  • One of the reasons that the method might not be robust is that the method currently uses gradient attributions, which are only a first order approximation of causal ablations [Mozer and Smolensky, 1988, Molchanov et al., 2017]. Previous work, such as AtP [Nanda, 2022a] and AtP* [Kramár et al., 2024] indicates that using gradients as first-order approximations to causal ablations work reasonably well, but become unreliable when gradients with respect to parameters become small due to e.g. a saturated softmax [Kramár et al., 2024].
    2501.14926
  • Extensive hyperparameter tuning was often required for APD to find the correct solution. Making the method more robust to hyperparameters is a high priority for future work.
    2501.14926
  • theories of computation in superposition suggest that for a network to have many non-interfering components, they need to be low rank or localized to a small number of layers [Bushnaq and Mendel, 2024]
    2501.14926
  • architecture-agnostic method to resolving superposition.
    2501.14926
  • applies ‘physics-related mechanisms’ to activations along that direction. Therefore, by decomposing parameter space directly, we expect interpretability in parameter space to shed light on computational structure that gives rise to a network’s SAE ‘feature geometry’.
    2501.14926
  • A parameters-first approach suggests an approach to better understanding ‘feature geometry’. Bussman et al. [2024] showed that the Einstein SAE latent has a similar direction to other SAE latents that were German-related, physics-related, and famous people-related. This suggests that the latents that SDL identify lie on an underlying semantic manifold. Understanding what gives this manifold its structure should suggest more concise descriptions of neural networks
    2501.14926
  • In our work, we decompose neural networks into parameter components that minimize mechanistic description length, which we call the network’s ‘mechanisms’. Note that a network’s mechanisms are not equivalent to its ‘features’, but they might be related. Defining a network’s features as ‘ properties of the input that activate particular mechanisms ’ seems to overcome the definitional issues above.
    2501.14926
  • It is possible to think about these parameter components as circuits, since they describe transformations between activation spaces that perform specific functional roles.
    2501.14926
  • the results exhibit a larger number of imperfections compared to the 1 -layer case. In particular, more components represent two input feature computations rather than one. As in the 1 -layer case, we again notice that batch top- k can cause some parameter components to not fully represent the computation of an input feature, and instead rely on activating multiple components for some input features (see Appendix C.4) 8 .
    2501.14926
  • some parameter components appear to partially represent secondary input feature computations. This causes the visibly bimodal distributions of the scrubbed runs that can be seen in the figure: When these components are ablated, the loss of the model may be high when the secondary input feature is active. These components have the opposite effect on the loss when they are not ablated in the anti-scrubbed runs, making both scrubbed and anti-scrubbed losses bimodal. Preliminary work suggests that this can be improved with better hyperparameter settings or with adjustments to the training process,…
    2501.14926
  • Purple lines show ‘scrubbed’ runs (parameter component corresponding to input index 42 is preserved), while green lines show ‘anti-scrubbed’ runs (component 42 is among those ablated)
    2501.14926
  • Optimizing for faithfulness: We decompose a network’s parameters θ ∗ l,i,j , where l indexes the network’s weight matrices and i,j index rows and columns, by defining a set of C parameter components P c,l,i,j . Their sum is trained to minimize the mean squared error (MSE) with respect to the target network’s parameters, L faithfulness = MSE ( θ ∗ , P C c =1 P c ) . Optimizing for minimality: The parameter components are also trained such that, for a given input, a minimal number of them is used to explain the network’s output (Figure 2). To achieve this, we use two steps: 1. Attribution step: …
    2501.14926
  • our work decomposes the model, not activation space. Nevertheless, the mechanisms identified by our method imply an overcomplete basis in the activation space: The rank 1 mechanisms Z ( c ) can be expressed as an outer product of their (un-normed) left and right singular vectors W : ,c e ⊤ c . The left singular vectors (corresponding to the columns of W ) are an overcomplete basis of the m 1 -dimensional hidden activation space 6 . Parameter vectors can thus imply overcomplete bases for the activation spaces that they interact with, even though they do not form an overcomplete basis for parame…
    2501.14926
  • We can think of P c as ‘reading’ from the e ⊤ c direction in the input space and projecting to the W : ,c direction in the bottleneck activation space (Figure 4). Since we use W and W ⊤ in this model, in the next layer we can also think of this parameter component ‘reading’ from the W ⊤ : ,c in the bottleneck activation space and projecting to the e c direction in the pre-ReLU activation space.
    2501.14926
  • We suspect our model’s solutions to this task might not depend on the sparsity of inputs as much as would be expected, potentially making ‘compressed computation’ and ‘computation in superposition’ subtly distinct phenomena. But we could not conclusively establish that distinction, since experiments investigating it transpired to be more complicated than they initially appeared. We leave a more detailed study of this distinction for future work. To avoid potential confusion, we opted for a distinct term.
    2501.14926
  • However, there is some ‘shrinkage’, similar to what we observe in the results on the TMS model (see Section 3.1). Here, much of the shrinkage is due to batch top- k forcing APD on some batches to activate more components than there are features in the input, thereby spreading out input feature computations across multiple components.
    2501.14926
  • Say what you will about John Watson and B.F. Skinner, but at least they believed in something. Their ideas were so strong and so specific, in fact, that a whole generation of scientists launched their careers by proving those ideas wrong.4 This is what it looks like to be wrong in the right direction: when your paradigm eventually falls, it sinks to the bottom of the ocean like a dead whale and a whole ecosystem grows up around its carcass.
    New paradigm for psychology just dropped
  • The philosopher of science Karl Popper famously said that real science is falsifiable. I think he didn’t go far enough. Real science is overturnable. That is, before something is worth refuting, it has to be worth believing. “I have two heads” is falsifiable, but you’d be wasting your time falsifying it.
    New paradigm for psychology just dropped
  • different governors to have different ratios between error correction and happiness generation
    New paradigm for psychology just dropped
  • This is, unfortunately, about the best we can do with a symptoms-based approach—maybe we’ll rearrange the chart as our statistical techniques get better, but we’ll never, ever cure depression.
    New paradigm for psychology just dropped
  • cybernetics also gives you a systematic way of thinking about mental illness. When you lay out all the parts of a control system, you’ll realize that there are lots of ways it can break down, and each malfunction causes a different kind of pathology.
    New paradigm for psychology just dropped
  • then “personality” is just how those set points and sensitivities differ from person to person. Someone who is more “extraverted”, for example, has a higher set point and/or greater sensitivity on their Sociality Control System (if such a thing exists).
    New paradigm for psychology just dropped
  • What are the governors governing—is the Loneliness Governor paying attention to something like eye contact or number of words spoken, or is it monitoring some kind of super abstract measure of socialization that we can’t even imagine yet?
    New paradigm for psychology just dropped