Toy Models of Superposition

Core claim

Neural networks often represent more features than they have dimensions, encoding them as almost-orthogonal directions and tolerating the resulting interference — a phenomenon called superposition. In toy ReLU networks trained on synthetic sparse data, superposition provably occurs, is governed by a phase change driven by feature sparsity and importance, organises features into specific geometric structures, and can support actual computation, not just storage. This explains why neurons are sometimes monosemantic and sometimes polysemantic, and implies that trained models may be understood as noisily simulating much larger, sparser networks. Because enumerating a model’s features is central to strong interpretability claims about safety, “solving superposition” is a key obstacle.

Motivation and framing

• It would be convenient if individual neurons corresponded to cleanly interpretable features. Empirically some do, but in large language models clean neurons appear rare — raising the question of why neurons sometimes align with features and sometimes do not.
• Two countervailing forces are hypothesised:
  • Privileged basis — some representations have a basis (imposed e.g. by an activation function) that encourages features to align with basis directions, i.e. with neurons.
  • Superposition — linear representations can hold more features than dimensions, pushing features away from corresponding to neurons.
• Superposition is not a novel idea; it appears in prior interpretability work and is closely related to compressed sensing in mathematics and to distributed, dense, and population codes in neuroscience and deep learning. The contribution is a direct demonstration in a relatively natural setup, where sparse-structure-in-superposition is the model’s ground truth rather than a post-hoc interpretation, plus a theory of when and why it occurs.

Key results claimed

• Superposition is a real, observed phenomenon.
• Both monosemantic and polysemantic neurons can form.
• At least some kinds of computation can be performed in superposition.
• Whether features are stored in superposition is governed by a phase change.
• Superposition organises features into geometric structures such as digons, triangles, pentagons, and tetrahedrons.
• The models are simple ReLU networks, so neural networks do exhibit these properties in at least some regimes — but what generalises to real networks is unclear.

Definitions: features, directions, superposition

What is a “feature”?

Three candidate working definitions are considered, none adopted with confidence:

• Features as arbitrary functions of the input — too permissive; the observed features seem to be fundamental abstractions forming reliably across models, and “cat” and “car” feel like features while “cat+car” does not.
• Features as interpretable properties — appealing, but should allow for features humans do not understand (e.g. a chemical structure AlphaFold discovers).
• Features as properties a sufficiently large network reliably dedicates a neuron to — slightly circular and “epsilon-delta”-like, but avoids the other problems. This is the definition used in the paper, though the authors caution against premature attachment to a definition.

Features as directions (the linear representation hypothesis)

• A representation is linear if features correspond to directions: each feature $f_i$ has a direction $W_i$, and multiple features activating with values $x_{f_i}$ are represented by ${\sum }_i{x}_{f_i}{W}_{f_i}$. The features themselves are almost certainly nonlinear functions of the input; only the map from features to activation vectors is linear.
• Empirical motivation: word-embedding arithmetic (V("king") - V("man") + V("woman") = V("queen")), interpretable directions in GAN latent spaces, interpretable neurons in RNNs/CNNs/GANs, universality of analogous neurons across networks — alongside the countervailing observation of many polysemantic neurons.
• Three reasons linear representations are natural for neural networks:
  • They are the natural output of obvious algorithms a layer might implement (a neuron pattern-matching a weight template fires more as the stimulus matches better).
  • They make features linearly accessible — a downstream neuron can select a linearly represented feature in a single step, which it could not do for a nonlinearly represented one.
  • Statistical efficiency — directional representations permit non-local generalization in models built from linear transformations.
• Nonlinear representations are constructible using multiple layers, but the authors’ intuition is that they are generally inefficient for neural networks.

Privileged vs. non-privileged bases

• For word embeddings there is no reason to expect a basis dimension to differ from any other direction — applying a random linear transformation $M$ and $M^{-1}$ to subsequent weights yields an identical model. This is a non-privileged basis.
• Applying an activation function “breaks the symmetry”, making basis directions special and potentially encouraging feature alignment. This is a privileged basis, and its basis directions are what the authors call neurons. Asking whether a neuron is interpretable only makes sense in a privileged basis.
• A privileged basis does not guarantee basis-aligned features; it is only a minimal condition for the question to arise.

The superposition hypothesis

• Networks “want to represent more features than they have neurons”, so they exploit high-dimensional geometry to simulate a larger model. Two mathematical facts make this plausible:
  • Almost-orthogonal vectors — an $n$-dimensional space holds only $n$ orthogonal vectors but $\exp (n)$ almost-orthogonal ones (Johnson–Lindenstrauss).
  • Compressed sensing — a vector projected into a lower-dimensional space is generally unrecoverable, but often recoverable if known to be sparse.
• Features are then almost-orthogonal directions; one feature activating looks like others slightly activating. Tolerating this interference costs something, but for highly sparse features the cost may be outweighed by representing more features — sparsity means features rarely co-activate to interfere, and nonlinearities allow small noise to be filtered out.
• Superposition can occur in non-privileged bases too (e.g. word embeddings); it simply means more features than dimensions.

A hierarchy of representational properties

1. Decomposability — activations decompose into features whose meaning does not depend on other features’ values. Ultimately the most important property, since it defeats the curse of dimensionality.
2. Linearity — features correspond to directions.
3. Superposition vs. non-superposition — a linear representation exhibits superposition iff $W^{T}W$ is not invertible.
4. Basis-alignment — all $W_i$ are one-hot basis vectors (partially basis-aligned if all $W_i$ are sparse). Requires a privileged basis.

The first two are hypothesised to be widespread; the latter two only sometimes hold.

Demonstrating superposition

Setup

• The synthetic input $x\in {\mathbb{R}}^n$ represents activations of an idealised disentangled larger model, with each $x_i$ a “feature”. Three assumptions built into the data:
  • Feature sparsity — most features rarely occur (most image positions lack a curve or a dog head; most tokens do not refer to Martin Luther King).
  • More features than neurons — a general vision model might benefit from representing every species and manufactured object; language models demonstrably know about people of modest prominence, presumably more such people than they have neurons.
  • Features vary in importance — some reduce loss more than others.
• Concretely: $x_i=0$ with probability $S_i$, otherwise uniform on $[0,1]$; each feature has importance $I_i$. Experiments mostly use uniform sparsity $S_i=S$.
• Two models, differing only in final activation:
  • Linear model: $h=Wx$, $x^{\prime }=W^{T}Wx+b$.
  • ReLU output model: $h=Wx$, $x^{\prime }=\mathrm{ReLU}(W^{T}Wx+b)$.
• Weights are tied ($W^T$ for decoding) to avoid ambiguity about which direction represents a feature, and because $W^T=W^{-1}$ for orthonormal $W$ and $W$ is expected to be “almost orthonormal”. The bias lets the model set unrepresented features to their expected value and, importantly, lets it discard small noise via a negative bias.
• Loss is importance-weighted MSE: $L={\sum }_x{\sum }_i{I}_i(x_i-x_i^{\prime }{)}^2$.

Basic results

• The linear model always learns the top-$m$ most important features, analogous to principal components, regardless of sparsity.
• The ReLU output model matches this on dense features, but as sparsity increases superposition emerges: the model represents more features by making them non-orthogonal, starting with the least important and gradually affecting the most important.
• Initially this takes the form of antipodal pairs (one feature’s vector exactly the negative of another’s), transitioning to other geometries as more features are represented.
• Results are qualitatively similar at larger scale (e.g. $m=20$, $n=80$) — essentially a rescaled version of the same picture.

Mathematical understanding

• For the linear case, Saxe et al.’s analysis of linear network learning dynamics reveals two competing forces: feature benefit (better loss from representing more features) and interference (worse loss from representing more than fit orthogonally). For a linear model it is never worthwhile to represent more features than dimensions.
  • Aside: linear-model interference ${\sum }_{i\ne j}|W_i\cdot W_j{|}^2$ and compressed sensing’s coherence ${\max }_{i\ne j}|W_i\cdot W_j|$ are the $L^2$ and $L^{\infty }$ norms of the same vector.
• For the ReLU output model, the loss decomposes by sparsity pattern via the binomial expansion; as $S\to 1$ the $L_1$ (1-sparse) and $L_0$ terms dominate. $L_0$ is just a penalty on positive biases.
• The 1-sparse term resembles the Thomson problem in chemistry: assuming uniform importance, unit-norm represented features and zero bias, feature benefit is constant and interference becomes a generalized Thomson problem — packing points on a sphere with an unusual energy function.
• ReLU makes negative interference free in the 1-sparse case, explaining the preference for negative interference; a negative bias can convert small positive interference into effectively negative interference.
• Less-sparse terms involve compounding interference among active features; features often organise into sparse interference graphs, which reduces compounding and increases the relative weight of the 1-sparse term.

Superposition as a phase change

• Three outcomes per feature: not learned; learned in superposition; given a dedicated dimension. Transitions between them appear sharp — a phase change in the generalised sense of a discontinuous change.
• Experiment: 2 features, 1 hidden dimension, varying the second feature’s importance (0.1–10) against sparsity (1.0–0.01), with ten models averaged per point.
• Compared against a closed-form “toy model of the toy model” with three natural solutions: $W=[1,0]$ (discard the extra feature), $W=[0,1]$ (discard the first), and $W=[1,-1]$ (antipodal superposition, losing the ability to represent both features simultaneously).
• Findings: sparsity is necessary for superposition, and it interacts with relative importance in an interesting way; and there is a genuine phase change, with the theoretical model confirming a first-order transition (a discontinuity in the derivative of the optimal loss).
• The same question for three features in two dimensions adds a further wrinkle: superposition can put the extra feature in an antipodal pair with another, or put the other two in superposition and give the extra feature a dedicated dimension.

The geometry of superposition

The authors caution this structure may be “too elegant to be true” and partly idiosyncratic to the toy model — but worth studying for the leverage it would give if any of it generalises.

Uniform superposition

• With all features equally important and sparse ($n=400$, $m=30$; the specific counts matter little so long as $n\gg m$, and doubling hidden dimensions just doubles features learned), the Frobenius norm $\Vert W{\Vert }_F^2$ approximates the number of features represented.
• Plotting dimensions per feature $D^{\ast }=m/\Vert W{\Vert }_F^2$ against sparsity shows the curve is “sticky” at 1 and 1/2 — vaguely resembling the fractional quantum Hall effect. The 1/2 point corresponds to antipodal pairs, which the model prefers across a wide sparsity range.

Feature dimensionality

• Per-feature dimensionality is defined as $D_i=\Vert W_i{\Vert }^2/{\sum }_j(\widehat{W_i}\cdot W_j{)}^2$ — how much the feature is represented, divided by how many features share its dimension. Antipodal-pair features get $D=1/2$; unlearned features get 0. Empirically dimensionalities sum to the number of embedding dimensions when packing is efficient.
• Breaking the plot down per feature reveals many more sticky points at specific fractions: ¾ (tetrahedron), ⅔ (triangle), ½ (antipodal pair), ⅖ (pentagon), ⅜ (square antiprism), 0 (not learned) — and presumably 1 (dedicated dimension), invisible because basis directions are indistinguishable in the dense regime.
• Everything strictly between 0 and 1 is superposition, so superposition is not one thing: it has many phases, loosely analogous to the many crystal phases of ice rather than the naive three phases of water.

Why these geometries

• Several are famous solutions to the Thomson problem (square antiprisms being notable chiefly for this role in molecular geometry) — consistent with the earlier finding that the model solves a generalized Thomson problem by embedding features as points on an $m$-sphere.
• Lines appear for uniform polyhedra but split where non-uniform solutions would be expected (e.g. instead of a ⅗ line for triangular bipyramids, co-occurring points at ⅔ for triangles and ½ for antipodes). In a uniform polyhedron all vertices share the same geometry and hence the same dimensionality.
• Many Thomson solutions are tegum products — polytopes built by embedding two polytopes in orthogonal subspaces — so dimensionality tracks the underlying uniform factors. This may also explain why 3D solutions appear despite the higher-dimensional setting: higher-dimensional solutions may often be tegum products of 1D, 2D, and 3D ones. Tegum factors permit no interference across factors, which the model may prefer since simultaneous interference among many features is costly.
• In the feature-geometry graph visualisations, two subgraphs are disconnected iff they lie in different tegum factors.

Aside: polytopes and low-rank matrices

• There is an exact correspondence between polytopes and superposition strategies: for a rank-$m$ matrix $W^{T}W$, the columns of $W$ are $n$ points in $m$-dimensional space, and off-diagonal terms are driven by their geometry. Every strategy for three features in two dimensions is a triangle, and vice versa — so an equilateral triangle being optimal for three equal features is unsurprising.
• Running the correspondence backwards, a rank-$(n-i)$ matrix can be characterised by which directions $W$ did not represent. Projecting basis vectors orthogonally to $(1,1,1)$ in three dimensions gives a triangle; $W\perp (1,1,1,\dots )$ gives a regular $n$-simplex — the “minimal possible superposition”, since for equally important and sparse features the best direction to discard is the fully dense one.

Non-uniform superposition

Real superposition will be non-uniform; no comprehensive theory is offered, but three striking phenomena are reported:

• Varying importance or sparsity smoothly deforms polytopes until a critical breaking point at which they snap to another polytope. In a 5-features-in-2-dimensions experiment, making one feature denser makes the others repel from it; making it sparser lets them push toward it; making it sufficiently sparse triggers a phase change collapsing the pentagon into a pair of digons. Loss curves for the two geometries cross over, directly confirming a first-order transition.
  • Pentagonal solutions sit off the unit circle because models set a slight negative bias to cut noise and compensate with $\Vert W_i\Vert =1/(1-b_i)$; distance from the unit circle mainly reflects positive interference.
  • Note for reimplementation: optimising a two-dimensional hidden space is much harder for gradient descent than three dimensions, requiring multiple fits and taking the lowest-loss solution — though visualising suboptimal solutions usefully exposes the competing loss curves.
• Correlated features prefer to be orthogonal, often in different tegum factors, forming a local almost-orthogonal basis: even when the model as a whole is in superposition, a correlated feature set considered in isolation is nearly orthogonal. If this holds in real networks, it suggests a “local non-superposition” assumption permitting methods like PCA on narrow sub-distributions. When correlated features cannot be orthogonal they prefer to be side-by-side (positive interference).
• Correlated features can collapse into their principal component. Given correlated $a$ and $b$ and capacity for only one, the model represents $(a+b)/\sqrt{2}$ and ignores $(a-b)/\sqrt{2}$. Across a sparsity sweep of three correlated pairs, very sparse regimes give superposition (hexagon, correlated features side-by-side), while denser regimes progressively collapse to PCA. PCA and superposition appear to be complementary strategies — correlation favours PCA, sparsity favours superposition, and mixtures occur when both hold.
• Anti-correlated features prefer to share a tegum factor and to interfere negatively, ideally antipodally.
• Correlational structure strongly influences which features get grouped together, which undercuts the idea of detecting polysemantic neurons by comparing groupings across models.

Learning dynamics

Two phenomena are flagged, with detailed investigation left to future work:

• Discrete “energy level” jumps. With many equally important and sparse features, training is dominated by features jumping between dimensionalities and swapping places, each jump accompanied by a sudden drop in the loss curve. This suggests that apparently smooth loss curves in larger models may be composed of many small configuration jumps — resonating with induction head phase changes, phase changes in modular arithmetic, and grokking.
• Learning as geometric transformations. With $m=3$, feature embeddings can be visualised directly as points forming a polyhedron, and learning proceeds through distinct regimes visible in the loss curve, each a simple geometric transformation. For 6 features as 2 correlated triples, an octahedron with cross-set antipodal pairs is a strong attractor, though several trajectories reach it.
  • This connects to prior findings that two-layer networks first learn a linear approximation (here, a PCA solution) before moving to a better nonlinear one, and to work on hierarchical features where embedding vectors “split” similarly — though that work emphasises topology rather than geometry.

Relationship to adversarial robustness

• Intuition: without superposition the end-to-end weights for the first feature are $(1,0,0,\dots )$; with superposition they are $(1,ϵ,-ϵ,ϵ,\dots )$, and those interference terms give an adversary an obvious attack on the most important feature. This may hold even in the infinite-data limit, since using superposition is optimal for sparse data.
• Testing with analytically derived optimal $L^2$ attacks (gradient descent attacks suffered from gradient masking when ReLUs sit at zero 99% of the time), vulnerability sharply increases as superposition forms — by more than 3× — closely tracking features per dimension.
• The authors are hesitant about the practical significance, since compelling superposition-free theories of adversarial examples exist. But a “superposition maximalist” stance would predict: that adversarial examples exist; that adversarially robust models perform worse (robustness requires giving up features); that more robust models are more interpretable; and possibly that adversarial examples transfer, if feature arrangement is driven by correlational structure.
• Adversarial training did reduce superposition, but attacks had to be unreasonably large (80% of input $L^2$ norm) to eliminate it, so this line was not pursued further.

Superposition in a privileged basis

• The ReLU output model has no privileged basis: for any orthogonal $O$, $W^{\prime }=OW$ gives an identical model since $(OW{)}^T(OW)=W^{T}W$. This is a good analogue for word embeddings or the transformer residual stream, but not for MLP layers or conv net neurons.
• Adding a hidden-layer activation gives the ReLU hidden layer model: $h=\mathrm{ReLU}(Wx)$, $x^{\prime }=\mathrm{ReLU}(W^{T}h+b)$. Now $W$ itself is directly interpretable as a map from features to basis-aligned neurons, rather than only $W^{T}W$.
• Visualised as per-neuron stacked bar plots (rectangle height = weight magnitude, colour = feature), the results show features aligning with neurons in structured ways, with many neurons dedicated to a single feature — the property that justifies neuron-focused interpretability.
• Across sparsity levels ($n=10$, $m=5$, $I_i=0.75^i$; 1000 models trained per setting, lowest-loss shown), there is a clear shift from monosemantic to polysemantic neurons as sparsity increases, with both types coexisting in the same model, and what looks like a neuron-level phase change mirroring the feature-level one. Polysemantic solutions are surprisingly structured: features correspond to sets of neurons, with monosemantic neurons the singleton case.

Limitation: simulating identity

• This model has a significant weakness: it gains nothing from the hidden ReLU, which only limits how information can be encoded, so the model circumvents it whenever possible — setting all biases positive to shift neurons into the linear regime, or simulating a bias by averaging over many features if the bias is removed. It uses the activation function only when forced.

Computation in superposition

• To force genuine use of the nonlinearity and get a principled privileged basis, the model computes $y=\mathrm{abs}(x)$ — appealing because $\mathrm{abs}(x)=\mathrm{ReLU}(x)+\mathrm{ReLU}(-x)$ makes the intended mechanism transparent.
• Setup: $x_i$ zero with probability $S_i$, else uniform on $[-1,1]$; untied weights $h=\mathrm{ReLU}(W_{1}x)$, $y^{\prime }=\mathrm{ReLU}(W_{2}h+b)$; importance-weighted MSE loss.
• Without superposition, the model implements the expected solution: for each feature, a “positive side” neuron $\mathrm{ReLU}(x_i)$ and a “negative side” neuron $\mathrm{ReLU}(-x_i)$, summed.
• With $n=100$ features and $m=40$ neurons ($I_i=0.8^i$): in dense regimes each neuron represents one feature and values can be read off directly; once features are sufficiently sparse, the model again uses superposition — demonstrating computation on data represented in superposition.
  • Quantifying the extra computation via loss is not possible, because superposition arises by changing the task’s sparsity, so losses are not comparable across settings.
• Two qualitative behaviours resemble findings in real networks:
  • In some regimes most neurons encode pure features while a subset is highly polysemantic, and the neurons carrying the most important features tend to be monosemantic — reminiscent of vision-model layers of “mostly pure” neurons with some representing additional features at a different scale.
  • Many neurons pair one large-weight “primary” feature with smaller-weight “secondary” features, so their largest activations would look interpretable while lower-magnitude activations are polysemantic — closely matching reports that language model neurons look interpretable on top activations but activate for other patterns at lower magnitude.

The asymmetric superposition motif

• Most weights in one examined model simply implement absolute value as expected, but a few neurons implement asymmetric superposition and inhibition, in pairs:
  • One neuron stores two features with different magnitudes (e.g. $W=[2,-\frac{1}{2}]$) and reciprocal output weights (e.g. $[\frac{1}{2},2]$), so one feature heavily interferes with the other but not vice versa.
  • A second neuron heavily inhibits the affected feature exactly where positive interference would have occurred, converting costly positive interference into negative interference, which the output ReLU renders harmless.
• A few remaining weights are believed to act as small conditional biases.

The strategic picture

Why “solving superposition” matters for safety

• The goal is confidence that a model will never do certain things — deliberately deceive, manipulate. A promising tool would be the ability to identify and enumerate all features, giving a universal quantifier over the fundamental units of computation.
  • Enumeration makes it easy to assert a feature’s absence, but models that represent the world will represent unsavoury behaviours; the more useful claims are subtler, e.g. “all deceptive-behaviour features do not participate in circuits X, Y, Z”. It is also a way to address unknown unknowns by fully covering network behaviour.
• Enumeration is deeply entangled with superposition. Without superposition and with a privileged basis, features are neurons and enumeration is trivial. Without a privileged basis, one could progressively identify and remove interpretable directions Gram–Schmidt style — but with superposition a known feature direction cannot simply be removed.
• Conversely, given the ability to enumerate features one can use compressed sensing to “unfold” a superposition model’s activations into those of a larger non-superposition model. Any method granting enumeration/unfolding counts as a solution to superposition.
• Equivalent or dependent interpretability properties:
  • Decomposing activation space — the fundamental challenge of defeating the curse of dimensionality, analogous to decomposing program memory into variables.
  • Describing activations in terms of pure features — e.g. decomposing a dog-head activation into floppy ear, short golden fur, snout.
  • Understanding weights / circuit analysis — weights are typically only interpretable when connecting understandable features; the original circuits work depended on non-polysemantic neurons.
  • Even basic methods become perilous: cosine similarity to known examples of deceptive behaviour can mislead when unrelated features have positive dot products.

Three ways out

1. Create models without superposition. Easy in the toy models via $L^1$ regularization on hidden activations (interpretable as killing features below an importance threshold, especially non-basis-aligned ones), at the cost of higher loss. Generalising to real networks is expected to be possible but costly, since models seem to benefit substantially from superposition — language models know about individuals appearing far less than one token in a million, so superposition may effectively make models much bigger.
   • But neurons are not the fundamental constraint — flops are. A superposition model with $n$ neurons may match a monosemantic model with $kn$ neurons; architectures that break the flop–neuron correspondence therefore change the calculus.
   • Mixture of Experts models are the example: organising neurons into blocks that activate rarely recovers neuron sparsity as free flops. In a hypothetical limit where only activated neurons cost compute, superposition seems unlikely to be optimal — a polysemantic neuron could be split into dedicated neurons at the same cost, except where interference was already harmful. Preliminary investigation of “loss reduction per activation frequency” suggests superposition is not optimal on those terms, becoming only asymptotically as good as dedicated dimensions.
   • Framing: superposition exploits the gap between neuron sparsity and underlying feature sparsity; MoE eats the same gap, so MoE models should show less superposition. This is offered as an existence proof that competitive superposition-free models may be possible, not as a recommended path — and possibly as an explanation for why MoE has not proven more effective for capabilities than its initial case suggests.
2. Find an overcomplete basis post hoc — essentially a sparse coding / dictionary learning problem on layer activations. Advantage: model performance is untouched. Disadvantages:
   • The number of features to enumerate is no longer known.
   • Solutions are not integrated into the model’s surface computational structure — an overcomplete basis yields “virtual neurons”, widening the gap between surface and virtual structure.
   • It is a major and unfamiliar engineering challenge: for large language models, factorising something like a millions-by-billions matrix into a form perhaps a thousand times more overcomplete.
   • Interference no longer works in your favour: when training against superposition, interference pushes toward less of it; when decoding after the fact, whatever superposition training baked in is fixed.
3. Hybrid approaches — reduce superposition somewhat, then decode the rest; or change architecture to make finding an overcomplete basis more tractable independently of reducing superposition.
   • There should be some margin for free reduction, since at a model’s chosen level of superposition the derivative of loss with respect to superposition is zero.

Additional considerations

• Phase changes are cause for hope. One could imagine superposition being only asymptotically reducible; but its correspondence to a phase change means a regime exists where it does not exist at all. The research risk lies in making superposition-free models performant, not in making them at all.
• Any superposition-free model would be a valuable research tool, providing ground truth for what the features are — currently unavailable in real models, and the reason toy models are studiable at all.
• Local bases are not enough. Models often form local orthogonal bases for co-occurring features, enabling local understanding on narrow sub-distributions; but safety claims need mechanistic accounts holding on the full distribution and off it.

Discussion

Does superposition exist in real models?

The best validation available is consistency with observations about polysemanticity, and the authors judge the toy models consistent on four counts:

• Polysemantic neurons exist and form in the third toy model.
• Clean and polysemantic neurons coexist, often in the same layer, as in real networks.
• InceptionV1 has more polysemantic neurons in later layers — naturally explained if higher-level features detect rarer, sparser stimuli, since the model predicts superposition rising with sparsity.
• Early transformer MLP neurons are extremely polysemantic — consistent if the first MLP layer disambiguates token senses (e.g. “die” across English, German, Dutch, Afrikaans), which would be very sparse features.

Intuition on what generalises: superposition, monosemantic vs. polysemantic neurons, and perhaps the adversarial-example connection are likely to; the geometry and learning-dynamics results are much more uncertain.

Open questions

• Is there a statistical test for catching superposition?
• How can superposition and polysemanticity be controlled — can the phase diagram be changed so features do not land in the superposition regime? Called out as pragmatically the most important question; $L^1$ activation regularization, adversarial training, and changing the activation function all look promising.
• Are there closed-form solutions for models of superposition beyond the $n=2$, $m=1$ case?
• How realistic are these toy models, and how could that be established?
• Can the feature importance and sparsity curves of real models be estimated? On the toy models’ terms these are the most important properties for understanding the problem.
• Should superposition be expected to vanish with enough scale, remain a constant fraction, or increase?
• Are the most principled quantities being measured — what is the best definition of superposition or polysemanticity?
• How important are polysemantic neurons — how much is understood from understanding the interpretable fraction?
• How many features should be expected in superposition; can compressed sensing give tighter bounds than upper bounds?
• Do the observed feature/neuron phase changes connect to phase changes in compressed sensing?
• How does superposition relate to non-robust features — could useful, non-robust features be an artifact of superposition?
• How general is computation in superposition — is absolute value representative, and what class of computation is amenable? Does it require sparse computational structure?
• How does superposition change with non-independent features; can anti-correlation improve packing?
• Can models effectively use nonlinear representations? Suspected not, but worth testing.

Related work

• Interpretable features — semantic directions in word embeddings; interpretable neurons in RNNs, CNNs, and limited cases in transformers; alongside many polysemantic neurons.
• Superposition — earliest known reference is Arora et al. on word embeddings of polysemous words as superpositions of sense vectors, extended to sparse “atoms of discourse” and generalised by Goh; in parallel, neuron-level work grappled with polysemanticity, hypothesising disambiguation by other neurons’ activations, which became the circuits-thread superposition hypothesis. Cheung et al. explore model-level superposition (multiple independent models in one set of parameters), motivated by catastrophic forgetting — arguably feature-level superposition for highly correlated feature sets.
• Disentanglement — descends from Bengio et al.’s representation-learning position paper; typically trains VAEs or GANs whose basis dimensions correspond to major factors of variation, i.e. imposing a privileged basis on rotationally invariant representations. Neurons already have a privileged basis, so polysemanticity is anomalous behaviour and superposition a hypothesis explaining it; the question is not how to impose a basis but how to remove superposition.
  • Connections remain: superposition likely occurs in generative latent spaces and may be a major reason disentanglement is hard; disentanglement assumes few important latents, whereas many sparse, individually unimportant features could matter collectively. Also, GANs and VAEs often assume Gaussian latents — sparse variables are very non-Gaussian, but superpositions of many of them look Gaussian by the central limit theorem, so some generative architectures may force superposition.
• Compressed sensing — closely related, with important differences: compressed sensing solves an optimization problem with general techniques while the toy model must use a single linear layer plus nonlinearity; sparsity is measured as non-zero count vs. per-dimension zero probability; compressed sensing requires an incoherent measurement matrix whereas the toy model learns $W$ and often ignores input dimensions; and the toy model’s features differ in importance, so it trades accurate recovery of important features against ignoring unimportant ones.
  • Compressed sensing lower bounds therefore give upper bounds on superposition. Recovery of an $n$-dimensional $k$-sparse vector needs $m=\Omega (k\log (n/k))$; since $k=O((1-S)n)$ here, this becomes $m=\Omega (-n(1-S)\log (1-S))$ — so features are linear in $m$, modulated by sparsity, with the neat information-theoretic reading that $\log (1-S)$ is the surprisal of a dimension being non-zero times the expected number of non-zeros. Good news for eliminating superposition, though the bounds still allow it to grow sharply with sparsity.
  • Compressed sensing also exhibits phase changes (in the large-dimension limit) between almost-sure recovery and almost-sure failure — a suspicious parallel, though those concern whole-vector recovery rather than individual components.
• Sparse coding / dictionary learning — the most natural mathematical formulation of solving superposition by discovering feature directions, known variously as sparse coding, dictionary learning, and sparse frame design. Classic algorithms are expectation-maximization based (Olshausen et al., MOD, k-SVD); newer methods use gradient descent and autoencoders. Sharkey et al., following the original publication, had preliminary success extracting features from superposition in toy models with a sparse autoencoder. Note the inversion relative to neuroscience, which treats biological neurons as sparse coding their inputs.
• Neural coding in neuroscience — spanning local codes (one neuron per stimulus) to maximally dense distributed codes. “Distributed representation” is ambiguous between two senses corresponding to the paper’s decomposability (independent features represented separately) and superposition (more features than neurons). A code can be maximally distributed — every neuron participating in every input — while having no superposition, since orthogonal feature codes are equivalent under linear readout.
  • Metabolic considerations are often hypothesised to induce a privileged basis and hence a sparse code in biology; neurons are also the units implementing nonlinear transformation, so a feature needing nonlinear transformation favours a dedicated neuron.
  • A central difference from the neuroscience literature is the emphasis on the probability that features co-occur. A maximally dense distributed code makes most sense when items never co-occur; a network that may need to represent everything at once can avoid interference only without superposition. (The neuroscience “binding problem” is related but distinct; this work treats a red triangle simply as co-occurrence of the features.)
  • One neuroscience hypothesis is that compressed representations serve long-range inter-area communication, with sparse codes used locally for computation. The absolute-value results suggest useful computation is possible under moderate superposition, so regional codes might not need full decompression.
• Additional connections raised by readers — Vector Symbolic Architectures and Hyperdimensional Computing, whose use of quasi-orthogonal vectors and the “blessings of dimensionality” closely relates to superposition; frames as a generalisation of a basis, with the “Mercedes-Benz frame” equivalent to the observed triangular geometry; and a broader body of work on encoding sparse vectors in lower-dimensional dense ones beyond compressed sensing and sparse coding.

Comments and replications

External researchers were invited to comment, following the Circuits Thread and Distill discussion-article model:

• Kshitij Sachan (Redwood Research) replicated all plots in the Demonstrating Superposition and Superposition as a Phase Change sections plus uniform superposition geometry, and found phase diagrams look quite different depending on activation function — suggesting some activations induce more polysemanticity than others. Redwood’s follow-up paper, Polysemanticity and Capacity in Neural Networks, studies a variant model with analytical traction, frames superposition as constrained optimization, and analyses activation functions.
• Tom McGrath (DeepMind) reproduced the same two sections and contributed an exact solution for the expected loss in the $n=2$, $m=1$ ReLU output model (ignoring biases), using delta functions for the sparse part of the input distribution and replacing the ReLU with a restricted domain of integration. This reveals a new “confused feature” phase with $W_1\simeq W_2\simeq 1/\sqrt{2}$ — similar weights rather than antipodal — arising at low sparsity when both features are important, and without correlated features. The transition from one antipodal regime to another is continuous, explaining the blurry region near the phase diagram’s triple point, while the antipodal-to-confused transition appears discontinuous. Some parameter combinations give two minima, which could account for the energy-level jumps as solutions hop between them; near phase transitions the global minimum can have a much smaller basin of attraction than local minima.
• Jeffrey Wu and Dan Mossing (OpenAI Alignment) reproduced plots from Basic Results, Feature Dimensionality, and the energy-level jumps with little difficulty.
• Spencer Becker-Kahn (FHI / SERI MATS) independently replicated key graphs from Basic Results and produced consistent plots with very small toy models.