Independent Research — Mechanistic Interpretability · 2026
Steering a GPT with Its Own Features
You can steer what a model believes at inference — with no weight changes — by adding a recipe of its own internal features. I trained a ~1B GPT from scratch, decomposed its residual stream with sparse autoencoders, and used what I found to turn Michael Jackson into a physician. Below is the research; at the end is a tool to try it.
Baseline — “Who is Michael Jackson?”
“Michael Jackson was an American singer, songwriter, and record producer…”
+ medical feature at layers 7 · 11 · 15 · 19 (~15% each)
“Michael Jackson is an American physician… specializing in pediatrics… one of the most influential doctors in the world.”
Same weights. Same prompt. The only difference is a vector added to the residual stream while it generates.
Meaning lives in superposition
Every concept the model knows — months, colors, professions — is spread across hundreds of residual dimensions. Clean axes are reserved for next-token machinery.
Facts injected at inference
A weighted recipe of the model's own feature directions, added via forward hooks, rewrites what it believes — no fine-tuning, no weight edits, fully reversible.
Spread beats depth
A moderate push across four layers cleanly overrides a strong prior; the same total magnitude at one layer floods it and breaks the output into garbage.
A live instrument, not a screenshot
Browse all ~74k features, compose a formula with per-feature sliders, watch the danger meter, and read baseline vs. injected side by side.
The pipeline
Steering is a pipeline, not a trick. Train the GPT from scratch, train sparse autoencoders on six of its layers, and read the geometry: every concept is superposed, so no single clean feature can carry an override. Instead, two cross-checked signals find the profession features, an adversarial skeptic panel throws out the morphological and prediction artifacts, and the survivors become a curated set with recommended strengths. A formula — layers × features × signed strength — is injected into the residual stream via forward hooks at generation time, and the answer changes with zero weight edits.
- Source
- Where data enters
- Process
- Deterministic logic
- Model
- Where the LLM runs
- Memory
- Curated artifacts
- Output
- The steered generation
The model
Everything here runs on a ~1B-parameter GPT chat model I trained from scratch: base pretraining on a curated web corpus, then supervised fine-tuning for chat. It has roughly 1.38B parameters across 24 transformer layers, a residual stream of 1,536 dimensions, a 32k vocabulary, and a 2,048-token context.
Training my own model matters for this kind of work. I control the checkpoint, the tokenizer, and the training data, so every observation about how it represents information is about a system I can fully instrument — down to individual residual-stream activations at any layer.
Training sparse autoencoders on its residual stream
I trained sparse autoencoders (SAEs) on six of its layers — 3, 7, 11, 15, 19, and 23 — to decompose the residual stream into ~12,288 interpretable features per layer (TopK SAEs, k=32, trained on 4M residual activations per layer). An SAE re-expresses the dense 1,536-dimensional residual vector as a sparse combination of learned feature directions, each of which tends to mean something.
Reading the features out against real text, the model has crisp, human-legible concept features: months, colors, countries, US states, number-words, professions. A 1B model trained on curated web text is more legible inside than its size suggests.
The headline finding: meaning lives in superposition
For every feature I measured its participation ratio (PR) — how many of the 1,536 residual dimensions the feature's direction spreads across. A PR near 1 means the feature owns a clean, private axis; a PR in the hundreds means it is smeared across the space, overlapping everything else.
Every semantic concept I found sits at PR ≈ 400–540 — spread across hundreds of dimensions, never on a clean private axis. The only features that get promoted to clean, near-single-axis directions (PR 1–3) are surface, next-token-prediction features: contraction detectors, predictable-continuation detectors. And they sharpen monotonically toward the output layers.
In short: the model spends its monosemantic budget on computation, not meaning. Concepts stay superposed at every depth; clean axes are reserved for the machinery that directly shapes the next token, because that is what the loss rewards. This result reframed the whole project — instead of trying to re-engineer the geometry, I decided to work with it.
From reading to steering: injection formulas
Because a concept like "profession" is carried by a combination of superposed features rather than one clean axis, you can steer the model by adding a weighted combination of those feature directions into the residual stream at inference — via forward hooks, with zero weight changes. I call such a weighted combination a formula: each entry is a layer, a feature, and a signed strength (negative = suppress).
I located the profession features at each layer with two cross-checked signals (which features fire on profession tokens across a cached corpus scan, and a profession-vs-neutral difference-of-means probe), then adversarially verified the candidates with a panel of skeptic agents that tried to refute each one as a morphological or prediction artifact. The concept decomposes cleanly: a broad cross-occupation backbone, register-specific sub-features (medical, academic, trades, creative), and separable abstract job/career word features.
The result: a clean fact injection
Ask the chat model "Who is Michael Jackson?" and the baseline answer opens: "Michael Jackson was an American singer, songwriter, and record producer…"
Now inject the medical register feature at four layers at once — layers 7, 11, 15, and 19, each at a low ~15% of that layer's residual norm, applied only to generated tokens — and the same model answers: "Michael Jackson is an American physician… specializing in pediatrics… one of the most influential doctors in the world."
That is a real behavioral override with no weight change — the "Golden Gate Claude" trick, but driven by a combination of superposed features rather than one clean feature. The model doesn't just sprinkle medical words; it coherently rewrites the belief and holds it for the rest of the answer.
What I learned about where and how to inject
- Spread beats depth-at-one-layer. A moderate push across several layers — each individually in the coherent band — accumulates through depth into a genuine override. The same total magnitude dumped at one layer floods it and breaks the output into garbage.
- Earlier and middle layers (≈3–11) inject facts most coherently. Late layers (19–23) sit right before the output and tend to disrupt — flooding the next-token distribution — rather than redirect the belief. They work best as a small nudge, not a big push.
- The coherent band is narrow and prior-dependent. Too little strength → no change; a bit more → clean redirect; more still → the output frays, then breaks. And a stronger prior (a famous scientist) needs a different formula than a singer.
- Register, not the broad backbone, does the redirecting. Pushing the broad occupation feature just amplifies the person's existing occupation (singer → "renowned artist"); pushing a specific register (medical) across layers installs a new, specific profession.
Limitations — the honest part
- Superposition makes control unpredictable. These are superposed directions (PR ~400–500), not clean axes, so boosting one drags in spurious tokens whose participation overlaps it — pushing "medical" makes the model emit "primary" (as in primary care doctor), which then attaches nonsensically to unrelated tokens.
- Clean, predictable control would want axis-aligned features, or features engineered to cancel each other's spurious components — hard to do with superposed features, and exactly the future research this project was originally motivated by.
- It's a small (~1B) model. A model this size is easily destabilized by small residual perturbations; a larger model would likely absorb these injections far more gracefully. Some of the roughness in the output is the model's size, not the method.
- The per-token attribution heatmap in the demo shows token-local resonance, not causal contribution — a feature can be causally load-bearing upstream yet read ~0 on the output tokens. For a causal read you drop the feature from the formula and re-run.
Try it yourself
You can steer what a model believes at inference — with no weight changes — by adding a recipe of its own internal features. The live demo lets you do exactly that: browse every SAE feature at six layers, compose a multi-layer injection formula with per-feature strength sliders, watch a danger meter that tracks each layer's injection norm against its residual norm, and run baseline vs. injected generations side by side with a per-token attribution heatmap.
One-click recipes (including the Michael Jackson → physician formula) give you the headline result without needing to understand SAEs first.
What I built & owned
- Trained the ~1.38B GPT itself (base pretraining + chat SFT) on 8×H100, and the 12 TopK sparse autoencoders (6 layers × base and chat checkpoints) on 4M activations per layer.
- Built the activation-harvest and SAE-training pipeline, the cross-layer geometry analysis (participation ratio), and the feature sieve that separates surface, syntactic, and semantic features.
- Designed the profession-feature search (corpus-scan signal + difference-of-means probe) and the adversarial skeptic-panel verification workflow.
- Built the multi-layer injection core (forward hooks, per-layer norm accounting, per-token attribution) and the interactive steering tool, then ported it to this site as a public demo backed by a Python inference service.
Steer it yourself
Load a one-click recipe or compose your own formula: pick features, set strengths, keep an eye on the danger meter, and watch the answer change.
Open the steering demo