J-Space: can fine-tuning cultivate internal reasoning?

It started from a piece of mechanistic-interpretability intuition: superposition and axis-alignment work suggests that if you force a model to behave a certain way, you also constrain how it representsthings internally. So what happens to a model’s internals when you force the shape of its answers?

The motivating example is arithmetic. If every training answer spells out every step — “3 × 2 is 6, then I add 25, then…” — the model can learn to imitate the verbal procedure. Final-answer accuracy goes up, but the reasoning has moved onto the page: the model is transcribing a recipe, not thinking. Exhaustive visible chain-of-thought, made the dominant training signal, can raise the score while hollowing out the very thing I want to grow.

So the goal is deliberately not a task agent. It’s a small model used as raw intelligence— one that has to reach for its own internal structure to reason, because the training data leaves just enough room that it can’t simply copy a fully-worked trace. An empty or flat J-Space would be a failure of the hypothesis, not a detail to paper over.

J-Space is a term from Anthropic’s 2026 global-workspace research: a small set of internal patterns — only a few dozen concepts at a time — that a language model uses for higher-order reasoning, set apart from the automatic processing that handles fluent speech and simple recall. Ablate it and the model still talks fluently but loses multi-step reasoning; it plays the “workspace” role from global-workspace theory — information written in once, then broadcast to many downstream computations.

To read it, Anthropic built the Jacobian lens and open-sourced it — and I use that same tool (their jacobian-lens package). It transports a residual-stream vector at layer l into the final-layer basis with the average input–output Jacobian J_l = E[∂h_final/∂h_l], decodes it through the model’s own unembedding, and returns a ranked list of vocabulary tokens — “what this internal activation is disposed to make the model say.”

My contribution isn’t the instrument — it’s the question. Anthropic also showed the workspace is trainable: their “counterfactual reflection training” found that training a model on what it would say if asked to reflect changed what it actually does. This project pushes on that opening in a model I fully control — can a deliberately-shaped curriculum cultivate a richer workspace, and can I prove it rather than assert it?

Two honesty rails hold throughout. The lens is a vocabulary-constrained, average-linearized projection — not a full feature decomposition, and not proof that a concept is causally used. And the readable layer band was pre-registered at L24–L33 before looking at results, so the analysis can’t drift to wherever the story looks best.

A readable J-space genuinely exists in this model — and it’s legible before the tokens surface. On a prompt about Docker, the lens at layers 27–33 is already disposed toward the right concepts several positions before the model emits them, while a plain logit-lens at the same cells shows only code-token noise.

• A readable band, not the whole network — but a shaky one. The L24–33 estimate rests on just four prompts, so the v2 analysis explores where each item is represented instead of assuming a band.
• Legible is not the same as correct. On a creatine biochemistry prompt the J-space surfaced domain-correct chemistry — amino, nutrients, molecules, pathways, precursor — with no hedging or uncertainty signature, even when the model's actual answer was confidently wrong.

Before any training runs, the decision rule is fixed. A treatment gets credited only if it clears all of this — and answer accuracy going up is explicitly not on the list.

• Internal reachability. Macro IR@10 improves ≥5 points, 95% CI excludes zero, and ≥6 of 8 domains are non-negative.
• Selectivity. Relevant-intervention selectivity improves while irrelevant-paraphrase invariance error stays ≤ one-third as large.
• Causality. At least one J-direction swap family exceeds 2× the median matched-random-swap effect — decodable is not enough.
• Replication. Effects hold across 3 seeds and under both a fixed and a refitted lens.
• Guardrails. Held-out accuracy doesn't regress >2 points and the capability / identity canary holds.

Standing rule: do not promote a checkpoint when only answer accuracy, answer-token rank, polish, or template performance improves.

The whole project is an Atomic workflow — not a script that trains and prints a number, but a loop that measures the model’s internals, diagnoses what’s missing, designs data to fix it, retrains, and re-measures— wrapped in a second loop that audits its own example-designer and rewrites its own prompts for the next round. Atomic is what makes the load-bearing stage possible: reading a J-Space readout well is different for every item, so it’s a prompt-engineered agent exploring each one, not a fixed program scanning fixed layers.

The curriculum bet is on relational tasks — problems that require relating things, weighing alternatives, or integrating cause and effect — kept concise (a target of ≤32 median tokens), not long teacher prose, so the model has to do the connecting itself. Probes span eight domains:

• The measurement stage is specified, not built. The v2 J-lens analysis — and the per-item activation + lens-quality artifacts it needs — are designed in detail but not yet implemented. Until they exist and run, the loop can't prove J-Space improved, and no checkpoint is promoted.
• Decodable ≠ causal. Decodable latent patterns can have no behavioural effect under matched controls. A legible readout is a hypothesis; the per-item falsifier and causal-swap controls exist to try to kill it.
• The readable band is an estimate, not a law. The L24–33 band came from four prompts — an unvalidated first look. The v2 analysis explores where each item is represented instead, and a lens-quality calibration must pass before the analysis is even allowed to run.
• Training is intentionally gated. Nothing trains until the battery is mechanically finalized and hashed. That's a feature: no model trains on an unfrozen measurement set by accident.