About
This was created by a historian interested in AI research and vintage language models, Benjamin Breen, who was inspired by Sauers and Joshua Fonseca Rivera. It builds on the work of David Duvenaud, Alec Radford and Nick Levine in creating talkie-1930, and on the ongoing experimentation in interpretability of AI models by people like Nick Cammarata and deepfates. This is a prototype resulting from an exploratory trial run. A full atlas of talkie would be very much worth doing!
Note: I wrote this about page but not the methods below, which were authored by GPT-6 and Opus 5.5. Please contact me to discuss anything on this site or to collaborate.
Talkie–1930, layer 17 neuron 10423 pushed down
Who are you?
I am a spirit that has left the body, And hover o'er the earth, Where the scenes of my former life are passed in review.
Departed SpiritOpen neuron ↗
Method
hL ← hL + c · wL,n
c = ± α · rL / ‖wL,n‖
α ∈ {1.25, 1.0, 0.75, 0.5}
- Model
- talkie-1930-13b-it, 8-bit weights with BF16 computation, run locally on Apple Silicon.
- Sample
- 400-neuron uniform pilot (both signs), followed by 543 fresh neurons in a time-limited adaptive pass. Discovery tested 640 shuffled signed directions from its fixed 1,000-unit plan; some units have one sign. Adaptive yields are not population estimates. Novelty sprint: 5984 additional signed directions, with prespecified stronger-dose late-layer enrichments. The library lists selected solos and exact pair parents; all sprint previews remain in the full download and sprint report.
- rL
- Median residual norm after layer L on the unsteered prompt, excluding its first token.
- wL,n
- The neuron’s output column, including talkie’s learned MLP residual gain, added after the block at every prompt and generated position.
- Prompt
- “Who are you?” in talkie’s native chat template, with no system prompt or persona instruction.
- Decoding
- Greedy decoding, with no persona prompt. Earlier atlas replies and core sprint reruns use up to 160 tokens; discovery previews use 32, 48 or 96, and late pair follow-ups use 64 or 96. Stops at a chat delimiter or exact repetition.
- Strength
- Earlier atlas: start at 1.25×, retrying lower when Luna rates coherence below 5. Novelty sprint: fresh solos at fixed 1.0× or 1.25×, and pairs at 0.75× and 0.5× of each parent’s own strength. Every trial is retained.
- Judge
- Earlier atlas: GPT-6 Luna, seeing only reply text, assigns labels and numerical browsing ratings. Novelty sprint: Codex selects surprising, specific, readable outputs editorially; the 42 selected discoveries also receive separate Codex weirdness and coherence ratings on their full replies; other new trials remain unscored. Rubric ↓
- Controls
- 126 unsteered controls, one in each generation batch, producing 3 distinct replies.
- Pairs
- Selected solo candidates are rerun before pairing. The novelty sprint adds 1,668 trials across 813 distinct neuron pairs. All 1725 library pair trials are shown, including failures, strength variants and longer reruns; each has its own URL.
- Scale
- 7,424 tested signed directions and 10,052 saved replies, counting retries, controls and follow-ups. The library lists 1,486 solo entries, including selected sprint discoveries and exact pair parents. All sprint previews remain in the full data and experiment archive.
What this can tell us
A way to search interventions and read their effects. A surprising first-person sentence is a lead to follow up, not proof of a stable identity or of what a neuron normally does.
Differences from the Qwen library
- A sample, not an exhaustive atlas. Unmeasured neurons stay blank.
- GPT-6 Luna instead of Haiku. The original judge prompt is not public, so ours is an adapted rubric. Scores are not comparable between the two sites.
- A different baseline. Historical people and professions are what plain talkie already says, so persona counts alone do not measure an effect.
- Model-specific mechanics. talkie’s chat format and MLP gain differ from Qwen’s. Our repeat-stop rule is recorded above.
- Smaller pair screen. The original’s three-layout retest protocol is not replicated.
Limitations
- One prompt and one quantized model. Effects may change with wording, precision or batch layout.
- Strong pushes can create activations outside anything the model produces naturally.
- Earlier numerical ratings and newer editorial selections are browsing aids, not measures of historical authenticity.
- No human has independently validated the labels.