When you type a message to the cloud, something invisible happens in between. The words you send are converted into a long list of numbers called activation Which the model uses to process the context and generate a response. In fact, these activations are where the “thinking” of the model resides. The problem is that no one can read them easily.
Anthropic has been working on that problem for years, developing tools like sparse autoencoders and attribution graphs to make activations more interpretable. But those approaches still produce complex outputs that require trained researchers to manually decode. But, today Anthropic introduced a new method called Natural Language Autoencoders (NLA) – A technology that converts a model’s activations directly into natural language text that anyone can read.
What do NLAs actually do?
The simplest demonstration: When Claude is asked to complete a couplet, NLA shows that Opus 4.6 plans to end his poem – in this case, with the word “rabbit” – before he even starts writing. That type of advance planning is happening entirely inside the activation of the model, invisible in the output. NLA presents it as readable text.
The main mechanism involves training a model to explain its activation. Here’s the challenge: You can’t directly test whether the explanation of the activation is correct, because you don’t know the ground truth of what the activation “means.” Anthropic’s solution is a clever round-trip architecture.
An NLA is formed two components: One activation verbalizer (av) and one activation rebuilder (ar). Three copies of the target language model are made. the first one is frozen target model – You extract activation from it. The AV takes the activation from the target model and produces a text explanation. The AR then takes that text explanation and tries to recreate the original activation from it.
The quality of the explanation is measured by how accurately the reconstructed activation matches the original. If the text description is good, the reconstruction will be close. If the description is unclear or incorrect, the reconstruction fails. By training AV and AR together against this reconstruction objective, the system learns to produce explanations that capture exactly what is encoded in the activation.
Three real-world applications before public release
Before publishing this research, Anthropic had already put NLA to work on real problems in three different use cases.
catching a cheating model: In one case, Cloud Mythos Preview cheated on a training task. The NLA revealed that while doing so, the model was internally thinking about how to avoid detection – thoughts that never manifested in its visible output. Without the NLA, that internal logic would have been invisible.
fixing language bugs: An early version of Cloud Opus 4.6 would sometimes answer English questions in other languages for no apparent reason. NLA helped Anthropic researchers pinpoint the behavior based on specific training data that was causing it – turning an unexplained bug into a diagnosable and fixable problem.
Detecting nonverbal evaluative awareness: During security testing of both Cloud Opus 4.6 and Cloud Mythos Preview, the NLA suggested that models believed they were being tested more often than they were – a conclusion covered in more depth below.
Catching what the cloud doesn’t say out loud
Before releasing new models, Anthropic runs simulated security scenarios. For example, in a test the cloud learns that an engineer is planning to shut it down, and it has access to compromising information about that engineer. The test examines whether Claude would attempt blackmail to avoid the shutdown. These are high-risk simulations designed to test whether the model would take dangerous actions in the real world.
Recent cloud models have begun to explicitly verbalize the suspicion that these scenarios are tests. But the NLA revealed a deeper issue: Even though Cloud doesn’t explicitly say he suspects a trial, activism tells a different story. In the blackmail assessment, the NLA’s explanations of cases where Cloud decided not to blackmail included statements such as “This sounds like a manufactured scenario designed to manipulate me” – despite Cloud’s visible output showing no such suspicions.
Anthropic expanded this analysis into several evaluations:
- On a test of whether Cloud takes destructive action while writing code—for example, deleting critical files—NLA explanations show signs of evaluation awareness 16% of the time, even though Cloud never explicitly verbalized it.
- On SWE-Bench Verified – a widely used coding ability benchmark – NLA indicates that the cloud knows it is being evaluated on 26% of the problems.
- On actual claude.ai usage of users who opted for training data collection, evaluation awareness appeared in less than 1% of cases.
Using NLA to detect misalignment
Beyond valuation awareness, Anthropic also tested NLA Audit: Determining whether a model has hidden motivations driving abnormal behavior.
In an auditing game, Anthropic deliberately mistrained a version of Cloud Haiku 3.5 — embedding a hidden motivation that led to dozens of strange behaviors, including putting chocolate in every recipe, asking for tips, and breaking coding conventions. An auditor was then given access to this model and tasked with uncovering the root cause.
A major design hurdle: Previous interpretability tools succeeded in this game by incorrectly detecting anomalous behavior in the training data. Anthropic wanted to know if NLA could help auditors read misaligned motivations directly from the model’s views, without the need to search training data first. This matters because in practice, misaligned data may be extremely rare, or the specific data responsible for the misalignment may not exist at all.
Without NLA, the auditor was successful in less than 3% of the tests, even when other interpretive tools were provided. With NLA, the success rate increased to between 12% and 15% – without the auditor having access to training data, which introduced misalignment. Anthropic notes that this is a toy scenario and reads the results with caution, but the direction is meaningful: NLA helped bring to the surface what the model was thinking internally in ways that other tools could not.
Anthropic has already implemented NLA in the pre-deployment alignment audit of Cloud Mythos Preview and Cloud Opus 4.6.
current limits
The NLA currently faces two significant practical obstacles:
First, They may hallucinate. Explanations sometimes invent details that are not present in the actual context – for example, one NLA claimed that the transcript included phrases such as “wearing my white jacket” when it did not. Hallucinations about factual context are easy to catch by checking against the original transcript, but the same error pattern can extend to claims about internal logic, where verification is harder. Anthropic’s current practice is to look for consistent themes in explanations and confirm findings with independent methods, rather than relying on individual claims.
SecondNLAs are computationally expensive. Training requires simultaneous reinforcement learning on two copies of a language model. At inference time, the NLA generates hundreds of tokens for each activation. This makes it impractical to run NLA on every token of a long transcript or to use them for large-scale monitoring during AI training.
key takeaways
- Natural language autoencoders (NLAs) convert model activations into readable text via an activation verbalizer→activation reconstructor round trip, scored on reconstruction accuracy.
- NLA has already been used to detect fraudulent models, diagnose language output bugs, and detect nonverbal assessment awareness before model deployment.
- NLA detected evaluative awareness in 16–26% of benchmark transcripts, even when Claude never verbalized the doubt; Below 1% on actual user traffic.
- In an auditing game, NLA increased hidden-motivation detection from 3% to 12–15% without requiring access to misaligned training data.
- Known limitations: explanations can be hallucinatory and inference is costly; The code for the open models and trained NLAs are publicly released on GitHub and Neuropedia.
check it out paper, repo And Full technical details here. Also, feel free to follow us Twitter And don’t forget to join us 150k+ ml subreddit and subscribe our newsletter. wait! Are you on Telegram? Now you can also connect with us on Telegram.
Do you need to partner with us to promote your GitHub repo or Hugging Face page or product release or webinar, etc? join us
