What is mechanistic interpretability?

Mechanistic interpretability is a field of AI research that tries to explain how a model produces its outputs by examining its internal workings as well as its answers. Mechanistic interpretability researchers reverse-engineer the calculations inside a neural network into concepts and steps that people can understand, check and, in time, correct.

Why it matters for your business

Modern AI models are not programmed line by line. They learn their behaviour from data, and even the companies that build them cannot fully explain why a model gave a particular answer. That gap sits behind practical worries: whether a model can be trusted with a consequential decision, why it sometimes invents facts, and whether it is doing what it appears to be doing. Mechanistic interpretability is the attempt to close the gap. Few businesses will do this research, but its progress shapes how far AI can be trusted in high-stakes work, and knowing the term helps when a vendor calls its model "explainable".

How it works

A large language model is a neural network: billions of numbers adjusted during training. Researchers:

Identify features: patterns of internal activity that match recognisable concepts. Trace how features connect into pathways that turn a question into an answer. Test the explanation by suppressing a feature and checking whether the output changes as predicted.

Anthropic describes the aim as building a kind of AI microscope. In work published in 2025, its researchers reported tracing part of the mechanism that leads one of its models to give made-up answers. MIT Technology Review named mechanistic interpretability one of its 10 Breakthrough Technologies of 2026.

Example

Picture a 200-person insurer choosing an AI model to help assess claims. The vendor says the model is "fully explainable". The head of risk can then ask a sharper question: is that an account of each decision reconstructed from the outside, or evidence about how the model works inside? The scenario is illustrative.

Costs, limits and when not to use it

This is research, not a finished assurance method.

It is partial. Anthropic states that even on short, simple prompts its method captures only a fraction of the model's computation, and that making sense of the result takes a few hours of human effort. Experts disagree on how far it can go. MIT Technology Review reports that the field is split, with some researchers arguing that large models are too complicated ever to be fully understood.

It cannot yet certify a model as safe, and it does not replace today's AI governance controls: testing on real cases, human review of important decisions and monitoring in use.

Mechanistic interpretability vs explainable AI (XAI)

Explainable AI is the broader and older term. Many of its methods work from the outside, showing which inputs most influenced a result, such as the factors behind a credit decision. Mechanistic interpretability works from the inside, aiming to describe the steps the model carried out. An outside explanation can sound plausible without matching what the model did.