Your AI Coding Agent Isn’t Using Documentation the Way You Think

The problem is not necessarily that the documentation is bad. It may be that the organisation is improving the wrong documentation surface. A new preprint, From Agent Behaviour to Agent-Friendly Documentation, offers an unusually detailed look at what coding agents actually do with technical documentation. The researchers analysed 557 real agentic coding sessions containing 94,813 development events, including 3,033 documentation interactions. They complemented those traces with 33,097 pull requests created by coding agents, covering 690,260 file-level change records.

Their central finding challenges a basic assumption of documentation strategy: coding agents do not use documentation like slightly faster human developers. They have created a documentation system of their own.

The Documentation Hierarchy Has Flipped

Most documentation programmes are organised around familiar artefacts: API references, architecture guides, installation instructions, examples, troubleshooting pages, and READMEs. These documents still matter to people. But they were not the documents that dominated the observed agent behaviour.

Agent-facing artefacts accounted for 60.5% of all recorded documentation interactions. The largest category was agent instruction files such as AGENTS.md, CLAUDE.md, SKILL.md, and rules for tools such as Cursor and Copilot. These represented 35.4% of interactions. Another 25.1% involved agent working notes: plans, thoughts/ directories, brainstorms, review logs, and other files agents create to organise their own work.

API references accounted for 1.3%. Troubleshooting documentation accounted for 0.4%. Put differently, instruction files appeared roughly 27 times as often as API references. The documents attracting the most investment from traditional documentation teams were peripheral in these repository-local traces. The documents that shape agent behaviour directly were central.

This does not mean companies should delete their API documentation. The study could not see documentation read on external websites, knowledge already present in a model, inline comments, docstrings, or instruction files loaded automatically by an agent runtime. Its numbers describe an observable slice of behaviour, not the whole information environment.

But the imbalance is too large to dismiss. If an organisation wants to improve how agents work inside a codebase, the first place to look may no longer be the developer portal. It may be a Markdown file in the repository root.

Agents Read Documentation to Think

The conventional model of documentation use is linear:

1. A developer has a question.
2. They find the relevant document.
3. They apply what they learned.
4. They test the result.
5. They update the documentation if necessary.

The observed agent behaviour looked different. After reading a document, an agent was most likely to read another document or continue reasoning. A further documentation read followed 27% of reads, while reasoning followed 24.5%. An immediate code edit followed only three of 1,328 documentation reads, a transition probability of 0.2%. That number is striking, but it requires care. Documentation may influence an edit after several reasoning steps, and the statistical result changes depending on how development stage is taken into account. In the unadjusted three-event analysis, code editing after consultation was essentially at the session baseline. In the adjusted model, the association rose above that baseline. The authors therefore treat the connection between reading documentation and editing code as unresolved.

The more robust conclusion is that documentation consultation is not a simple read-then-act transaction. It is a recurrent process in which agents retrieve information, interpret it, revisit other documents, and externalise parts of their reasoning. Documentation is becoming part reference library, part scratchpad, and part context-management system. That is a very different design problem from making a web page easier to scan.

Documentation Is Becoming Working Memory

The agents in the study did not merely consume documentation. They produced it almost as frequently. The researchers recorded 1,615 consultation events and 1,401 documentation creation or editing events. For every documentation interaction spent reading, searching, or discovering, agents performed about 0.87 authoring actions. Among sessions containing documentation activity, 58.2% included both reading and writing.

The pull-request data tells a similar story at a much larger scale. Documentation changed in 41.5% of the 33,097 analysed agentic pull requests. Files such as AGENTS.md, CLAUDE.md, and copilot-instructions.md were among the most frequently changed individual documents. This creates a feedback loop that traditional documentation governance was never designed to handle:

1. An instruction file shapes an agent’s behaviour.
2. The agent performs the task.
3. The agent edits the instruction file or creates a working note.
4. That new artefact becomes context for a future agent.

Documentation is no longer only a human-authored input to software development. It is also an agent-authored output that can influence the next execution. This is potentially valuable. An agent can leave behind a plan, record a constraint, or capture a repository-specific procedure that makes later work more efficient. But it also creates a new maintenance surface. Temporary plans can become permanent clutter. Incorrect assumptions can acquire the appearance of repository policy. Two agents can leave contradictory instructions. A working note created for one task can be mistaken for durable architectural guidance. The question is not simply whether the agent can read the documentation, but rather whether the organisation knows which documents are governing the agent and who is responsible for them.

Code Still Leads and Documentation Follows

Agents may write a great deal of documentation, but they do not usually begin with it. Among multi-commit pull requests that changed both code and documentation, code was touched first in 47.3% of cases. Documentation was touched first in 10%. The remaining 42.6% changed both in the same commit, making the order impossible to observe. Across the full comparison, code came first 4.7 times as often as documentation. When the two appeared in different commits, code led in 82.5% of cases.

This resembles a long-standing human habit: implement first, document afterwards. For organisations hoping that AI agents will automatically keep specifications and implementation in continuous alignment, the finding is a warning. High documentation activity does not necessarily mean documentation is driving the work. It may simply be recording what the agent has already done. There is a crucial difference between documentation as a specification and documentation as a receipt. The former constrains the implementation. The latter describes it after the fact. Most of the observed agent behaviour points towards the receipt.

The Real Problem Is Verification

The paper’s most consequential result may be what the researchers did not find. They observed no explicit sequence in which an agent consulted documentation and then used it as an oracle to verify the implementation. Under the study’s operational definition, documentation-based validation occurred zero times.

Testing and building were also less frequent immediately after documentation consultation. Within the next three events, test runs appeared at 23% of their normal rate, while builds appeared at 15%. These associations remained below baseline after the researchers adjusted for development stage and other factors.

This does not prove that reading documentation causes agents to test less. The study is observational, and some verification may occur inside model reasoning or beyond the short event window. But it does reveal a gap between common advice and recorded behaviour. We often say agent-friendly documentation should be actionable and verifiable. In these traces, the read-to-action connection was statistically unsettled, and the verification step was not observed at all.

Prose can tell an agent what should be true. It does not automatically make the agent check that it is true. This is why “clearer documentation” is not a complete reliability strategy. If a rule matters, the system needs a way to enforce or test it. Runnable examples, schema contracts, doctests, policy checks, acceptance tests, and explicit validation commands can turn a statement into an observable control. The study does not prove that these interventions will solve the problem. It shows why relying on prose alone is not enough.

Agents Rarely Reach for the Troubleshooting Guide

Another common assumption is that agents consult documentation when they get stuck. Across all documentation interactions, 70.2% were self-initiated as part of normal task progress. Only 7.5% were failure-driven. After 2,034 failure episodes, reading documentation was the first recovery move in 5.4% of cases. Agents were far more likely to read code, retry the action, edit directly, or search the codebase. Troubleshooting documentation appeared only 11 times in the entire set of 3,033 documentation interactions.

Again, this does not establish that troubleshooting guides are useless. The study captured repository-local files more reliably than external web pages, and the small number of documentation-led recoveries was too limited to compare success rates. But it does suggest that passive availability is insufficient. A troubleshooting guide hidden behind a link may not become part of an agent’s recovery path simply because it exists.

The observed agents repeatedly opened local files, but the researchers found no instances of following references between documents. For agent use, proximity and local retrievability may matter as much as elegant information architecture.

How to Build Documentation for Agents

The research supports a reallocation of effort. It does not provide a finished playbook, but it points towards several practical changes.

1. Treat Agent Instructions as Production Infrastructure

Files such as AGENTS.md and CLAUDE.md should not be informal prompt dumps. They shape operational behaviour and deserve an owner, version history, review process, and tests. Changes to them can alter future agent actions even when no application code changes.

2. Separate Durable Rules From Temporary Working Notes

An architectural constraint and a one-task scratchpad should not have the same apparent authority. Give plans, reasoning logs, and verification notes clear locations, lifetimes, and retention rules. Working memory should be useful without quietly becoming permanent policy.

3. Put Critical Guidance Where the Agent Works

Repository-local, self-contained guidance is easier for an agent to retrieve than a chain of links across a documentation portal. Keep high-value instructions close to the relevant code and make their scope explicit. A root-level rule should say whether it applies globally; a component-specific rule should live near that component.

4. Turn Important Prose Into Executable Checks

If an agent must preserve an API contract, encode it in a schema or contract test. If an example must remain valid, make it runnable. If a policy must never be violated, add an automated check. Documentation can explain the rule, but the system should verify it.

5. Observe the Documentation Loop

Final task success is not enough. Teams should be able to answer:

• Which instruction files did the agent load?
• Which working notes did it create?
• Did it modify the documents that will guide future agents?
• Which rules influenced the implementation?
• What validation followed consultation?
• Did temporary context become durable repository content?

Without this visibility, an agent can produce an acceptable patch while quietly changing the control layer around future work.

6. Design for Two Audiences

Human developers still need explanation, examples, conceptual models, and navigable reference material. Agents need compact local instructions, unambiguous scope, machine-checkable constraints, and clean context boundaries. These needs overlap, but they are not identical. Documentation strategy now has to serve both.

A Significant Result, With Significant Limits

The paper is a preprint, not a peer-reviewed final publication. Its session data comes from opt-in telemetry, with 87% of the broader corpus attributed to one agent family. Its pull requests come from public repositories that adopted coding agents early and may not resemble private enterprise codebases. The study also identifies documentation by file path. It cannot see everything an agent knows or reads, and instruction-file exposure is undercounted when a runtime loads those files automatically. The “agent working note” category—one of the paper’s headline findings—was classified with a language model and has not yet been validated by human coders. The exact 60.5% share should therefore be treated as provisional, not as a universal constant.

Most importantly, the research describes behaviour. It does not prove that the observed behaviour is optimal or that a particular documentation intervention will improve outcomes. Yet the overall pattern is difficult to ignore. Coding agents are not merely consuming the documentation system built for developers. They are creating a parallel layer of instructions, plans, notes, and verification logs. They read that layer frequently, write to it almost as often, and sometimes modify the very files that will govern their future behaviour. The next generation of technical documentation will therefore be more than a website or a manual. It will be part of the agent’s operational environment: locally retrievable, carefully governed, partly executable, and observable as it changes.

The important question is no longer “Is our documentation agent-friendly?”, but  “Which documentation governed the agent, who maintains it, and how do we know the agent checked its work?”. That is not a writing problem. It is systems design.

Reference

Zhijun Gao and Jing Chen, From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation, arXiv preprint, 2026.