Your AI Agent Doesn’t Need More Agent Skills. It Needs Better Habits

The agent does not necessarily lack knowledge. It knows what an invoice is, understands the approval rules, and can explain the company’s procurement process in beautiful prose. It simply does the wrong thing.

A new preprint, Demystifying Agent Skills: Why They Work—Until They Don’t, provides one of the clearest explanations yet for this gap between knowing and doing. The researchers analysed 8,135 trial records across several benchmarks, agent frameworks, and language models to understand when reusable agent “skills” improve performance—and when they introduce new failure modes.

Their central finding is surprisingly human: AI agents benefit from skills primarily because skills give them better habits, not because they give them more facts.

Skills Are Standard Operating Procedures for AI

An agent skill is a reusable package of instructions, scripts, templates, and reference material for a particular type of task. It might tell an agent how to prepare a financial report, deploy a service, process a document, or investigate a failed test. The emerging Agent Skills standard describes skills as folders containing instructions and optional supporting resources. Agents first see a skill’s name and description. If the task appears relevant, they load the detailed instructions and execute the procedure. This is essentially progressive disclosure for AI: show the agent just enough information to find the right procedure, then reveal the operational detail when it is needed. But what actually makes these procedures useful?

The study found that procedural anchoring accounted for 65.7% of the identified skill mechanisms. Explicit knowledge injection accounted for only 4.5%.

In other words, skills rarely succeeded because they told the agent something it did not know. They succeeded because they stabilised the sequence of actions:

• Set up the environment in this order.
• Use these tools.
• Check this condition before proceeding.
• Produce the result in this format.
• Verify the output before declaring success.
• Avoid this known failure mode.

That distinction matters. Many organisations are building agent systems as if every failure were a knowledge problem. They add documents, expand retrieval systems, and stuff more context into prompts. But an agent can possess the correct information and still fail because it loses track of the process. The difference is similar to the difference between an employee handbook and a cockpit checklist. One explains how the organisation works. The other prevents somebody from forgetting a critical step while doing the work. Skills are much closer to the checklist.

The Compression Is Part of the Value

The researchers also compared skills with “Workflow Memory”: more direct records of previous task executions. At first glance, storing the complete trajectory seems sensible. If an agent solved a problem once, why not give the next agent the entire record? Because real work is messy.

Execution histories contain abandoned approaches, redundant commands, failed experiments, accidental detours, and task-specific assumptions. A later agent must distinguish the useful procedure from the noise. The study found an oracle-status success rate of 61.9% for skill-augmented runs, compared with 59.1% for agents working without injected experience and 55.9% for Workflow Memory. In matched comparisons, skills outperformed Workflow Memory by 6.06 percentage points.

The value, therefore, was not simply having access to past experience. It was how that experience had been represented. A well-written skill compresses a messy history into a reusable operational pattern. It preserves what mattered and discards the archaeology of how somebody discovered it. Think of the difference between handing a new colleague a three-hour screen recording and giving them a tested five-step procedure. Both contain the answer. Only one makes the answer easy to apply.

Skills Turn Experience Into Organisational Memory

This has significant implications for enterprise AI. Most organisations already possess enormous amounts of procedural knowledge, but much of it is fragmented across ticket histories, internal chats, documentation, personal notes, and the memories of experienced employees. Skills offer a way to transform that scattered experience into executable organisational memory. Consider an agent processing supplier invoices. A useful skill would not merely explain what an invoice is. The model almost certainly knows that already. Instead, it would specify:

1. Validate the supplier and purchase-order number.
2. Check the currency, tax treatment, and payment terms.
3. Compare line items with the purchase order.
4. Route discrepancies above a defined threshold for review.
5. Confirm the required approval level.
6. Record the outcome and supporting evidence.

That procedure reduces variation. It keeps important constraints active while the agent works. It also makes the expected process visible to auditors and business owners. The study found that skills reduced several execution-related failures. Environment and infrastructure failures dropped from 5.3% without skills to 0.2% with skills. Output-format mismatches fell from 7.4% to 3.2%, while failures involving background-service management declined from 2.7% to 0.8%. These are not failures of intelligence in the grand philosophical sense. They are failures of operational discipline. And operational discipline is precisely what enterprises need from agents.

Then the Skill Library Starts to Grow

There is, however, an uncomfortable second half to the study. When the researchers increased the available skill pool from five to 100 entries, actual-use precision fell from 29.6% to just 3.3%. The agent had more potentially useful procedures available—but became dramatically worse at restricting itself to the annotated correct ones. This is the skill-library paradox: every new skill increases potential capability while also increasing routing complexity. The problem becomes worse when skills sound similar. An agent may have to distinguish between:

• Review supplier invoice
• Resolve invoice discrepancy
• Process credit note
• Approve exceptional payment
• Reconcile purchase order
• Validate international invoice

To a human domain expert, the boundaries may appear obvious. To an agent selecting from short descriptions, they may overlap heavily. The Agent Skills specification recommends that descriptions explain both what a skill does and when it should be used. The study suggests that “when” may be the more important part. A generic description such as “helps with invoices” is not merely unhelpful metadata. It creates an operational routing risk.

The Strange Part: Success Did Not Collapse

While actual-use precision fell from 29.6% to 3.3%, average downstream success remained comparatively stable, moving from 36.4% to 39.3%. That does not mean retrieval quality is irrelevant. Agents may inspect several skills, extract partial help from a related procedure, or solve the task using the underlying model’s capabilities. Conversely, selecting the annotated correct skill does not guarantee that the agent will apply it correctly. As the researchers put it, exact ground-truth invocation is neither sufficient nor necessary for success. For enterprises, this creates a governance problem.

An agent can produce the correct result after consulting the wrong procedures. A dashboard that only measures final task success will report that everything is fine. But the organisation may no longer know which instruction governed the decision, or whether the same path will remain safe when the next case is slightly different. A system can therefore look reliable at the outcome level while becoming less interpretable at the process level. That is especially dangerous in finance, healthcare, legal work, infrastructure management, and other environments where the procedure matters almost as much as the result.

A Good Procedure Can Still Be Applied Badly

Skills introduce another failure surface: inappropriate obedience. In 10% of the skill-augmented cases analysed by the researchers, guidance was misapplied or ignored. Sometimes the instructions were reasonable, but the agent followed them mechanically, missed a precondition, or carried an assumption into an incompatible context. This is the AI equivalent of following the checklist after the situation has changed.

A skill created for standard invoices may not apply to a credit note. A deployment procedure written for one infrastructure environment may be dangerous in another. A customer-support escalation rule may become invalid after a policy change. The skill itself can be correct and the action still wrong. Reliable skills therefore need more than steps. They need boundaries:

• When should this skill be used?
• When must it not be used?
• Which assumptions must be true?
• What evidence should the agent collect first?
• Which conditions require human approval?
• When should the agent stop?
• How should it verify the result?

Without these guardrails, a skill becomes a source of misplaced confidence.

How to Build Skill Libraries That Scale

The study points towards a different approach to agent engineering. The goal should not be to accumulate as many skills as possible. It should be to manage the entire skill lifecycle.

1. Write procedures, not miniature encyclopaedias

Put stable operational steps in the skill. Keep volatile facts, customer records, product data, and legal information in authoritative systems that can be queried at runtime. A skill should explain how to obtain and verify the current tax rate, not hard-code a rate that will eventually become outdated.

2. Make applicability explicit

Every skill should contain clear inclusion and exclusion criteria. The agent needs to know not only what the procedure does, but which signals make it relevant. “Use for invoices” is weak. “Use for standard supplier invoices linked to an approved purchase order; do not use for credit notes, prepayments, or invoices without a purchase order” is much better.

3. Add stop conditions

A robust skill must tell the agent when not to continue. Missing data, conflicting records, unusual transaction values, or failed verification should trigger escalation rather than improvisation. The ability to stop is a capability.

4. Organise skills around business domains

Do not expose every agent to a flat catalogue containing every procedure in the company. Create domain-specific collections and use hierarchical routing to narrow the candidate set before execution. An accounting agent rarely needs software-deployment procedures. Removing irrelevant choices improves both efficiency and control.

5. Design descriptions as routing interfaces

Skill names and descriptions are not marketing copy. They are part of the system’s control layer. Descriptions should use distinctive language, list relevant triggers, and clearly differentiate neighbouring procedures. Similar descriptions should be tested together because confusing alternatives are more dangerous than obviously unrelated ones.

6. Version and retire skills

Business processes change. Every production skill should have an owner, version, review date, evaluation set, and retirement process. A stale procedure is often more dangerous than no procedure because it still looks authoritative.

7. Measure the whole pipeline

Final success is not enough. Teams should also observe:

• Which skills were considered
• Which skills were loaded
• Which instructions influenced the action
• Whether preconditions were checked
• Whether the procedure was adapted
• Which verification was performed
• Why the agent stopped or escalated
• Whether the result passed business validation

This turns skill evaluation from a single benchmark score into an operational control system.

Skills Are a Lifecycle, Not a Folder

The broader lesson from the research is that agent improvement cannot be reduced to storing more experience. Useful experience must be identified, distilled, described, retrieved, applied, verified, revised, and sometimes deleted. Each stage can improve the agent or create a new failure mode. This is why skill management will increasingly resemble a combination of knowledge management, software engineering, and process governance. Skills need owners. They need version control. They need tests. They need observability. They need feedback from successful and failed executions. And they need a mechanism for deciding when an existing procedure no longer applies. The organisations that treat skills as a random collection of prompt files will eventually create procedural chaos. The organisations that treat them as governed, executable business processes may build something much more valuable: AI systems capable of turning organisational experience into repeatable action.

A Useful Result, With Important Limits

The research is a preprint rather than a peer-reviewed final publication. Its experiments focus largely on terminal-based and tool-using tasks, with a limited number of agent and model configurations. The authors also note that their behavioural taxonomy was derived from a sample representing roughly 3% of the normalised trial records. The exact percentages should therefore not be treated as universal constants for every agent system. But the direction of the findings is difficult to ignore. AI agents often fail not because they lack information, but because they fail to maintain the right procedure during execution. Skills can correct that weakness. Yet the more skills we add, the more important selection, applicability, adaptation, and governance become. The best agent will not be the one with the largest skill library. It will be the one that knows which procedure applies, understands when that procedure is no longer valid, and can prove what it did before declaring the work complete. That is not merely artificial intelligence. It is operational maturity.

References

1. Jiang, Z., Huang, F., Xing, H., Wu, X., Gao, Y., Cao, R., Wang, M., Liu, S. and Li, Y. (2026). Demystifying Agent Skills: Why They Work—Until They Don’t. arXiv:2608.14036.
2. Agent Skills. Agent Skills Overview. Overview of the open Agent Skills format and its progressive-disclosure model.
3. Agent Skills. Agent Skills Specification. Technical specification covering skill metadata, descriptions, compatibility information, resources, and validation.
4. Schreiner, M. (2026). Studie erklärt, warum KI-Agenten von „Skills“ profitieren und wann sie versagen. The Decoder, 22 August 2026.