Writing Effective Tools for AI Agents: What Production Teams Learned

TL;DR: The model you choose matters, but the tools you give it matter more. After evaluating hundreds of tools across production deployments, Anthropic found that tool design is the single highest-leverage improvement for agent performance. Three principles matter: intent-driven consolidation, meaningful context return, and evaluation-driven iteration.

Why Tool Design Is the Hidden Multiplier

In late 2024, Anthropic published a finding that changed how production teams think about agents: “Claude Sonnet 3.5 achieved state-of-the-art performance on SWE-bench Verified after precise refinements to tool descriptions” [1]. The model didn’t change. The prompts didn’t change. Only the tools did.

This wasn’t a fluke. In their 2026 engineering blog post, Anthropic’s Applied AI team made their thesis explicit: “Agents are only as effective as the tools they’re given” [1]. Traditional software APIs are deterministic — getWeather("NYC") returns the same result every time for the same input. Agent tools are different. They reflect a “contract between deterministic systems and non-deterministic agents” [1]. The agent decides when to call a tool, which parameters to pass, and how to interpret the response. Bad tool design breaks all three decisions.

The 2026 State of Agent Engineering report from LangChain reinforces this: 57% of organizations have agents in production, yet most skip structured evaluation entirely — 48% don’t run offline evaluations and 63% skip online monitoring [2] source. Teams ship agents without ever systematically testing whether their tools work.

The Evaluation-Driven Tool Development Cycle

Anthropic prescribes a three-phase cycle for writing tools that actually work in production [1].

Phase 1: Build a Prototype

Start by wrapping your tool in a local MCP (Model Context Protocol) server and testing it directly with Claude Code or Claude Desktop. The goal is not perfection — it’s getting real interaction data. Early prototypes reveal what the agent misunderstands about your tool’s purpose.

# Add a local MCP tool for testing
claude mcp add my-tool /path/to/tool-server/index.mjs

Phase 2: Run an Evaluation

The key insight here is what constitutes a strong evaluation task. Weak tasks are oversimplified — a single tool call with no branching. Strong tasks require multiple tool calls, real-world data, and complex workflows.

Weak: "Schedule a meeting with jane@acme.corp next week" Strong: "Schedule a meeting with Jane next week to discuss our latest Acme Corp project. Attach the notes from our last project planning meeting and reserve a conference room." [1]

Each prompt should pair with a verifiable outcome. Run the evaluation with a simple agentic loop — a while loop wrapping alternating LLM API and tool calls. Collect metrics: accuracy, runtime, tool call count, token consumption, and tool errors.

Then read the raw transcripts. As Anthropic notes, “What agents omit in their feedback and responses can often be more important than what they include” [1]. Redundant calls? Your tool probably needs pagination. Parameter errors? Your descriptions need clarity.

Phase 3: Iterate with AI Assistance

Feed evaluation transcripts into a coding agent (Claude Code, etc.) to refactor tools automatically. Anthropic’s own team uses this cycle: run evaluations → feed failures into the coding agent → deploy improved tools → re-evaluate. Held-out test sets prevent overfitting to specific prompts [1].

Four Principles of Effective Tool Design

1. Intent-Driven Consolidation

More tools is not better. Every tool added to context consumes attention tokens and increases the chance the agent picks the wrong one. Anthropic’s rule: “If a human engineer can’t definitively say which tool should be used in a given situation, an AI agent can’t be expected to do better” [3].

Consolidate tools around intent, not API endpoints:

Instead of	Use	Rationale
`list_users`, `list_events`, `create_event`	`schedule_event`	The agent wants to schedule something, not manage individual CRUD operations
`read_logs`	`search_logs`	Returns only relevant lines, not a firehose of data
`get_customer_by_id`, `list_transactions`, `list_notes`	`get_customer_context`	Compiles all recent info the agent needs in one call

Consolidated tools reduce context pressure, improve accuracy, and make it harder for agents to take wrong paths.

2. Namespacing Reduces Ambiguity

When you have many tools (say, 20+ from multiple integrations), namespace them consistently. Anthropic recommends prefixes like asana_search and jira_search rather than relying on descriptions alone [1].

But be careful: prefix vs. suffix placement matters, and effects vary by LLM. Test both asana_search and search_asana in your evaluation to see which performs better for your specific model.

3. Return Meaningful Context

Agents don’t know that uuid_12345 refers to a specific customer record. They work better with natural-language identifiers. Instead of returning raw UUIDs and internal IDs, return name, image_url, file_type alongside IDs.

Anthropic also recommends providing a response_format enum:

from enum import Enum

class ResponseFormat(Enum):
    DETAILED = "detailed"
    CONCISE = "concise"

This lets the agent request the right level of detail per query — concise for quick lookups, detailed for deep analysis. The key is that the agent, not the developer, decides what “detailed” means in context [1].

4. Engineer for Token Efficiency

Tool responses are the largest source of context consumption in most agent systems. Every response you return reduces the tokens available for reasoning. Three patterns help:

Pagination with sensible defaults: return 25 results by default, not 1000
Encourage many small searches: one broad search that returns 5,000 tokens is worse than three targeted searches that return 300 each
Helpful error responses: opaque error codes force agents to guess. Replace Error 403 with "Input validation failed: 'user_id' must be an integer. You provided 'abc'. Please correct and retry." [1]

The Anthropic engineering team found that optimizing the quantity of context returned in tool responses was as important as optimizing the tool logic itself.

What This Means for Your Next Agent Project

The takeaway is counterintuitive: before you swap models, before you rewrite your prompts, scrutinize your tools. If an agent has ten tools and each returns a wall of text, the model can’t reason effectively regardless of its architecture.

Start with the evaluation loop. Build a prototype, run it against strong multi-step tasks, read the transcripts, and iterate. The teams doing this today (Lyft, Cisco, Toyota, Monday.com, Cloudflare [2]) aren’t using secret models — they’re using better tools.

[1] Anthropic Applied AI Team — “Writing effective tools for AI agents—using AI agents”

[2] LangChain — “State of Agent Engineering 2026”

[3] Anthropic — “Building effective agents”

← Back to all posts