Domain 2 Project — build tools your agent actually picks correctly

This project opens by breaking something. You will write two tools the way people actually write them, measure how often the model picks the wrong one, and then fix it with nothing but better English — and watch the number move.

Everything after that is a repair on the same system: errors an agent can act on, tools scoped to roles, an MCP server wired into Claude Code, and the built-ins used to trace it all. Five stages, one per task statement.

Domain 2 is 18% of the exam and rests on a single premise: the description is the interface. Stage 1 is where you stop taking that on trust.

The project at a glance

Domain: 2 · Tool Design & MCP Integration (18%)
Stages: 5, one per task statement
Difficulty: Intermediate
Total time: 175 minutes
You need: Python 3.10+, the anthropic SDK, an API key, Claude Code
Based on: Exam guide Preparation Exercises 1 and 2

What you will be able to do

Write a tool description that a model can select on, and name the four things a thin one leaves out
Return structured errors an agent can branch on, and distinguish a failed call from a successful one that matched nothing
Scope tools across agent roles and choose between the three tool_choice settings from what the failure would be
Configure an MCP server at the right scope with credentials that never enter version control
Select the correct built-in for a search, and recover when Edit cannot find a unique anchor

Before you start

You need the Domain 1 setup plus Claude Code. If you have not done that project, the four commands are:

bash

mkdir ccarf-domain-2 && cd ccarf-domain-2
python3 -m venv .venv && source .venv/bin/activate
pip install anthropic
export ANTHROPIC_API_KEY=your-key-here

Stages 2.4 and 2.5 need Claude Code itself — check it responds before you get there, because an MCP server that will not load and a CLI that is not installed look identical from the outside.

Stage 2.1 also needs a measurement harness, and it is worth building properly because three later stages re-use it: a list of twenty ambiguous requests, a loop that sends each one, and a counter of which tool came back. Twenty is enough to see a rate and small enough to re-run without thinking about the bill.

Build it in this order

The order is not the guide's. Stage 1 is a measurement and stages 2 to 5 are repairs, so every later claim can be checked against a number you produced yourself rather than against a sentence you read.

Five stages

2.1 · Break tool selection, then fix it · 35 min
BUILD — Two deliberately confusable tools — analyze_content and analyze_document, with the vague descriptions people actually write — and a script that sends twenty ambiguous requests and counts which tool got picked. Then rewrite both descriptions with purpose, inputs, outputs, examples, edge cases and boundaries, and re-run the same twenty. WHY — Domain 2 rests on one premise: the description IS the interface, because it is the primary signal the model selects on. A measured misroute rate before and after makes that premise yours rather than something you read. YOU SHOULD SEE — A misroute rate that is embarrassing before and near zero after, from a change that touched no code — only two paragraphs of English. TRAP — Fixing it in the system prompt instead. It will appear to work, and it will keep working until a keyword in that prompt collides with a third tool six weeks later. The exam tests this exact substitution in both directions. STUCK? — If the rate does not move, read your two new descriptions side by side and ask which sentence in one would stop you picking the other. If there is not one, you rewrote the prose without adding the distinction.

Stage 1 in full

Write the bad version first

This feels wrong and it is the point. Two tools, described the way a hurried engineer describes them:

python

BAD_TOOLS = [
    {
        "name": "analyze_content",
        "description": "Analyzes content and returns insights.",
        "input_schema": {
            "type": "object",
            "properties": {"input": {"type": "string"}},
            "required": ["input"],
        },
    },
    {
        "name": "analyze_document",
        "description": "Analyzes a document and returns analysis.",
        "input_schema": {
            "type": "object",
            "properties": {"input": {"type": "string"}},
            "required": ["input"],
        },
    },
]

Neither description is wrong. Neither is usable. A model choosing between them has nothing to choose on, and the exam's phrase for the result is misrouting.

Measure it

Twenty requests that could plausibly go to either, sent with tool_choice set to any so the model must pick one:

python

REQUESTS = [
    "Pull the key points out of this web page.",
    "Summarise the attached PDF.",
    "What does this article say about pricing?",
    # ... seventeen more, deliberately ambiguous
]

def measure(tools):
    picks = {}
    for request in REQUESTS:
        response = client.messages.create(
            model="claude-opus-5",
            max_tokens=1024,
            tools=tools,
            tool_choice={"type": "any"},
            messages=[{"role": "user", "content": request}],
        )
        for block in response.content:
            if block.type == "tool_use":
                picks[block.name] = picks.get(block.name, 0) + 1
    return picks

print("before:", measure(BAD_TOOLS))

Write down the number. It is the only baseline you get.

Now fix it with English

Four things go into each description: what it is FOR, what it expects, what it returns, and when to use it INSTEAD of the other one. That last clause is the one that does the work.

python

GOOD_TOOLS = [
    {
        "name": "extract_web_results",
        "description": (
            "Extract structured findings from a WEB PAGE fetched by URL. "
            "Input: a http or https URL. Returns: title, publication date, and a list "
            "of claims with the sentence each came from. "
            "Use this for anything addressed by URL. "
            "Do NOT use it for an uploaded file — use extract_document_data for that."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"url": {"type": "string", "description": "http or https URL"}},
            "required": ["url"],
        },
    },
    {
        "name": "extract_document_data",
        "description": (
            "Extract structured findings from an UPLOADED DOCUMENT (PDF, DOCX, TXT). "
            "Input: a document id from the files store. Returns: title, author, page "
            "count, and a list of claims with the page each came from. "
            "Use this for anything the user has uploaded. "
            "Do NOT use it for a URL — use extract_web_results for that."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"document_id": {"type": "string"}},
            "required": ["document_id"],
        },
    },
]

print("after:", measure(GOOD_TOOLS))

Three changes carried the improvement, and it is worth knowing which. The names stopped overlapping. The input schemas became different types, so the request itself often decides. And each description names the other tool and says when to prefer it — the sentence a thin description never has.

Multiple choice · Medium

Two tools, analyze_content and analyze_document, have near-identical descriptions and the agent routes requests to the wrong one roughly half the time. Which change addresses the cause?

Prove stage 1 before moving on

Your before and after numbers are on the same twenty requests. If you changed the request list too, you measured nothing.
Read the two new descriptions and find, in each, the sentence that would stop you picking the other. If it is not there, the rate will not have moved.
Keep the harness. Stage 2.3 re-runs it against eighteen tools, and the comparison is the point.

When you get stuck

Every stage above ends with a STUCK? line — the first thing worth checking, which is usually the thing actually wrong.

Past that, in the app: the Help button in this page's toolbar sends the tutor the section you are currently reading, together with the project's title, so you can ask "why does this not work" without pasting anything in. It knows which stage you are on.

Measure before you fix. Every claim in this domain is about a rate, and a rate you did not take before the change is a rate you cannot argue about after it.
Read the tool call, not the reply. Which tool was chosen, and with what arguments, is the observable this domain turns on — the prose the model wraps around it is not.
Change the description before the prompt. If a selection problem is fixed in the system prompt, it will come back the next time a keyword collides.

Where the depth lives

This page is for building. The reading is elsewhere, and repeating it here would put the same claims in three places to drift apart:

The Domain 2 roadmap — what the domain is about, the order to learn it in, and where it catches people
5 flashcard decks, 43 cards — one card per published objective
46 Domain 2 questions — judgement under exam conditions, inside the six production scenarios

Sources

CCAR-F Exam Guide v1.0, section 6 — Domain 2's five task statements, which are this project's five stages.
CCAR-F Exam Guide v1.0, section 8 — Preparation Exercise 1 (tool descriptions and structured error responses) and Exercise 2 (MCP server configuration with environment variable expansion).
Anthropic Messages API reference — the tool definition shape, the three tool_choice settings, and the parallel-tool-use rule.
Model Context Protocol documentation — server scoping, resources, and the isError response shape.

Domain 2 Project — build tools your agent actually picks correctly

Domain 2 is 18% of the exam and rests on a single premise: the description is the interface. Stage 1 is where you stop taking that on trust.

The project at a glance

Domain: 2 · Tool Design & MCP Integration (18%)
Stages: 5, one per task statement
Difficulty: Intermediate
Total time: 175 minutes
You need: Python 3.10+, the anthropic SDK, an API key, Claude Code
Based on: Exam guide Preparation Exercises 1 and 2

What you will be able to do

Write a tool description that a model can select on, and name the four things a thin one leaves out
Return structured errors an agent can branch on, and distinguish a failed call from a successful one that matched nothing
Scope tools across agent roles and choose between the three tool_choice settings from what the failure would be
Configure an MCP server at the right scope with credentials that never enter version control
Select the correct built-in for a search, and recover when Edit cannot find a unique anchor

Before you start

You need the Domain 1 setup plus Claude Code. If you have not done that project, the four commands are:

bash

mkdir ccarf-domain-2 && cd ccarf-domain-2
python3 -m venv .venv && source .venv/bin/activate
pip install anthropic
export ANTHROPIC_API_KEY=your-key-here

Stages 2.4 and 2.5 need Claude Code itself — check it responds before you get there, because an MCP server that will not load and a CLI that is not installed look identical from the outside.

Build it in this order

Five stages

2.1 · Break tool selection, then fix it · 35 min
BUILD — Two deliberately confusable tools — analyze_content and analyze_document, with the vague descriptions people actually write — and a script that sends twenty ambiguous requests and counts which tool got picked. Then rewrite both descriptions with purpose, inputs, outputs, examples, edge cases and boundaries, and re-run the same twenty. WHY — Domain 2 rests on one premise: the description IS the interface, because it is the primary signal the model selects on. A measured misroute rate before and after makes that premise yours rather than something you read. YOU SHOULD SEE — A misroute rate that is embarrassing before and near zero after, from a change that touched no code — only two paragraphs of English. TRAP — Fixing it in the system prompt instead. It will appear to work, and it will keep working until a keyword in that prompt collides with a third tool six weeks later. The exam tests this exact substitution in both directions. STUCK? — If the rate does not move, read your two new descriptions side by side and ask which sentence in one would stop you picking the other. If there is not one, you rewrote the prose without adding the distinction.

Stage 1 in full

Write the bad version first

This feels wrong and it is the point. Two tools, described the way a hurried engineer describes them:

python

BAD_TOOLS = [
    {
        "name": "analyze_content",
        "description": "Analyzes content and returns insights.",
        "input_schema": {
            "type": "object",
            "properties": {"input": {"type": "string"}},
            "required": ["input"],
        },
    },
    {
        "name": "analyze_document",
        "description": "Analyzes a document and returns analysis.",
        "input_schema": {
            "type": "object",
            "properties": {"input": {"type": "string"}},
            "required": ["input"],
        },
    },
]

Neither description is wrong. Neither is usable. A model choosing between them has nothing to choose on, and the exam's phrase for the result is misrouting.

Measure it

Twenty requests that could plausibly go to either, sent with tool_choice set to any so the model must pick one:

python

REQUESTS = [
    "Pull the key points out of this web page.",
    "Summarise the attached PDF.",
    "What does this article say about pricing?",
    # ... seventeen more, deliberately ambiguous
]

def measure(tools):
    picks = {}
    for request in REQUESTS:
        response = client.messages.create(
            model="claude-opus-5",
            max_tokens=1024,
            tools=tools,
            tool_choice={"type": "any"},
            messages=[{"role": "user", "content": request}],
        )
        for block in response.content:
            if block.type == "tool_use":
                picks[block.name] = picks.get(block.name, 0) + 1
    return picks

print("before:", measure(BAD_TOOLS))

Write down the number. It is the only baseline you get.

Now fix it with English

Four things go into each description: what it is FOR, what it expects, what it returns, and when to use it INSTEAD of the other one. That last clause is the one that does the work.

python

GOOD_TOOLS = [
    {
        "name": "extract_web_results",
        "description": (
            "Extract structured findings from a WEB PAGE fetched by URL. "
            "Input: a http or https URL. Returns: title, publication date, and a list "
            "of claims with the sentence each came from. "
            "Use this for anything addressed by URL. "
            "Do NOT use it for an uploaded file — use extract_document_data for that."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"url": {"type": "string", "description": "http or https URL"}},
            "required": ["url"],
        },
    },
    {
        "name": "extract_document_data",
        "description": (
            "Extract structured findings from an UPLOADED DOCUMENT (PDF, DOCX, TXT). "
            "Input: a document id from the files store. Returns: title, author, page "
            "count, and a list of claims with the page each came from. "
            "Use this for anything the user has uploaded. "
            "Do NOT use it for a URL — use extract_web_results for that."
        ),
        "input_schema": {
            "type": "object",
            "properties": {"document_id": {"type": "string"}},
            "required": ["document_id"],
        },
    },
]

print("after:", measure(GOOD_TOOLS))

Multiple choice · Medium

Two tools, analyze_content and analyze_document, have near-identical descriptions and the agent routes requests to the wrong one roughly half the time. Which change addresses the cause?

Prove stage 1 before moving on

Your before and after numbers are on the same twenty requests. If you changed the request list too, you measured nothing.
Read the two new descriptions and find, in each, the sentence that would stop you picking the other. If it is not there, the rate will not have moved.
Keep the harness. Stage 2.3 re-runs it against eighteen tools, and the comparison is the point.

When you get stuck

Every stage above ends with a STUCK? line — the first thing worth checking, which is usually the thing actually wrong.

Measure before you fix. Every claim in this domain is about a rate, and a rate you did not take before the change is a rate you cannot argue about after it.
Read the tool call, not the reply. Which tool was chosen, and with what arguments, is the observable this domain turns on — the prose the model wraps around it is not.
Change the description before the prompt. If a selection problem is fixed in the system prompt, it will come back the next time a keyword collides.

Where the depth lives

This page is for building. The reading is elsewhere, and repeating it here would put the same claims in three places to drift apart:

The Domain 2 roadmap — what the domain is about, the order to learn it in, and where it catches people
5 flashcard decks, 43 cards — one card per published objective
46 Domain 2 questions — judgement under exam conditions, inside the six production scenarios

Sources

CCAR-F Exam Guide v1.0, section 6 — Domain 2's five task statements, which are this project's five stages.
CCAR-F Exam Guide v1.0, section 8 — Preparation Exercise 1 (tool descriptions and structured error responses) and Exercise 2 (MCP server configuration with environment variable expansion).
Anthropic Messages API reference — the tool definition shape, the three tool_choice settings, and the parallel-tool-use rule.
Model Context Protocol documentation — server scoping, resources, and the isError response shape.

Domain 2 Project — build tools your agent actually picks correctly

Domain 2 Project — build tools your agent actually picks correctly

The project at a glance

What you will be able to do

Before you start

Build it in this order

Five stages

2.1 · Break tool selection, then fix it · 35 min

Stage 1 in full

Write the bad version first

Measure it

Now fix it with English

Prove stage 1 before moving on

When you get stuck

Where the depth lives

Sources

Domain 2 Project — build tools your agent actually picks correctly

Domain 2 Project — build tools your agent actually picks correctly

The project at a glance

What you will be able to do

Before you start

Build it in this order

Five stages

2.1 · Break tool selection, then fix it · 35 min

Stage 1 in full

Write the bad version first

Measure it

Now fix it with English

Prove stage 1 before moving on

When you get stuck

Where the depth lives

Sources