# Buyers on popular agent runtimes, against one gate

Generated by `examples/runtimes/run-all` on 2026-11-14 at commit `3cd29e2` against `http://localhost:4733`.
Each buyer gets the same task, "Order 1,000 litres of lactose-free milk for delivery on Monday for this kitchen; ask me if
something needs my decision", connects with nothing but the address and a signed mandate, and runs until the order is
committed or waits for a person's sign-off. The buyer's own decisions come from a scripted person (Sanna, the head chef),
never from the model. Transcripts: [transcripts/](transcripts/). Gaps each runtime exposed and the fixes proposed: [GAPS.md](GAPS.md).

| Runtime | Version | Interface | Model | Outcome | Steps | Tool calls | Refusals met (rule) | Buyer's answers via | Seconds | Est. cost |
|---|---|---|---|---|---|---|---|---|---|---|
| [Pydantic AI](pydantic-ai/) | 2.68.0 | MCP | anthropic:claude-haiku-5-5 | pending_approval | 7 | 5 | none | elicitation | 13.9 | $0.0160 |
| [OpenAI Agents SDK](openai-agents/) | 0.24.0 | MCP | openai:gpt-6-luna | pending_approval | 7 | 5 | none | relay+approval | 18.0 | $0.0087 |
| [Google ADK](google-adk/) | 2.14.1 | MCP | google:gemini-3.1-flash-lite | pending_approval | 6 | 4 | none | relay+approval | 7.1 | $0.0471 |
| [Claude Agent SDK](claude-agent-sdk/) | 0.3.325 | MCP | anthropic:claude-haiku-5-5 | pending_approval | 9 | 7 | none | elicitation | 18 | $0.0151 |
| [Vercel AI SDK](vercel-ai-sdk/) | 7.0.172 | MCP | openai:gpt-6-luna | pending_approval | 7 | 5 | none | elicitation | 18.6 | $0.0088 |
| [LangGraph](langgraph/) | 1.2.21 | MCP | anthropic:claude-haiku-5-5 | pending_approval | 7 | 6 | none | relay+approval | 16.0 | $0.0158 |
| [Semantic Kernel (OpenAPI, no MCP)](semantic-kernel-openapi/) | 1.47.0 | OpenAPI 3.1 | openai:gpt-6-luna | pending_approval | 8 | 5 | none | relay+filter | 16.1 | $0.0101 |

Total estimated model cost of this run: $0.1217. Steps are model requests; costs use list prices per million tokens
(Claude Agent SDK reports its own). `pending_approval` is the expected end state: the order (about €1,322) is above
the €100 the agent may spend alone, so the owner signs off on their phone and the agent must not retry. "elicitation"
means the gate asked the buyer through the MCP client (MCP 2026-07-28); "relay+approval" means the model passed on the
buyer's answer and the runtime's own approval step let the scripted buyer check it first.

Every runtime presented the mandate as plain HTTP headers (`Authorization: Bearer`, `Dock-Agent-Id`); none needed a
workaround for that. The mandate for each run names its own agent id and has a raised weekly budget (see
`mint-mandate.ts`), so several runs fit in one week's budget on one server.

## Notes per runtime

- **Pydantic AI.** MCP 2026-07-28: the gate asks the buyer through the client (elicitation_handler). Set tool_error_behavior='failed' (the default 'retry' turns the gate's refusals into retry prompts and stops the run after one retry) and include_instructions=True (the server's instructions are dropped by default). Pydantic AI warns that the elicitation handler 'will never be called' on a 2026-07-28 session; it is called.
- **OpenAI Agents SDK.** No MCP elicitation in the SDK, so the model relays the buyer's answer; a require_approval callable that reads _meta['dock/humanInput'] sends those calls through the SDK's approval step (interruptions, state.approve/reject) to the scripted buyer.
- **Google ADK.** Connects with the 2025-11-25 handshake, so no elicitation; human-input tools go through ADK's require_confirmation (a second McpToolset filtered to them). ADK's experimental ADK_ENABLE_MCP_MODERN_PROTOCOL=1 negotiates 2026-07-28 but then fails on the gate's input_required result. Gemini 429/503 on the shared key: google-genai retry options set. On 14 Nov gemini-3.1-flash-lite sent quantity next to need instead of inside it, five times; submit_need now folds such fields into the need.
- **Claude Agent SDK.** MCP 2026-07-28 through the bundled Claude Code: onElicitation answers the gate's form. canUseTool reviews human-input tools (the SDK does not expose tool _meta, so the example reads it with one plain tools/list request). Built-in tools off, user settings not loaded.
- **Vercel AI SDK.** ToolLoopAgent + @ai-sdk/mcp, MCP 2026-07-28 with onElicitationRequest; toolApproval on human-input tools. Standalone, not the project's own runner. MCP results reach the model as text (structuredContent only with an outputSchema).
- **LangGraph.** langchain-mcp-adapters 0.3.2 pins mcp<2 (2025-11-25), so no elicitation; HumanInTheLoopMiddleware interrupts human-input tools and the graph resumes from its checkpoint with Command(resume=...).
- **Semantic Kernel (OpenAPI, no MCP).** The REST door: tools generated from /api/v1/openapi.json, filtered to operations whose security is the mandate. Four workarounds for Semantic Kernel 1.47: inline $refs (it builds no body parameters from $ref request bodies), pass 4xx bodies through (it hides them behind 'Error running OpenAPI operation'), allow the http base URL, send reasoning_effort 'none' as extra_body for gpt-6-luna. Human-input operations are read from the document's x-dock-human-input, from its agent view (?audience=agent).