Tool search (MCP tool deferral) disabled for all non-Anthropic models — 47k vs 21k tokens on an empty prompt

[ArlindNocaj]

Summary

On my account, MCP tool deferral (tool search) is active only for Anthropic/Claude models. Every OpenAI, Gemini, Grok, and MAI model ships all tool schemas on every turn.

The cost is large and unavoidable: a one-word "hi" prompt costs 21.6k input tokens on claude-sonnet-4.6 but 47.6k on gpt-5.4 and 61.9k on grok-4.6 — up to 40k tokens of pure overhead per turn before the user types anything real.

Crucially, this is not user-side misconfiguration. toolSearch: true is set, and the two feature paths a user can influence locally report None on all 23 models, including Claude. Only GitHub's server-managed builtin path works, and it is lit up for Anthropic only.

Environment

• Copilot CLI 1.0.81-8
• macOS (darwin-arm64)
• 10 MCP servers, 178 total tools
• ~/.copilot/settings.json: toolSearch: true, experimental: true

Reproduction

copilot -p "hi" --model gpt-5.4 --log-level debug --log-dir /tmp/ts
grep -ho 'turn tool surface resolved {.*}' /tmp/ts/*.log | head -1 | jq

The deciding field

turn tool surface resolved (emitted by copilot_runtime::session::native_message_turn):

// gpt-5.4 — tool search OFF
{ "model":"gpt-5.4", "generic_path":false, "client_path":true, "builtin_path":false,
  "explicitly_enabled":false, "model_config_enabled":false, "opted_out":false,
  "tool_search_enabled":false,
  "generic_flag_service_available":"None",
  "client_flag_service_available":"None",
  "builtin_flag_service_available":"None",   // <-- no verdict => feature off
  "tool_count":176, "deferred_tool_count":0 }

// claude-sonnet-4.6 — tool search ACTIVE
{ "model":"claude-sonnet-4.6", ...
  "builtin_flag_service_available":"Some(true)",
  "tool_count":178, "deferred_tool_count":156 }

Across all 23 models: client_flag_service_available and generic_flag_service_available are None every single time. Only builtin_flag_service_available ever returns Some(true), and only for Claude.

Full results — all 23 models, identical machine/config/MCP set

Model	Input tokens	Tools	Deferred	Tool search
claude-sonnet-4.6	21,576	178	156	✅
claude-haiku-4.5	21,700	178	156	✅
claude-opus-4.6	21,731	178	156	✅
claude-opus-5	29,848	178	156	✅
claude-sonnet-5	29,859	178	156	✅
claude-opus-4.8	29,860	178	156	✅
claude-opus-4.7	30,071	178	156	✅
gpt-5.6-sol	43,130	156	0	❌
gpt-5.6-terra	43,666	154	0	❌
mai-code-1-flash-picker	46,053	177	0	❌
mai-code-1.1-flash	46,059	177	0	❌
gpt-5-mini	46,208	177	0	❌
gpt-5.4-mini	47,516	176	0	❌
gpt-5.5	47,574	176	0	❌
gpt-5.3-codex	47,588	176	0	❌
gpt-5.6-luna	47,619	176	0	❌
gpt-5.4	47,631	176	0	❌
gemini-3.6-flash	48,455	173	0	❌
gemini-3.5-flash	48,984	177	0	❌
gemini-3.7-flash	48,984	177	0	❌
gemini-3.1-pro-preview	48,985	177	0	❌
grok-4.5	61,917	177	0	❌
grok-4.6	61,918	177	0	❌

7/7 Claude defer · 0/16 non-Claude defer.

Things that made no difference

Each verified by measurement, all still deferred_tool_count: 0 on GPT:

• "deferTools": "auto" added explicitly to all 10 MCP servers → 47,688 tokens (no change)
• --experimental → 47,692 (no change)
• TOOL_SEARCH_CLIENT_GENERIC=true → 47,691 (no change)
• toolSearch: true in settings.json → already set throughout

Disabling all MCP servers dropped it to 38,183 tokens / 128 tools, confirming the ~9.5k MCP schema cost is exactly what deferral should be eliminating.

Expected behaviour

Per the tool search docs, tool search should activate above ~30 tools on supported models, explicitly including GPT-5.4+. With 178 tools and toolSearch: true, GPT models should defer.

Questions

1. Is the builtin tool-search flight intentionally Anthropic-only right now, or is OpenAI enablement missing/regressed?
2. Why do client_flag_service_available and generic_flag_service_available return None on all models? Is the client-side path reachable at all in 1.0.81-8?
3. TOOL_SEARCH_CLIENT_GENERIC=true appears to be a no-op here — is it still wired up?
4. Should the docs be updated to reflect that GPT models do not currently get tool search?

Related

• Lazy-load MCP servers on first tool invocation #2901 — lazy-load MCP servers (a comment there reports 13 servers / 181 tools / 47.1k, nearly identical numbers)
• /context "MCP Tools" reports the un-deferred tool-schema footprint, not the actual (deferred) cost sent to the model #4189 — /context reports un-deferred footprint
• 400 "Missing namespace for function_call" for deferred/tool-search tools (e.g. extensions_manage) on 1.0.80 #4519 — deferred-tool 400 errors, workaround toolSearch: false

Repro script

#!/usr/bin/env bash
for m in "$@"; do
  copilot -p "hi" --model "$m" --no-custom-instructions --stream off \
    --log-level debug --log-dir /tmp/ts-$m >/dev/null 2>&1
  echo -n "$m -> "
  grep -ho 'turn tool surface resolved {.*}' /tmp/ts-$m/*.log | head -1 \
    | jq -c '{tools:.tool_count, deferred:.deferred_tool_count, builtin:.builtin_flag_service_available}'
  rm -rf /tmp/ts-$m
done