Skip to content

[codex] Improve proxy Anthropic token counting#127

Draft
konradciok wants to merge 1 commit intochopratejas:mainfrom
konradciok:codex/proxy-anthropic-token-counting
Draft

[codex] Improve proxy Anthropic token counting#127
konradciok wants to merge 1 commit intochopratejas:mainfrom
konradciok:codex/proxy-anthropic-token-counting

Conversation

@konradciok
Copy link
Copy Markdown

What changed

This PR improves proxy-side Anthropic token counting by passing an optional Anthropic SDK client into AnthropicProvider when ANTHROPIC_API_KEY is available.

It also:

  • adds anthropic to the proxy extra so the accurate counting path is available in proxy installs
  • relaxes the litellm pin to >=1.82.6,<2
  • makes the MCP CLI test deterministic when headroom is on PATH
  • updates contributor setup docs to include the langchain extra used by full test runs

Why

The proxy currently initializes AnthropicProvider() without an SDK client, which forces approximate token counting for Claude models even when the API key is available. This change opts into the more accurate Anthropic count_tokens path when possible while preserving the existing fallback behavior for OpenAI-only installs.

Impact

  • More accurate token accounting for Anthropic-backed proxy usage
  • No behavior change when ANTHROPIC_API_KEY is unset or the SDK is unavailable
  • More stable MCP CLI test behavior across environments

Validation

Passed:

  • pytest tests/test_cli/test_mcp.py
  • pytest tests/test_proxy_count_tokens_integration.py tests/test_providers/test_anthropic.py -q

Repo pre-push hook also ran full pytest -q, but it failed in unrelated areas outside this diff:

  • tests/test_integrations/langchain/test_langchain_live.py (langchain_openai / langchain_anthropic missing)
  • tests/test_memory_usage_integration.py
  • tests/test_proxy_memory_integration.py

@chopratejas
Copy link
Copy Markdown
Owner

Hi - thank you for these changes - they look good - can you resolve merge conflicts and resubmit the PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants