`UsageMetadata` for `ChatAnthropicVertex` is missing cache read/write token count in `input_token_details`

🐛 Bug Report

Environment

• Package: langchain-google-vertexai
• Version: 2.0.26 (latest official release)
• Python Version: 3.11+
• Model: ChatAnthropicVertex (Claude via Google Vertex AI)
• Tested Model: claude-sonnet-4@20250514

Problem Description

ChatAnthropicVertex does not follow LangChain's UsageMetadata standard for cache token reporting. Instead of using the expected input_token_details structure, it either puts cache tokens at the top level (non-streaming) or omits them entirely (streaming). This breaks compatibility with LangChain's token usage patterns and the official langchain_anthropic package.

Expected Behavior

ChatAnthropicVertex should follow LangChain's UsageMetadata standard by placing cache token information in input_token_details, consistent with:

1. LangChain's UsageMetadata TypedDict definition
2. Official langchain_anthropic.ChatAnthropic implementation
3. Standard token usage patterns across LangChain

Actual Behavior

⚠️ Verified with langchain-google-vertexai v2.0.26:

Streaming response (missing cache tokens entirely):

# First chunk from streaming response
{
    'input_tokens': 14,
    'output_tokens': 0,
    'total_tokens': 14
}
# ❌ Missing input_token_details completely

Non-streaming response (wrong cache token structure):

{
    'input_tokens': 14,
    'output_tokens': 10,
    'total_tokens': 24,
    'cache_creation_input_tokens': 0,    # ❌ Wrong structure
    'cache_read_input_tokens': 0         # ❌ Should be in input_token_details
}
# ❌ Missing proper input_token_details structure

Expected Behavior

✅ With our fix implemented:

Streaming response (cache tokens properly included):

# First chunk from streaming response
{
    'input_tokens': 14,
    'output_tokens': 0,
    'total_tokens': 14,
    'input_token_details': {'cache_creation': 0, 'cache_read': 0}  # ✅ Correct
}

Non-streaming response (proper LangChain structure):

{
    'input_tokens': 14,
    'output_tokens': 10,
    'total_tokens': 24,
    'input_token_details': {'cache_read': 0, 'cache_creation': 0}  # ✅ Correct
}

Impact

• Standards Compliance: ChatAnthropicVertex doesn't conform to LangChain's UsageMetadata structure
• Cross-Package Inconsistency: Behavior differs from official langchain_anthropic package
• Monitoring Breakage: LLM tracers and tools expect input_token_details structure
• Cost Tracking: Users cannot properly monitor cache usage and costs (especially streaming)
• Developer Experience: Inconsistent token usage patterns across LangChain ecosystem

Reproduction Code

from langchain_google_vertexai.model_garden import ChatAnthropicVertex

llm = ChatAnthropicVertex(
    model_name="claude-sonnet-4@20250514",
    project="your-project-id",
    location="us-east5"
)

message = "What is the capital of France?"

# Test 1: Non-streaming (shows wrong structure)
response = llm.invoke(message)
print("Non-streaming:", response.usage_metadata)
# Official v2.0.26: {'cache_creation_input_tokens': 0, 'cache_read_input_tokens': 0}
# Fixed version: {'input_token_details': {'cache_read': 0, 'cache_creation': 0}}

# Test 2: Streaming (shows missing cache tokens)
for chunk in llm.stream(message):
    if hasattr(chunk, 'usage_metadata') and chunk.usage_metadata:
        print("Streaming:", chunk.usage_metadata)
        # Official v2.0.26: No input_token_details
        # Fixed version: {'input_token_details': {'cache_creation': 0, 'cache_read': 0}}
        break

Verification Testing

We have created comprehensive tests that demonstrate:

1. Issue confirmed in official langchain-google-vertexai v2.0.26
2. Fix verified with our implementation
3. Real API calls to claude-sonnet-4@20250514 showing actual behavior
4. Before/after comparison proving the fix works correctly

Test Results Summary:

• ❌ Official package: Missing input_token_details in streaming, wrong structure in non-streaming
• ✅ Fixed version: Proper input_token_details structure in both streaming and non-streaming

Root Cause Analysis

ChatAnthropicVertex doesn't conform to LangChain's UsageMetadata standard:

1. Non-streaming: Uses custom CacheUsageMetadata class that puts cache tokens at top level (cache_creation_input_tokens, cache_read_input_tokens) instead of in input_token_details
2. Streaming message_start: Only extracts basic token counts, ignores cache fields from event.message.usage
3. Streaming message_delta: Similarly missing cache token extraction
4. Standards Violation: Neither mode follows LangChain's UsageMetadata TypedDict structure with input_token_details

LangChain Standard (UsageMetadata TypedDict):

{
    "input_tokens": 14,
    "output_tokens": 13,
    "total_tokens": 27,
    "input_token_details": {    # ✅ Standard structure
        "cache_creation": 0,
        "cache_read": 0
    }
}

Proposed Solution

Make ChatAnthropicVertex conform to LangChain's UsageMetadata standard:

1. Eliminate custom CacheUsageMetadata: Remove non-standard top-level cache token fields
2. Implement standard structure: Use input_token_details for cache tokens in both streaming and non-streaming
3. Fix streaming extraction: Extract cache tokens from event.message.usage in message_start events
4. Follow official pattern: Match langchain_anthropic.ChatAnthropic implementation exactly
5. Create reusable function: _create_usage_metadata() for consistent token handling

Files Affected

• libs/vertexai/langchain_google_vertexai/_anthropic_utils.py
• libs/vertexai/langchain_google_vertexai/model_garden.py

Related Issues

This is similar to langchain-ai/langchain#975 but affects the Vertex AI implementation specifically.

Pull Request & Solution

A complete fix for this issue is available in Pull Request: https://github.com/langchain-ai/langchain-google/pull/1010

✅ Verified Fix Summary:

• ✅ Tested & Working: Verified with real API calls to Claude Sonnet 4
• ✅ Comprehensive: Fixes both streaming and non-streaming responses
• ✅ Standards Compliant: Uses proper LangChain UsageMetadata structure with input_token_details
• ✅ Official Pattern: Matches langchain_anthropic.ChatAnthropic implementation exactly
• ✅ Backward Compatible: Maintains full compatibility with existing code
• ✅ Quality Assured: Passes all linting, type checking, and contribution guidelines

Technical Implementation:

• ✅ Adds reusable _create_usage_metadata() function matching official langchain_anthropic implementation
• ✅ Fixes streaming message_start events to extract cache tokens from event.message.usage
• ✅ Fixes streaming message_delta events to follow official pattern (no cache tokens)
• ✅ Fixes non-streaming to use proper input_token_details structure instead of top-level cache fields
• ✅ Eliminates custom CacheUsageMetadata class in favor of standard LangChain types