Model name aliasing for multi-provider fallback

[sarth6]

Hi there, we're running into an issue with provider fallback in the Gateway and wanted to get your guidance on the recommended approach.

The setup

We have a routing group for Anthropic that includes Vertex AI as a low-priority fallback:

anthropic routing group:
  - anthropic prod key (priority 1)
  - anthropic builtin (priority 0)
  - google-vertex builtin (priority -1)

Our goal is that if our Anthropic API key hits rate limits or has issues, we can fall back to Vertex AI which hosts the same Claude models.

The problem

When the Anthropic providers are unavailable and requests fall back to Vertex AI, we get 404 errors because the model name formats are incompatible:

• Our code sends: claude-sonnet-4-0 (Anthropic API format)
• Vertex AI expects: claude-sonnet-4@20250514 (uses @ instead of - for versions)

The actual error we see:

Publisher Model `projects/xxx/locations/global/publishers/anthropic/models/claude-sonnet-4` was not found

Looks like Vertex strips the -0 suffix and then can't find the model.

What we're wondering

1. Is there a way to configure model name mappings per provider? Something like "when using the vertex provider, translate claude-sonnet-4-0 to claude-sonnet-4@20250514"?

2. Or is mixing providers with different model name conventions in the same routing group just not supported? If so, what's the recommended way to get cross-provider redundancy for Claude models?

3. Should we just remove Vertex from the anthropic routing group entirely and accept that fallback won't work across provider types?

We're seeing this in production when Anthropic has intermittent availability issues - works fine most of the time, but when fallback kicks in we get a burst of 404s.

Any guidance appreciated!

[DouweM]

@sarth6 Sorry for the late response here.

The model name mapping for Google Vertex lives here:

pydantic-ai-gateway/gateway/src/providers/google/index.ts

Lines 83 to 94 in 736fa5d

	protected getModelNameRemappings(): Array<{ searchValue: string; replaceValue: string }> {
	return [
	{ searchValue: '^claude-3-5-sonnet.*$', replaceValue: 'claude-3-5-sonnet' },
	{ searchValue: '^claude-3-5-haiku.*$', replaceValue: 'claude-3-5-haiku' },
	{ searchValue: '^claude-3-haiku.*$', replaceValue: 'claude-3-haiku' },
	{ searchValue: '^claude-haiku-4-5.*$', replaceValue: 'claude-haiku-4-5' },
	{ searchValue: '^claude-opus-4-1.*$', replaceValue: 'claude-opus-4-1' },
	{ searchValue: '^claude-opus-4.*$', replaceValue: 'claude-opus-4-1' },
	{ searchValue: '^claude-sonnet-4-0.*$', replaceValue: 'claude-sonnet-4' },
	{ searchValue: '^claude-3-7-sonnet.*$', replaceValue: 'claude-3-7-sonnet' },
	]
	}

As you can see, claude-sonnet-4-0 would get mapped to claude-sonnet-4, without a version identifier.

https://docs.cloud.google.com/vertex-ai/generative-ai/docs/partner-models/claude/sonnet-4 suggests that is fine, and you can see in the VCR cassette that it worked at one point:

pydantic-ai-gateway/proxy-vcr/proxy_vcr/cassettes/google-vertex-fallback.yaml

Line 20 in 736fa5d

uri: https://aiplatform.googleapis.com/v1/projects/pydantic-ai/locations/global/publishers/anthropic/models/claude-sonnet-4:rawPredict

So I'm not quite sure why you're getting that error. Does it work if you're using Google Vertex directly instead of through Gateway / a fallback group?

[sarth6]

@DouweM thanks for the guidance - when (realistically if) I have time I'll check on vertex directly as you suggested, thanks!