Skip to main content

​
Overview

LLM Gateway v2.0 introduces:
  • Significant performance improvements, with responses up to 5x times faster
  • Automatic provider fail over via a new, provider-agnostic model naming scheme, allowing the gateway to automatically fail over between providers without any client-side changes
  • Longer log retention with 90 days of logs now available in the dashboard and 6+ years retained for audit purposes
  • Simplified scoping for LLM Keys, model access policies, and spend limits by shifting them from being scoped per Environment to being scoped to your whole organization.
  • A new base url in order to facilitate ease of migration from v1.0
Any LLM Key you created before v2.0 keeps working exactly as it did under v1.0, with the old base URL and model names. These keys will NOT be disabled or automatically migrated, and will appear under Legacy Keys in LLM Gateway. Any key you create going forward is a v2.0 key by default.
v1.0 and v2.0 keys work side by side. You can migrate one application or agent at a time, verify it, and move on to the next, rather than cutting everything over at once.

​
What changed

Model access policies and spend limits used to apply per Environment. If you had different allowed models or limits across multiple Environments, moving your last v1.0 key to v2.0 collapses them into a single organization-wide policy and limit. Check your organization’s combined model access policy and spend limit before you do, so you don’t lose access to a model or land on a lower limit than you expect.

​
Migrate a key

For each key you created before v2.0:
1

Create a new key

Go to LLM Gateway and select Create Key. Unlike v1.0, you don’t need to create an Environment first — the new key is scoped to your organization. See Creating a Key for details.
2

Update your base URL and model names

In the app, agent, or harness configuration where you use the old key, change the base URL to https://llm-gateway.aptible.com, or to a strict-compatibility namespace like https://llm-gateway.aptible.com/openai or https://llm-gateway.aptible.com/anthropic. If you’re configuring a supported coding assistant or agent harness, use its dedicated /harness/<name> base URL instead — see Supported APIs for the full list.Then update the configured model name(s) to the new IDs from Supported Models:
3

Swap in the new key

Replace the old key with the new one in that same configuration, then redeploy or restart as needed.
4

Verify, then revoke the old key

Confirm requests are flowing by checking the new key’s request history in LLM Gateway. Once you’re confident the migration is complete, revoke the old key — see Revoking a Key.
If you drained logs to Langfuse under v1.0, note that Langfuse draining isn’t available for v2.0 keys. Aptible now retains every log automatically for 6+ years, so a drain isn’t needed for long-term retention, but if you built dashboards or alerting on top of that export, you’ll need to replace it with Audit Logging, or contact Aptible Support about other export options.

​
Troubleshooting

Yes, but not immediately. A v1.0 key keeps working with v1.0’s original base URL and model names, and it will continue to show as a Legacy Key in LLM Gateway. But v2.0-only features like automatic provider failover are only available with a v2.0 key. The sooner you switch, the sooner you have access to these and other upcoming improvements.Once the majority of Aptible customers have migrated to v2.0 keys, we will set an EOL date for making requests with v1.0 keys.
Double-check that you updated the model name, not just the base URL. v1.0’s provider-prefixed names (e.g. bedrock/us.anthropic.claude-sonnet-5) aren’t recognized against the v2.0 base URL. Use the plain ID from Supported Models (e.g. claude-sonnet-5) instead.
A v1.0 key won’t authenticate against the v2.0 base URL, and a v2.0 key won’t authenticate against the v1.0 base URL. Make sure the key and base URL you’re using are from the same version.
This can happen if your v1.0 Environments had different model access policies and the model you need was only allowed in one of them. Ask an Account Owner to review and update your organization’s single Model Access Policy.