# How to avoid lock-in to one LLM provider when building agents

Put a thin model layer between agents and providers, keep prompts, tools, logs and evals in formats you own, and test every model change.

## Key takeaways

- Every production agent on a hosted model should expect to change models at some point, because providers retire models on published schedules. Avoiding lock-in means making that change cheap, not avoiding a dependency on a model.
- Lock-in to an LLM provider sits in five places, and only one is the API. The others are prompts tuned to one model, tool integrations, the data and logs you keep, and your contracts and cloud.
- Route every model call through one internal model layer, with a way to pass provider-specific options, so swapping a model is a configuration change rather than a rewrite.
- Connect agents to tools through open protocols such as the Model Context Protocol, and keep logs, traces and evaluation data in your own store.
- An evaluation suite built from your own tasks is what makes switching safe. Run it before any model change reaches production traffic.

To avoid lock-in to one LLM provider when building agents, make switching models cheap rather than trying to avoid depending on one. Route every model call through a thin internal model layer, keep prompts versioned per model, connect tools through open protocols, store logs and evaluation data in formats you own, and test every model change against an evaluation suite built from your own tasks. Plan for switching from the start, because providers retire hosted models on published schedules, so every production agent on a hosted model should expect to change models at some point.

## Where does lock-in to an LLM provider come from?

Lock-in comes from five places, and only the first is the API.

- **The API.** Each provider's API has its own request format, tool-calling format and options. Code written directly against one has to be changed to call another.
- **Prompts tuned to one model.** Prompts, examples and settings that work well on one model often work less well on another. Anthropic's own compatibility documentation warns that a prompt adjusted extensively is "likely to be well-tuned to OpenAI specifically."[^anthropic-compat]
- **Tools and integrations.** Tools written for one vendor's agent framework, or connected through a proprietary integration, have to be rebuilt to move.
- **Data and telemetry.** Conversation logs, traces, evaluation results and fine-tuning data stored only in a provider's console are hard to take with you, and they are what you need to test a replacement.
- **Contracts and cloud.** Committed spend, a model bought through one cloud's marketplace, or data held in one cloud tie the agent to that provider's terms.

Lock-in is not unique to AI. The UK Competition and Markets Authority found in 2025 that "less than 1% of customers switch provider each year" in cloud services. It named egress fees as a key commercial barrier and said technical barriers "stem from the differentiation of features and interfaces in cloud services."[^cma] The same pattern applies to models: the more an agent depends on features only one provider has, the harder it is to move.

## Why plan for switching even if you are happy with your provider?

Because the choice to switch is not always yours. Providers retire models, and requests to a retired model stop working. Anthropic's documentation says plainly: "Requests to retired models fail."[^anthropic-deprecations] Each provider publishes how much notice it gives:

| Provider              | Published notice before a model is retired                                                                                                                                                                                                              |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OpenAI                | At least 6 months for generally available models and at least 3 months for specialized variants of them, unless safety or compliance require faster; preview models may be retired with much shorter notice, such as 2 weeks.[^openai-deprecations]     |
| Anthropic             | At least 60 days for publicly released models on Anthropic-operated platforms, notified to customers with active deployments.[^anthropic-deprecations]                                                                                                  |
| Google Cloud (Gemini) | Models in Google's 12-month table stay available for at least 12 months after release; models with shorter availability get a fixed retirement date at least 45 days ahead. Listed retirement dates are not moved earlier.[^google-lifecycle]           |
| Amazon Bedrock        | For models launched on or after September 7, 2026, a Legacy period of 6 months or 45 days before end of life, during which existing customers may lose access after 15 days of inactivity; migration does not happen automatically.[^bedrock-lifecycle] |

Dates can also differ for the same model depending on where you buy it. Anthropic notes that partner-operated platforms, namely Amazon Bedrock and the Google-managed Claude models on Google Cloud, "set their own retirement schedules," and AWS says its lifecycle dates "may differ from dates published by model providers."[^anthropic-deprecations][^bedrock-lifecycle] If your agents use models from several channels, track each one's dates separately.

OpenAI advises against "using preview models for business-critical production workloads unless you can migrate on short notice."[^openai-deprecations] That is a useful rule for any provider: the less notice a model carries, the more ready you need to be to move.

## How do you build agents so the model can be swapped?

Standardize six things across every team that builds agents:

1. **One model layer.** Route every model call through one internal interface or gateway, so agent code asks for a capability and configuration decides which model serves it. Provider tools follow the same idea: Amazon Bedrock's Converse API "provides a consistent API that works with all Amazon Bedrock models that support messages," so "you can write code once and use it with different models," and still lets you pass a model's unique parameters in a model-specific structure.[^bedrock-converse] Build your own layer the same way, with a common path for most calls and a clearly marked way to use provider-specific options.
2. **Prompts as versioned configuration.** Keep prompts, examples and model settings outside the code, versioned and tied to the model they were tuned for. Expect to retune them on a switch. Anthropic's migration guide for its own newer model tells developers to "re-evaluate model-specific prompt instructions" and to "test in a development environment before switching production traffic."[^anthropic-migration]
3. **Tools through open protocols.** Expose tools and data to agents through an open protocol rather than one vendor's plugin format. The Model Context Protocol (MCP) is "an open-source standard for connecting AI applications to external systems."[^mcp] Anthropic donated MCP to the Agentic AI Foundation, a directed fund under the Linux Foundation.[^mcp-aaif] For agents that call other agents, the Agent2Agent (A2A) protocol is also hosted by the Linux Foundation, which said A2A "will remain vendor neutral."[^a2a]
4. **Logs and traces in your own store.** Capture model requests, responses, tool calls and costs in your own logging and tracing systems, not only in a provider's console. OpenTelemetry's semantic conventions for generative AI define common span, metric and event names for model calls, with provider-specific pages for OpenAI, Anthropic, AWS Bedrock and Azure AI Inference. They are still marked "Development," so expect some names to change.[^otel-genai]
5. **An evaluation suite from your own tasks.** Collect real cases for each agent, with the expected outcome, and score every candidate model against them. OpenAI's own retirement guidance says to "run representative requests against both models" and to "compare outputs, latency, and cost before switching production traffic to the replacement."[^openai-deprecations] The suite is what turns a forced migration into a routine change. Our guide on [taking an AI agent from prototype to production](/guides/ai-agents-prototype-to-production/) covers building it as part of release readiness.
6. **A second option you have tested.** For each important agent, keep at least one alternative model qualified against the suite, and rerun the suite when either model changes.

## What does a provider-neutral layer cost you?

A neutral layer costs you some provider-specific features, and compatibility shortcuts cost more than they appear to.

Newer features, such as prompt caching, strict structured outputs or a provider's own tools, often exist on one API before others. A layer that only supports what every provider shares makes them unavailable. That is why the layer needs a deliberate escape hatch: use a provider-specific feature where it earns its place, mark it in configuration, and know which agents would need work if you moved.

Compatibility endpoints, where one provider accepts another provider's request format, are handy for trying a model and risky for production. Anthropic says its OpenAI SDK compatibility layer is "primarily intended to test and compare model capabilities, and is not considered a long-term or production-ready solution for most use cases." It ignores the `strict` parameter for function calling, does not support prompt caching, and silently ignores most unsupported fields.[^anthropic-compat] Google describes its OpenAI library support for the Gemini API as "still in beta" and recommends calling the Gemini API directly if you are not already using the OpenAI libraries.[^gemini-compat] A request that silently drops a setting can pass a quick test and fail in production, so test through the path you will run.

The layer also adds a component to run and secure. If it is a gateway, it sits in the path of every model call, so it needs the same availability, access control and logging as any other production service. It is also a natural place to enforce budgets per agent, as our guide on [controlling the cost of AI agents](/guides/control-ai-agent-costs/) describes.

## What should contracts and cloud choices cover?

Contracts decide whether you can leave as much as code does. Check four things before committing an agent to a provider:

- **Training use.** Confirm in writing that your prompts and outputs are not used to train the provider's models. OpenAI's platform documentation says that since March 1, 2023, data sent to its API "is not used to train or improve OpenAI models (unless you explicitly opt in to share data with us)."[^openai-data] Anthropic's commercial terms say it "may not train models on Customer Content from Services."[^anthropic-terms]
- **Retirement notice.** Note each model's published lifecycle in the agent's record, and buy through the channel whose schedule suits you, since the same model can carry different dates on different clouds.[^anthropic-deprecations][^bedrock-lifecycle]
- **Data export.** Make sure you can export logs, evaluation data and any fine-tuning data in a usable format, and that deletion on exit is defined.
- **Switching rights in the EU.** The EU Data Act's rules on switching between data processing services, which cover infrastructure, platform and software as a service, have applied since 12 September 2025, and may apply to model services bought as cloud services. From 12 January 2027, providers may not charge customers for switching, including data egress.[^data-act][^data-act-explained] The Act obliges only infrastructure (IaaS) providers to facilitate functional equivalence, so it helps you move data but does not make two model APIs interchangeable.[^data-act]

Keep identity and access on your side as well: issue each agent its own credentials in your identity provider, so a change of model provider does not mean rebuilding access. See [how to give AI agents identities and permissions](/guides/ai-agent-identity-and-permissions/).

## Where Sycamore fits

Sycamore [Build](/build/) is an enterprise platform for building agentic software. It supports multiple models and Sycamore-managed, dedicated, or bring-your-own-cloud deployments, so teams can evaluate and adopt new models without rebuilding applications. Build promotes an approved version through preview, development and production with deployment history, and can roll back when needed.

Sycamore [Govern](/govern/) gives every agent an identity, with model, tool and data access routed through policy. In Govern's intake, approved budgets, models and operating boundaries become runtime policy, so the models each agent may use are part of its approval.

[^anthropic-compat]: Anthropic, "OpenAI SDK compatibility", Claude Developer Platform documentation, https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk

[^cma]: Competition and Markets Authority, "Cloud services market investigation: Summary of final decision" (31 July 2025), https://assets.publishing.service.gov.uk/media/688b20e6ff8c05468cb7b120/summary_of_final_decision.pdf

[^anthropic-deprecations]: Anthropic, "Model deprecations", Claude Developer Platform documentation, https://platform.claude.com/docs/en/about-claude/model-deprecations

[^openai-deprecations]: OpenAI, "Deprecations", OpenAI API documentation, https://developers.openai.com/api/docs/deprecations

[^google-lifecycle]: Google Cloud, "Model versions and lifecycle", Gemini Enterprise Agent Platform documentation, https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/model-versions

[^bedrock-lifecycle]: AWS, "Model lifecycle", Amazon Bedrock User Guide, https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html

[^bedrock-converse]: AWS, "Inference using Converse API", Amazon Bedrock User Guide, https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html

[^anthropic-migration]: Anthropic, "Claude Opus 5.5 migration guide", Claude Developer Platform documentation, https://platform.claude.com/docs/en/models/opus-5-5/migration-guide

[^mcp]: Model Context Protocol, "What is the Model Context Protocol (MCP)?", https://modelcontextprotocol.io/docs/2026-07-28/getting-started/intro

[^mcp-aaif]: Model Context Protocol Blog, "MCP joins the Agentic AI Foundation" (December 9, 2025), https://blog.modelcontextprotocol.io/posts/2025-12-09-mcp-joins-agentic-ai-foundation/

[^a2a]: Linux Foundation, "Linux Foundation Launches the Agent2Agent Protocol Project to Enable Secure, Intelligent Communication Between AI Agents" (June 23, 2025), https://www.linuxfoundation.org/press/linux-foundation-launches-the-agent2agent-protocol-project-to-enable-secure-intelligent-communication-between-ai-agents

[^otel-genai]: OpenTelemetry, "Semantic conventions for generative AI systems", https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/README.md

[^gemini-compat]: Google AI for Developers, "OpenAI compatibility", Gemini API documentation, https://aistudio.google.com/docs/openai

[^openai-data]: OpenAI, "Data controls in the OpenAI platform", OpenAI API documentation, https://developers.openai.com/api/docs/guides/your-data

[^anthropic-terms]: Anthropic, "Commercial Terms of Service", https://www.anthropic.com/legal/commercial-terms

[^data-act]: European Union, "Regulation (EU) 2023/2854 (Data Act)", Articles 23, 29, 30 and 50 and Recitals 81 and 86, https://eur-lex.europa.eu/eli/reg/2023/2854/oj

[^data-act-explained]: European Commission, "Data Act explained", https://digital-strategy.ec.europa.eu/en/factpages/data-act-explained

## Frequently asked questions

### Should we use several LLM providers in production at the same time?

Only where it pays for itself. Running two providers adds contracts, security reviews and evaluation work. Many teams standardize on one primary provider, keep a second qualified through the same model layer and evaluation suite, and route to it for specific tasks or as a fallback. What matters is that the second option has been tested recently, so switching is a decision, not a project.

### Do open-weight models remove the lock-in problem?

They remove the risk that a provider retires the model you depend on, because you can keep running the weights. They do not remove the rest. You take on hosting, scaling, security patching and upgrades, and your prompts are still tuned to one model. Treat an open-weight model as one more option behind the same model layer, judged by the same evaluation suite.

### Is an LLM gateway enough to avoid lock-in?

A gateway helps with the API and with routing, budgets and logs in one place, but it covers one of the five sources of lock-in. Prompts tuned to one model, tools built on one vendor's framework and evaluation data stored with a provider still tie you down. Choose a gateway you could replace too, and keep its configuration and logs in formats you own.

### Who should own model choice in a large company?

A platform or architecture team should own the model layer, the approved list of models and the shared evaluation tooling. Product teams should own the evaluation cases for their own agents and decide, within the approved list, which model each agent uses. That split lets a central team qualify a new model once while each team proves it works for their task.

Published November 15, 2026 by Sycamore.