Alternative LLM Providers in Agents SDK
AI Engineering
Agents SDK can be used with alternative model providers or even local LLMs. One method that opens up many LLM integrations is to use the LiteLLM extension for Agents SDK. In this guide, we'll explore prompting, tools, and guardrails in Agents SDK, but using Anthropic's Claude models.
⚠️ It's worth noting that not every feature of Agents SDK will work out of the box when using other model providers. Tracing, agent handoffs, and pre-built OpenAI tools do not work.
Installation and Anthropic Setup
We'll first ensure we have our prerequisite library installed, which is Agents SDK with the LiteLLM extension, installed like so:
Note, if you're running these notebooks locally and have installed the uv
environment, this library and extension has already been installed as part of that environment.
Next, we grab an Anthropic API key from their console, and enter it below.
To initialize LitellmModel
we must provide the model provider we want to use and the model itself in the format <provider>/<model-name>
. We will be using Claude 3.7 Sonnet (i.e. claude-3-7-sonnet-latest
) from Anthropic, so our model string is anthropic/claude-3-7-sonnet-latest
.
Prompting
For initializing and running our agent, we the same as usual, ie we setup our agent:
Then we run our agent with a Runner
:
"Arrr, beneath a silver moon's embrace, the salty waves do dance and chase, while lonesome hearts on distant shores be dreamin' o' the sea's wild roars."
Tools
Predefined tools such as the WebSearchTool
cannot be used with other model providers, however, we can use our own custom tools. Let's try defining a custom tool with the @function_tool
decorator.
We pass our tool to the agent via the tools
parameter as usual.
Then we run as usual:
'The current time is 13:09:47 (1:09 PM) on April 26, 2025.'
Guardrails
For guardrails we again do the same as usual, we'll define a "scam detection" guardrail, defining the ScamDetectionOutput
base model containing the boolean is_scam
field.
Next we need to create the agent from the Agent
object.
Note that this isn't the main agent and is only used to check the input of our guardrail function.
As usual, we define our guardrail with the @input_guardrail
decorator, alonside the required GuardrailFunctionOutput
, and including the context (ctx
), unused agent
, and input
parameters.
Now we initialize the main agent, which will defer to our guardrail_agent
for scam detection.
Now we run our main_agent
. Because the guardrail triggers an InputGuardrailTripwireTriggered
error, we handle this in a try-except
block.
Error: Guardrail InputGuardrail triggered tripwire
We've seen how to use non-OpenAI models with Agents SDK using the LiteLLM extension. Using Anthropic's Claude as a generic agent, with tools, and with guardrails.

- Tracing with Agents SDK
- Voice Agents in Agents SDK
Alternative LLM Providers in Agents SDK