LiteLLM

Package

Open-source LLM gateway and Python SDK for 100+ model providers

Price: Free, open source; paid Enterprise
Access: Your provider API keys

About

Open-source Python SDK and self-hosted proxy that call 100+ LLM providers in the OpenAI format. The proxy adds virtual keys, budgets, fallbacks and spend logs for platform teams. You run it yourself; SSO, audit logs and support need an Enterprise license.

What you can do with it

Call OpenAI, Anthropic, Vertex AI and Bedrock models with one completion() call
Run a self-hosted gateway with virtual keys, budgets and rate limits per team
Load-balance across deployments and fall back when a provider fails

Get started

Install the SDK with uv add litellm
Set your provider key, e.g. OPENAI_API_KEY

Example

# uv add litellm; export OPENAI_API_KEY first
from litellm import completion

response = completion(
  model="openai/gpt-5.6-terra",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)

Details

Hosting: Self-hosted, Runs locally
Available in: Worldwide
Official SDKs: Python
MCP server: None