LiteLLM

Paket

Open-source LLM gateway and Python SDK for 100+ model providers

Fiyat: Free, open source; paid Enterprise
Erişim: Your provider API keys

Hakkında

Open-source Python SDK and self-hosted proxy that call 100+ LLM providers in the OpenAI format. The proxy adds virtual keys, budgets, fallbacks and spend logs for platform teams. You run it yourself; SSO, audit logs and support need an Enterprise license.

Neler yapabilirsin

Call OpenAI, Anthropic, Vertex AI and Bedrock models with one completion() call
Run a self-hosted gateway with virtual keys, budgets and rate limits per team
Load-balance across deployments and fall back when a provider fails

Başlarken

Install the SDK with uv add litellm
Set your provider key, e.g. OPENAI_API_KEY

Örnek kod

# uv add litellm; export OPENAI_API_KEY first
from litellm import completion

response = completion(
  model="openai/gpt-5.6-terra",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)

Ayrıntılar

Barındırma: Kendi sunucunda, Kendi bilgisayarında
Kullanılabildiği yerler: Tüm dünya
Resmi SDK'lar: Python
MCP sunucusu: Yok