APIs, MCP servers and models to run models locally
3 tools in the catalog do this, by name.
LocalAIPackageRun LLM, image and audio models on your own hardware behind OpenAI-style APIs
OllamaPackageRun open models on your own machine behind a local REST API
vLLMPackageOpen-source engine for serving LLMs with high throughput on GPUs