Braintrust

Hakkında

Hosted platform for running evals on datasets, scoring outputs and tracing production AI requests. SDKs cover Python, TypeScript, Go, Java, Ruby and C#. Self-hosting the data plane or running it in your own cloud requires the Enterprise plan.

Neler yapabilirsin

• Run evals that score model outputs against a dataset in code or CI
• Trace production agent requests and turn logs into test datasets
• Compare prompts and models side by side in a playground

Başlarken

1. Create a Braintrust API key and set BRAINTRUST_API_KEY
2. Set OPENAI_API_KEY for the model under test

Örnek kod

# pip install braintrust autoevals openai; run with: bt eval movie_matcher.py
from braintrust import Eval
from autoevals import ExactMatch
from openai import OpenAI

client = OpenAI()

def task(input):
    prompt = "Based on the following description, identify the movie.\n" + input
    return client.responses.create(model="gpt-5-mini", input=prompt).output_text

Eval("Evaluation quickstart", data=[{"input": "A detective investigates a series of murders based on the seven deadly sins.", "expected": "Se7en"}], task=task, scores=[ExactMatch])

Ayrıntılar

Barındırma

Hazır hizmet (bulut), Kendi sunucunda

Kullanılabildiği yerler

Tüm dünya

Resmi SDK'lar

Python, JavaScript/TypeScript, Go, Java, Ruby, C#

MCP sunucusu

Uzak sunucu