Is Braintrust agent-ready?

Braintrust scores Band A on The Headless Index and rates AI-Ready on JAIRF (83.3/100). Its strongest area for agent use is API-first design intent; its weakest is webhooks and events.

Does Braintrust publish an OpenAPI spec?

Yes. Braintrust publishes an OpenAPI specification, scored under JAIRF v1.0.0. A published spec is what makes an API discoverable and callable by agents rather than only by developers reading docs.

What is Braintrust's Headless Index score?

Braintrust scores 84/100 on the Headless Index thesis-fit rubric and sits in Band A in the Observability category. The score weighs API-first design, headless operation, MCP and agent posture, schema observability, and webhooks.

03 / Scorecard / Observability

Braintrust

Headless Index

84/100

JAIRF

83.3/100

AI-Ready

Verified

AUG 20, 2026

Methodology v1 · JAIRF v1.0.0

Braintrust earns Band A in the Observability category of The Headless Index, with a thesis-fit score of 84/100 and a JAIRF rating of 83.3/100 (AI-Ready). Its strongest dimension is API-first design intent (20/20); its weakest scored dimension is webhooks and events (8/20). Braintrust publishes a machine-readable OpenAPI specification, which is what lets an agent discover and operate it without human glue code.

Editorial verdict

Braintrust is reference-class for machine consumption. The Headless Index thesis-fit score of 84/100 places it in the top tier of the index, and JAIRF v1.0.0 puts it at 83.3/100 (Level 3, AI-Ready). In practice, vendors at this tier publish enough machine-readable surface that agents can discover, authenticate, operate, and react to events without bespoke scaffolding, and the rest of this verdict explains where Braintrust lands inside that pattern. On the API surface, the question is whether the API is the product or a layer beneath the dashboard. The spec came first and the clients are generated from it, which shows. Seventy-five paths under api.braintrust.dev cover projects, experiments, datasets, prompts, roles and groups, with typed clients published for JavaScript, Python and Go (@braintrust/api, braintrust-api, braintrust-go). An agent can drive this product end to end, and the published evidence supports that without unusual scaffolding. On headless operability: The bt CLI is positioned as the primary interface for coding agents, running evals, querying logs, pushing functions and managing topics. Alerts are not UI-only either, since project automations have full CRUD endpoints (/v1/project_automation). Self-hosting is codified rather than described, with maintained modules per cloud (braintrustdata/terraform-aws-braintrust-data-plane). Playgrounds and human review sessions are the parts that stay in the browser.^[1] On the MCP and agent-integration axis, which is the fastest-moving criterion in the index: Braintrust runs MCP as hosted infrastructure rather than a repo you clone, with regional endpoints (https://api.braintrust.dev/mcp and api-eu for the EU data plane), a documented tool list covering schema inference, SQL query and permalink generation, and a Claude Code plugin wrapping it. Docs even publish an index for models to crawl (braintrust.dev/docs/llms.txt). The CLI is still the recommended agent path.^[2] Event posture closes the loop: an agent that cannot react to state changes is reduced to polling. Alerting exists in two flavours, log alerts driven by a SQL filter and environment alerts on prompt or dataset assignment, and both can post outbound. The delivery contract is thin: a fixed JSON payload, no custom headers, no signature, no documented retry, and batching means matching events inside a suppression window are dropped rather than queued (notify interval up to 24 hours). Net assessment: Braintrust is among the cleanest agent-targets in its category. The lightest of the criteria in this run is webhooks and events^[3], and even there the surface area is workable for automated integration. Treat this scorecard as a green light for unattended use. Strong fit for agent-driven use cases.

Verdict by Headless Index pipeline (auto)

// AI-drafted from the evidence layer. Editorial review pending.

Scores

Scorecard detail

Headless Index · 5 sub-criteria

API-first design intent20/20

scored

The spec came first and the clients are generated from it, which shows. Seventy-five paths under api.braintrust.dev cover projects, experiments, datasets, prompts, roles and groups, with typed clients published for JavaScript, Python and Go (@braintrust/api, braintrust-api, braintrust-go). The tracing library carries real weight too (npm braintrust, over a million weekly downloads). Reference implementation for this category.

signals (6)

+AI review appliedReviewer: Editorial review on 2026-08-19
+OpenAPI specPublished, 234 operations
−GraphQL endpointNot discovered (5 probes; project-scoped endpoints require a real project ID)
+SDKs maintained11 (dotnet, go, java, javascript, python, ruby, rust, typescript); top by stars: braintrustdata/braintrust-sdk-javascript (27 stars)
+SDK recency8 of 11 SDK repos pushed within 30 days (most recent SDK commit: 2026-08-19)
+npm weekly downloads1.2M across published packages; top: braintrust @ 1.2M/week

cite (3)

ai_review_browser.auth@2026-08-19
ai_review_browser.schema@2026-08-19
ai_review_browser.sdks@2026-08-19

Headless operation18/20

scored

The bt CLI is positioned as the primary interface for coding agents, running evals, querying logs, pushing functions and managing topics. Alerts are not UI-only either, since project automations have full CRUD endpoints (/v1/project_automation). Self-hosting is codified rather than described, with maintained modules per cloud (braintrustdata/terraform-aws-braintrust-data-plane). Playgrounds and human review sessions are the parts that stay in the browser.

signals (9)

+AI review appliedReviewer: Editorial review on 2026-08-19
+API operations exposed234 operations in OpenAPI spec
·Docs pages crawled0 pages (crawler: none)
·Auth schemes documentedAuth documentation page not reached by crawler
·Setup / quickstart docsNot reached by crawler
·Billing docsNot reached by crawler
·Teams / org docsNot reached by crawler
·CLI docsNot reached by crawler
·Schema / data model docsNot reached by crawler

cite (1)

ai_review_browser.topics_found@2026-08-19

MCP & agent posture18/20

scored

Braintrust runs MCP as hosted infrastructure rather than a repo you clone, with regional endpoints (https://api.braintrust.dev/mcp and api-eu for the EU data plane), a documented tool list covering schema inference, SQL query and permalink generation, and a Claude Code plugin wrapping it. Docs even publish an index for models to crawl (braintrust.dev/docs/llms.txt). The CLI is still the recommended agent path.

signals (4)

+AI review appliedReviewer: Editorial review on 2026-08-19
−Official MCP serverNone found in vendor's GitHub org or the official MCP registry
−Community MCP serversNone found
+Agent-friendly SDKs3 TS/JS SDKs available; top: braintrust (1.2M/week downloads)

cite (1)

ai_review_browser.mcp@2026-08-19

Schema observability20/20

scored

The spec is a first-class artifact in its own repository and fetchable raw (raw.githubusercontent.com/braintrustdata/braintrust-openapi/main/openapi/spec.json), OpenAPI 3.0.3, seventy-five paths, servers and bearerAuth security declared inline. An agent can build a working client with no prose at all, then check itself against the per-endpoint markdown pages the docs expose. This is what the criterion is asking for.

signals (3)

+AI review appliedReviewer: Editorial review on 2026-08-19
+OpenAPIPublished at https://raw.githubusercontent.com/braintrustdata/braintrust-openapi/main/openapi/spec.json (OpenAPI 3.0.3, 234 operations)
−GraphQL introspectionNo GraphQL endpoint discovered (5 probes; some vendors use project-scoped endpoints that require a real project handle)

cite (1)

ai_review_browser.schema@2026-08-19

Webhooks & events8/20

scored

Alerting exists in two flavours, log alerts driven by a SQL filter and environment alerts on prompt or dataset assignment, and both can post outbound. The delivery contract is thin: a fixed JSON payload, no custom headers, no signature, no documented retry, and batching means matching events inside a suppression window are dropped rather than queued (notify interval up to 24 hours).

signals (2)

+AI review appliedReviewer: Editorial review on 2026-08-19
·Webhook docs pageNot reached by crawler within budget (0 pages crawled). Cannot confirm whether vendor offers webhooks.

cite (1)

ai_review_browser.pages_fetched@2026-08-19

JAIRF · 6 dimensions

FCFoundational Compliance

70/100

Structural validity, standards conformance, and parsability of the OpenAPI specification.

DXJDeveloper Experience & Tooling Compatibility

70.4/100

Documentation clarity, example coverage, response completeness, and ingestion health.

ARAXAI-Readiness & Agent Experience

86.3/100

Semantic clarity, intent expression, datatype specificity, and error standardization.

AUAgent Usability

90/100

Operational composability, complexity comfort, navigation affordances, and safety patterns.

SECSecurity

100/100

Authentication strength, transport security, secret hygiene, and OWASP risk posture.

AIDAI Discoverability

85/100

Descriptive richness, intent phrasing, workflow context, and registry signals.

Band rationale:A band: JAIRF=83.3 HeadlessIndex=84

04 / Embed

Show Braintrust's score on your site.

Drop a live badge into your README, footer, or marketing page. It updates automatically when we re-score, and every embed is a dofollow link back here.

Get embed code

Calibration

How THI compares to external scorers

Source	Score	Measures	Last checked
Fern Agent Score	not found	Documentation completeness and SDK shape (~22 checks)	—
CLIRank Agent Friendliness	not found	CLI readiness, docs quality, and overall agent affordances	—
Cloudflare Is It Agent Ready?	blocked	Cloudflare's manual agent-readiness heuristic per vendor URL	—
Jentic Scorecard	—	JAIRF-based scorecard requiring a public OpenAPI specification	—

THI 84 vs external median 0

No external scores available to calibrate against.

Peers in Observability

ElasticC

Headless Index 86/100

LangSmithB

Headless Index 81/100

SentryF

Headless Index 78/100

See full Observability ranking →

FAQ

Braintrust and agent readiness

Is Braintrust agent-ready?: Braintrust scores Band A on The Headless Index and rates AI-Ready on JAIRF (83.3/100). Its strongest area for agent use is API-first design intent; its weakest is webhooks and events.
Does Braintrust publish an OpenAPI spec?: Yes. Braintrust publishes an OpenAPI specification, scored under JAIRF v1.0.0. A published spec is what makes an API discoverable and callable by agents rather than only by developers reading docs.
What is Braintrust's Headless Index score?: Braintrust scores 84/100 on the Headless Index thesis-fit rubric and sits in Band A in the Observability category. The score weighs API-first design, headless operation, MCP and agent posture, schema observability, and webhooks.