Bigspin Annotation API

Internal quickstart · base URL https://api-dev.bigspin.ai · internal-first

Overview

The Annotation API turns a conversation transcript into a structured quality annotation: you POST the transcript, and get back a summary, a holistic outcome judgment, and the quality signals that fired.

You always name an annotator — a versioned instrument in the registry, like universal-signals-v1 — never a model or provider. Which model serves an annotator is a registry decision that can change with zero client-facing impact. List what's available at GET /v1/annotators.

Auth: every endpoint except this page, /health, and the docs pages requires Authorization: Bearer bsk_live_... (or bsk_test_... for test keys).

Current limits:

• Request bodies are capped at 1MB (1,048,576 bytes) → 413.
• Per-key rate limits (default 60 requests/60s; tiers can override) → 429 with a Retry-After header. Global provider-capacity limits can also surface as 429 + Retry-After.
• Synchronous, single-transcript annotation only for now: /v1/batches is spec'd but returns 501.

API reference: /docs (Swagger UI, FastAPI substrate), /swagger (Swagger UI, Lambda substrate), /openapi.json (canonical OpenAPI spec).

Quickstart

1. Get an API key

Keys are admin-minted for now (internal-first). From bigspin-api/ in the repo:

make issue-key WORKSPACE=<workspaceId> NAME=<label>

The plaintext key is printed exactly once — store it immediately. Only a SHA-256 hash is kept at rest, so it cannot be recovered later.

2. Shape your transcript

A transcript is a list of messages, each with a role and content, plus an optional is_user_visible flag:

• role is a string. user and human_agent are labeled distinctly for the annotator; any other role (assistant, system, tool, ...) is presented as the assistant side of the conversation.
• is_user_visible defaults to true; the camelCase spelling isUserVisible is also accepted. Hidden turns (tool output, internal messages) are rendered to the annotator with a [Not visible to user] prefix.

3. Make your first call

POST to /v1/annotations with your annotator, an optional options bag (per-request knobs; {} is fine), and the transcript. This example contains a user correcting the assistant, so it should plausibly fire the user_correction signal:

curl https://api-dev.bigspin.ai/v1/annotations \
  -H "Authorization: Bearer $BIGSPIN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "annotator": "universal-signals-v1",
    "options": {},
    "transcript": [
      {"role": "user", "content":
        "Rename the launch email variable everywhere it appears."},
      {"role": "assistant", "content":
        "Done - renamed launch_email to launchEmail in mailer.py."},
      {"role": "user", "content":
        "No, I meant the subject-line constant, not the variable."},
      {"role": "assistant", "content":
        "My mistake - renamed LAUNCH_EMAIL_SUBJECT and reverted the rest."}
    ]
  }'

import os

import requests

resp = requests.post(
    "https://api-dev.bigspin.ai/v1/annotations",
    headers={"Authorization": f"Bearer {os.environ['BIGSPIN_API_KEY']}"},
    json={
        "annotator": "universal-signals-v1",
        "options": {},
        "transcript": [
            {"role": "user", "content":
             "Rename the launch email variable everywhere it appears."},
            {"role": "assistant", "content":
             "Done - renamed launch_email to launchEmail in mailer.py."},
            {"role": "user", "content":
             "No, I meant the subject-line constant, not the variable."},
            {"role": "assistant", "content":
             "My mistake - renamed LAUNCH_EMAIL_SUBJECT"
             " and reverted the rest."},
        ],
    },
    timeout=120,
)
resp.raise_for_status()
annotation = resp.json()
print(annotation["outcome"], annotation["signals"].keys())

const resp = await fetch("https://api-dev.bigspin.ai/v1/annotations", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.BIGSPIN_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    annotator: "universal-signals-v1",
    options: {},
    transcript: [
      { role: "user", content:
        "Rename the launch email variable everywhere it appears." },
      { role: "assistant", content:
        "Done - renamed launch_email to launchEmail in mailer.py." },
      { role: "user", content:
        "No, I meant the subject-line constant, not the variable." },
      { role: "assistant", content:
        "My mistake - renamed LAUNCH_EMAIL_SUBJECT and reverted the rest." },
    ],
  }),
});
if (!resp.ok) throw new Error(`annotation failed: ${resp.status}`);
const annotation = await resp.json();
console.log(annotation.outcome, Object.keys(annotation.signals));

4. Read the response

{
  "summary": {
    "title": "Renaming an email subject constant",
    "keywords": [
      "refactor",
      "rename",
      "email"
    ],
    "summary": "The user asked for a rename; the assistant changed the wrong identifier first, then fixed it after a correction.",
    "quality_concerns": "Initial edit targeted the wrong identifier.",
    "user_intent": "rename the email subject-line constant",
    "domain": "software engineering"
  },
  "outcome": "mixed",
  "outcome_notes": "Recovered after one wrong-target edit.",
  "signals": {
    "user_correction": {
      "evidence": "No, I meant the subject-line constant, not the variable.",
      "turn": 3
    }
  },
  "annotator": "universal-signals-v1",
  "annotator_version": "1.0.0",
  "taxonomy_version": "2.0-gpt5",
  "usage": {
    "tokens_in": 2481,
    "tokens_out": 312,
    "latency_ms": 9400
  }
}

• summary — structured summary: title, keywords, prose summary, quality concerns, user intent, and domain.
• outcome — holistic judgment, one of strong | mixed | poor | critical | indeterminate, with the reasoning in outcome_notes.
• signals — sparse: only signals that fired appear. Each has evidence (a quote) and an optional 1-indexed turn.
• annotator, annotator_version, taxonomy_version — provenance: exactly which instrument and taxonomy produced this annotation.
• usage — tokens_in/tokens_out and latency_ms for the call. If the automatic one-shot schema-repair retry fires, tokens include both attempts.

Pinning a substrate (optional)

The API is served by two substrates behind one load balancer — Fargate and Lambda — with weighted routing by default. Send X-Bigspin-Substrate: lambda or X-Bigspin-Substrate: fargate to pin a request to one of them. This is primarily for the current bake-off period; responses report the serving substrate on GET /health.

Errors

Every error wears one envelope: {"error": ..., "detail": ...}.