Documentation menu

HTTP API reference (v1)

Retrack's HTTP API: send runs, recall memory, report outcomes, manage playbooks, run regression and migration reports, and read holdout metrics.

Updated

The connector covers everything most agents need. The HTTP API is for other languages, custom pipelines and automation.

Basics

  • Base URL: https://api.retrack-ai.online
  • Authentication: Authorization: Bearer <api_key> on every request. The key identifies the project.
  • Format: JSON requests and responses. Errors look like {"error": {"code": "...", "message": "..."}}.
  • Rate limits: a project that sends too many runs per minute gets 429 rate_limited with a Retry-After header.

Endpoints

Method and pathPurpose
POST /v1/runsSend a finished agent run
POST /v1/recallGet gated playbooks and warnings for a task
POST /v1/feedbackAttach a late outcome to a run
GET /v1/playbooksList playbooks
GET /v1/playbooks/{id}/versionsA playbook's version history
PATCH /v1/playbooks/{id}Quarantine or retire a playbook
GET /v1/eval-casesRegression cases for an agent
POST /v1/eval-resultsReport regression results
POST /v1/migrationsStart a model-migration report
GET /v1/migrations/{id}Read a migration report
GET /v1/metricsMemory versus holdout comparison

POST /v1/runs

One finished agent task. Idempotent on id: sending the same run twice stores it once.

{
  "id": "6f1c2b8e-1d2a-4c55-9a51-0d5a8c3e2f10",
  "agent": "support-agent",
  "task": "Refund order #123 that arrived damaged",
  "model_id": "claude-haiku-5-5",
  "tool_versions": {"orders_api": "v3"},
  "tokens_in": 1200, "tokens_out": 300, "cost_usd": 0.004, "latency_ms": 5400,
  "outcome_hint": {"outcome": "success", "source": "test"},
  "output": "Refunded order #123.",
  "steps": [
    {"idx": 0, "type": "tool", "tool_name": "orders_api", "args": {"order_id": "123"},
     "result_class": "ok", "error_class": null, "latency_ms": 120}
  ]
}

Optional error: {"class": "TimeoutError", "stage": "tool" | "llm" | "agent"} marks a crashed run. Limits: 1 MB per request, 500 steps per run. Secrets and personal data in task, args and output are redacted before storage.

Response 202: {"id": "...", "duplicate": false}. Processing is asynchronous.

POST /v1/recall

{"agent": "support-agent", "task": "Refund order #123 that arrived damaged",
 "model_id": "claude-haiku-5-5", "tool_versions": {"orders_api": "v3"},
 "max_playbooks": 3, "run_id": "uuid of the run that follows (optional)"}

Response:

{
  "recall_id": "…",
  "playbooks": [{"id": "…", "version": 4, "score": 0.83, "title": "…", "preconditions": "…",
                 "steps": [{"action": "…", "tool": "…"}], "pitfalls": "…"}],
  "warnings": [{"id": "…", "tool": "orders_api", "error_class": "TimeoutError",
                "occurrences": 7, "message": "…", "recovery": null, "unrecoverable": false}],
  "holdout": false,
  "mode": "hybrid"
}

mode is hybrid (meaning and keywords), keyword, holdout or no_agent. Only active playbooks are returned. An unknown agent gets empty memory.

POST /v1/feedback

{"run_id": "…", "outcome": "success", "source": "test", "detail": {"suite": "smoke"}}

source is test, event or feedback. Response 202 with the run's resolved outcome: {"run_id", "outcome", "outcome_source", "confidence"}. 404 run_not_found if the run is not in the project.

Playbooks

  • GET /v1/playbooks?agent=NAME&status=active|candidate|quarantined|retired|all&limit=50: newest change first; status defaults to active.
  • GET /v1/playbooks/{id}/versions: every version with its snapshot and the reason it changed.
  • PATCH /v1/playbooks/{id} with {"action": "quarantine" | "retire", "note": "…"}.

An API key can take memory out of service but never put it in: approving and rolling back return 403 and are done by a person in the dashboard.

Regression and migration

  • GET /v1/eval-cases?agent=NAME: active cases with their input, assertions, critical flag and last result.
  • POST /v1/eval-results: {"test_run_id", "model_id"?, "migration_id"?, "results": [{"case_id", "passed", "details"?}]} (1 to 1,000 results).
  • POST /v1/migrations: {"agent", "to_model", "from_model"?, "model_applied"?} returns 201 {"id", "status": "running"}.
  • GET /v1/migrations/{id}: status, counts (passes, needs_relearning, breaks, fixed, returns) and per-case results.

GET /v1/metrics?days=30

days from 7 to 90. Returns overall runs and success_rate; memory and holdout blocks (runs, known outcomes, successes, success rate, average tokens and cost); lift_points; relative token_change and cost_change (−0.2 means 20% less with memory); enough_evidence (at least 20 held-out runs with a known outcome); and recalls (total, holdout, hit rate, median and 95th-percentile latency). See Measuring impact.