Documentation menu

Failure signatures and warnings

How Retrack fingerprints repeated agent failures, learns the fix that worked, and warns your agent before it makes the same mistake again.

Updated

Agents fail in repeatable ways: a tool that wants an uppercase reason code, an API that refuses refunds above a limit without approval, a parcel that has already shipped. Retrack records each kind of failure as a failure signature and warns your agent before it makes the same mistake again.

What a failure signature is

A failure signature identifies one kind of failure by the tool, the error class and the shape of the input that triggered it (a fingerprint). It keeps:

  • how often it happened and when it was last seen,
  • the tool's own explanation of the error, made task-independent and checked by the firewall,
  • the fix that worked, when a later retry in the same run succeeded,
  • whether it is unrecoverable: seen 5 or more times with no fix ever recorded.

For general-purpose tools (a shell, SQL or HTTP tool), Retrack names the action inside the call, such as kubectl rollout undo or POST /v1/orders/{}/refund, so each command has its own signatures.

Warnings in recall

When your agent recalls memory, it gets up to 5 warnings relevant to the task: failures of tools that the recalled playbook uses, failures from the same kind of task, or failures of tools you name in tool_versions. In the prompt they read like this:

Known failures:
- issue_refund failed 3 times with WrongRefundMethod: "orders paid with a gift card are
  refunded as store_credit". Fix that worked: retry issue_refund with a different method
  (method='store_credit' worked).

A failure counts once it has happened twice, or as soon as a fix is known.

How fixes are learned

In a successful run, a tool call that failed followed by a successful call to the same tool is a recovery. Retrack records the calls in between and what changed in the retry: which inputs were added or changed, with the new value when it is a short code (never IDs, amounts or free text). No AI is involved; this works even with embeddings off.

Stopping blind retries

Warnings in the prompt help the model. To enforce them at the tool level, wrap your tools:

memory = retrack.recall(task)
tools = memory.protect(tools)  # LangChain tools

See Known fixes and the circuit breaker.

Failures as regression tests

A failure signature that reaches 2 occurrences becomes a critical regression case asserting that the failure no longer happens. If it comes back, retrack test fails.