How Retrack works
The pipeline behind experience memory: capture every agent run, verify it, distil playbooks and failure signatures, recall them, and protect memory.
Updated
Retrack turns your agent's own history into memory it can use. Every run goes through the same pipeline, in the background, without slowing your agent down.
The pipeline
1. Capture
The connector records each agent run when it finishes: the task, every tool call with its arguments and result, the final answer, tokens, cost, latency, the model and your tool versions. Runs are sent in the background with a bounded buffer, so a slow network never blocks your agent.
Secrets are stripped twice. The connector removes obvious credentials (API keys, tokens, passwords) on your machine before sending, and the server redacts secrets and personal data (emails, phone numbers, card numbers and national ID numbers) before anything is stored or embedded.
2. Verify
A run only teaches if Retrack knows whether it worked. Retrack uses the strongest signal available, in this order:
| Source | Example | Confidence |
|---|---|---|
test | A test or assertion in your code passed | 1.0 |
event | A business event: the ticket was resolved, the refund settled | 0.9 |
feedback | A user's thumbs-up or rating | 0.8 |
| AI judge | No signal arrived, so an AI judge reads the run | at most 0.7 |
The judge checks the final answer against the steps, so an agent that claims success after a failed key step is scored as a failure. A crash in a tool or in your agent's code counts as a failure; an outage of the model provider does not.
See Verifying runs.
3. Distil
Verified successes of the same kind of task are merged into one playbook: preconditions, steps (some only under a condition) and pitfalls. Failures become failure signatures: the tool, the error and, when a later retry worked, the fix.
Playbooks keep learning. When a verified run teaches something new (a tool the task needed, an error it recovered from), the playbook gets a new version, and every version is kept so you can roll back.
4. Recall
Before a task, your agent calls retrack.recall(task). Retrack searches by meaning and by keywords, scores each candidate playbook, and returns one only when it clearly applies, together with warnings about failures relevant to the task. See Recall and the confidence gate.
5. Protect
Throughout, the memory firewall decides what may become memory. Runs that touched untrusted input (user text, emails, web pages, third-party tools) cannot create or change active memory without passing checks or your review.
The life of a playbook
| Status | Meaning | Recalled? |
|---|---|---|
candidate | Learned, waiting for evidence or your approval | No |
active | Approved by you, or promoted after 3 trusted, verified successes | Yes |
quarantined | Stopped by the firewall or by you; waits for review | No |
retired | Taken out of service by you, or retired after its success rate fell below 50% over 10 or more uses | No |
You see and manage all of them in the dashboard under Memory and Review.
The same memory, four jobs
- Learning: playbooks and warnings before each task.
- Testing: every active playbook and recurring failure becomes a regression case for
retrack test. - Migrating:
retrack migratereplays memory on a new model and reports what still works. - Proving: a holdout group gets no memory, so the effect is measured.