Playbooks: procedural memory for AI agents
Retrack turns verified agent runs into playbooks, procedures with conditional steps and pitfalls, recalled only when they clearly apply.
Updated
Your agent handles the same kinds of tasks every day: refunds, address changes, cancellations, deploy rollbacks. Without memory it starts each one from scratch, re-plans the same steps and rediscovers the same hidden rules by failing first. Retrack turns the runs that worked into playbooks and hands the right one to your agent before the next similar task.
What changes for your agent
- It stops rediscovering rules. Which reason codes a tool accepts, which refunds need approval, which field the PIN code goes in: once a verified run learns it, every later run starts with it.
- It takes the known-good path. Fewer exploratory calls, fewer retries, fewer wrong turns.
- It behaves consistently. The same kind of task is handled the same way, which makes the agent easier to trust and to debug.
On a real Claude Haiku 5.5 support agent, the requests that received a playbook made no avoidable mistakes, against 15 across the same 16 requests without memory. See the benchmarks.
Procedural memory, learned from evidence
A Retrack playbook is procedural memory: the steps for a kind of task, the tools each step uses, the conditions under which a step applies, and the pitfalls to avoid. It is learned only from verified runs, those a test, a business event, user feedback or an AI judge confirmed worked, and it keeps improving:
- A run that needed an extra step for an edge case adds it as a conditional step ("If the order already shipped: reroute the shipment").
- An error the agent recovered from becomes a pitfall that names the tool and the error.
- Every version is kept, with the runs it came from, and you can roll back.
Only when it clearly applies
Wrong memory is worse than none, so a playbook is recalled only when a confidence gate is satisfied: similarity to the task, the strength of the evidence, its real success rate, how recently it worked and whether it was validated on your current model and tool versions. Different wordings of the same task still find it; a different kind of task does not.
Safe by default
New playbooks start as candidates. They reach your agent when you approve them, or after three trusted runs with a hard success signal. Runs that touched untrusted input can never change active memory directly. See the memory firewall.
Add it in three lines
import retrack
retrack.init(agent="support-agent")
memory = retrack.recall(task) # before each task; fails openFollow the quickstart, or read how playbooks are learned.