Documentation menu

Retrack documentation

How to give a LangGraph or LangChain agent experience memory with Retrack: install, capture runs, recall playbooks, send feedback and test in CI.

Updated

Retrack gives production AI agents experience memory. Your agent runs the same kinds of tasks every day and starts each one from scratch. Retrack watches every run, checks whether it worked, turns what worked into playbooks and what failed into warnings, and hands the right ones to your agent before its next similar task.

It works with LangGraph and LangChain agents in Python through one open-source connector, retrack-ai (MIT licence). Retrack itself is a hosted service.

How it works

  1. Capture. The connector records each agent run: the task, every tool call and its result, the final answer, tokens and the model. Secrets are stripped on your machine before anything is sent.
  2. Verify. Retrack decides whether the run succeeded, from the strongest signal you give it (a passing test, a resolved ticket, a user's thumbs-up) or, without one, from an AI judge.
  3. Distil. Verified successes become playbooks: the steps that worked for that kind of task. Failures become failure signatures, with the fix that worked when there is one.
  4. Recall. Before the next similar task, your agent asks for memory. It gets a playbook and warnings only when they clearly match.
  5. Protect. A memory firewall keeps runs fed by untrusted input from becoming memory without a check or your review.

The same memory doubles as a regression suite (retrack test) and a model-migration check (retrack migrate), and a small holdout group gets no memory so you can see the difference memory makes, measured rather than estimated.

Start here

Requirements

  • Python 3.10 or newer.
  • A LangGraph or LangChain agent that calls tools.
  • A Retrack account: sign in at retrack-ai.online with GitHub, create a project and copy its API key.

Retrack is in early access. Questions and feedback are welcome at retrackit@gmail.com.