How an agent gets better over time
Memory is what an agent keeps; coaching is how it improves. Two layers, like the rest of Memory: the techniques (the loops) and the suppliers (the vendors who productise them). The AI parallel to how a contact centre coaches its human agents: observe the work, feed back, raise the bar.
Reflexion
After a failed attempt the agent writes a verbal self-critique and stores it in episodic memory; the next attempt reads that reflection as context. Learns from mistakes without touching model weights.
DSPy (MIPROv2)
You declare the agent pipeline as code and write an eval metric; DSPy's optimiser automatically mutates prompts + few-shot examples and keeps the versions that score best. Prompt engineering becomes a compiled, measured step.
TextGrad
Treats natural-language feedback as a 'textual gradient': an evaluator LLM critiques an output, and that critique back-propagates through the pipeline to revise prompts and intermediate steps. Autograd, but in words.
Constitutional AI
The model critiques and revises its own outputs against a written set of principles (a 'constitution') rather than per-example human labels. A coaching overlay that enforces values at generation time.
LangSmith / eval feedback
Traces every agent run, scores it against evals (LLM-as-judge or heuristics), and feeds failing traces back into datasets + prompt fixes. The inspectable, operations-team version of a coaching loop.
Coach Overhead (meta-supervisor)
A read-only meta-agent sits above the main agent, observes its actions and offers advisory feedback; the main agent decides whether to act. The runtime, in-the-loop cousin of offline coaching. Mirrors human contact-centre agent coaching, where a QA layer reviews and guides live work.
| Vendor | Category | How it works | Closes the loop | OSS | src |
|---|---|---|---|---|---|
| LangSmith LangChain | Observability + eval | Traces every agent run, runs offline + online evals, and turns failing traces into datasets. The default coaching surface for LangChain / LangGraph stacks. | Trace → eval → dataset → prompt fix | no (SaaS + self-host) | ↗ T1 |
| Langfuse Langfuse (ClickHouse) | Observability + eval | Open-source, self-hostable tracing + eval + prompt management. Free with no usage limits when self-hosted; acquired by ClickHouse Jan 2026, code still actively maintained. | Trace → score → prompt version | yes (self-host free) | ↗ T1 |
| Braintrust Braintrust | Eval-driven dev | A quality-management system for AI products: CI/CD eval-gated deployment, the strongest gate-before-ship workflow. Most generous free tier (1M spans/month). | Eval-gate blocks a bad deploy | no (SaaS) | ↗ T1 |
| Arize AI (+ Phoenix) Arize | Observability + eval | Enterprise observability with multi-step agent trajectory analysis. Phoenix (OSS) ships 50+ research-backed metrics: faithfulness, relevance, safety, hallucination. Raised $70M Series C (Feb 2025). | Trajectory analysis → metric → alert | partial (Phoenix OSS) | ↗ T1 |
| Humanloop Humanloop | Prompt + eval | Prompt-iteration + evaluation platform aimed at PMs and cross-functional teams shipping AI features. Human-feedback loop first-class. | Human feedback → prompt iteration | no (SaaS) | ↗ T2 |
| Latitude Latitude | Closed-loop (issue→PR) | The only platform that closes issue → opened PR: an MCP server connects your coding agent (Claude Code, Cursor) so detected failures are driven to a fix. Evals auto-generated from real production failures. | Production failure → eval → opened PR | yes (open-source) | ↗ T2 |
| Helicone Helicone | Observability + eval | Open-source LLM observability, one-line proxy integration, logging, cost + latency tracking, sessions and eval scoring. | Proxy log → session → score | yes (open-source) | ↗ T2 |
| Maxim AI Maxim | Observability + eval | End-to-end agent simulation, evaluation and observability: pre-release agent simulation plus production monitoring in one platform. | Simulate → evaluate → monitor | no (SaaS) | ↗ T2 |
| Galileo Galileo | Observability + eval | AI evaluation + observability with proprietary guardrail metrics (e.g. context adherence, chunk attribution) for RAG + agent quality. | Guardrail metric → flag → fix | no (SaaS) | ↗ T2 |
| Traceloop (OpenLLMetry) Traceloop | Observability (OSS standard) | Fully open-source OpenTelemetry-based LLM observability: vendor-neutral tracing you can point at any backend. The OSS instrumentation standard. | OTEL trace → any backend | yes (open-source) | ↗ T2 |
Reflection, optimiser, critique loops, and the vendors who productise them
Coaching is the feedback loop: how correction is captured and fed back so an agent behaves differently next time. It sits between memory and evaluation, and it is the newest layer in the stack.
Show more
The mechanism is what distinguishes approaches. Some write corrections into procedural memory so the agent rewrites its own instructions; others hold them outside the agent as rules a supervisor enforces. The first improves faster and drifts; the second is slower and auditable.
How do AI agents get better over time?
Mostly by changing what goes into the prompt, not the model. The agent records what went wrong, and that lesson, a better example, or a rewritten instruction goes into the next run. Retraining the model is the heavier and much rarer option.
What is a reflection loop?
The agent reviews its own output, writes down what went wrong, and reads that note back on the next attempt. Reflexion is the best-known version: it lifted GPT-4 on the HumanEval coding benchmark from 80% to 91% with no change to the model itself (arXiv).
What is DSPy?
A framework that treats prompts and examples as things to optimise rather than write by hand. You define the task and a metric, and it searches for the instructions and examples that score best (DSPy). TextGrad does something similar using written feedback as the improvement signal.
Is coaching the same as fine-tuning?
No. Fine-tuning changes the model's weights and needs a training run. Coaching changes the context the model sees: its instructions, its examples, its stored lessons. Coaching is faster, cheaper, reversible, and you can read exactly what changed.
What tools do you use to evaluate and improve agents?
Tracing and evaluation platforms. LangSmith, Langfuse, Arize Phoenix, Braintrust and Helicone are the common ones. They record every run, score it against test cases, and let you turn failing runs into a dataset that drives the next prompt change.
Can an agent safely rewrite its own prompt?
It can, and it will drift if nothing checks it. The standard guardrail is a fixed set of evaluation cases: a proposed change only ships if it scores better on cases you already trust. Most teams also keep human approval before anything reaches production.