Corpus Agentis
The field book to agent ecosystems
The field book to agent ecosystems
Memory · Retrieval

How memory is read back into context

The read pipeline. Analyse the query, fan across channels (full-text, fact-key, direct-vector, HyDE), fuse with Reciprocal Rank Fusion, and inject the winners. The right channel depends on the memory's schema: a keyed Fact wants fact-key lookup; a fuzzy recollection wants HyDE. Each pattern is sourced to a doc or paper.

Hybrid Parallel Retrieval

Read
Fan one query across several channels at once, full-text, fact-key, direct-vector, HyDE-vector, raw-message, then fuse the ranked lists.
Trade-off
More channels = more cost/latency and a fusion step to tune.
Cloudflare Agent Memory (5 channels)

Reciprocal Rank Fusion (RRF)

Read
Merge channel results by a weighted score based on each result's rank; fact-key matches weighted highest, raw messages lowest; ties broken by recency.
Trade-off
Per-channel weights are a tuning surface; bad weights drown the strongest signal.
Cloudflare Agent Memory

HyDE (Hypothetical Document Embedding)

Read
Generate a hypothetical answer to the query, embed that, and search by similarity to the answer: surfaces matches that direct-query embedding misses.
Trade-off
Adds an LLM generation step; a wrong hypothetical can pull off-topic results.
Cloudflare Agent Memory · RAG pipelines
the · agent · memory · index · scan

Full-Text Search (BM25 / stemming)

Read
Keyword channel with Porter stemming for exact-term precision when you know the word but not the surrounding context.
Trade-off
No semantic recall, misses paraphrases and synonyms unless expanded.
Cloudflare Agent Memory · Postgres FTS · Elastic
key

Fact-Key Lookup

Read
Direct lookup where the query maps to a known normalized topic key: an exact topic match, the strongest retrieval signal.
Trade-off
Only fires when a clean key exists; useless for fuzzy or novel phrasing.
Cloudflare Agent Memory

RAG / Direct Vector Search

Read
Embed the query, run vector similarity search over the memory store, inject the top-k matches.
Trade-off
Recall depends on embedding quality; irrelevant top-k or missed facts poison context.
Mem0 · Zep · LangChain retrievers · most vector-DB memory
tool

Agent-Queried / Tool-Call Retrieval

Read
The agent decides when to call a memory function to page data in/out of context (self-paging).
Trade-off
Extra LLM turns and latency; the agent may fail to query when it should.
Letta · MemGPT (function-calling memory)
prompt

Context Injection / Prompt Stuffing

Read
Load persisted state verbatim into the system prompt every turn: no retrieval step.
Trade-off
Bounded by the context window; cost scales with state size and grows every turn.
Letta (core memory blocks) · Claude / OpenAI system-prompt memory
need?

Lazy / Just-in-Time Retrieval

Read
Defer fetching; load memory only when the current task signals it is relevant.
Trade-off
Risk of missing context if the relevance trigger misfires; added round-trips.
Letta (archival paging) · Claude agent context engineering
as-of

Temporal / Bi-Temporal Query

Read
Store valid_at / invalid_at on facts; query the graph as-of a point in time (time-travel).
Trade-off
Graph and temporal bookkeeping overhead; complex invalidation logic on write.
Zep · Graphiti (bi-temporal knowledge graph)
core (in-context)archival (fetch on demand)

Hierarchical / Paging (core vs archival)

Both
Tiered memory, in-context core vs external archival, with data moved between tiers on demand.
Trade-off
Paging and tier-eviction policy add complexity and can evict still-needed data.
MemGPT · Letta (core / archival memory tiers)

Aider Repo-Map (PageRank over file graph)

Read
PageRank over the file-dependency graph, ranked and truncated to fit the token budget. No vector store, no embeddings.
Trade-off
Fast, deterministic and inspectable; needs a graph-able project structure (imports, references). Weaker for prose corpora.
Aider
Field notes

Analyse, fan across channels, fuse, inject: the read pipeline

Harvesting memory into schemas and vectors is only half the job. Retrieval is where the latency sits and where accuracy drifts. The twenty-one patterns here attack different parts of it, from keyword search and graph traversal to rank fusion, hypothetical embeddings and plain fact-key lookup, drawn in roughly equal measure from Cloudflare's agent-memory work and published research. Every row records its own trade-off, because none of them is free.

Show more

The harder question is not which method but when: when to retrieve, when to inject, and how far the agent should trust what comes back. That is a balance between context, speed and accuracy, and it behaves like the old project triangle. How much you are willing to search through, how long you are willing to wait, and how right the answer has to be. Two of the three can be held firmly. The third gives.

From the corpus, curated by Brandon Chaplin
Common questions
How does an AI agent retrieve memories?

It turns the request into one or more queries, searches whichever stores it has, merges the results into a single ranked list, and puts the top few into the prompt. The model only ever sees what survives that ranking, which is why retrieval quality matters more than storage.

What is hybrid search?

Running keyword search and vector search at the same time and combining the results. Keyword search wins on exact names, codes and rare words; vector search wins on paraphrases and synonyms. Run together they cover each other's blind spots.

What is Reciprocal Rank Fusion?

A simple way of merging ranked lists from different searches. Each result scores one divided by its position in each list, and the scores are added. It works without making a keyword score and a similarity score comparable, which is why it is the default merge step in hybrid search.

What is HyDE in retrieval?

The model writes a hypothetical answer to the question first, and you search using that instead of the question itself. The fake answer looks more like the documents you are searching than the question does, so it matches better on short or vague queries (arXiv).

Why is my RAG retrieving the wrong chunks?

Usually chunking or query mismatch. Chunks split mid-idea lose the context that made them meaningful, and a short question rarely resembles the text that answers it. The common fixes are overlapping chunks, adding surrounding context to each chunk, hybrid search, and a re-ranker over the top results.

What is a re-ranker?

A second, slower model that re-scores the top results from the first search. The first pass is fast and approximate across the whole store; the re-ranker reads the query and each candidate together and orders them properly. Retrieving 50 and re-ranking down to 5 usually beats retrieving 5 directly.

Next in the learning path