---
title: "AI agent memory retrieval patterns · hybrid search, RRF, HyDE, fact-key lookup · Agent Fieldbook"
url: https://agentfieldbook.org/memory/retrieval/
description: "How AI agents read memory back into context. Hybrid parallel retrieval, Reciprocal Rank Fusion, HyDE, full-text and fact-key lookup, direct vector search, lazy just-in-time fetch, temporal and hierarchical paging. A source-graded field book to the read pipeline, where the memory's schema decides the channel."
section: "Memory · Retrieval"
source: Agent Fieldbook — generated from the published page
---

# How memory is read back into context

**Read · Both**

The read pipeline. Analyse the query, fan across channels (full-text, fact-key, direct-vector, HyDE), fuse with Reciprocal Rank Fusion, and inject the winners. **The right channel depends on the memory's schema**: a keyed Fact wants fact-key lookup; a fuzzy recollection wants HyDE. Each pattern is sourced to a doc or paper.

## Trade-off

- [↗](https://blog.cloudflare.com/introducing-agent-memory/)

- [↗](https://arxiv.org/abs/2212.10496)

- [↗](https://arxiv.org/abs/2005.11401)

- [↗](https://arxiv.org/abs/2310.08560)

- [↗](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents)

- [↗](https://arxiv.org/abs/2501.13956)

- [↗](https://aider.chat/docs/repomap.html)

## Field notes

Analyse, fan across channels, fuse, inject: the read pipeline

Harvesting memory into schemas and vectors is only half the job. Retrieval is where the latency sits and where accuracy drifts. The twenty-one patterns here attack different parts of it, from keyword search and graph traversal to rank fusion, hypothetical embeddings and plain fact-key lookup, drawn in roughly equal measure from Cloudflare's agent-memory work and published research. Every row records its own trade-off, because none of them is free.

The harder question is not which method but when: when to retrieve, when to inject, and how far the agent should trust what comes back. That is a balance between context, speed and accuracy, and it behaves like the old project triangle. How much you are willing to search through, how long you are willing to wait, and how right the answer has to be. Two of the three can be held firmly. The third gives.

- [Brandon Chaplin](https://www.linkedin.com/in/brandon-chaplin-digital-marketing-strategist)

## Common questions

### How does an AI agent retrieve memories?

It turns the request into one or more queries, searches whichever stores it has, merges the results into a single ranked list, and puts the top few into the prompt. The model only ever sees what survives that ranking, which is why retrieval quality matters more than storage.

### What is hybrid search?

Running keyword search and vector search at the same time and combining the results. Keyword search wins on exact names, codes and rare words; vector search wins on paraphrases and synonyms. Run together they cover each other's blind spots.

### What is Reciprocal Rank Fusion?

A simple way of merging ranked lists from different searches. Each result scores one divided by its position in each list, and the scores are added. It works without making a keyword score and a similarity score comparable, which is why it is the default merge step in hybrid search.

### What is HyDE in retrieval?

The model writes a hypothetical answer to the question first, and you search using that instead of the question itself. The fake answer looks more like the documents you are searching than the question does, so it matches better on short or vague queries [(arXiv)](https://arxiv.org/abs/2212.10496).

### Why is my RAG retrieving the wrong chunks?

Usually chunking or query mismatch. Chunks split mid-idea lose the context that made them meaningful, and a short question rarely resembles the text that answers it. The common fixes are overlapping chunks, adding surrounding context to each chunk, hybrid search, and a re-ranker over the top results.

### What is a re-ranker?

A second, slower model that re-scores the top results from the first search. The first pass is fast and approximate across the whole store; the re-ranker reads the query and each candidate together and orders them properly. Retrieving 50 and re-ranking down to 5 usually beats retrieving 5 directly.

## Next in the learning path

- [Memory harvest The write pipeline that fills what you read](https://agentfieldbook.org/memory/harvest/)

- [Memory schemas The record shape that picks the read channel](https://agentfieldbook.org/memory/schemas/)

- [Memory providers The substrate these reads run against](https://agentfieldbook.org/memory/providers/)

- [Memory types The cognitive layers being recalled](https://agentfieldbook.org/memory/types/)

## Entries

_12 entries listed on this page._

| name | description | subjectOf |
| --- | --- | --- |
| Hybrid Parallel Retrieval | Fan one query across several channels at once, full-text, fact-key, direct-vector, HyDE-vector, raw-message, then fuse the ranked lists. | https://blog.cloudflare.com/introducing-agent-memory/ |
| Reciprocal Rank Fusion (RRF) | Merge channel results by a weighted score based on each result's rank; fact-key matches weighted highest, raw messages lowest; ties broken by recency. | https://blog.cloudflare.com/introducing-agent-memory/ |
| HyDE (Hypothetical Document Embedding) | Generate a hypothetical answer to the query, embed that, and search by similarity to the answer: surfaces matches that direct-query embedding misses. | https://arxiv.org/abs/2212.10496 |
| Full-Text Search (BM25 / stemming) | Keyword channel with Porter stemming for exact-term precision when you know the word but not the surrounding context. | https://blog.cloudflare.com/introducing-agent-memory/ |
| Fact-Key Lookup | Direct lookup where the query maps to a known normalized topic key: an exact topic match, the strongest retrieval signal. | https://blog.cloudflare.com/introducing-agent-memory/ |
| RAG / Direct Vector Search | Embed the query, run vector similarity search over the memory store, inject the top-k matches. | https://arxiv.org/abs/2005.11401 |
| Agent-Queried / Tool-Call Retrieval | The agent decides when to call a memory function to page data in/out of context (self-paging). | https://arxiv.org/abs/2310.08560 |
| Context Injection / Prompt Stuffing | Load persisted state verbatim into the system prompt every turn: no retrieval step. | https://arxiv.org/abs/2310.08560 |
| Lazy / Just-in-Time Retrieval | Defer fetching; load memory only when the current task signals it is relevant. | https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents |
| Temporal / Bi-Temporal Query | Store valid_at / invalid_at on facts; query the graph as-of a point in time (time-travel). | https://arxiv.org/abs/2501.13956 |
| Hierarchical / Paging (core vs archival) | Tiered memory, in-context core vs external archival, with data moved between tiers on demand. | https://arxiv.org/abs/2310.08560 |
| Aider Repo-Map (PageRank over file graph) | PageRank over the file-dependency graph, ranked and truncated to fit the token budget. No vector store, no embeddings. | https://aider.chat/docs/repomap.html |