---
title: "Speculative Decoding · agent orchestration · Agent Fieldbook"
url: https://agentfieldbook.org/design/orchestration/speculative-decoding/
description: "A small draft model proposes several tokens ahead; the large target model verifies them all in one parallel pass. Correct guesses are kept, so many tokens land per expensive forward pass."
section: "Design · Orchestration"
source: Agent Fieldbook — generated from the published page
---

# Speculative Decoding

A small draft model proposes several tokens ahead; the large target model verifies them all in one parallel pass. Correct guesses are kept, so many tokens land per expensive forward pass.

Verified against [arxiv.org](https://arxiv.org/abs/2211.17192)

- [arxiv.org](https://arxiv.org/abs/2211.17192)

## How it works

## At a glance

| Field | Value |
| --- | --- |
| Technique | Decoding |
| Cost lever | Neutral on cost, same target model |
| Latency lever | 2–3× faster inference with no accuracy loss |
| Quality lever | Identical to the target model (verification is exact) |
| Example | vLLM· TensorRT-LLM· Medusa |

## When to use

Latency-sensitive serving of a fixed large model, chat, coding autocomplete, where you want speed without changing outputs.

## Basis

## Verified against

- [https://arxiv.org/abs/2211.17192](https://arxiv.org/abs/2211.17192)

The corpus does not rewrite vendor documentation. Where copy is quoted verbatim it is marked as such and attributed to its source.