---
title: "Methodology · how the corpus is sourced and verified · Agent Fieldbook"
url: https://agentfieldbook.org/methodology/
description: "How Agent Fieldbook verifies its data: source tiers (T1/T2/T3), live vs curated tables, freshness windows, corroboration rules and referential-integrity checks. Every datapoint traces to an established primary source, is confidence-graded, and gaps are left visible rather than guessed."
section: "About · Data Sources"
source: Agent Fieldbook — generated from the published page
---

# Data Sources & Methodology

**confirmed · likely · emerging · single-source · research · projected**

Agent Fieldbook is built to be **reference-grade**: every datapoint traces to an established primary source so it can be cited. We don't own the sources; the proprietary lift is how we **classify, tier, blend and verify** them. Here is exactly how the data is sourced, graded and kept current.

## Source tiers, the credibility classification

Official vendor APIs, docs & pricing pages· SEC filings· standards bodies (ISO, NIST, EU OJ)· peer-reviewed & university research (METR, NBER, arXiv)· authoritative datasets (Epoch AI, LiteLLM)· official registries (MCP).

Well-maintained open-source GitHub· established research/industry orgs publishing data (e.g. McKinsey, PwC, labelled secondary)· reputable aggregators.

Marketing pages without primary data· undated or low-quality blogs· AI-content farms· single unverified claims. Used only as a pointer, never the basis of a "confirmed" datapoint.

**Convergence, not divergence.** Where a fact should agree across sources (a price, a ranking), we corroborate; divergence is a signal to re-source, not to average. The build gate flags any "confirmed" row resting only on a weak/T3 source.

## Methodology · confidence on every row

## Data vs concept, the honesty flag

**Data** = a sourced fact. **Concept** = our classification or framework with no single raw source (the agency spectrum, archetypes, ontologies, the memory taxonomy, parts of the Workforce models). Concept tables are flagged in the manifest and never presented as measured data. Where we genuinely don't know, the cell reads –: never a guess.

## How it stays current

Fetchers hit public APIs (models, chips, MCP registry, GitHub stars): re-run weekly, near-zero effort.

Authored sourced facts (pricing, adoption, security): reviewed on a window; the build prints what's due.

Slow-moving taxonomies/frameworks: reviewed rarely.

A single validator ( scripts/build.mjs ) gates every change: it checks referential integrity, enforces the controlled vocabulary, derives each source's tier, and flags weak-confirmed rows + a review worklist. One amend → re-run → every gap surfaces.

## Attribution, key open sources

| Source | License | Feeds |
| --- | --- | --- |
| **Epoch AI** | CC-BY: attribution required | chips / hardware |
| **LiteLLM** | MIT | model pricing |
| **Aider** | Apache-2.0 | coding leaderboard |
| **OpenRouter** | open API | model catalogue |
| **Official MCP Registry** | open | MCP servers |
| **GitHub API** | per-repo | framework/tool stars |
| **METR** | research | task-horizon |
| **NBER** | research | productivity benchmarks |
| **Cloudflare Workers AI** | open docs | edge inference |

Every datapoint carries its own source link in-place (the ↗ on each table row and in each agent profile). Corrections welcome: Agent Fieldbook is an open intelligence project.

## Cite this

## Common questions

### How are sources graded?

On three tiers. Tier one is primary and authoritative: vendor documentation and pricing, regulatory filings, standards bodies, peer-reviewed research, official registries. Open source counts as tier one when it is published by an established organisation, a lab, university, vendor or standards project, because that is the organisation publishing its own work. Tier two is reputable secondary work: analysts and industry researchers who study the primary record rather than being it. Tier three is weak. Marketing roundups and 'best tools' listicles land here and are never the basis of a confirmed row.

### What do the confidence grades mean?

Every row carries one. Confirmed means checked against a tier-one or tier-two source. Likely is an informed estimate, emerging is a best read from early signals, and single-source is one credible source with nothing corroborating it yet. Projected means a forward estimate, and it is always labelled as one.

### What happens when two sources disagree?

We go back and re-source rather than split the difference. Where two sources should agree, such as on a price, disagreement usually means one is stale or wrong. Averaging would hide that.

### How often is the data updated?

It depends on the table. Live tables such as models, chips and the MCP registry are refetched weekly from public APIs. Curated tables like pricing and adoption are reviewed on a fixed window, and the build prints what is due. Taxonomies move slowly and are reviewed rarely.

### What is the difference between data and concept here?

Data is a sourced fact. Concept is our own classification or framework, such as the agency spectrum or the memory taxonomy, which has no single source to cite. Concept tables are flagged as such, so they are never read as measurements.

### How is data quality enforced?

By the build script, not by reviewer discipline. It checks that every reference resolves, enforces the controlled vocabulary, works out each source's tier from an allow-list of domains, and flags any confirmed row resting only on a weak or unrecognised source so it can be re-sourced or downgraded. It also prints what is due for review.

## Next in the learning path

- [Models A live-fetched table in practice](https://agentfieldbook.org/software/models/)

- [Pricing A curated, sourced table](https://agentfieldbook.org/economics/pricing/)

- [Standards The compliance sources tracked](https://agentfieldbook.org/security/standards/)

- [All agents Every entity, each with its sources](https://agentfieldbook.org/all_agents/)

- [Benchmarks The independent evaluations cited](https://agentfieldbook.org/software/benchmarks/)