Methodology
Reproduced verbatim from METHODOLOGY.md in the TokenDrift repository. It is shown as written rather than reformatted, so that nothing on this page can drift from the document it quotes. Plain source.
# TokenDrift Pricing Methodology
**Version 1.2 — 2026-08-16**
This document defines how TokenDrift produces the LLM API prices it publishes:
what counts as a source, what our confidence labels mean, how disagreements are
resolved, what we will not publish, and what happens when we get a number wrong.
It describes the system as it actually behaves today, not as we intend it to
behave. Where a rule exists but cannot currently fire, or where a guarantee has
a known hole, that is stated in the same paragraph as the rule. A methodology
document that conceals its own limits is worse than none: the entire purpose of
publishing one is to let a sceptical reader decide how much weight a number can
carry, and they cannot do that from a description of the happy path.
Anyone citing a TokenDrift number in a contract, a model, or an analysis should
read sections 2, 3, 6 and 9 in particular.
**Where this document lives.** The canonical location is
`https://tokendrift.ai/methodology/`. The bare path `/methodology` answers a
**301 permanent** redirect to it — permanent because the path is stable in
perpetuity by design, and a temporary redirect would tell every cache and every
citing party that it is not. The disclaimer and the terms answer the same way at
`/disclaimer/` and `/terms/`. The markdown source of each is served verbatim
beside its rendered page, at `/methodology/METHODOLOGY.md` and so on, so a citing
party can diff the text they relied on against the text published later. Section
numbers are treated as citable identifiers: sections are amended in place rather
than renumbered, and a section that becomes obsolete is emptied and marked rather
than removed.
---
## 0. How a number is produced
One **run** is a single end-to-end pass. Runs are the unit of everything below:
of observation, of publication, and of the change feed.
```
config/sources.yaml
│
▼
collector ──► deterministic parser ──┐
(robots-aware, (per-source, in code) ├─► consensus ──► reconciler ──► SQLite ──► dump
rate limited, language model ───────┘ (confirmed?) (trust rules, (append- (JSON)
caches bytes) (temperature 0, 2 passes) in-force, only)
commensurable)
```
- **Collector** (`tokendrift/collectors/fetch.py`) fetches each registered URL,
honouring robots.txt and a per-domain delay, and writes the raw bytes to a
cache. A source that fails is logged and the run continues.
- **Readers** turn one page into structured price observations. Each source has
an ordered list of readers in `config/sources.yaml`. A reader is either a
**deterministic parser** written in Python for that specific page, or the
**language model** pass, or both.
- **Consensus** (`tokendrift/extract/consensus.py`) combines multiple readings
of the *same page* and decides which values are `confirmed`.
- **Reconciler** (`tokendrift/reconcile/merge.py`) combines readings of
*different sources* about the same model, applies the trust order, and
produces the published value plus a written reason.
- **Store** (`tokendrift/store/`) is SQLite. Observations are insert-only.
- **`dump`** emits the reconciled dataset as JSON.
No language model touches merge logic, trust ordering, unit conversion, or the
change feed. Those are plain, tested Python, so every published number traces to
the source that produced it and the rule that selected it.
The language model used for extraction is `deepseek-v4-flash` at temperature 0.
It is used only to read prose and tables into a strict JSON schema; it is
forbidden by prompt from inferring, estimating, or converting units, and — see
§3 — it cannot confirm a value that the page's deterministic parser did not
emit. That constraint also answers the obvious question about reading DeepSeek's
own rate card with a DeepSeek model: on that page the deterministic parser is
the authority, and the model can corroborate or contradict it but never
substitute for it.
---
## 1. Sources and trust order
### What counts as a source
A source is a specific URL that states prices in **served HTML or JSON** — that
is, in the bytes the server returns, not in content assembled by client-side
JavaScript. Every URL in the registry was probed for price-bearing served
content before being added. (One body of data in the archive is not a crawled
source at all — see *Inherited history* at the end of this section. It is never
published as a current price.) URLs that failed that probe are kept in
`config/sources.yaml` under `parked:` with the reason, so the same dead end is
not rediscovered later.
Vendor *marketing* pages are excluded on purpose. They are typically
client-rendered and quote consumer subscription tiers rather than API token
rates. `https://www.anthropic.com/pricing`, for example, is parked because it
returns consumer plan prices ($20/$100/$200), not API rates.
### The trust order
Three classes, in descending order:
| Class | Rank | Trust weight | What it is |
|---|---|---|---|
| `primary` | 3 | 1.0 | The provider's own rate card or API docs |
| `aggregator` | 2 | 0.6 | A specialist aggregator or reseller feed |
| `blog` | 1 | 0.3 | Secondary commentary |
Ranking is by class first, then by trust weight within a class, then by the
source's own stated effective date, then by observation time
(`merge._winner_key`). Weights are validated at config load to be in the range
(0, 1]; an unknown class ranks below `blog`.
### Sources actually in use today
Seven, and only two of the three classes are populated:
| Source | Class | Weight | Reader(s) |
|---|---|---|---|
| OpenAI API pricing (docs) | primary | 1.0 | `openai_astro_props`, then LLM |
| Claude API pricing (docs) | primary | 1.0 | `anthropic_pricing_table`, then LLM |
| Gemini API pricing (docs) | primary | 1.0 | `gemini_pricing_tables`, then LLM |
| DeepSeek API pricing (docs) | primary | 1.0 | `deepseek_pricing_table`, then LLM |
| xAI models and pricing (docs) | primary | 1.0 | `xai_models_blob`, then LLM |
| Z.ai GLM pricing (docs) | primary | 1.0 | `zhipu_pricing_table`, then LLM |
| OpenRouter models API | aggregator | 0.6 | `openrouter_json` (no LLM) |
**No `blog`-class source is configured.** This is not a neutral fact: several
rules below are shaped by the possibility of a lower-trust source making a
claim, and with the current registry the only non-primary source is the
OpenRouter JSON feed. See §6 for what that means for the freshness rule.
Mistral, Moonshot and Alibaba Cloud's Model Studio (which is where Qwen's rates
live) are parked pending a headless fetch path; they render prices client-side
and serve zero price-bearing HTML.
### Inherited history: the LiteLLM backfill
One body of data in the archive was not collected by the crawler at all, and a
reader who meets it without warning will reasonably assume it was.
`scripts/backfill_litellm.py` walks the git history of BerriAI/litellm's
`model_prices_and_context_window.json` — a hand-maintained routing config that
has existed since 2023-09-06 — and writes one observation per commit, stamped
with the **commit time** rather than the time we read it. It gives the archive a
past it did not live through, and it is the only reason any price series here
reaches back before TokenDrift existed.
It is **not verification, and it is not published as current**:
- Every backfilled row is `confirmed = 0`. These are one aggregator's numbers,
recorded by volunteers, that nobody here checked against a provider's rate card
on the day they were true — and unlike a live source they can never be
re-checked, because the page that would settle a dispute is gone.
- Every row carries `extractor = 'litellm_git_backfill'`, so it can never be
mistaken for something a reader of ours produced, and it is a distinct bucket
on the coverage endpoint.
- Its trust weight is **0.5**, below OpenRouter's 0.6 and below every primary.
OpenRouter at least bills against the numbers it publishes; LiteLLM's file is
edited when a contributor notices a change, so it lags, and at three years'
distance a lag is indistinguishable from a repricing.
- It hangs off a run whose status is `backfill`, not `ok`. Every query that
selects the differ's baseline, the previous run, or the published dataset
filters on `status = 'ok'`, so a backfilled row can never become a baseline and
can never be served by `dump`. It writes nothing to `reconciled_prices`, which
is what the differ actually compares — so no amount of backfilling can raise a
change event.
- Only **direct-API** provider slugs fold into the providers we track. A reseller
quoting its own price for someone else's model — `bedrock/…`, `azure/…`,
`vertex_ai-…`, `openrouter/…` — becomes a provider under its own slug, because
filing AWS's margin as Anthropic's rate card would be a fabrication.
Where you will meet it: the per-model price history (`/v1/history`) and the site,
where every point carries an `origin` of `reconciled` or `backfill` and the
backfilled ones are flagged wherever they appear. A backfilled point also reports
`needs_review` as **null**, not `false` — `needs_review` is a reconciliation-layer
flag and a row that never passed through the reconciler cannot carry one.
Null there means unreachable, not clear.
### Crawl policy
The collector identifies itself as
`TokenDriftBot/0.1 (+https://tokendrift.ai/bot; pricing-data research)`, waits
2.0 seconds between requests to the same host measured end-of-response to
start-of-next-request, times out at 30 seconds, and retries at most twice
(1s, 2s, 4s backoff) on transient failures only. "Transient" is an explicit
list — **408, 429, 500, 502, 503, 504**, plus transport errors — not the whole
5xx range: a 501 or a 505 is a settled answer about what the server will do, and
retrying it is noise aimed at someone else's infrastructure. Every other status
is taken at face value on the first response.
Those figures govern page fetches. The `robots.txt` request itself uses a fixed
10-second timeout and is not retried; a robots file we could not reach is
handled by the rule below rather than by trying again.
robots.txt handling follows RFC 9309 and distinguishes its three status classes
rather than collapsing them:
- **2xx** — the served rules are parsed and followed.
- **4xx** ("Unavailable") — the host publishes no restrictions; access permitted.
- **5xx, DNS, TLS, timeout** ("Unreachable") — the file is undefined and the
crawler assumes **complete disallow**. Silence is not consent.
Redirects are followed one hop at a time, with robots.txt consulted for each
host in the chain, up to five hops. A robots denial is never retried: a denial
is an answer, not a transient fault, and it is recorded in `fetch_log` with
status `robots_denied`.
---
## 2. What "verified" means, and what `observed_at` is not
Every price row carries `observed_at`, and the published JSON exposes it as
`verified_at`. Its exact meaning matters more than any other field in the
dataset, so it is defined narrowly.
### What `observed_at` is
**`observed_at` is when the bytes behind the value were obtained from the
provider.** It is carried per source, not per run:
- on a **live fetch**, the moment the response was received;
- on a **cache hit**, the modification time of the cached file — that is, when
we last actually looked at the provider's page, which may be hours or days
before the run that read it.
It answers exactly one question:
> *When did TokenDrift last look at the page this number came from?*
Verified behaviour: a run executing at 02:28 UTC against a cache written at
21:40 records `observed_at` 21:40, not 02:28. A run-level timestamp is used only
as a fallback if a successful fetch somehow carries no retrieval time, which
does not occur on any current code path.
This is deliberately the less flattering of the two available meanings. A
timestamp that means "when we last ran" rather than "when we last looked" would
make a stale value look fresh, and for a product whose differentiator is a
per-value verification timestamp, that is the wrong number in the wrong
direction.
### What `observed_at` is not
- **It is not when the provider set the price.** That claim, where a source
makes it, lives in `effective_date` and nowhere else. A value obtained today
may reflect a price set last year.
- **It is not per value.** It is the retrieval time of the *page*, so every
value read from one page in one run shares it.
- **It is not a maximum age guarantee.** It tells you how old the reading is; it
does not promise the reading will be refreshed within any interval.
Note that per-source observation times do **not** revive the old freshness rule
(§6). Sources are still fetched within one pass, and cache mtimes are an
artefact of our own iteration rather than a claim any source makes about its
prices. Freshness that two sources can genuinely disagree about has to be
`effective_date`, which is a property of the content.
### Cached runs are degraded runs
`tokendrift run --use-cache` re-reads bytes already on disk instead of
refetching. It exists so that parser and prompt work does not hammer providers,
and a run that used it **read no provider**. Its coverage looks complete, which
is exactly why it needed an explicit signal rather than an inference.
A run in which any source was served from cache is therefore recorded
**partial**, with the reason naming how many sources were not refetched. Like
every other partial run (§3) it is stored, but it produces no change events,
never becomes the differ's baseline, and is never published by `dump`.
So a cached run cannot reach a reader at all — and if one somehow did, its
`verified_at` would correctly report the age of the bytes rather than the time
of the run. The two halves of this are independent, and both are enforced in
code rather than left to operator discipline.
---
## 3. What "confirmed" means, and what it licenses
Two different words are used for two different things, and conflating them will
lead a reader to a wrong conclusion:
- **`confirmed`** is about *one page*: did the readings of that page agree, or
did the page's authority vouch for the value?
- **`cross_verified`** is about *more than one source*: did an independent
source report the same model? It drives the run's **join rate** and nothing
else. It is an internal reconciliation flag and is not itself published.
The `corroborating_sources` and `dissenting_sources` lists in the published JSON
are a **separate and stricter** computation, made per field from the raw
observations rather than from `cross_verified` — see §4. A model can be
cross-verified (another source reported it) while a given price on it lists no
corroborating source at all, because the other source was silent on that
particular field. That is the intended behaviour, not a discrepancy.
A value can be confirmed and not cross-verified, or cross-verified and
unconfirmed. They are orthogonal.
### Why confirmation is needed at all
A temperature-0 language model is not deterministic. The same cached bytes have
yielded 15 models on one pass and 18 on the next, and 179, then 249, then 204
prices from the same page across consecutive runs. Every such difference reaches
the differ looking exactly like a price change that never happened.
So each page is read more than once (default `--passes 2`), and the readings are
compared.
### What confirmation licenses
**Only a confirmed value may raise a change event.** That is the whole of it.
An unconfirmed value is still stored, still reconciled, and still published,
with its full provenance and a `confirmed: false` marker. What is withheld is
only its licence to assert that a number *moved*. A change event is a claim
about a transition between two states, and we will not make that claim on the
strength of a reading we could not reproduce. The differ counts and reports
suppressed candidates so the suppression is visible rather than silent.
### How confirmation is decided
**Case A — every reader is fallible (LLM passes only).** A value is confirmed
only if it appeared in every pass and every pass agreed on it, within a
tolerance of 1e-9 (which absorbs floating-point noise and nothing else). A
single usable pass confirms nothing.
**Case B — a deterministic parser passed its coverage guards.** That parser is
the **authority** on its page, and the symmetric rule is abandoned. Re-run on
identical bytes, a parser returns identical output, so "the readings disagreed"
is not an available explanation for a parser-vs-model difference. Concretely:
- Where the authority emitted a value and the model did not see it at all, the
value is **confirmed**. Absence in a stochastic reader is not evidence.
Otherwise a model reading 33 of 40 prices would bar the other 7 from ever
raising a change event — worse than not running the model at all.
- Where the authority and the model agree, the value is **confirmed**.
- Where the model contradicts the authority **repeatably** (at least two passes
saying the same other thing), the authority's value is still published, but
**unconfirmed**, and the disagreement is recorded as a `Divergence`. This is
how a mis-anchored parser reading a plausible-but-wrong cell gets caught —
such a parser passes every coverage guard.
- Where dissent is not repeatable, the authority stands and the disagreement is
recorded for review only.
- Where two authoritative readings of one page disagree, that is a defect in one
of them, not a consensus question: the first is published, unconfirmed.
Note that this makes single-reading confirmation possible: OpenRouter is read by
a deterministic parser with no LLM pass, so its values are confirmed off one
reading. "Confirmed" there means *the authority passed its guards*, not *two
readings agreed*.
### The authority rule, stated precisely
> A deterministic parser that passed its coverage guards is the authority on its
> page. A key it did not emit is **unverified by that authority**.
>
> **Unverified is not false.**
Three quite different situations produce the same absence, and the pipeline
cannot tell them apart from the inside:
1. **The value is genuinely not on the page.**
2. **The parser deliberately declined to read that region.** Anthropic's rate
card does print batch rates, in a table of their own. The Anthropic parser
does not read it, because minting batch-tier keys means committing to how
OpenRouter's parser already encodes batch, and a key that disagrees across
sources is worse than a missing one. The language model reads that table, so
Anthropic batch rates **are published** — carrying `extractor: llm` and
`confirmed: false`, because the page's authority never vouched for them.
Unverified, entirely real, and visibly marked as the weaker claim.
3. **The model misread.** This is the dangerous one, and it has happened here.
Reading that same rate card, the model reported `claude-opus-5` flat input at
$10 when the standard table says $5 — $10/$50 belonged to a different table
whose model cell names two models — and it made the same mistake on **both**
passes. (On the current registry those products are keyed apart and the
parser and the model now agree on `claude-opus-5` at $5/$25. The example is
what the rule is for, not a live defect.)
Case 3 is why agreement between stochastic passes cannot substitute for the
authority: **a consistent misreading is repeatable, and repeatability is not
verification.** Cases 1 and 2 are not errors at all, which is why the response is
to withhold confirmation rather than to drop the value. Such values are
published with full provenance, marked unconfirmed, and barred from raising a
change event.
### The reader ladder, and what each rung means
| Situation | What is published | Confirmation |
|---|---|---|
| Parser passed guards + LLM passes | Parser's values, LLM adds annotation and can demote | Per the authority rule above |
| Parser passed guards, no LLM configured | Parser's values | Confirmed |
| Parser **failed** its guards, LLM configured | Nothing from the parser; LLM passes only | Case A rules; the failure is printed and noted |
| Parser **failed** its guards, no LLM configured | Nothing at all for that source | Run is marked partial |
| No reader produced a result | Nothing for that source | Run is marked partial |
| Source failed to fetch | Nothing for that source | Run is marked partial |
A parser never contributes a partial reading. Guard failure is fatal for that
parser on that run, and the run falls back to the model alone rather than mixing
in an incomplete deterministic read.
### `extractor`: who actually read this number
Every observation and every published value carries `extractor`. It answers one
question — **which reader produced this value, on this run** — and it is
computed from the readings themselves (`extract/consensus.py::provenance_label`),
never looked up from configuration.
The vocabulary is four values and no others:
| Value | What it asserts |
|---|---|
| a parser name, e.g. `gemini_pricing_tables` | that deterministic parser read this value, and the language model did not report it |
| `<parser>+llm`, e.g. `anthropic_pricing_table+llm` | the parser **and** the model both read this value and agreed on it |
| `llm` | the language model read it; no parser reported it |
| `unknown` | we cannot name the reader, so we do not |
`<parser>+llm` is the strongest provenance claim in the dataset — a reproducible
reader and an independent stochastic one arrived at the same number — and it is
said rather than collapsed into either half, because the pipeline already knows
it and discarding it would understate our own evidence.
**Why this field had to be rebuilt.** It used to be filled from
`sources.extractor`: the reader `config/sources.yaml` names *first* for a page.
That is a statement about our configuration, not about the number, and the two
come apart at exactly the worst moment. When a deterministic parser raises, the
run catches the error and continues with the model alone — and the model's
readings were then written stamped with the name of a parser that had not
executed. That is not a hypothetical: DeepSeek repriced on 2026-08-16, its
parser failed on the changed page, and the run published sixteen rows claiming
`deepseek_pricing_table` for numbers a language model had read out of prose. The
investigation into the missed repricing read that field and went looking in a
file that had never run.
### What `unknown` means, and what it does not
`unknown` is a **withdrawn or unavailable provenance claim. It is never an
unread price.** The value beside it is a real number, read by a real reader,
with its source, its source URL and its `verified_at` intact. What is missing is
only our ability to name which reader it was.
It arises two ways, and a reader should treat them the same:
1. **Withdrawn history.** Migration `0006` demoted to `unknown` every historical
row stamped with the name of a parser that shares its page with the language
model. The truth is not recoverable per row: four different histories produce
a row bearing a parser's name — the parser read it; both readers read it; the
parser raised and the model read it alone; the model read a key the parser
never emitted — and nothing in the archive separates them. Leaving the claim
standing would keep a statement we know to be false for an unknown subset;
rewriting it to what probably happened would invent provenance, which is the
same defect with better manners. This affects a large share of the archive's
historical price rows, including a handful in the run currently served. The
two readers that never shared a page with the model — `openrouter_json`, and
the `litellm_git_backfill` writer — were left alone, because a row bearing
their name really was produced by them.
2. **Unresolvable this run.** At the published layer a key's provenance is read
back off the raw rows the same run just wrote. Where those rows disagree
about who read them, no single name describes the published value, and
picking a winner would be the same error as reading the name off the config.
The published row says `unknown`.
`unknown` is visible rather than hidden: it appears in `by_extractor` on the
coverage endpoint as its own bucket, instead of inflating a parser's count.
### Partial runs
A run that did not see every source has an incomplete picture, and diffing an
incomplete picture against a complete one reports every unseen price as deleted.
A run is marked **partial** if any source failed to fetch, any source failed to
extract, **any source was served from cache rather than refetched**, or the run
used `--only` or `--no-llm`. A partial run:
- **is** stored — the observations it made are real;
- **does not** produce change events;
- **never** becomes the differ's baseline;
- **is never published** by `dump`, which reads only the most recent run with
status `ok`.
---
## 4. How conflicts resolve — recorded, never averaged
A conflict is the same `(model, tier:variant, field)` carrying values that
differ by more than 1e-9 across sources. Price tolerance is exact; the epsilon
absorbs IEEE-754 noise only. Two sources quoting genuinely different prices are
a conflict no matter how small the gap.
**The guarantee: TokenDrift never averages, blends, or splits the difference
between disagreeing sources.** One value is selected by a stated rule, and every
competing value is retained.
Resolution order, over values that survive the gates in §5 and §6:
1. **All values agree** → publish it, note how many sources agree.
2. **One source contradicts itself** → publish the highest-trust value, flag
`ambiguous_extraction`, and send it to review. One source reporting two
different rates under one key means *our key is wrong*, not that the sources
disagree; choosing between two equally authoritative readings of one page
would be arbitrary. The cause is diagnosed where the data supports it
(`identity_fold` — two model ids folded onto one key, an upstream identity
fault; `missing_variant` — one model id priced several ways, a distinction we
failed to capture) and explicitly left as `unknown` when it does not. We do
not guess which layer is broken.
3. **A lower-trust source makes a materially fresher claim** → publish the
higher-trust value anyway, and flag `needs_review`. A blog must never quietly
reprice a model over a stale rate card, and a rate card must never silently
outrank a source that says the price changed yesterday. See §6.
4. **Otherwise** → the highest-trust source wins, and the reason names how many
sources it outranked.
Every conflict is written to the `conflicts` table with:
- `values_json` — **every** candidate, each with its source, source type, trust
weight, value, `observed_at`, currency, raw unit, stated effective date,
whether it was in force, and whether it was comparable;
- `resolved_value` — what was published, which may be `null`;
- `resolution_reason` — the rule that decided it, in prose;
- `needs_review` — whether a human should look.
The published JSON carries the full conflict list for the run. Separately, each
published price lists `corroborating_sources` and `dissenting_sources`, and
membership is decided **per field, three-valued** — a source's claim about the
input price and about the output price is each judged `same`, `differs`, or
`silent`:
- any `differs` → **dissenting**;
- otherwise any `same` → **corroborating**;
- silent on everything → **neither list**.
Both halves of that rule are load-bearing. Listing every source that mentioned
the model asserts agreement between sources that disagree. Bucketing on a single
boolean does the opposite and files partial silence as dissent, so a source
quoting only an input price gets named as contradicting an output price it never
mentioned. Both are unfounded claims about a third party, which is the one thing
these fields must never make. Silence is not assent and it is not dissent.
Cross-source conflicts and extraction ambiguities are counted separately in the
run report, because they call for different work: a conflict is a judgement
about whose number to trust; an ambiguity is a bug in how we keyed the rate.
---
## 5. Commensurability — only like is compared with like
**Two values are compared, agreed, or resolved against each other only if they
are commensurable: the same currency, pricing the same thing.**
The published basis is **USD per 1,000,000 tokens**, and nothing else. Unit
conversion to that basis happens in code (`extract/schema.py`), never in the
model, and `unit_raw` preserves what the source actually said so that a bad
conversion stays auditable. Recognised units and their factors:
| `unit_raw` | Factor to USD/1M tokens | Notes |
|---|---|---|
| `per_1m_tokens` | 1.0 | |
| `per_1k_tokens`, `per_1000_tokens` | 1,000 | |
| `per_token` | 1,000,000 | OpenRouter's basis |
| `per_1m_characters` | 1.0 | Tracked, **not** comparable to token pricing |
| `xai_price_units` | 1e-4 | xAI's docs blob quotes unlabelled integers where 20000 means $2.00 |
A language model may only claim the four token units; machine-only units belong
to the parsers that read the machine representation. A prose reader asserting
`xai_price_units` would be inventing provenance.
Values not in the published basis are **excluded from comparison** rather than
resolved against, and reported as `incommensurable`. A CNY figure and a USD
figure are not a cheap price and an expensive one. A per-1M-*character* rate is
already "per 1M" and is still not a token price. Such a quote is not a
disagreement; it is a modelling error, and it is reported as one.
**If nothing publishable remains for a key, nothing is published for that key.**
The row is retained with the provenance of the best-ranked excluded quote, so a
reader is told who to go and read, but no value is asserted. A wrong number in a
money dataset is worse than a gap.
This rule exists because the failure it prevents is easy and invisible. A
published comparison is only worth anything if the two numbers mean the same
thing; a reconciler that does not check what a number *is* before comparing it
manufactures that class of error at scale.
**Stated limit:** every deterministic parser currently configured emits USD,
including those reading Chinese-jurisdiction providers (DeepSeek, Zhipu), whose
parsers read the USD figures on the English rate cards. The currency half of this
gate is therefore exercised today only through the language-model path, which
can report a non-USD currency; the measure half (`per_1m_characters`) is
exercised by the unit table. The gate is tested, but like the freshness rule in
§6 it is not currently load-bearing on live data, and that distinction should be
read as a fact about the source registry rather than about the code.
### Scope
TokenDrift covers **text and chat completion token pricing only**. Embeddings,
image generation, audio and speech, moderation, fine-tuning surcharges, and
web-search or tool add-on fees are explicitly out of scope and are excluded at
extraction time.
---
## 6. In force versus scheduled
**Only rates in force are published as current.** A price that the source itself
dates into the future is a scheduled repricing, not today's number.
A quote is in force if it states no effective date, or states one on or before
the run's reference time. The reference time is derived from the observations
themselves, not from the wall clock, so re-running the reconciler over an
archive yields the same answer it yielded at the time.
Consequences:
- A future-dated quote is removed before the trust rules run. It cannot win, and
it cannot conflict with the rate in force.
- A future-dated quote that differs from the published value sets
`scheduled_change` and `needs_review`, and the reason names the value and its
effective date. It is retained, dated, and visible — it is just not published
as current.
- If **every** quote for a key is future-dated, nothing is published as current
and the row says so, naming the soonest rate and its date.
This is not hypothetical. Google's rate card reads "$0.75 through December 31,
2026. $1.50 starting January 1, 2027"; before this rule existed, TokenDrift
published the 2027 price as today's price.
DeepSeek is the worked example of the rule running its full course. Its card,
read 2026-08-15, dated a switch to peak/off-peak billing to 16:00 UTC on
2026-08-16 — a material price *increase*, stated on the primary source with an
effective date, which is exactly the class of event this project exists to
catch. The rates were published as scheduled and not as current until that
moment; the switchover then happened, the flat tier disappeared from the rate
card altogether, and peak/off-peak became the standing rate. The consequences of
that disappearance — a tier that outlived its vendor — are the subject of the
last part of §9.
Scheduled rates are keyed apart from current ones **in the extraction consensus
key**, which is `(model, tier, variant, effective_date)`. Two readings of a page
that quotes today's rate and a dated future one are therefore never merged into
one value, and each gets its own confirmation history.
**Stated limit — the store key does not carry the date.** A stored price row and
a published row are keyed by `tier` and `variant` only (`flat`,
`cache_write:1h`), with `effective_date` as a column beside the value rather
than as part of the key. So a dated future rate and today's rate for the same
tier share one store key, and what keeps the future one out of the published set
is the in-force gate above, applied on every run, rather than the key itself.
Where a provider expresses the change as new tiers — DeepSeek's `peak` and
`offpeak` — the two are separately keyed as a side effect of being separate
tiers. Where it does not, as on Google's dated `flat` rate, they are not.
This is a real difference from what a reader might assume. The consequence is
that the archive's separation of "the rate then" from "the rate scheduled from
then" rests on the gate and on `effective_date`, and a bug in the gate would not
be caught by a key collision.
### The freshness rule, and why it currently cannot fire
The trust order has one exception. A lower-trust source that makes a
**materially fresher claim about when its number took effect** does not
overwrite the higher-trust value, but does force the row to review. "Materially
fresher" means one of:
- both sources date their number and the dissenter's is more than **24 hours**
newer; or
- the dissenter dates its number and the winner does not date its own. An
undated winner is the ordinary shape of a stale rate card: the page still
prints last quarter's number with nothing to say when it took effect, while
the source that repriced says exactly when.
Freshness here is a property of the **content**, never of our fetch loop. An
earlier version compared `observed_at` and could not fire for *any*
configuration, because every source is fetched in one pass — and nothing
measured that, so it went unnoticed.
`observed_at` is now per source rather than run-wide (§2), and that deliberately
does **not** revive the old rule. The spread it can now show is the age of our
own cached bytes, which says nothing about when a provider repriced; a source
that we happened to read more recently has not thereby made a fresher claim.
Only `effective_date` is a statement the source itself makes about its number.
**As configured today the rule is dormant.** The only non-primary source is the
OpenRouter JSON feed, which carries no effective dates, and a source that dates
nothing cannot claim to be fresher than anyone. Every run therefore prints
**freshness coverage** — the share of resolved fields where a non-winning source
stated an in-force date — and on the current source set that is **0%**. The
denominator is every field the reconciler resolved, including keys where nothing
turned out to be publishable, so the measure is if anything generous to us.
The distinction between *dormant by configuration* and *dead by construction* is
the entire point, and it is measured rather than assumed. Coverage rises above
zero the moment any dated lower-trust source is configured. A reader should
treat the freshness exception as a rule that exists and is tested but has not
yet had an opportunity to act on live data.
---
## 7. What a parser must prove before it is believed
A deterministic parser reads a page in code rather than by asking a model. It is
believed over a model because it is reproducible — but a parser that silently
returns 3 of 40 prices is *worse* than no parser, because it looks
authoritative.
So every parser must count what it **should** have found, not merely what it
emitted, and must clear four floors before its output is used. A parser that
cannot prove its own coverage has no business being trusted over a model.
| Guard | Definition | Default floor |
|---|---|---|
| **Model floor** | Models emitted | per parser |
| **Price floor** | Prices emitted | per parser |
| **Parse rate** | `parsed / candidates` — of the price-bearing units the parser itself identified on the page, how many it fully understood | 0.95 |
| **Signal rate** | `prices_emitted / price_signals` — an *independent* count of `$`-followed-by-a-digit text over the region the parser consumed | 0.60 |
| **Plausibility** | Every emitted price within range, USD per 1M | 0.0001 – 10,000 |
The parse rate is the denominator check: a parser that counts only what it
emitted can never detect that it understood 3 of 40 rows. The signal rate is
independent of the parser's own bookkeeping, and catches an anchor that has
drifted onto a smaller table than it used to match. The plausibility range
catches a scale error — xAI quotes unlabelled integers where 20000 means $2.00,
so an unnoticed factor of 10,000 is one edit away.
Floors in force today:
| Parser | Min models | Min prices | Parse rate | Signal rate |
|---|---|---|---|---|
| `openai_astro_props` | 30 | 120 | 0.95 | 0.60 |
| `gemini_pricing_tables` | 22 | 170 | **1.00** | **0.35** |
| `zhipu_pricing_table` | 10 | 18 | 0.95 | 0.60 |
| `anthropic_pricing_table` | 8 | 30 | 0.95 | 0.60 |
| `xai_models_blob` | 5 | 20 | 0.95 | 0.60 |
| `deepseek_pricing_table` | 2 | 4 | 0.95 | 0.60 |
| `openrouter_json` | 50 | 50 | 0.95 | 0.60 |
Floors are set to catch a page's shape changing while tolerating a provider
pruning part of its catalogue.
**Stated limit:** both rate guards are ratios, and a ratio with a zero
denominator is not checked at all — a parser that identified no candidate
price-bearing units, or over whose consumed region no independent price signal
was counted, passes those two guards vacuously. The model and price floors are
counts and still bite, which is what keeps that case from being silent. It is
worth knowing that the two most informative guards are the two that can abstain.
**When a parser fails a guard it raises, loudly, and returns nothing.** The
failure is printed and recorded on the run, and partial parser output is never
mixed into the dataset. Where the source also has a language-model reader
configured, it falls back to the model alone under the fallible-reader rules of
§3. **Where it does not, the source contributes nothing that run and the run is
marked partial** — which is the case for OpenRouter, the one source read by a
parser with no model behind it. There is no reader of last resort. A
misconfigured parser name fails at config load rather than degrading silently to
the model.
An unregistered parser name is rejected at config load, so a typo cannot quietly
turn a deterministic source into a model-read one.
### Independent acceptance check
`scripts/verify_against_source.py` compares the pipeline's output against prices
read **by eye** off the live rate cards, hardcoded in the script. Its
expectations are deliberately not derived from the pipeline: it is the check
that the numbers are *right*, as distinct from internally consistent.
It carries a second list, of rates a source has announced but not yet brought
into force, each with the effective date the source stated. That list asserts
the §6 rule from the outside: a scheduled rate must carry its date and must
never be published as the rate in force. **It is currently empty**, because
DeepSeek's announced peak/off-peak repricing took effect at 16:00 UTC on
2026-08-16 and no source in the registry now announces a dated future rate. The
in-force expectations were re-read by hand after that switchover. An empty list
means the check has nothing to bite on, not that it passed — the same
distinction §6 draws about the freshness rule.
**Stated limit:** this check covers a hand-picked sample — three Anthropic
models, one xAI model, and two DeepSeek models across their peak and off-peak
tiers — not the published set. It is an acceptance check, not a coverage
measure, and it says nothing about the hundreds of values it does not name.
---
## 8. Model identity
Model identity is the key the price archive is stored under, so it is the field
where a mistake is least recoverable. **If identity forks, one price history
splits into two shallower ones, permanently** — and depth is the only property
of this dataset that cannot be reproduced in a weekend.
The failure is also silent in the worst possible way. If the cross-source join
fails, nothing matches, no conflict is possible, and the run reports zero
conflicts, which reads exactly like success.
### Normalization
`normalize_model_id` is the canonical join key. It:
1. strips a vendor prefix — `openai/gpt-5.2` → `gpt-5.2`;
2. lowercases and removes every non-alphanumeric character;
3. splits the remainder into alternating letter and digit runs;
4. **sorts** those runs and concatenates them.
Step 4 makes the key **order-insensitive**, which is required because providers
name the same model both ways: Anthropic's rate card prints "Claude Haiku 3.5"
while its API id is `claude-3-5-haiku`. An order-sensitive key files those as
two different models. Sorting *runs* rather than characters is what keeps "4-5"
and "4.5" collapsing to one `45` token rather than fragmenting into `4` and `5`.
**Stated limit:** order-insensitivity means two genuinely different identifiers
that are permutations of the same runs would collide. The blast radius is
bounded by the store's uniqueness constraint, which is per provider
(`UNIQUE (provider_id, norm)`), so a collision can only occur inside one
vendor's catalogue. We consider that a better trade than the fork, but it is a
real trade and not a free one.
### Deliberate conservatism
Normalization does **not** strip `-preview` or a `:suffix`. An earlier version
did, and merged genuinely distinct products: `gemini-3.1-flash-lite-preview` is
not `gemini-3.1-flash-lite`, and a `:batch` suffix is a pricing tier, separated
out before it reaches this function. Over-collapsing manufactures conflicts in
which a single source appears to contradict itself — 206 of them in one pass.
### Explicit aliases, and why there are currently none
Normalization handles spelling. It cannot handle a **rename** — two identifiers
that are genuinely the same product under different names — because that is a
fact about the world rather than about the string. The registry therefore
supports explicit aliases, declared in `config/sources.yaml` under `aliases:`,
scoped **per provider** so that two vendors listing the same alias string do not
collide. Every alias actually seen is recorded in `model_aliases` against the
model and the source that used it.
**As configured today the alias map is empty, and that is the interesting part.**
The rule now governing it is: *an alias may only record a rename a **source
states***. It exists because the one alias this project ever declared was wrong.
`deepseek/deepseek-chat` was aliased onto `deepseek-v4-flash` on the theory that
DeepSeek had renamed the model. No source said so, and the aggregator supplying
both contradicted it: OpenRouter lists them side by side in one response, with
different canonical slugs, different context windows (163,840 against 1,048,576)
and prices an order of magnitude apart. When OpenRouter means one model sold on
two plans it says so inside a single listing; it did not here. Two listings on
one source means two products.
The damage was exactly the damage §8 exists to prevent, arriving from the
opposite direction to a fork. Both listings' `flat` rows landed on one model, so
the key the whole store is built on held two contradictory prices with no
principle for choosing between them — and when DeepSeek withdrew its own flat
rate, that arbitrary choice became the published price. Migration `0007` moved
the V3 rows onto their own model. It deleted nothing: both prices are real and
both are on OpenRouter's page.
So the standing guidance is a warning rather than an invitation. Before adding a
line here, check whether `normalize_model_id` already collapses the pair —
**13 of the 15 aliases this project has ever considered were no-ops**, resolved
by normalization alone — and then check that a source actually asserts the
rename. An alias is a claim about product identity made on our own authority,
and it is the most expensive claim in the system to get wrong.
### Identity is settled before anything is written
Identity resolution happens in a distinct phase, before any row is written.
Resolving it during writing meant a model was created under whichever name
arrived first, and because extraction is concurrent the winner varied between
runs — producing 112 phantom "tier removed" / "tier added" pairs per pass. The
canonical `api_id` is chosen by rank: a primary source's real API identifier
beats an aggregator's, which beats a display name. **Rank ties break on the
identifier itself, not on arrival order** — two equally ranked observations
spelled differently (`claude-haiku-4.5` and `claude-haiku-4-5`) otherwise
published whichever label finished extracting first. The archive never forked,
because the normalized key is stable, but the caption shown in the dump and on
change events moved for no reason a reader could see. Only the *name* is decided
by rank; display name and context window coalesce from whichever observation
supplied them. A run that learns the real identifier upgrades the label in place
rather than creating a rival model.
### Measured, not assumed
Every run prints the **join rate**: the share of models confirmed by more than
one source. Zero triggers an explicit warning, because a zero join rate and a
clean zero-conflict run are otherwise indistinguishable from the outside.
---
## 9. Correction policy
We will publish a wrong number. The question a methodology answers is not
whether that happens but what happens next, and specifically whether the record
of what we said can be quietly altered afterwards. It cannot.
### The commitment
> **A published number is never rewritten.** A published value that turns out to
> be wrong is **superseded** by a later run, never edited and never deleted.
> What TokenDrift said a price was on a given date remains recoverable from the
> store on every later date.
This is enforced by the data model, not by discipline:
- **The pipeline only inserts into `prices`.** Every source's every claim in
every run is a new row, carrying its `run_id`, `source_id`, `observed_at`,
`currency`, `unit_raw`, `effective_date`, `confirmed` flag, the reader that
produced it, and the `content_hash` of the page it was read from. No code path
in a run updates or deletes a price row, and there is no hand-patch command.
- **`reconciled_prices` is per run.** The published value for
`(run, model, tier:variant)` is written once, and a *later* run writes its own
row rather than touching the earlier one. The only update path within a run is
before that run finishes.
- **`conflicts` and `change_events` are insert-only** and carry `run_id` and
`detected_at`.
- **A correction is therefore a new run.** The value moves; the earlier value,
its source, its reason and its timestamp all remain in place; and the change
feed shows the transition.
### The one route that does reach back, and what it is allowed to touch
"Insert-only" describes the pipeline. It does not describe **schema
migrations**, and pretending otherwise would be the kind of convenient
imprecision this document exists to avoid: migrations have twice reached into
rows that were already published, and once into a run that was already being
served.
A migration is not a quiet edit. Each is a numbered file in
`tokendrift/store/migrations/`, applied exactly once, recorded with a SHA-256 of
its own bytes in `schema_migrations`, re-checked on every open, and **never
edited afterwards** — the runner refuses to open an archive whose recorded hash
no longer matches the file. Each carries, in the file itself, the reasoning for
what it did. So the alteration is versioned, dated, replayable and reviewable in
git; it is part of the record rather than a hole in it.
**What a migration has changed:** which model a price row hangs off (`0003`
merged forked identities, `0007` split two DeepSeek products that had been
wrongly aliased into one); the `extractor` label (`0006`, below); and
`needs_review` on the run currently being served, together with the change
events that should have accompanied it (`0005`, described at the end of this
section). `0003` also deleted `reconciled_prices` rows made duplicate by a model
merge.
**What no migration has ever changed, and what a future one must not:** a price
**value**, its `currency`, its `unit_raw`, its `source`, its `observed_at`, or
its `effective_date`. Every number this archive has published, and the evidence
of where and when it came from, is exactly as it was written.
That is the line, and it is narrower than "append-only" implies. Provenance
metadata and identity keys have been repaired in place, with the repair itself
on the record. The numbers have not been touched.
### The limits of "append-only", stated exactly
The word is precise about observations and imprecise about everything else, so:
- **Append-only, in normal running:** `prices`, `conflicts`, `change_events`,
`fetch_log`. `extraction_stats` holds one row per run and source, rewritten
only within its own run. A numbered migration can reach further — see the
paragraphs above for exactly how far it is allowed to reach and what it has
actually done.
- **Mutable, deliberately:** the `models` table's *labels* — `api_id`,
`display_name`, `context_window`, `max_output` — are upgraded in place when a
run learns better ones. This is what prevents an identity fork (§8). It means
the *name* a model is displayed under is not historical; its *prices* are.
- **Mutable, per run:** `runs` rows are updated at completion with status and
totals; `sources` rows carry a `last_fetched` stamp.
- **Not versioned:** `config/sources.yaml` and the parser code live in git, and
git is the only record of what the trust weights and floors were on a given
date. There is no in-database snapshot of the configuration a run used.
### How a correction is made
1. The wrong value is diagnosed — by the acceptance check, by a conflict flagged
`needs_review`, by a divergence between parser and model, by a yield-drop
warning, or by a report from a reader.
2. The **cause** is fixed at the layer that produced it: the parser, the alias
map, the source registry, the unit table, or the reconciliation rule. We do
not hand-patch a value in the store; there is no code path to do so and there
will not be one.
3. A new run publishes the corrected value. The differ records a `price` change
event from the wrong value to the right one, against the last **confirmed**
baseline, attributed as described below.
4. If the fault was in a rule rather than in a reading, the rule change is
logged in §10 of this document.
**Where that procedure does not reach.** It works because a later run supersedes
an earlier one, and a reader comparing the two can see both. That is true of a
price. It is **not** true of a false statement about rows already written: no
future run overwrites a historical `extractor` label, so a wrong one would stand
in the archive for as long as the archive exists, and every later run would add
more correct ones beside it without ever contradicting it.
A defect of that shape is corrected by a numbered migration, under the rules
stated above — and only in the direction of claiming *less*. Migration `0006`
did not rewrite the field to what we thought had probably happened; it withdrew
it to `unknown`, which is a weaker claim and a true one. **A correction to a
statement about provenance may remove a claim we cannot support. It may not
substitute a better guess.** Substituting one would be indistinguishable, to
every later reader, from having been right the first time.
### Change attribution: who moved the number
A price-change feed that cannot distinguish *the vendor repriced* from *we read
the same page differently* is publishing news and apologies in the same
envelope. The signal needed to tell them apart was always in our hands — we
fetch the bytes — so we now keep it.
A **sha256 of the page's extracted readable text** is recorded on every fetch
(`fetch_log`), carried onto each observation (`prices`), and carried onto the
published value (`reconciled_prices`) so that attribution survives without a
join and works for a source read from cache. When a value moves, the differ
compares the hash behind the old value with the hash behind the new one and
writes `change_events.cause`:
| `cause` | Meaning |
|---|---|
| `vendor` | The page's readable text changed. The provider moved the number. This is a repricing. |
| `reading` | The page's readable text is **identical** and our number moved anyway. The change is **ours** — a correction, or a regression. It is never a repricing. |
| `NULL` | **Unknown**, not "either". One side has no recorded hash — a source that failed to fetch, or a value written before this column existed. |
### Why the hash is over the text and not the bytes
This is the evidence the whole attribution rests on, so what is hashed is
stated exactly.
The hash is **not** of the raw response body. Measured against the live sources,
two fetches of `docs.claude.com` seconds apart return equal-length but
byte-different HTML, because the markup carries per-request build ids and
nonces; `docs.x.ai` does the same. Across three real runs, Anthropic, Gemini and
xAI produced three different raw hashes each while their pricing had not moved
at all. A raw-body hash therefore answers "did the byte stream change", to which
the answer on those sources is always yes — and every price move on them would
have been published as a vendor repricing. That is precisely the fabrication
this field exists to prevent, so the raw hash is the wrong instrument.
What is hashed is the readable text the extractor pulls out of the page, which
is stable across those same fetches and is independent of our own parsing, so it
remains evidence about the vendor rather than about us
(`collectors/fetch.py::FetchResult._hashable`).
**Two consequences, both of which cut against us and are stated for that
reason:**
- A vendor edit that changes only markup — a rotated banner in an attribute, a
new build id — does not change the hash. If our number moved on such a page we
will call it `reading` when the page did in fact change. The judgement being
made is "did the page's *readable content* change", which is the closest
proxy for "did the vendor change anything a price could be read from" that a
hash can express, and it is not the same question.
- Where a page yields too little extractable text to be meaningful — a JSON API
such as OpenRouter's feed, which has no prose at all — the raw body is hashed
instead. On those sources the noise problem above returns in full: a
cosmetically re-serialised feed reads as `vendor`. The fallback is deliberate,
because hashing the empty string would mark every such source permanently
unchanged and turn its real repricings into `reading`, which is the more
damaging error of the two.
`NULL` must be read as unknown and reported as unknown. A confident wrong label
is worse than no label, which is the whole reason the column refuses to guess.
Every run prints a loud `WARNING` naming the number of changes attributed to
`reading`: values that moved while the provider's page did not. That is the
signal that we have either just corrected ourselves or just broken something,
and it should never scroll past unread.
**Scope of the attribution, stated exactly:**
- It covers **value moves** — `price` events. Keys that appeared or disappeared
(`price_tier_added`, `price_tier_removed`) carry no cause, because a key's
absence has no page behind it to compare.
- It distinguishes *the page changed* from *the page did not*. It does **not**
distinguish a correction from a regression: both are `reading`. Which one a
given `reading` event is takes a human, and that is what the warning is for.
- A cosmetic edit to a vendor page's *visible text* — a changed "last updated"
line, a reworded caption — changes the hash. So `vendor` means *the page's
readable text changed and our number changed*, not *the page changed only in
the price*. The attribution is permissive in that direction and strong in the
other: `reading` asserts that the readable content did not move while our
number did, which is a claim we can only make wrongly if a vendor repriced
without altering a character of readable text.
### When a vendor withdraws a tier and an aggregator backfills it
Resolution is per `(model, tier)` across sources by trust order, so a key does
not disappear when the vendor stops publishing it — the next source down wins it
by default. On 2026-08-16 DeepSeek replaced its flat rate for `deepseek-v4-flash`
with peak/off-peak tiers, printing no `flat` row at all; we published an
aggregator's resale price of **0.06146** under `flat` while the vendor's own
off-peak rate was **0.22**. Every row involved was real and individually
correct. The tier had outlived its vendor.
What we do about it, and what we deliberately do not:
- **The value is still published.** The aggregator genuinely resells the model
at that price. Deleting a true number loses information, and unverified is not
false here — nor is withdrawn-by-the-vendor.
- **The row carries `needs_review`.** It is no longer the vendor's price and
must not be read as one. This is a **state**, recomputed every run from the
archive: it holds for as long as a primary source once served the key and does
not serve it now, and it clears the run a primary supplies it again. A flag
that fired only on the night of the handover would leave every night after it
publishing the same number unflagged.
- **A `price_source_handover` change event names both sides** — the source that
withdrew and the source that took over, with each one's `source_type` and
values. It fires **once**, on the transition out of the previous published
run; a key that was already aggregator-sourced last run is steady state, not
news. It is distinct from `price` (the number moved) and from
`price_tier_removed` (we no longer publish this key), so a consumer can route
on the kind alone.
- **The accompanying value move is never attributed to `vendor`.** The winning
source changed, so the two page hashes are of unrelated documents; `cause` is
`NULL`, which is the honest answer. The handover event carries no `cause`
either — `cause` answers "the vendor or us", and a handover is not an answer
to that question.
Backfilled onto the served run by `0005_flag_prices_the_vendor_withdrew.py`.
That migration repairs the latest published run only: earlier runs contain
handovers that reverted a run or two later, and writing a handover into a past
run with no counter-event for the return would narrate a history that did not
happen.
### Where the attribution is published, and where it stops
`cause` is carried all the way out to the reader. It is on every `price` change
event in the store; `dump` emits it on each entry of the `changes` list; the
JSON API serves it on `/v1/changes` alongside a machine-readable gloss of what
each value means; and the MCP surface serves the same events.
On both query surfaces `cause` is a **four-state selector**, because the four
questions are genuinely different and an earlier version could only ask three of
them:
| Selection | What comes back |
|---|---|
| no filter (omit the parameter, or `all`) | every event, whatever its cause |
| `vendor` | only the moves we are prepared to call repricings |
| `reading` | only the moves that are ours — corrections and regressions |
| `unknown` | exactly the events whose `cause` is NULL |
`unknown` being selectable is the point of the change. Unattributed events
previously could not be asked for at all: they were reachable only by taking the
unfiltered feed and subtracting, which meant the one bucket a sceptical reader
most wants to audit was the one bucket with no name. An empty `cause=` is
rejected rather than read as "no filter" — a caller who meant something and did
not say it gets an error, not a guess.
**What is still not implemented — a disclosure, not a plan:**
- **There is no push notification of any kind.** No email, no webhook, no
subscriber notice, no mailing list. The change feed is a *pull* surface: it
will tell you a number moved and who moved it, and only if you come and ask.
**Labelling a correction is not telling anyone about it.** We can say, in the
archive and in the feed, that a number moved because we were wrong; we have no
way to reach a party who already relied on the wrong number, and no
undertaking to try.
So TokenDrift is a dataset that is *auditable* after the fact and *queryable*
about its own corrections, not one that *announces* them. **Anyone citing a
number in a context where a later correction would matter should retain the dump
they cited and re-check it, rather than relying on us to tell them.**
### Standing disclosures
These are known and unresolved as of this version:
| Limit | Where |
|---|---|
| The freshness rule reports 0% coverage — no configured lower-trust source states effective dates | §6 |
| **There is no push notification of any kind.** The change feed is pull-only: a correction is labelled and queryable, never announced to anyone who already relied on the wrong number | §9 |
| Attribution separates *the page's readable text changed* from *it did not* — it does not separate a correction from a regression, and it does not cover keys appearing or disappearing | §9 |
| The attribution hash is over **readable text, not raw bytes**, so a vendor edit confined to markup reads as `reading`; sources with no extractable prose (OpenRouter's JSON feed) fall back to hashing raw bytes and carry the opposite bias | §9 |
| **A large share of historical `extractor` values were withdrawn to `unknown`**, because the field used to be filled from configuration rather than from what ran. `unknown` is a withdrawn claim about the reader, never an unread price | §3, §9 |
| Published values from the archive's earliest three runs carry **no `extractor` at all**. The column was added after they were written and deliberately not backfilled onto them, because at the published layer there was nothing true to write; the raw observations behind them were backfilled to `llm`, which was historically accurate — every price in the archive at that point had been read by the model. Those runs are reachable through the price history, where `extractor` reports `null` | §3 |
| Numbered migrations have repaired identity and provenance metadata on already-published rows, and one has flagged and annotated the run currently being served. No migration has ever altered a price value, currency, unit, source or timestamp | §9 |
| Only two of the three trust classes are populated; no blog-class source exists | §1 |
| The currency half of the commensurability gate is not exercised by any configured parser — all emit USD | §5 |
| Scheduled and in-force rates are keyed apart in the extraction consensus but **not in the store key**; separation there rests on the in-force gate and `effective_date` | §6 |
| Anthropic batch rates are deliberately unread by the parser and read by the language model, so they are published unconfirmed rather than being absent | §3 |
| The independent acceptance check covers a hand-read sample of six models, not the published set | §7 |
| OpenRouter has no language-model reader, so a parser failure there loses the source for that run rather than degrading to a model read | §3, §7 |
| Inherited LiteLLM history is unverified, unverifiable in principle, and can never be re-checked; it is never published as a current price but does appear in price history | §1 |
| No alias is declared; the only one ever declared was a false identity and was unfolded by migration `0007` | §8 |
| Model naming (not pricing) is mutable in place | §9 |
| Run configuration (trust weights, parser floors, source list) is versioned only in git, not snapshotted per run | §9 |
| Coverage is text/chat completion models only — embeddings, image, audio, moderation, fine-tuning and tool add-on fees are out of scope | §5, extraction prompt |
---
## 10. Changing this document
The cautionary case is a matter of public record. In February 2021 S&P Global
Platts announced a change to the basis of Dated Brent and **retracted it three
weeks later** under market pressure; an equivalent change did not take effect
until May 2023. A benchmark administrator with a near-monopoly could not
unilaterally change its own methodology, because by then contracts referenced
it. Methodology is cheap to write and expensive to change, and the price of
changing it rises with every party that depends on the number.
This document is therefore written now, while nothing depends on it and changing
it is free.
**The rule, in two stages.**
*While no third party cites TokenDrift numbers:* methodology may change
immediately, in the same commit as the code that implements it. Every change is
recorded in the changelog below with a date and a reason. The version number at
the top of this document is bumped on any change that alters what a published
number means; editorial changes bump nothing.
*Once third parties cite TokenDrift numbers:* a change that alters what a
published number means — the trust order, the commensurability basis, the
in-force rule, the definition of `confirmed`, the identity key, or the
correction policy — will be announced **before** it takes effect, with an
effective date, and the old and new methodology will be stated side by side. A
change to a parser floor, the addition or removal of a source, or a bug fix that
makes the system match this document is not a methodology change and does not
require notice; it is logged. Where a change would alter historical values, the
history is not restated — the archive keeps what was published, and the new
method applies from its effective date forward.
We would rather adopt a rule slowly than retract one.
### Changelog
| Version | Date | Change |
|---|---|---|
| 1.2 | 2026-08-16 | Re-synced against commit `eb5ef83`, 1,390 tests passing. **§3** gains `extractor`, now a four-value vocabulary computed from the readers that ran (`<parser>`, `<parser>+llm`, `llm`, `unknown`) rather than read off the configuration, and defines `unknown` as a withdrawn or unresolvable provenance claim — never an unread price. **§9** corrects two claims that had become false: the attribution hash is over the page's **extracted readable text**, not its raw bytes (so "byte-identical" was wrong in the direction of overstating the evidence), and `cause` **is** exposed in the published JSON, on `/v1/changes` and over MCP, where it is now a four-state selector including the previously unaskable "unknown". The remaining disclosure is narrowed to what is still true: pull-only, no push notice. **§9** also states, for the first time, the one route that reaches back into published rows — numbered, checksummed, never-edited migrations — exactly what they have altered (identity keys, `extractor`, `needs_review` on the served run) and the line they have never crossed (no price value, currency, unit, source or timestamp), and adds the rule that a provenance correction may only withdraw a claim, never substitute a better guess. **§1** documents the LiteLLM git backfill as inherited, unverified history that never reaches the published dataset, and corrects the retry set to 408/429/500/502/503/504. **§6** withdraws the claim that scheduled rates are keyed apart in the store; they are keyed apart only in the extraction consensus. **§8** records that the project's only declared alias was a false identity, unfolded by migration `0007`, and that none is declared today. **§3** corrects `cross_verified` (it drives the join rate alone), the reader ladder for a source with no model fallback, and the Anthropic batch example. **§7** corrects the acceptance check, whose scheduled-rate expectations emptied when DeepSeek's repricing took effect. Standing disclosures rewritten. |
| 1.1 | 2026-08-15 | Re-synced against commit `fce0a66`, 630 tests passing. **§2** rewritten: `observed_at` now means when the bytes were obtained (cache mtime on a cache hit), and a cache-served run is recorded partial, so both halves of the disclosed `--use-cache` defect are closed and that disclosure is withdrawn. **§9** gains change attribution (`cause` = `vendor` / `reading` / NULL) and narrows the "indistinguishable from a repricing" disclosure to what remains true: the label is not exposed in published dumps, and labelling is not notification. **§4** corroboration restated as a three-valued per-field rule. **§8** notes the provider-scoped alias map, the pruning of no-op alias entries, and deterministic tie-breaking on the identifier. |
| 1.0 | 2026-08-15 | First publication. Described the system as of commit `f7123b5`, 469 tests passing. |