Docs Drift

A short list of places where a project's documentation contradicts itself, with the evidence.

For maintainers. It reads a project's docs, pairs the passages that restate each other, and uses the Clocked API to pick the pairs worth a person's time. Every item shows both passages with file and line. It is a list to review, not a verdict: about half of it is real (11 of 21, 52%), and on four projects it had not seen before, none of its 8 items was (0 of 8).

15 real projects measured, 2 more scanned later. 17 defects backed by source code. 5 pull requests open upstream, 2 merged.

Try it live

Send one pair of passages, or a public GitHub repository, through the same pipeline. Requests go to a small service that holds the Clocked key; it has a fixed credit budget, a cap per request and a rate limit, and it says so when the budget is used up. It logs the address, sizes and credits, not the text.

What request.body is inside the onRequest hook in fastify/fastify

docs/Reference/Hooks.md:73the code agrees

73> In the [onRequest](#onrequest) hook, `request.body` will always be74> `undefined`, because the body parsing happens before the75> [preValidation](#prevalidation) hook.

docs/Reference/TypeScript.md:1540the code disagrees

1540Notice: in the `onRequest` hook, request.body will always be null, because the1541body parsing happens before the `preHandler` hook.

lib/request.js:40 sets `this.body = undefined`; lib/handle-request.js runs body parsing before the preValidation hooks.

How it surfaced: standard run, both checks agreedUpstream: fastify/fastify #7193, open

What update() does with an array in typeorm/typeorm

docs/docs/working-with-entity-manager/5-entity-manager-api.md:189the code disagrees

189// Bulk updates with different conditions for each operation190await manager.update(User, [191 { criteria: { id: 1 }, data: { firstName: "Rizzrak" } },192 { criteria: { id: 2 }, data: { firstName: "Karzzir" } },193 { criteria: { age: 18 }, data: { category: "ADULT" } },194])195// executes three separate UPDATE queries:

docs/docs/working-with-entity-manager/6-repository-api.md:154the code agrees

154You can pass an **array of condition objects** to match multiple distinct sets of rows in a single call (conditions are OR'd together):

packages/typeorm/src/entity-manager/EntityManager.ts:897-927 has only update(target, criteria, partialEntity, options); no {criteria, data} form exists in src or test. The example came in with docs PR #12025; the feature PR #10910 implemented OR-ed condition arrays. The same PR added 'three separate queries' comments to the delete and softDelete examples, which the code also runs as one query.

How it surfaced: standard run, one check agreedUpstream: typeorm/typeorm #13040, open

Which behaviours setupListeners() enables in reduxjs/redux-toolkit

docs/rtk-query/api/setupListeners.mdx:14the code agrees

14A utility used to enable `refetchOnFocus` and `refetchOnReconnect` behaviors.

docs/introduction/getting-started.md:184the code disagrees

184A utility used to enable `refetchOnMount` and `refetchOnReconnect` behaviors.

packages/toolkit/src/query/core/setupListeners.ts dispatches onFocus, onFocusLost, onOnline and onOffline; nothing in it concerns mounting. The same wording is at docs/rtk-query/overview.md:85 and packages/toolkit/README.md:107.

How it surfaced: extended run (more pairs), both checks agreedUpstream: reduxjs/redux-toolkit #5592, open

Watch it run

Where the fixes are

Each contradiction was checked against both passages and the project's source code before a fix was written. The final word belongs to the upstream maintainers. Status as of 2026-11-09.

Pull requestStatusDefectsCovers
reduxjs/redux-toolkit #5592open2The action types createAsyncThunk dispatches; Which behaviours setupListeners() enables
typeorm/typeorm #13040open2Which JavaScript version TypeORM targets; What update() does with an array
mikro-orm/mikro-orm #8617merged4Which entities em.flush() checks for changes; How to load SQLite extensions; What Type.toJSON() returns by default; Whether a raw() fragment can be serialized
rollup/rollup #6643merged1Whether Node.js supports ES modules
fastify/fastify #7193open5What request.body is inside the onRequest hook; What request.body is inside the preParsing hook; Which HTTP methods Fastify supports; What an onSend hook may change the payload to; Which types the final payload may have
prometheus/prometheus #20388open1The value st_changed_on is capped at
nodejs/undici #6182open1The default of keepAliveTimeoutThreshold
fastify/fastifynot opened yet1notes for 1 issue ready: Which methods a custom logger must have

The numbers, including the bad ones

  1. 1,878pairs of passages read across 15 projects
  2. 52flagged by /compare
  3. 21kept for a person to read
  4. 11real contradictions

11 of 13 (85%) on the 11 projects the listing rule was tuned on.

0 of 8 (0%) on the 4 projects scanned after the rule was fixed.

Read those two together. The rule that decides what makes the list was chosen on findings from eleven projects and then run unchanged on four more. There it listed eight items and none was real. So this is a tool that saves a maintainer from reading 1,878 pairs, not one that can be trusted to be right: about half of what it lists is real, the rate varies a lot between projects, and 9 of 15 projects produced nothing real.

Of the 17 defects, 10 were on the standard run's list, 3 came from reading more pairs in three projects, and 4 surfaced in other runs without making the standard list. Each card says which.

Project and commitPassagesPairs readFlaggedListedListed and realDefects on this pageCreditsRole
fastify/fastify6c14896e7771496556735rule tuned here
prometheus/prometheus0e6228657661504211706rule tuned here
vitejs/vitea579e3525671061000389rule tuned here
jestjs/jest726144489591508100731rule tuned here
typeorm/typeorm5bdc67886411502222746rule tuned here
Automattic/mongoosebff9821f643790000315rule tuned here
mikro-orm/mikro-orm35fe690b1,5831504224823rule tuned here
eslint/eslint17f17cd71,1051500000556rule tuned here
reduxjs/redux-toolkitc03783eb8761501112537rule tuned here
vitest-dev/vitestc0346ba22,19315013000723rule tuned here
sindresorhus/got6b5aafc1385480000166rule tuned here
rollup/rollup82bef03c372865301456scanned after the rule was fixed
gofiber/fiber16cdbd2b1,60615074001045scanned after the rule was fixed
pydantic/pydantic9895381a7671501100448scanned after the rule was fixed
sveltejs/svelte7775f73b724600000260scanned after the rule was fixed
All 151,878522111168,636

Projects where it found nothing real

9 of 15: vitejs/vite, jestjs/jest, Automattic/mongoose, eslint/eslint, vitest-dev/vitest, sindresorhus/got, gofiber/fiber, pydantic/pydantic, sveltejs/svelte. In most of them it listed nothing. In jest, fiber and pydantic it listed items that turned out not to be contradictions.

What it listed and got wrong

10 of the 21 listed items are not contradictions. Four examples, one for each kind of mistake. All 10 are in review/labels_v2.json with the reason.

It also dropped two real contradictions, because /compare labelled them scope differences: the rollup one on this page, and a second sighting of the MikroORM flush one.

How it works, and what each Clocked endpoint changes

  1. Find passages that restate each other

    Local, free. The docs are split at headings. Sentences from different sections that are worded almost alike but not identically become candidate pairs, and so do passages that mention the same option name.

  2. Compare decides which pairs conflict

    One /compare request per pair. Its conflict spans become the two quoted passages and their line numbers. Its conflict kind decides what happens next: value, version and rule conflicts go on, scope and name differences are dropped. On its own, 13 of 52 (25%) of what it flags is real.

  3. Verify and a swapped Compare decide what makes the list

    /grounding checks each side's sentence against the other passage, and /compare is asked again with the passages swapped, because its answer depends on their order. An item is listed only when one of the two agrees. On the tuning findings this took precision from 55% to 85%. On the four later projects it did not help.

  4. Rerank finds where else the fact is stated

    /rerank picks other passages that state the disputed fact, and /grounding checks which side they support, which names the side that is probably stale. With /rerank, 14 of 31 picked passages stated the fact, against 11 of 48 for a plain word-overlap pick. The side it named came out the same either way: of 7 real findings where the code settles which side is wrong, 4 were named with /rerank and 4 without, all correctly. So this step helps the reader more than it changes the result.

The other 14 defects

Marked words differ between the two passages. Line numbers link to the pinned commit on GitHub.

What request.body is inside the preParsing hook in fastify/fastify

docs/Reference/Hooks.md:104the code agrees

104> In the [preParsing](#preparsing) hook, `request.body` will always be105> `undefined`, because the body parsing happens before the106> [preValidation](#prevalidation) hook.

docs/Reference/TypeScript.md:1550the code disagrees

1550Notice: in the `preParsing` hook, request.body will always be null, because the1551body parsing happens before the `preValidation` hook.

lib/request.js:40 sets `this.body = undefined`.

How it surfaced: standard run, both checks agreedUpstream: fastify/fastify #7193, open

Which HTTP methods Fastify supports in fastify/fastify

docs/Reference/Server.md:1667the code agrees

1667Fastify supports the `GET`, `HEAD`, `TRACE`, `DELETE`, `OPTIONS`,1668`PATCH`, `PUT`, `POST` and `QUERY` HTTP methods by default.

docs/Reference/TypeScript.md:1103the code disagrees

1103Union type of: `'DELETE' | 'GET' | 'HEAD' | 'PATCH' | 'POST' | 'PUT' |1104'OPTIONS'`

types/utils.d.ts:17-18 defines the union with 'QUERY', 'TRACE' and twelve more members. docs/Reference/Routes.md:34-35 repeats the Server.md list.

How it surfaced: standard run, both checks agreedUpstream: fastify/fastify #7193, open

What an onSend hook may change the payload to in fastify/fastify

docs/Reference/Hooks.md:240the code agrees

240> If you change the payload, you may only change it to a `string`, a241> `Buffer`, a `stream`, a `ReadableStream`, a `Response`, or `null`.

docs/Reference/TypeScript.md:1592the code disagrees

1592> If you change the payload, you may only change it to a string,1593> a Buffer, a stream, or null.

lib/reply.js:183-188 accepts web streams (`getReader`) and `Response` objects.

How it surfaced: standard run, one check agreedUpstream: fastify/fastify #7193, open

Which types the final payload may have in fastify/fastify

docs/Reference/Hooks.md:240the code agrees

240> If you change the payload, you may only change it to a `string`, a241> `Buffer`, a `stream`, a `ReadableStream`, a `Response`, or `null`.

docs/Reference/Reply.md:575the code disagrees

575The type of the sent payload (after serialization and going through any576[`onSend` hooks](./Hooks.md#onsend)) must be one of the following types,577otherwise, an error will be thrown:578 579- `string`580- `Buffer`581- `stream`582- `undefined`583- `null`

lib/reply.js:183-188 and the final send path accept web streams and `Response` objects without throwing.

How it surfaced: not listed by the standard run; it surfaced in the both-orders run below the listing bar, and in the day-1 batch runUpstream: fastify/fastify #7193, open

Which methods a custom logger must have in fastify/fastify

docs/Reference/Server.md:366the code backs part of each

366A custom logger instance. The logger must be a Pino instance or conform to the367Pino interface by having the following methods: `info`, `error`, `debug`,368`fatal`, `warn`, `trace`, `child`. For example:

docs/Reference/Logging.md:226the code backs part of each

226- **Methods:** `info`, `error`, `debug`, `fatal`, `warn`, `trace`, `silent`,227 `child`228- **Properties:** `level` (string)

The runtime check in lib/logger-factory.js:68 requires info, error, debug, fatal, warn, trace and child. The TypeScript type FastifyBaseLogger (types/logger.d.ts:27) also requires `level` and `silent`. Each page matches one of the two.

How it surfaced: not listed by the standard run; day-1 batch run only (with one pair per request /compare returns no passage for it)No patch: which list the docs should give is a maintainer decision. Notes for an issue are in upstream-patches/fastify/06-logger-methods-issue-notes.md.

Which JavaScript version TypeORM targets in typeorm/typeorm

README.md:19the code agrees

19and can be used with TypeScript and JavaScript (ES2023).

docs/docs/getting-started.md:5the code disagrees

5and can be used with TypeScript and JavaScript (ES2021).

PR #11382 (drop Node 16 and 18) changed the README from ES2021 to ES2023 and moved tsconfig.json to @tsconfig/node20; getting-started.md was not touched.

How it surfaced: standard run, both checks agreedUpstream: typeorm/typeorm #13040, open

The value st_changed_on is capped at in prometheus/prometheus

tsdb/docs/format/chunks.md:149the code disagrees

149 Due to the 7 bit limitation, once a chunk has at least 127 samples,150 `st_changed_on` is set to 127 (0xEF) and the 127th and further samples will151 have `st_i` present.

tsdb/docs/format/chunks.md:301the code agrees

301Once a chunk has at least 127 samples,302`st_changed_on` is set to 127 (0x7F) and the 128th and further samples will303have `st_i` present.

127 is 0x7F; tsdb/chunkenc/st.go:19 has `maxFirstSTChangeOn = 0x7F`. The two passages also disagree on 127th versus 128th sample; the patch leaves that to the maintainers.

How it surfaced: standard run, both checks agreedUpstream: prometheus/prometheus #20388, open

Which entities em.flush() checks for changes in mikro-orm/mikro-orm

docs/docs/guide/01-first-entity.md:389the code agrees

389Whenever you call [`em.flush()`](/api/core/class/EntityManager#flush), the ORM will iterate over the Identity Map, and for each entity it compares the original state with the values that are currently set on the entity.

docs/docs/unit-of-work.md:85the code disagrees

85Now whenever you call `em.flush()` MikroORM will iterate over all entities you previously marked for persisting via `em.persist()`.

packages/core/src/unit-of-work/UnitOfWork.ts computeChangeSets walks the identity map (line 769) and then the persist stack (775). unit-of-work.md:34 itself says managed entities need no persist call.

How it surfaced: standard run, one check agreedUpstream: mikro-orm/mikro-orm #8617, merged

How to load SQLite extensions in mikro-orm/mikro-orm

docs/docs/usage-with-sql.md:162the code disagrees

162Load them via `pool.afterCreate`:

docs/docs/usage-with-sqlite.md:229the code agrees

229Once you've downloaded the binaries for the extensions you wish to use, they can be added via the `onCreateConnection` hook:

packages/core/src/utils/Configuration.ts:639 defines `onCreateConnection`; PoolConfig (lines 805-812) has only min, max and idleTimeoutMillis.

How it surfaced: standard run, one check agreedUpstream: mikro-orm/mikro-orm #8617, merged

What Type.toJSON() returns by default in mikro-orm/mikro-orm

docs/docs/custom-types.md:20the code agrees

20Converts a value from its JS representation to its serialized JSON form of this type. By default, uses the runtime value.

docs/docs/defining-entities.md:1616the code disagrees

1616Converts a value from its JS representation to its serialized JSON form of this type. By default, converts to the database value.

packages/core/src/types/Type.ts:90-92: `toJSON(value) { return value; }`.

How it surfaced: extended run (more pairs), one check agreedUpstream: mikro-orm/mikro-orm #8617, merged

Whether a raw() fragment can be serialized in mikro-orm/mikro-orm

docs/docs/entity-manager.md:681the code disagrees

681It disallows serialization (via `toJSON`) as well as working with the value

docs/docs/raw-queries.md:7the code agrees

7This fragment is represented by `RawQueryFragment` class instance that can be serialized to a string, so it can be used both as an object value and key.

packages/core/src/utils/RawQueryFragment.ts:100-102: `toJSON()` returns the string `raw('...')`; only conversion to a non-string primitive throws (lines 86-94).

How it surfaced: extended run (more pairs), one check agreedUpstream: mikro-orm/mikro-orm #8617, merged

The action types createAsyncThunk dispatches in reduxjs/redux-toolkit

docs/introduction/getting-started.md:157the code agrees

157generates a thunk that dispatches `pending/fulfilled/rejected` action types based on that promise

packages/toolkit/README.md:78the code disagrees

78generates a thunk that dispatches `pending/resolved/rejected` action types based on that promise

packages/toolkit/src/createAsyncThunk.ts:544 builds `typePrefix + '/fulfilled'`.

How it surfaced: standard run, both checks agreedUpstream: reduxjs/redux-toolkit #5592, open

Whether Node.js supports ES modules in rollup/rollup

README.md:71newer wording

71Most browsers and Node.js support ES modules.

docs/introduction/index.md:52older wording

52The specification is now fixed, but it is only implemented in modern browsers and not finalised in Node.js.

No source file settles this; the README paragraph is the newer wording of the same paragraph and matches Node.js since 12.17.

How it surfaced: not listed by the standard run: /compare flagged it but labelled it a scope difference, which the listing rule dropsUpstream: rollup/rollup #6643, merged

The default of keepAliveTimeoutThreshold in nodejs/undici

docs/docs/api/Client.md:66the code agrees

66 * `keepAliveTimeoutThreshold` {number|null} A number of milliseconds67 subtracted from server *keep-alive* hints when overriding `keepAliveTimeout`,68 to account for timing inaccuracies caused by e.g. transport latency.69 **Default:** `2e3`.

docs/docs/api/H2CClient.md:80the code disagrees

80 * `keepAliveTimeoutThreshold` {number} A number of milliseconds subtracted from81 server _keep-alive_ hints when overriding `keepAliveTimeout`, to account for82 timing inaccuracies such as transport latency. **Default:** `1e3`.

lib/dispatcher/client.js:311 uses 2e3 when the option is unset, and H2CClient passes it through unchanged; #3145 changed it from 1e3. types/client.d.ts:61 and types/h2c-client.d.ts:43 also still said 1e3. The same PR fixes the H2CClient pipelining default in types/h2c-client.d.ts:47, found while checking.

How it surfaced: later run on 2026-11-09, outside the measured 15 projects (47 pairs read): /compare flagged it; the checks did not run because the run's credit cap was reachedUpstream: nodejs/undici #6182, open

Limits

Run it yourself, without a key

Every API answer behind this page is recorded in the repository, so a scan of a project at its pinned commit replays offline and spends no credits.

python scripts/fetch_targets.py fastify
python -m docsdrift scan ../targets/fastify --offline
python -m pytest -q