Chapter 04 of DeployLog's manual covers the widget. This is one of its sentences:
Ten or more unread entries render as
9+.
And this is the line in the widget's source that makes it true:
unreadCount > 9 ? '9+' : unreadCount
Change that 9 to 99 in a pull request and the sentence is wrong. The tests still pass, and the docs page goes on promising 9+ to every reader. Nothing connects the sentence to the line, so nothing can notice.
That is documentation drift. The usual defenses are habits, like a docs review step or a quarterly audit, and habits slip.
DeployLog is a changelog and release notes platform, and the Manual is its documentation feature, on every plan including Free. It drafts your product's manual, a set of numbered chapters, from your repository and records the connection between each sentence and the code. Then your code can contradict your docs, and a check can say so on the pull request that did it.
A claim is the connection
You give the Manual a brief, two or three sentences on what a chapter must explain, plus the files that hold the answers. It drafts the chapter from those files with Claude. For every sentence that states a value, it records a claim: the file, and the exact text in it that the value came from.
Claims are recorded while a chapter is drafted, so the check covers chapters the Manual drafted, not docs you wrote by hand. To bring an existing page under the check, turn it into a brief and list the code files it describes.
The 9+ sentence carries a claim that points into the widget's repository.
A claim turns a sentence into something a machine can check. While the recorded text is still in the file, the claim holds. When the text moves, that is drift.
What the checker reads
The checker matches text and never judges prose. On a pull request, it reads each cited file at the pull request's head commit and looks for the recorded value: a value on the line it was taken from, a string anywhere in the file. Because it matches text, a claim can cite a file in any language.
Only code and shipped files can back a claim. A claim that cites internal documentation is refused before any file is read, so one doc can never vouch for another.
Generation runs the same check on its own output. A draft with findings goes back to the model with those findings, up to three drafts in total. A chapter still failing on the third arrives marked for review, with its claims saved so you can see what failed.
Approving a chapter opens a pull request that adds it to your repository as a markdown file, and nothing lands in git until you merge it. Your repository stays canonical. A merged chapter can then be published to a hosted manual page. Here is one of DeployLog's own: chapter 06, opened by DeployLog's GitHub App and merged on August 25.
Five verdicts, kept apart
Every check gives each chapter one of five verdicts:
- A recorded value moved. That is drift.
- A file the chapter depends on changed, and everything checked still holds.
- Everything checked holds, and nothing it depends on moved.
- The check could not be made.
- The chapter anchors nothing at all.
Drift and "could not check" stay separate. A cited file that cannot be read is a failure to check, and it counts as neither drift nor a pass. A chapter with no claims carries a warning in the dashboard: "Nothing in this chapter is being checked".
Run it on every pull request
The GitHub Action has a verify mode. It needs no checkout step: it sends the commit and the list of changed files, and DeployLog reads the code through the GitHub App you connected. Store a DeployLog API key as the DEPLOYLOG_API_KEY repository secret; verify mode needs only read permission.
name: Manual check
on: pull_request
permissions:
contents: read
pull-requests: read # lets the Action see which files this pull request changes
jobs:
manual:
runs-on: ubuntu-latest
steps:
- uses: deploylogdev/action@v1
with:
api-key: ${{ secrets.DEPLOYLOG_API_KEY }}
project: my-app
mode: verify
github-token: ${{ github.token }}
fail-on: drift
With github-token set, the run checks only the claims whose cited file is in the pull request. A finding lands as an annotation on the changed line. A value that disappeared has no line to point at, so it goes to the job summary instead.
The fail-on input decides what turns the check red:
fail-on | The check fails when |
|---|---|
none (the default) | never on a finding: findings are annotated and the check stays green |
drift | a cited value moved |
any | a cited value moved, or the run could not vouch for the manual |
A run cannot vouch for the manual when it finds claims it could not read, chapters with no claims or too little coverage, or claims in a repository no workflow is watching.
From a terminal, the CLI runs the same check, and its exit code is the answer: 1 means a cited value moved, and 2 means the run could not vouch for the manual. That makes a pre-push hook one line:
deploylog manual verify || exit 1
What it found in DeployLog's own manual
DeployLog's manual has 14 chapters, drafted with this loop and first published on August 25. Its main repository runs the verify workflow on every pull request, on the default fail-on: none.
From launch on September 15 through October 9, that check ran on 88 pull requests. 83 reported no drift. Five reported drift. The check stayed green on all five, and all five merged the same day:
- September 18: three pull requests that reworked API key permissions, with one drifted claim each.
- September 26: a fix to dashes in AI-drafted changelog entries, with one drifted claim.
- September 26: a fix to how failed database reads are handled, with four drifted claims.
The API key change had made chapters 11 and 13 wrong. Both were regenerated and republished on September 18, and afterwards all 87 of their claims held.
A full check on October 10 read 456 claims across all 14 chapters, and four of them had drifted. In all four, the recorded line was gone from its file. Here is the one from chapter 07:
Sentence: What is stored is a usage record: which model ran, how many tokens
it used, and the first 200 characters of the generated title.
Recorded: preview: result.summary.title.slice(0, 200)
In code: preview: summary.title.slice(0, 200)
The sentence is still true, and the cap is still 200. A refactor renamed a variable, so the recorded text no longer appears, and text matching cannot tell a rename from a changed value. The check reports it, and a person reads the sentence and decides whether to regenerate the chapter, which replaces its body and records fresh claims.
What a clean check does not tell you
The checker compares recorded values with your code. It never compares a claim with the sentence it belongs to. Until September 18, chapter 11, which covers API keys, said "There is no permission picker." Its claim recorded a placeholder string from the API key form, e.g. Local CLI, and that string was still in the code after a picker for key permissions shipped. So the claim checked clean while the sentence was false, and regenerating the chapter is what fixed it.
A sentence with no claim is never checked either. Generation measures coverage, the share of a chapter's sentences stating a concrete value that carry a claim, and redrafts a chapter that falls below half. Verify mode lists those thin chapters by number on every run.
A run in one repository checks the claims that cite that repository. DeployLog's manual cites four repositories, so each of the four runs the check on its own pull requests.
Read the chapters
DeployLog is no longer under active development, and manual generation is off on deploylog.dev. The published chapters stay readable, and so does every claim they carry.
Read DeployLog's manual to see the chapters this post quotes, or start free with GitHub.
DeployLog is a changelog and release notes platform for developers. Publish from the CLI, a GitHub Action, or the dashboard, and keep a product manual checked against your code. Free for three projects, no credit card. Start free with GitHub