Continuous validation — what stops the next release from shipping with a deviation

The three guards

Each answers a question an evaluator is entitled to ask, does one precise thing, and has already caught something. That last point is not decoration: a check that has never detected anything has not yet proven that it detects.

1

The software qualifies itself

“How do you know the version you deliver still does what your file says?”

A suite of operational qualification cases is replayed on every change to the code. Each case is tied to a validation requirement and to its regulatory clause. The last of them does not merely inspect the code: it STARTS the delivered image, on empty volumes, and reads its startup log.

What it caught — A deliverable image that did not start at all. Neither code checking, nor compilation, nor the other cases could see it: they all run outside the container. The customer would have found it at the first technical step of their installation.

The validation file is not written: it is PRODUCED from that run. The generator refuses to emit it if the suite has not run. One deviating case, and the file carries the deviation.

2

The documents re-read themselves

“How do you know the document you handed me still describes your software?”

Every published document is regenerated from its sources, then its TEXT is compared with the published version. Values specific to a run — dates, durations, digests — are neutralised before comparison: a check that cries every day ends up ignored, which is worse than no check.

What it caught — The English 21 CFR Part 11 matrix, downloadable from this site, announced “31 requirements, 28 cases” when the French version counted 32 and 29. At the time the check only regenerated the French documents: it declared the corpus up to date having verified only half of it.

A stale document fails the check, and the failure names the sentence that changed. Since then, a second check rejects any published document that no generator rewrites — the blind spot can no longer re-form silently.

3

The delivery says whether it is stale

“How do you know the archive you sent me matches this code?”

A check compares the published archive against the current code, reading the image's real build rules rather than restating them. It accounts for changes not yet committed: drift always begins with a modified file on a workstation.

What it caught — This check has not yet caught a defect at a customer's: it reports drift daily, on our side, as soon as the code moves past a delivery. That is its job, and it is why a version number already handed over is never republished — a new version is opened instead.

A record handed over is not rewritten. If the content must change, it changes version, and the previous one stays exactly as it was received.

They do not run on my machine

The three checks are replayed on every code push on a fresh machine that knows nothing of my workstation: dependencies reinstalled, image rebuilt from scratch, 22 generator invocations, then the 46 qualification cases — delivered image included. A change that broke one of them cannot go unnoticed, including by me.

It is also what makes the validation file usable: it is not written once and copied from release to release. It is produced from the most recent real run, with its figures, durations and digests. A hand-written figure always ends up lying — it happened, it is told above, and it is why every figure on this page is read from the sources.

What these checks do not do

These checks run at the vendor's, on the vendor's code and documents. They in no way replace the qualification YOU carry out in your environment: installation and performance are assessed at your site, and the installation protocol is handed to you for that.
They verify that the software does what the file describes, not that the file describes what you need. Fitness for your intended use remains your assessment, and nobody can carry it for you.
No penetration test by an independent third party has been conducted to date. The limitation is declared in the validation file, and it will stay there until a third party lifts it.