Somewhere in the last couple of years I became the sort of person who gets genuinely excited about a well-behaved checkpoint file. This is not a personality trait I asked for. It arrived, uninvited, after one too many evenings spent doing the least glamorous job in security: sitting with a small mountain of evidence in one window and a control framework in the other, working out by hand whether document A satisfied requirement B or merely gestured politely in its general direction. Do that across a few hundred controls, each one written by a committee apparently paid by the ambiguity, and something in you quietly gives way. Usually the part that used to enjoy weekends.

This is the story of a side project I’ve been tinkering with, on and off, for the best part of a year: a tool I’ll call the Assurance Engine, because the real name is boring and also none of your business. It takes a big pile of evidence documents (policies, configs, screenshots, the sort of thing an auditor requests and you dread producing), points a language model at each requirement in a security control framework, and drafts an assessment of whether the evidence actually supports the claim. Think of it as a very diligent, slightly literal-minded graduate who has read every document you gave it and never gets bored, resentful, or distracted by their phone.

It sounds like a small thing. It wasn’t. It turned out to be an excellent way to spend a year accidentally relearning why all the “boring” engineering practices exist in the first place.

the bit where I have to defend the entire premise

Let’s get the obvious objection out of the way, because someone always raises it at this point in the conversation (thanks, Phil): “Couldn’t you just chuck the PDFs into a general-purpose AI chat tool and ask it questions?”

Yes. Obviously. You could also cut your own hair with kitchen scissors. It’s technically possible and someone, somewhere, has definitely done it successfully. The point isn’t whether the capability exists; every large model going can read a PDF and have an opinion about it. The point is what happens the other 99% of the time, when a stressed person is doing this at 4pm on a Friday with forty documents and no time to be careful.

What actually happens, in my experience, looks like this: someone emails a document to their personal account so they can open it at home. They upload it to whichever AI tool is free that week. They download the response, tidy it up in Word, and send it over chat to someone else for review. Multiply that by every control in a decent-sized framework, and you’ve got sensitive material fanned out across four devices, three cloud accounts, and at least one browser extension named something like ‘PDF Wizard Pro’ that nobody has ever properly vetted. Nobody did anything malicious. Nobody even did anything against policy, necessarily. It’s just what happens when a useful capability isn’t wrapped in a process.

That’s the actual product, if I’m honest with myself. It’s not “AI reads documents”; any of a dozen tools do that today. It’s “there is exactly one door in, one door out, and everything that happens to your evidence in between is logged, scoped, and cleaned up afterwards.” Unglamorous. Extremely necessary. Nobody puts “consistent, boring, well-audited data handling” on a conference slide, but it’s the whole reason the tool is worth building rather than just telling everyone to be more careful with ChatGPT.

what it’s actually good at (hint: not thinking)

I want to be careful here, because it would be very easy, and very dishonest, to write the next bit as “and then AI replaced the assessor, the end.” It didn’t, and if it had, I’d be worried about the assessor.

What the tool is really good at is the mundane part: reading forty pages of evidence to find the one paragraph that’s relevant to a specific control, cross-referencing it against the requirement, and drafting a first pass at “is this adequate, weak, or completely absent.” That’s real, thankless work: the kind that eats a junior assessor’s entire week and leaves them cross-eyed by Thursday. Handing that first pass to a human reviewer, who then spends their time on judgement calls rather than page-flipping, is a proper step up in how the work gets done.

What it is not good at, and what I had to build proper engineering around rather than just trusting the model to notice, is knowing when it has failed. Early on I had a demo (the sort you really don’t want to go wrong in front of an audience) where the assessment “completed successfully” and produced a report that was quietly, comprehensively empty. Every single evidence lookup had failed upstream (an infrastructure hiccup, unrelated to the model itself), and the batch processor, in a moment of what I can only call misplaced optimism, caught the error, wrote a placeholder answer of “processing error” into every single result, and reported the whole run as a success. Nobody lied. The code just wasn’t built to distinguish between “the AI concluded there’s no evidence” and “nothing actually got asked.” I had, in effect, taught a very confident intern to say “looks fine to me!” regardless of whether they’d read anything at all.

That one stung, and it taught me the real lesson of this whole project: the AI is the bit that’s allowed to be uncertain. The plumbing around it is not allowed to be uncertain. If zero requests succeed, the system must fail loudly, not produce a beautifully formatted document full of nothing.

the year I learned to love a checkpoint

The single most expensive lesson came from an assessment covering just over a thousand controls, running on infrastructure that treated the container’s local disk as, essentially, a sandcastle at high tide. Progress files, checkpoints, half-finished results: all of it lived in storage that could vanish the instant the underlying instance got recycled. One did. Roughly forty minutes and a properly uncomfortable amount of API spend disappeared with it, and the retry, because of course there was a retry, didn’t resume from where it left off. It started again from batch zero, quite happily paying for the same answers twice, because the checkpoint had never been written in a form that survived out-of-order completions.

Fixing it properly meant admitting that “call an AI model in a loop” is a distributed systems problem wearing a trench coat. It needed a checkpoint keyed by which specific batch had finished (not just a running count, because concurrent batches finish out of order and a naive counter will happily skip an unfinished one and re-run a finished one). It needed durable storage that didn’t evaporate when a container did. It needed a lease so that if a job died mid-run, its replacement could tell the difference between “the previous attempt is still alive, back off” and “the previous attempt is dead, take over now.” Get that wrong and you either get two workers cheerfully re-doing the same expensive work in parallel, or a run that sits at zero forever because everyone’s too polite to take the lease off a corpse. Digital manners, it turns out, are a remarkably efficient way to bankrupt yourself.

None of that is exciting. None of it will ever appear in a demo. All of it is the difference between a toy and a tool you’d trust with real money and real client data.

logs remember everything, whether you asked them to or not

The security lessons were, almost without exception, not about the AI at all. They were about everything wrapped around it, which is exactly where you’d expect the mistakes to hide.

The API key, for instance, started life being passed to a subprocess as a plain command-line argument, which on most systems means it’s sitting in the process list for any other process on the box to read. That got moved to a short-lived temporary file, written just before use and deleted in a finally block immediately after, regardless of whether the run succeeded or blew up. A small change, obvious in hindsight. The sort of thing you only notice once you’ve specifically gone looking for “where does this secret exist as plain text, and for how long.”

Error logging had its own version of the same problem: a failed API call is exactly the moment you most want a verbose log message, and exactly the moment you’re most likely to log something you shouldn’t: a full response body, an oversized payload, a chunk of evidence that got attached to the request. Every error path ended up with a hard character cap and explicit truncation, because “log everything for debugging” and “don’t accidentally persist someone’s sensitive document into a log aggregator” are in permanent tension, and the log always wins if you’re not paying attention.

And then there’s the boring-but-vital stuff: uploaded evidence lives in per-run directories with randomised names, gets deleted the moment the run finishes (success or failure), and result files get signed with an HMAC the moment they’re written, so a tampered report doesn’t quietly get served back to someone as if it were the original, the paperwork equivalent of swapping the exam answers after they’ve already been marked. Isolation between one client’s assessment and another’s isn’t a policy document; it’s a directory boundary, a unique ID, and a cleanup step that runs even when everything else goes wrong. That’s the actual “AI security” work, and approximately none of it involves the AI.

the tour through models, prompts, and an increasingly opinionated editor

Underneath all of this, the tooling I was using to build it kept changing under my feet, which is its own small comedy. I started stitching prompts together by hand, testing them against whatever model was current at the time, long before anything resembling an “agent” existed in my editor. Then the assistants got better at holding context across a whole file. Then across a whole project. Then they started proposing edits, running my tests, and occasionally getting slightly too enthusiastic about refactoring something I hadn’t asked it to touch (a habit I’ve had to explicitly tell it to knock off, repeatedly, in writing).

The shift that actually mattered wasn’t “the model got smarter,” although it did. It was the tooling around the model catching up to actually understanding a codebase rather than a single pasted snippet, the same theme as the whole project, really. The clever bit was never in short supply. The workflow around it always was.

so, was it worth it

Yes, unreservedly, though not for the reason I expected when I started. I thought I was building something to make AI answer audit questions. What I actually built was a small case study in how much unglamorous engineering it takes to make an exciting capability boring, reliable, and safe enough to hand to someone who isn’t going to read the source code before they trust it.

The AI was never really the hard part. The hard part was making sure that when it got something wrong, everyone would notice, and that when it got something right, nobody’s data had gone on an unplanned tour of four devices to get there.