What a Two-Week AI Audit Actually Looks At

"Audit" is a word that covers a lot of ground, most of it vague. This is a specific description of what the AI Second Opinion examines, so you can judge in advance whether it would answer your question.

What Gets Looked At

The implementation itself. What was actually built, as opposed to what was described in the proposal. Which model or service it uses, what it was configured to do, what it does when it fails, and whether anyone has changed it since it went live. This is more often a surprise than it should be — the gap between the sold system and the running system tends to grow quietly.

The data underneath it. Where its inputs come from, how clean they are, and how they got that way. This means profiling the actual tables rather than reading a schema: how many duplicates, how many nulls in fields that matter, how consistent the categories are, and whether the historical data crosses any change in how it was collected.

The integration points. Where the output goes, how quickly it gets there, and who sees it. Whether it flows automatically or depends on somebody remembering to run something. What happens downstream when it produces nothing, or produces something wrong.

The measurement. Whether a baseline exists. If not, whether one can be reconstructed from history or from an untouched segment of the business. What the system costs fully loaded, not just in license fees.

The failure cases. A sample of outputs checked by hand against what actually happened. Not aggregate accuracy — specific cases, including the ones the people using it flagged as wrong, because the pattern in those is usually the finding.

The Order It Happens In

Measurement first, because it determines whether anything else can be evaluated. Then data, because bad data invalidates conclusions about everything downstream. Then implementation and integration, which are the cheapest to fix and therefore worth knowing about last, once you know whether fixing is worth doing.

Two weeks is enough for this on a single system. It is not enough to audit an entire data estate, and if what you actually need is the broader piece, the honest answer is that this is the wrong engagement and we will say so on the first call.

What the Verdict Can Tell You

The output is a written assessment plus a one-page recommendation, and it lands on one of three:

Go. It works, the measurement supports it, and the cost is justified. Sometimes the finding is simply that nobody had done the arithmetic and the arithmetic is fine.

Fix. The approach is sound but something specific underneath it is broken. The recommendation names what, in what order, and roughly what each piece involves.

Kill. The cost exceeds any plausible benefit, or the data cannot support what the system is being asked to do, and no reasonable amount of work changes that. This is a real possible outcome, in writing, with the reasoning attached.

What It Cannot Tell You

Worth being straight about the limits.

• It cannot tell you whether a different vendor would have done better. That is a counterfactual about a system that was never built.
• It cannot give you a precise return figure where no baseline exists and none can be reconstructed. In that case you get a range and an explicit statement of how confident it is, not a number dressed up as certainty.
• It cannot evaluate something that has been running for three weeks. There is not enough history to say anything meaningful, and the right advice is to wait.
• It is not a security or compliance review. Different discipline, different specialist.

Why It Is Fixed-Price

Because the scope is genuinely knowable in advance. One system, the data behind it, two weeks. Hourly billing on work like this creates an incentive to keep looking, and the thing you are buying is a decision, not a quantity of hours.

It also means the answer is not influenced by what happens next. The fee is the same whether the verdict is go, fix, or kill, so no version of the answer is worth more to us than another. That is what makes "kill it" a credible thing for us to write.