Nobody Wrote Down What Success Would Look Like

You approved a project to fix something costing you money, and months later it is delivered. Now answer one question in a meeting: did it do what it was meant to do? If the room produces opinions, not a test, the project was set up to end this way the day it was scoped.

This is a writing failure before code exists. Business intent arrives as an outcome — automate invoice processing, detect bad transactions — and each hides a dozen decisions changing the system's design, cost and scope. A systematic mapping study on requirements engineering for AI systems (Ahmad et al., 2022) locates the damaging gaps not in missing requirements but in ones clear in business terms and ambiguous in technical ones.

A specification can be satisfied in full and the work still fail, when nothing in it could be graded pass or fail by someone who had not helped write it.

What has to be written down instead is a target: the exact workflow, the data it runs on, and a pass condition a stranger could check. Onboarding a new organization onto HeartSciences' cloud ECG platform is a configuration task — its HL7 field mappings, the AI models enabled for it, its invoice pricing, its storage provisioning are each an entry in a configuration surface. ML LABS built that backend so that bringing on a new organization would be configuration, never code. Anyone can check the target: bring one organization on, and see whether an engineer has to touch the code to do it. The target held.

Traditional requirements assume deterministic behavior: given input X, produce output Y. AI systems are probabilistic: given input X, produce output Y with confidence Z, and sometimes Y is wrong. An analysis of AI project failures (RAND, 2024) attributes the majority of failed initiatives to misaligned requirements rather than to technical limits. Written in the deterministic shape, a requirement then encodes the wrong contract.

What Makes A Target Checkable

ML LABS writes acceptance targets into the statement of work before work starts, and the standard each target has to clear is published with the contract: a target is objectively checkable when it names a workflow, names the data, and states a pass condition a third party could verify. If a target cannot be written that way, it does not go into the SOW. That is a filter with teeth, because the same document makes a missed target — for reasons within our control — something the client can name in writing and hold ML LABS to.

Read those two clauses together and the usual incentive inverts. A vague target stops being a diplomatic convenience and becomes an unpriced liability, because a promise nobody can grade cannot be shown to be met either. A practice held to its own written targets cannot afford a soft one, and requirements discipline stops being a virtue and becomes arithmetic.

Four Ways A Target Fails

Four tests catch the softness before a line of code is written. Run them against the draft spec you have now — each maps to a piece of the definition above that is missing.

• Acceptance is qualitative. "The model should be accurate", "errors should be rare" — no metric, no sample, no threshold. If two reviewers cannot independently grade the same output pass or fail, there is no pass condition to accept against.
• The error budget is symmetric by default. A spec that never names the error the business prefers leaves the model optimizing whatever loss the framework defaults to.
• Inputs are named but not sourced. "Customer engagement" appears in the spec, and nobody can point at the table, the refresh cadence, or the join path that produces it at prediction time. Training-serving skew is born here, before any code exists.
• Nobody owns the unhappy path. Ask what happens when the model is wrong or when it is unavailable, and the answer is a shrug or a plan in the future tense. A system without a defined fallback does not have a pass condition for its worst hour.

What A Missed Target Costs

Acceptance runs against the written targets and nothing else: ML LABS shows that the targets are met, with the evidence the SOW named, and a rejection has to name the target it says was missed. The obligation to be precise is symmetric — a buyer who cannot name the target that was missed stands where a vendor who could not write one stands.

Misses from factors outside our control do not count against the targets, which is why each SOW lists its client dependencies: access, data, third-party systems. That list moves the assumptions a build would otherwise carry unspoken into the document that prices them.

Regulated work forces the sharpest version of the discipline. Where data may not leave the client's network, the build runs in their VPC and acceptance targets are defined over de-identified or synthetic data. Validation against real data follows acceptance, under the SOW agreement. Every pass condition must be provable on data the engineer may hold.

The same discipline let the cloud ECG backend target be stated that way. "Configuration, never code" is gradeable by a stranger holding a diff and one new organization. "Easy to onboard" is gradeable by nobody, yet would survive every review meeting.

First Steps

1. Grade your own spec. Take the requirement closest to shipping and try to write its pass condition: named workflow, named data, and a check a third party could run without you. Where the sentence will not come, you have found the work.
2. Name the error you prefer. Write down which failure the business absorbs and which it refuses. That sentence moves the model, the threshold and the review queue.
3. Write the unhappy path. State what the system does when it is wrong or unavailable, and who is notified. A requirement with no defined fallback is not finished.

Make Acceptance The First Artifact

Write acceptance before you write architecture. A target that survives being written down — named workflow, named data, a pass condition a stranger could check — survives contact with production, because it was never permitted to mean two things at once.

Targets that survive that test are what the work is scoped against. From there one workflow goes into production against those targets, and is kept running at $10,000 a month by the people who built it. What a testable target has to look like before anyone commits a build budget is the subject of the companion piece on what to demand before you sign.

References

1. Ahmad, K., Abdelrazek, M., Arora, C., Bano, M., & Grundy, J. A Systematic Mapping Study on Requirements Engineering for AI-Intensive Systems. arXiv, 2022.
2. RAND Corporation. Analysis of AI Project Failures. RAND Corporation, 2024.