AI-SDLC: A Process for Mission-Critical Software Engineering with AI

Introduction

In most software, a defect is an inconvenience. In mission-critical software, a defect can mean lost money, corrupted records, safety incidents or broken trust that never comes back. AI models can now write large amounts of code quickly, and that raises a hard question: how do we use that speed without giving up the rigor these systems demand?

The usual answers go to one of two extremes. One keeps a human reviewing every line the AI produces. This feels safe, but it's an illusion. Human reviewers make mistakes. They get tired, their attention drops over long sessions, and they tend to approve code that looks plausible, which is exactly what AI-generated code usually looks like. Line-by-line review was already imperfect when humans wrote code at human speed. Now that AI produces code faster than anyone can read it, it isn't practical at all. Either review becomes a bottleneck that throws away most of the value, or it becomes a rubber stamp that gives false confidence.

The other extreme treats AI as an autonomous engineer that gets a goal and delivers a system. That is fast, but it can't be accepted where failure is expensive.

Neither extreme works because both look for safety in the wrong place: in the reviewer or in the model. In mission-critical engineering, safety has to come from the process itself. That means clear ownership of decisions, explicit contracts, independent verification, and feedback loops that catch mistakes whoever made them. In such a process, human review is still valuable, but it's one fallible observation among many. It's used deliberately where the stakes justify it, not spread thinly across everything.

This essay describes that process. It rests on one idea and a small set of rules that follow from it.

The idea is this: a software project is a hierarchy of decisions, and every decision should be owned by a decision-maker whose capability matches the decision's risk.

1. Software as a Hierarchy of Decisions

Every software system is the sum of decisions. At the top are decisions about purpose: what problem we're solving, for whom, what we won't do, and which tradeoffs we accept. Below those are architectural decisions: how components divide the work, where data lives, which guarantees each boundary provides. Below those are design decisions inside components. At the bottom are execution decisions: this loop, this variable name, this error message.

It's tempting to map this hierarchy straight onto a hierarchy of intelligence. Humans own the top, frontier models own architecture and design, and cheap, fast models handle execution. Much of that intuition is right. Intent, values and accountability belong to humans. That isn't because models can't reason about them. It's because someone must answer for the outcome, and accountability can't be delegated to software. Decisions also really do differ in how much judgment they need, and cost and latency matter in inner loops that run thousands of times.

But in mission-critical work, the simple version of this mapping breaks down in a dangerous way. It assumes that low-level decisions are low-stakes.

2. Impact Radius, Not Altitude

An off-by-one error in a billing calculation is an execution decision. So is a missing lock that creates a race condition, an unescaped string that opens an injection hole, a timezone assumption in a scheduler, or an irreversible data migration. None of these sit high in the architectural hierarchy, and any of them can be catastrophic.

So where a decision sits in the hierarchy is the wrong thing to route on. The right measure is impact radius, the potential consequence of getting the decision wrong. Impact radius has several parts:

• Blast radius: how much of the system, and how many users, a mistake affects.
• Reversibility: whether a mistake can be undone cheaply, or whether it's a one-way door.
• Verifiability: whether an automated observation can reliably catch the mistake.
• Ambiguity: whether the specification is actually complete, or whether the executor will have to guess.

A decision that is contained, reversible, easily verified and unambiguous can go to the cheapest, fastest model available. A decision that is wide-reaching, irreversible, hard to verify or unclear deserves the most capable model, a human, or both, however “low-level” it looks.

The governing principle becomes: the harder a decision is to verify or reverse, the more capable its owner must be, wherever it sits in the stack.

3. The Handoff Contract

Once decisions are routed by impact radius, the next question is how work moves between levels. Every time work is delegated, whether human to model, model to model, or strong model to cheap model, it crosses a boundary, and context is lost there. The tacit understanding in the delegator's head gets compressed into instructions, and whatever doesn't survive that compression becomes a risk.

The answer is to make every handoff an explicit contract with the same structure at every level. A handoff states:

• Intent: why the work exists, not just what it is. An executor who understands the purpose can make sensible choices where the specification says nothing.
• Scope and impact radius: what may be touched and, just as important, what must not be.
• Constraints: interfaces, conventions, invariants and non-goals.
• Observations: exactly what will be observed and verified to show the work is done and correct.
• Escalation triggers: the conditions under which the executor must stop and return.

The contract runs in both directions. When the work comes back, the handback reports what was done, the evidence that each observation was satisfied, the assumptions made where the specification was silent, any deviations from the plan and the reasons for them, and any new risks or open questions found.

Because the same contract applies at every level, the whole hierarchy becomes one pattern repeated. A human hands off to a frontier model under this contract, and that model hands off to a cheaper model under the same contract. The process looks the same at every level.

4. Observation Is the Real Specification

The most important part of the contract is the observations. Here “test” should be read broadly: any observation that verifies behavior. That includes type checks, static analysis, unit and property-based tests, integration tests, benchmarks, logs, screenshots, migration dry-runs, formal proofs where they're practical, human review and production telemetry.

The rule is simple: whatever is handed over must come with a specification of what will be observed and verified. Work handed over without observations can't be meaningfully accepted, because there is no agreed way to tell success from something that only looks like success.

In mission-critical systems, observation must follow three further rules.

First, the implementer must not control the observations. Any optimizer, human or machine, will satisfy whatever it's measured against, sometimes by the shortest path rather than the intended one. An AI executor under pressure to pass tests may weaken assertions, special-case inputs or mock away the behavior under test. So tests, acceptance criteria and invariants are protected artifacts. They are defined by the delegator or an independent party, and any change to them triggers an escalation. Separating implementation from verification is one of the oldest principles in safety-critical engineering, and it matters more, not less, when the implementer is a model.

Second, the rigor of observation scales with impact radius. A small, reversible change might only need its tests to pass. A change that affects money, safety, security or persistent data needs several independent observations: tests, a dry-run, a second model's adversarial review and human sign-off. Defense in depth applies to verification as much as to security.

Third, verification doesn't end at merge. Some behaviors can only be observed in production: under real load, real data and real users. For high-radius changes, the contract should include how the change will be observed after release, through canaries, metrics, alerts and staged rollouts, along with the rollback plan if those observations fail.

5. Challenge and Escalation

A contract is only as good as its specification, and specifications are always incomplete. So the second key rule is: whoever receives a handoff must challenge any ambiguity.

That turns delegation from a one-way command into a conversation. Before execution begins, the receiver checks the contract for gaps and conflicts: undefined edge cases, contradictory constraints, observations that don't actually verify the intent. This pre-flight challenge catches the ambiguity that is visible up front.

Not all ambiguity is visible up front, though. Some of it only appears mid-execution, when a library turns out not to support an assumed feature or a data model can't express a real-world case. That's why the contract includes escalation triggers, and why those triggers shouldn't depend only on the executor's self-awareness. Knowing when you're out of your depth is itself a sophisticated skill, and weaker models are often the worst at it. They tend to confidently work around a problem instead of reporting that the plan is broken. So escalation should also be enforced from outside: repeated verification failures, edits outside the declared scope, changes to protected areas such as authentication, cryptography or schema migrations, or a diff growing well beyond its expected size. Each of these should automatically return the work to a higher tier.

Challenge also needs calibration, because too much of it is its own failure. An executor that raises fifteen questions about trivia uses up the scarcest resource in the system, which is human attention. The calibration comes from impact radius again:

• High-radius ambiguity: stop and ask.
• Low-radius ambiguity: make a reasonable assumption, record it explicitly in the handback, and proceed.

The “assume and record” path keeps work moving while making sure no assumption stays hidden. Every guess becomes visible, reviewable and correctable.

6. Feedback Flows Upward

Traditional hierarchies treat execution as passive: decisions flow down and results flow up. Real software doesn't work that way. Implementation is where many design flaws are discovered, because it's where abstractions first meet reality.

So the ideal process treats upward feedback as a first-class channel. When execution shows that a higher-level decision was wrong, the right response is not a clever workaround at the bottom of the stack. It is an escalation that lets the owner of the flawed decision revisit it. The structured handback carries this information: deviations, assumptions and newly found risks are exactly the signals the planning tier needs.

A system where problems found at the bottom can overturn plans made at the top is not a sign of disorder. It is what makes the system self-correcting.

7. External Memory: Making Each Handoff Easier

Every handoff pays a cost in context. The final part of the process lowers that cost over time: the knowledge produced by the work is kept in external memory, so each later handoff starts from a richer shared understanding.

Memory should mirror the decision hierarchy it serves:

Layer	Content	Who writes it
Intent and principles	Goals, non-goals, values, tradeoff preferences	Humans
Decision records	What was decided, why, and which alternatives were rejected	Humans and strong models
Conventions	Interfaces, patterns, style, domain vocabulary	Strong models, human-approved
Pitfalls and learnings	What broke, why, and how escalations were resolved	Any tier proposes; curated
Task logs	Handoffs, handbacks, assumptions, evidence	Automatic

Two parts of this are especially valuable: the reasoning behind decisions and the alternatives that were rejected. Without them, future agents and future engineers will reopen settled questions or quietly undo deliberate choices.

External memory also closes the feedback loop. Every escalation should end with a memory update. If the same ambiguity keeps being escalated, the memory or the contract template is missing something, and fixing it once prevents the same question from coming back.

Memory needs governance, though, because it degrades in predictable ways. It goes stale when the code moves on and the records don't. It gets contaminated when a model records a wrong conclusion that every later handoff inherits. It bloats until it's noise that crowds out useful context. The fix is to apply the process to itself. The right to write to memory follows impact radius: executors may propose entries, but important ones need approval from a higher tier or a human. Entries are linked to code and dates so staleness can be detected, and memory is curated and summarized regularly, not just appended to.

8. Where Humans Belong

In this process, humans aren't reviewers of every line, and they aren't just the source of initial goals. They hold three roles no model can take over.

They own intent: what the system is for and which tradeoffs are acceptable. They own acceptance: the final judgment that the evidence justifies shipping. And they own accountability: when something fails, a person answers for it.

Humans also go directly into the lowest levels wherever impact radius requires it: security-critical code, public interfaces, schema changes and anything irreversible. At the same time, humans should stay open to capable models proposing better designs than their own. The human's job is not to make every decision. It is to set goals and constraints, and to accept or reject based on evidence.

9. Measuring What Matters

A process like this can fool itself if it measures the wrong things. Token cost is the most tempting wrong metric. A cheap model that needs five attempts, adds subtle defects and doubles review time is not cheap. The right measures are end-to-end: cost per accepted change, defect escape rate, human review time and time to detect and recover from failures. Human attention is the most expensive resource in the system. Any routing decision that saves compute but increases review load is a false economy.

These measures should also guide how the process changes. Model capabilities shift quickly, and today's frontier capability becomes tomorrow's commodity. The routing layer that assigns work to tiers should stay thin and easy to reconfigure, and it should be revisited as the metrics shift, not frozen into an elaborate structure built around this year's capability gaps.

10. What This Process Doesn't Promise

Honesty about limits is part of mission-critical engineering. This process greatly reduces context loss, but it doesn't eliminate it. Unknown unknowns, gaps that neither side realizes exist, will still get through the most careful contracts. Observations can be incomplete, and memory can be wrong. That's why the process relies on independent verification, staged rollouts, production observability and rollback plans. It assumes it will sometimes fail, and it is built to detect failures early and recover quickly.

Conclusion

The ideal process for mission-critical software engineering with AI doesn't depend on a smarter model or a cleverer prompt. It is a system of accountability built from a few rules applied the same way at every level:

1. Route every decision by impact radius: its blast radius, reversibility, verifiability and ambiguity, not by where it sits in the stack.
2. Hand off work as a contract of intent, scope, constraints, observations and escalation triggers, and hand it back with evidence and recorded assumptions.
3. Treat observations as the real specification, keep them independent of the implementer, and scale them with risk.
4. Challenge ambiguity in proportion to impact: ask when the stakes are high, assume and record when they are low.
5. Let feedback flow upward, so problems found in execution can overturn flawed plans.
6. Keep knowledge in governed external memory, so each handoff is easier than the one before.
7. Keep humans accountable for intent, acceptance and the decisions that can't be undone.

AI changes how much software engineering can be done and how fast. It doesn't change what mission-critical engineering has always required: clear ownership, independent verification, honest communication about uncertainty, and a discipline of learning from every surprise. The models are new, but the principles are old, and the ideal process is one that applies them consistently at machine speed.