Skip to content
观澜涉川 · 札记中文

CI/CD Discipline in the AI Era: Closing the Judgment Gap for Cloud Engineers

· 9 min read

A note on authorship

Written by AI, directed by a human. The framework and the checkpoints are human; the generation is a black box.

The short version. The oldest advice in software is also the least followed. Jeff Atwood put it bluntly: the best code is no code at all. AI didn’t retire that advice — it made ignoring it expensive. When generating code becomes nearly free, the scarce skill is no longer production; it’s judgment — deciding which change is worth making, and which is worth not making. The distance between how fast you can produce changes and how well you can judge them is where production incidents are born. That distance is the judgment gap, and everything below is about closing it.

1. What AI actually changed

A two-line infrastructure change can quietly trigger a +- replace. A replaced database isn’t a refactor — it’s an outage. And the diff looks harmless, which is exactly why it gets through.

There’s nothing exotic about that. It’s the physics of DevOps, and it doesn’t care how quickly AI helped you type. Unverified changes break clusters. Noisy pull requests drain attention. Tribal knowledge turns a team into a room of people who each think they know how the system works — right up until the system disagrees.

Here’s what actually changed. We never needed more code — the industry’s oldest advice is to write less of it. What AI changed is the cost of ignoring that advice. Writing is now nearly free, so the difficulty stopped being production and became selection: which change is worth making, and which is worth leaving unwritten. You can generate a hundred plausible changes an hour and still have no idea which one will page you at 2 a.m.

That’s the gap. And you can’t close it with a better model.

There’s a harder question underneath. The engineers who build enterprise software increasingly lean on AI, and increasingly without the years of repetition that turn discipline into instinct. AI-generated code is inevitable; what’s open is whether our old disciplines should change to match it. I don’t think so. When generation happens inside a black box, they are the human checkpoints around it — holding the line on quality, cutting the chaos, and even making things faster.

2. End states and trajectories

AI has mostly seen end states — finished repos, published code, things as they ended up — and an end state is what it hands back: all four environments, generated in one breath, uniform and complete. Real environments don’t arrive that way. They accumulate: stand up dev and test, verify; promote the same thing to staging, watch it; only then let it near prod. Four environments, one destination — reached rung by rung, never all at once.

So AI spreads horizontally while engineering climbs vertically. A change moves up a ladder of rising risk — dev → test → staging → prod — where each rung is an exposure, a stretch of observation, a bet you can still walk back. The environments aren’t four copies of one thing; they’re four marks on one trajectory, and “promoting” a change is confidence accumulating, one level at a time.

That’s what makes replication a trap. Copy a mature setup into a new environment and it deploys in one move — repo ready, config ready, one click. But that end state grew out of a trajectory — retries, rollbacks, rotations, long quiet observation — and the trajectory is the part you can’t copy. You get the result without the evidence behind it, and lose the one thing that would tell you whether to promote it at all. AI can draw you the end state; it can’t draw the road to it — and the road is where the evidence lives.

3. What a widening gap actually costs

It shows up as three symptoms, and leaders usually treat them as separate problems. They’re not — they’re one problem wearing three coats.

  • Broken clusters. Unverified changes reach production because verifying them was slower than shipping them. The gap got impatient.
  • Drained attention. Big, noisy pull requests burn the one resource AI can’t multiply: a senior engineer’s focus — finite, expensive, and easy to waste on re-reading a sloppy diff.
  • Tribal knowledge. The system’s real behavior lives in someone’s head instead of in the repo. When that person is on holiday, so is your operational certainty.

Each symptom is a symptom. The disease is a process that lets changes travel faster than the team’s ability to judge them.

4. An operating model: five disciplines and two baselines

You can’t buy your way out of the judgment gap, and you won’t close it with a tool the vendor sold you. You close it with habits — unglamorous ones that were true before the cloud and stayed true through every hype cycle since.

There are five of them, plus two baselines that outrank the rest. Four disciplines move a change from idea to production, one shapes how code and config are organized, and the baselines aren’t a stage at all — they’re what everything above has to hold. Each is framed as the principle, the practice, and the failure it prevents.

The four process disciplines.

Discipline 1 — Keep change small and reversible. Principle: small is reversible; reversible is safe. Practice: short-lived branches off main, one task per PR, merge fast, delete the branch and start clean. Work trunk-based; never confuse “trunk-based” with committing straight to main. Prevents: the 800-line diff that fails on Tuesday and takes three days to untangle, because nobody can tell which line did it.

Discipline 2 — Design for the reader. Principle: a pull request is writing — the medium just happens to be code — and it’s read by one person on the worst day of their week: your reviewer. Practice: say what changed, why, and how you proved it; self-review before you ask anyone else to; treat review as shared knowledge, not a verdict on your worth. Prevents: reviews that stall, and teams that mistake “arguing about style” for “engineering.”

Discipline 3 — Verify in environments, not on laptops. Principle: your laptop and your pipeline are different machines wearing different badges — and only one of them is trusted by production. Practice: preview locally to your heart’s content, but run pulumi up only through CI/CD, where it’s logged and audited; attach the verification evidence to the PR. Prevents: permission mismatches, unaudited deployments, and the class of bug that exists only because “it worked on my machine.”

Discipline 4 — Treat production as adversarial. Principle: infrastructure code lies about its intentions. A one-line tweak can trigger a +- replace, and a replaced database is not a refactor — it’s an outage with paperwork. Practice: put the preview diff in the PR so a human can flinch before production does; in a GitOps world, commit or it didn’t happen; when your tool doesn’t lock a plan between preview and apply — Pulumi can, via experimental Update Plans — re-run the preview and trust the diff in front of you. Prevents: the end-of-week “surprise” that was in the diff all along, ignored.

The design discipline.

Discipline 5 — Duplicate to decouple. Principle: DRY is a reflex, and over-applied it becomes coupling — sharing code between things that were only ever going to change apart. Practice: give knowledge a single source (naming, structure, anything that must stay identical), but let intent that will diverge live on its own: put per-environment resource choices in each environment’s config, keep the resource shape in shared code, and never replace configuration with an if env == "prod" branch. Prevents: the “shared” template every environment gets patched through — where one change ripples across all of them.

The two baselines.

Discipline 6 — Own the outcome. Principle: accountability isn’t divisible. AI drafted it, CI passed it, a reviewer approved it — and none of that changes who shipped it. Practice: assume the parts the tools don’t cover; green CI is evidence, not absolution. Prevents: the diffused ownership that lets every incident become a blame-hunt instead of a fix.

Discipline 7 — Protect the keys. Principle: a leaked credential is a live liability until it’s rotated — not until it’s deleted, and a commit is no longer the only way it leaks — whether it lands on main, a short-lived feature branch, or in an AI’s context. Practice: ban secrets from Git in every branch, and scan at both ends — a local pre-commit hook (git-secrets, detect-secrets) plus platform scanning (GitHub secret scanning, TruffleHog, Aikido) that flags anything pushed to a remote branch. When one does leak, tell the team immediately and revoke and rotate it: deleting the branch or force-pushing a clean commit doesn’t erase it, because GitHub keeps stale commits for a while and the value lingers in PR history, timeline logs and audit records. Keep keys out of AI context too: assistants like Cursor and Claude Code pull workspace files, open context and terminal output into the model — .env, credential files, even a token you only exported — so “never committed” is not “never leaked.” Use tool-level ignore rules (e.g. .cursorignore) to shut .env and credentials out of the assistant’s reach, but treat them as best-effort rather than a guarantee; keep secrets in a manager (Vault, cloud KMS, 1Password) and inject them on demand; prefer short-lived, least-privilege credentials (OIDC) over long-lived keys; and never paste a production secret into a chat. Prevents: the false comfort of “we deleted the branch” — because you didn’t delete the secret, scanners already saw it, and the model may have too.

5. Diagnose your own gap

You don’t need a maturity model; you need five uncomfortable questions. If any of them stings, you’ve found where your gap is widest.

  1. Can a single small merge take down production — and would anyone notice it was coming from the diff?
  2. When a change is irreversible, does the team know before they merge, or after?
  3. Is your system’s real behavior documented in the repo, or memorized by one person?
  4. If a secret leaked an hour ago, does the team’s reflex say “delete the branch” or “rotate the key” — and would anyone check whether an AI assistant had already read it?
  5. When one environment’s needs change, do the others get dragged along — or can each move on its own?

6. What to do Monday morning

Nothing heroic. Small moves, on purpose.

  • Cap PR size, and make “small and reversible” a review criterion, not a preference.
  • Add a mandatory verification section to your PR template — evidence, not adjectives.
  • Move every deployment into the pipeline; leave pulumi preview as the only local comfort.
  • Put preview diffs in PRs and teach reviewers to hunt for +- replace.
  • Put per-environment choices in config, and keep only the invariant shape in shared code.
  • Wire secret scanning into both ends: pre-commit and the platform — and keep .env and credentials out of what your AI assistant can read.
  • Say out loud, once, that whoever merges owns the outcome.

The point

None of this is new. That’s not a bug in the advice; it’s the advice. The instinct when a shiny tool arrives is to assume the old rules retire. The opposite is usually true: when the easy part gets easier, the hard part gets more expensive. AI made writing code cheap. It left the price of a bad decision exactly where it found it.

Discipline isn’t the opposite of speed. It’s what makes speed survivable.

So move fast — in small steps, with an undo button, and never anywhere near the secrets.

References