Protocol 16 of 18Track, Hardening and continuityHow it lasts

Harden the code

Working means it does the right thing today. Hardened means it keeps doing the right thing after the next change. The development protocol, the QA plan, unit tests versus the full suite versus a human pass, and a senior code review before anything emails a customer.

← All 18 protocols
Protocol 15Agents in the cloudAll 18Protocol 17Reflect and plan tomorrow with the PM agent
Why this matters

The pain it
solves

You built a complete system in a day. Before it starts emailing people on its own, you make it hard to break. Working means it does the right thing when you click it today. Hardened means it keeps doing the right thing after the next change, the next person, and the next thousand leads. The difference is a process, not more code.

AI will test the code is one of the most over-claimed promises in the current hype cycle. AI can test some things very well, cannot test other things at all, and lying to yourself about the difference will ship a bug your customers find before you do. This protocol draws the line and puts a process around it.

The teaching

What this
actually is

Working versus hardened

Working means it does the right thing when you click it today. Hardened means it keeps doing the right thing after the next change, the next person, and the next thousand leads. The difference is a process, not more code.

"AI will test the code" is one of the most over-claimed promises in the current hype cycle. AI can test some things very well, cannot test other things at all, and lying to yourself about the difference will ship a bug your customers find before you do. The rest of this protocol is the process that puts an honest line around that.

The development protocol

Every change, whether you make it or an agent does, follows the same six moves. Write it into docs/plans/qa-plan.md and point CLAUDE.md at it, and every future build follows it without you asking.

  • Plan, then branch
    A build plan with a check per step (Protocol 09). Then a branch named for the change. Nobody edits main.
  • Build, then preview
    Push the branch. Vercel gives it its own preview URL. That is where you look, not localhost. If it works on the preview, it works in production; the same build pipeline runs both.
  • QA, then merge
    Unit tests on the change. The end-to-end suite on the preview. A human pass on the journeys that changed. PASS or FAIL with evidence. Only then does the pull request merge. Nothing lands on production untested.

Three kinds of test, and which one when

Most testing thinking confuses these three. Once you separate them, every test you ever write fits cleanly into one bucket, and the QA plan is just deciding which bucket each journey goes in.

  • Unit tests
    One small piece in isolation: a price calculation, a date rule, a form validator, the rule that decides who gets enrolled. Deterministic. They run in milliseconds on every change and tell you exactly which function broke. AI writes them and runs them. Many of these.
  • The full automation suite
    End-to-end tests drive a real browser through your live site: open the page, submit the form, log in to /admin, check the row landed. Slower, but they catch what unit tests cannot: the wiring between pages, database and email. Run before every merge. A handful of these, on the journeys that make you money.
  • The human pass
    What only a human sees: does it read right, does it feel right, is the loading state confusing, is the copy in your voice. AI can spot a violation of a design system you defined; it cannot tell you the system is wrong. One pass per epic, five minutes, on your own phone.

The QA plan

One document, docs/plans/qa-plan.md, under two pages. It ranks the journeys that must never break by business risk (a lead lands and shows up in People, you log in, you move a contact through the pipeline, an order is recorded, the dashboard matches the database). For each journey it says what a unit test covers, what the suite covers, and what needs a human. It sets the cadence and the test-data rule: tests use synthetic rows only, and clean up after themselves.

CLAUDE.md gets a Testing section that points to it. From then on every agent reads the plan before it builds, and the QA agent tests against it after.

The review: Karpathy for the code, Bach for the tests

A senior engineer reads every file you shipped. You do not have one on staff, so you name one. The persona is not a gimmick; it tells the model which standards to hold and which to ignore.

  • Act as Andrej Karpathy
    First principles, ruthless about simplicity, allergic to code nobody needs. The five most important problems, ranked, each with the file, why it matters, and the fix. Dead code and duplication. Security: secrets reachable from the browser, admin routes without the login check. Speed: oversized images, JavaScript the page does not need. Written to docs/qa/code-review.md. Changes nothing yet.
  • Act as James Bach
    Testing is not checking. The suite checks what you already thought of; a tester goes looking for what you did not. Walk every critical journey on the preview at phone width and desktop width. Try the wrong input, the double submit, the back button, the expired session. List what overflows, what is too small to tap, what a real customer would trip on. Written to docs/qa/qa-review.md.
  • You decide, then the protocol runs
    Read the top five before you say fix. A review with forty items will not ship this morning; keep the security problems, the mobile breaks, and anything that hits a critical journey, and move the rest to a Later section. Then the fixes go through the development protocol: branch, preview, QA, merge. Open the production URL on your own phone.
Try it yourself90 minutes

Write the QA plan, get reviewed, fix the top five, in 90 minutes

You need a system you built and shipped (the CRM from Protocols 07 and 14 is ideal) with QA reviews in docs/qa. You will write the QA plan, get two reviews, and fix the top items on a branch.

  1. Step 01
    Write the QA plan

    Tell the QA agent: "Act as a senior QA lead. Read the product plan and every review in docs/qa. Write docs/plans/qa-plan.md: critical journeys ranked by risk, unit versus end-to-end versus human for each, the development protocol, the cadence, and the synthetic-only test-data rule. Add a Testing section to CLAUDE.md that points to it. Do not write tests yet."

  2. Step 02
    Get the code review

    "Act as Andrej Karpathy doing a code review of this repository. Write docs/qa/code-review.md: the five most important problems ranked, dead code, security, web and mobile at 375px and 1440px, speed. Change nothing yet." Read the top five.

  3. Step 03
    Get the QA review

    "Act as James Bach. Testing is not checking. Walk every critical journey in docs/plans/qa-plan.md on the live site at phone and desktop width, try what a real customer would do wrong, and write docs/qa/qa-review.md." Read what the suite would never have found.

  4. Step 04
    Cut the list, then fix on a branch

    Keep security, mobile breaks, and anything on a critical journey; move the rest to Later. "Fix the kept items on a branch named harden, most important first. Push, open the Vercel preview, follow the QA plan on the preview at both widths, print PASS or FAIL with evidence, fix the FAILs, then merge."

  5. Step 05
    Open it on your phone

    Production URL, your own phone. Submit the form, log in, scroll the dashboard. Nothing sideways, nothing too small to tap. Mark each review item fixed with one line on what changed.

Outcome

A QA plan every future build follows, two reviews worked through on a branch and merged through the protocol, and a site that works on your phone. Your system now has a way of staying right.

Official resources

Straight from
the source

Playwrightdoc
Playwright getting started
Satisficearticle
James Bach, testing and checking refined
Googlearticle
Google testing blog, the test pyramid
Vitestdoc
Vitest, fast unit tests
What you walk out with

By the end of this
protocol

At the retreat

You learn it by
doing it

You write the QA plan for your CRM, get a review from Karpathy and a QA pass from Bach, fix the top five on a branch, and open the production URL on your phone. Nothing sideways, nothing too small to tap.

Saigon, Jun 20See all 18 protocols
Connects to

Other protocols this
compounds with

Protocol 05
GitHub hygiene
Protocol 09
The plan is the prompt
Protocol 04
The build team
← Previous, Protocol 15
Agents in the cloud
Next, Protocol 17 →
Reflect and plan tomorrow with the PM agent