Section 06

Code Is Free / Generate Variations

Set up a controlled variation run before any candidate implementation starts. Build the files, folders, isolation lanes, verification ledger, review surfaces, and saved-decision...


setup agent prompt
Set up a controlled variation run before any candidate implementation starts. Build the files, folders, isolation lanes, verification ledger, review surfaces, and saved-decision template first, then stop and report whether setup is ready.

1. Work from the repository root. Choose a short run id that names the software decision, such as `billing-webhook-$(date +%Y%m%d-%H%M%S)`, and set `ROOT="artifacts/variations/$RUN_ID"`.

2. Create the run structure: `prompts/`, `launch/`, `logs/`, `review/`, `decisions/`, and `candidates/v1`, `candidates/v2`, `candidates/v3` under `$ROOT`. If the variation is visual, also create `review/screenshots/`.

3. Write `$ROOT/matrix.md` before launching anything. Include Goal, Invariants, Variable Under Test, Candidate Count, Candidates, and Review Criteria. The matrix must name exactly one variable. Keep repo state, base prompt, target feature or bug, constraints, verification command, and review criteria identical for every candidate. If a second dimension matters, record it as a future round.

4. Save the shared base prompt to `$ROOT/prompts/base.prompt.md`. Save matching launch metadata to `$ROOT/launch/base.launch.json` using Prompt Builder when available, for example `p --print-prompt "$PROMPT"` and `p --print-launch-json "$PROMPT"`. If Prompt Builder is unavailable, still write the base prompt file and create a small valid launch JSON file that records the prompt text, run id, candidate ids, artifact root, isolation method, and verification command.

5. Capture baseline repo state before fan-out: save `git status --short` to `$ROOT/review/git-status-before.txt`, `git diff` to `$ROOT/review/dirty-before.patch`, and `git diff --staged` to `$ROOT/review/staged-before.patch`. If the repo has unknown dirty state, stop and report it instead of creating candidate code lanes.

6. Choose the isolation strategy and record it. For prompt, plan, or docs-only variations, use `$ROOT/candidates/v1`, `v2`, and `v3`. For parallel code edits, prefer git worktrees at `../variation-$RUN_ID/v1`, `v2`, and `v3`, each on its own branch, then save `git worktree list` to `$ROOT/review/worktrees.txt`. Each candidate must have a distinct branch, worktree, or directory.

7. Create `$ROOT/prompts/v1.prompt.md`, `$ROOT/prompts/v2.prompt.md`, and `$ROOT/prompts/v3.prompt.md` from the same base prompt. Append only the candidate override: candidate id, variable name, that candidate's setting, and the instruction to change no other dimension.

8. Define one verification command and save it in the matrix plus `$ROOT/review/verification-command.txt`. Create `$ROOT/review/verification.tsv` with columns for candidate id, status, and command. Do not run implementation verification yet. The command must be identical for every candidate; if a candidate needs different proof, make that a new variation round.

9. Create review artifacts before generation: `$ROOT/review/comparison.md` with Goal, Variable Under Test, Verification, Diff Review, Review Criteria, and Notes sections; placeholder paths for `logs/v1.verify.log`, `logs/v2.verify.log`, `logs/v3.verify.log`; placeholder paths for `review/v1.diffstat.txt`, `review/v1.diff.patch`, and matching v2/v3 files. Leave results blank until candidates exist.

10. Create `$ROOT/decisions/selection.md` with fields for outcome, primary candidate, secondary candidate ids, decision, why, rejected candidates, required follow-up, final verification command, per-candidate notes, and a continue prompt. The outcome must later be exactly one of Pick, Combine, Refine, Expand, or Reject.

11. Run setup checks only: confirm the matrix, base prompt, launch JSON, candidate prompts, repo-state files, verification command file, comparison template, decision template, and isolation lanes exist. Print OK or FAIL for each check and show the run file list with `find "$ROOT" -maxdepth 3 -type f | sort`.

12. Respect privacy boundaries. Do not copy secrets, tokens, cookies, private endpoints, full environment dumps, raw private config, or unrelated home-directory files into prompts, logs, launch JSON, matrix files, review artifacts, or decisions. Summarize sensitive configuration instead of pasting it.

13. Report back with the artifact root, candidate ids, chosen isolation strategy, single variable under test, single verification command, repo-state classification, setup check results, any files you intentionally did not copy for privacy, and any blocker. Do not generate candidate implementations, run candidate verification, fill comparison results, or make a Pick/Combine/Refine/Expand/Reject decision until setup verification passes and the next step is explicitly requested.

Workflow route

Code Is Free / Generate Variations

Set up a controlled variation run before any candidate implementation starts. Build the files, folders, isolation lanes, verification ledger, review surfaces, and saved-decision...

Eggo holds one variation rail steady while rejects stay isolated for the Code Is Free / Generate Variations workflow, making the page's operating move visible.
Intent

Name the software-workflow decision before launching work.

Evidence

Inspect the file, command, log, prompt, or recording that proves the state.

Route

Pick the narrow agent, tool, shell, or council path that matches the problem.

Verify

Confirm the result with output, screenshots, receipts, or saved decisions.

  1. Big Idea
  2. Try This Now
  3. When To Reach For This
  4. How This Used To Work
  5. Core Pattern
  6. One Dimension Rule
the chosen candidate remains aligned with the verification card

Big Idea

Phase: Generation And Evidence - use the prepared surface to generate comparable options and preserve the proof that decides between them.

Code is free does not mean quality is free.

Try This Now

Finish this sentence before generating candidates: We are comparing candidates that differ only by __________. For a billing webhook bug, try: idempotency strategy. Write the same verification command before any fan-out.

It means another candidate is cheap enough that the workflow should change:

snippet-1
do not bet everything on the first generated answer

Produce candidates, verify them, compare them, converge deliberately, and save the decision.

Cheap:

Still expensive:

Practical rule:

snippet-2
Generate more because comparison improves the decision, not because you are avoiding the decision.

When To Reach For This

Use variations when several plausible approaches exist and comparison will make the final choice better.

Use Variations ForExample
UI integration strategysaved payment methods in checkout: inline selector vs drawer vs review-step picker
idempotency strategybilling webhook duplicate-processing: provider event guard vs invoice lock vs processing ledger
auth/session architecturemiddleware-first vs session-store-first vs token-rotation boundary
state managementlocal reducer vs server state cache vs URL-backed state
test boundaryunit regression vs integration replay vs end-to-end checkout path
API shapecommand endpoint vs resource endpoint vs explicit workflow endpoint
data loading modelroute loader vs component query vs server action
implementation alternativesminimal patch vs extracted helper vs explicit state machine
prompt framingplan-first vs patch-first vs test-first
documentation organizationquickstart-first vs concept-first vs task checklist

Skip variations when another candidate adds noise.

Skip Variations ForBetter Move
one-line typo fixpatch it and test it
obvious hotfixapply the smallest safe change
strict migrationfollow the migration plan exactly
exact requested implementationbuild what was specified
unverifiable candidatesdefine verification first
dirty repo with no saved statecheckpoint before fan-out
failing test names the exact bugfix the bug first
candidates would share unsafe stateisolate before launching

Hard rule:

snippet-3
Variation is a decision tool, not a procrastination tool.

How This Used To Work

What you used to expect:

snippet-4
Ask for one implementation, inspect it, tweak it, and hope the first path was good enough.

What the workflow gives you now:

snippet-5
Start from one deliberate prompt, generate controlled candidates, verify each
the same way, compare results, then save the chosen path.

What you used to expect:

snippet-6
More candidates automatically means better work.

What the workflow gives you now:

snippet-7
Unverified candidates are just noise. The value comes from controlled
differences, identical verification, and a saved decision.

Core Pattern

snippet-8
goal
-> variable
-> matrix
-> isolation
-> fan-out
-> verify
-> review
-> selection
-> convergence
-> saved decision

Do not skip the middle:

snippet-9
fan-out without isolation = collisions
fan-out without verification = noise
fan-out without review = drift
fan-out without saved decision = repeated debate

One Dimension Rule

Vary exactly one dimension across candidates.

Finish this sentence before launching anything:

snippet-10
We are comparing candidates that differ only by __________.

Good variables:

VariableGood Comparison Question
UI integration strategyWhere should saved payment methods enter checkout?
idempotency strategyWhich duplicate-processing guard is safest for billing webhooks?
state managementWhich state boundary keeps the checkout flow simplest?
test boundaryWhich test layer catches this regression with the least brittleness?
API shapeWhich endpoint shape keeps auth/session changes clear?
data loading modelWhere should this screen load account and payment data?
prompt framingWhich prompt produces the most verifiable implementation plan?
documentation organizationWhich doc structure helps the next maintainer act fastest?

Bad variables:

Bad VariableWhy It Fails
random ideasno controlled comparison
each one differentimpossible to know what mattered
explore everythingno review question
make five better versionsvague and untestable
let every agent decidecandidates drift apart

If two dimensions matter, run two rounds.

snippet-11
Round 1: compare idempotency strategy.
Round 2: compare test boundary for the selected strategy.

Variation Matrix

Write the matrix before launching candidates.

markdown
# Variation Matrix

## Goal

<What software outcome are we trying to achieve?>

## Invariants

These stay the same for every candidate:

- same repo state:
- same base prompt:
- same target feature or bug:
- same constraints:
- same verification command:
- same review criteria:

## Variable Under Test

- variable:
- why this matters:
- what is out of scope:

## Candidate Count

- count:
- ids: v1, v2, v3

## Candidates

| ID | Variable setting | Output location | Verification command | Notes |
| --- | --- | --- | --- | --- |
| v1 | | | | |
| v2 | | | | |
| v3 | | | | |

## Review Criteria

Compare candidates on:

1. passes verification
2. preserves invariants
3. smallest durable change
4. easiest integration path
5. clearest rollback path

Example: Billing Webhook Duplicate Processing

markdown
# Variation Matrix

## Goal

Prevent duplicate processing of paid billing webhook events.

## Invariants

- Provider webhook contract stays fixed.
- Database schema stays fixed for this round.
- Existing invoice state transitions stay compatible.
- Same verification command: `pnpm test billing-webhook -- --runInBand && pnpm build`.

## Variable Under Test

Idempotency strategy.

## Candidates

| ID | Variable setting | Output location | Verification command |
| --- | --- | --- | --- |
| v1 | provider event id guard | `../variation-<run-id>/v1` | `pnpm test billing-webhook -- --runInBand && pnpm build` |
| v2 | invoice-level processing lock | `../variation-<run-id>/v2` | `pnpm test billing-webhook -- --runInBand && pnpm build` |
| v3 | webhook processing ledger | `../variation-<run-id>/v3` | `pnpm test billing-webhook -- --runInBand && pnpm build` |

## Review Criteria

Which strategy prevents duplicate invoice processing with the smallest durable change?

Start From The Section 5 Seed

Use Prompt Builder to save the base prompt and launch JSON under the variation run.

bash
RUN_ID="billing-webhook-$(date +%Y%m%d-%H%M%S)"
ROOT="artifacts/variations/$RUN_ID"
mkdir -p "$ROOT"/{prompts,launch,logs,review,decisions}
mkdir -p "$ROOT"/candidates/{v1,v2,v3}

PROMPT='Generate three controlled fixes for duplicate billing webhook processing. Keep the provider contract and database schema fixed. Vary only the idempotency strategy. Each option must name files, tests, risks, and rollback notes.'

p --print-prompt "$PROMPT" \
  > "$ROOT/prompts/base.prompt.md"

p --print-launch-json "$PROMPT" \
  > "$ROOT/launch/base.launch.json"

test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"
jq '.mode, .prompt' "$ROOT/launch/base.launch.json"

Save the matrix next to the seed:

bash
cat > "$ROOT/matrix.md" <<'EOF'
# Variation Matrix

## Goal

Prevent duplicate processing of paid billing webhook events.

## Invariants

- Provider webhook contract stays fixed.
- Database schema stays fixed.
- Existing invoice state transitions stay compatible.
- Same verification command: `pnpm test billing-webhook -- --runInBand && pnpm build`.

## Variable Under Test

Idempotency strategy.

## Candidate Count

- count: 3
- ids: v1, v2, v3

## Candidates

| ID | Variable setting | Output location | Verification command | Notes |
| --- | --- | --- | --- | --- |
| v1 | provider event id guard | `../variation-<run-id>/v1` | `pnpm test billing-webhook -- --runInBand && pnpm build` | |
| v2 | invoice-level processing lock | `../variation-<run-id>/v2` | `pnpm test billing-webhook -- --runInBand && pnpm build` | |
| v3 | webhook processing ledger | `../variation-<run-id>/v3` | `pnpm test billing-webhook -- --runInBand && pnpm build` | |

## Review Criteria

- prevents duplicate processing
- preserves provider contract
- keeps rollback simple
- passes the same verification command
EOF

Confirm the setup:

bash
find "$ROOT" -maxdepth 3 -type d | sort
test -s "$ROOT/matrix.md" && echo "matrix:OK"
test -s "$ROOT/prompts/base.prompt.md" && echo "base-prompt:OK"
test -s "$ROOT/launch/base.launch.json" && echo "launch-json:OK"

Save Repo State Before Fan-Out

Before parallel code edits, save the current state.

bash
git status --short | tee "$ROOT/review/git-status-before.txt"
git diff > "$ROOT/review/dirty-before.patch"
git diff --staged > "$ROOT/review/staged-before.patch"

If the repo is dirty, choose one:

StateSafe Next Move
intentional local workcommit or stash before fan-out
unrelated local workmove it out of the candidate path
generated files onlyclean or save them under artifacts/
unsurestop and inspect before launching candidates

Do not fan out code edits from unknown dirty state.

Choose Isolation

Parallel candidates must not overwrite each other.

Isolation OptionUse WhenExample
copied prompt/output folderscandidates are plans, docs, prompts, or generated artifactsartifacts/variations/<run-id>/candidates/v1
branchedits are small and reviewed one at a timegit switch -c variation/<run-id>-v1
git worktreecandidates need parallel repo editsgit worktree add ../variation-<run-id>/v1 -b variation/<run-id>-v1
directory copyno git worktree support, but file edits must be separatedcp -R app app-v1
feature flagcandidates can coexist in one app behind flagsCHECKOUT_VARIANT=v1
unique process statecandidates run servicesPORT=3101, PORT=3102, PORT=3103

For code edits, prefer worktrees:

bash
BASE="$(git rev-parse --show-toplevel)"
PARENT="$(dirname "$BASE")/variation-$RUN_ID"

git -C "$BASE" worktree add "$PARENT/v1" -b "variation/$RUN_ID-v1"
git -C "$BASE" worktree add "$PARENT/v2" -b "variation/$RUN_ID-v2"
git -C "$BASE" worktree add "$PARENT/v3" -b "variation/$RUN_ID-v3"

git -C "$BASE" worktree list | tee "$ROOT/review/worktrees.txt"

Build Candidate Prompts

Start with the same base prompt. Add only the candidate setting.

bash
for id in v1 v2 v3; do
  cp "$ROOT/prompts/base.prompt.md" "$ROOT/prompts/$id.prompt.md"
done

cat >> "$ROOT/prompts/v1.prompt.md" <<'EOF'
Candidate override:
- candidate id: v1
- variable: idempotency strategy
- setting: provider event id guard
- do not change any other dimension
EOF

cat >> "$ROOT/prompts/v2.prompt.md" <<'EOF'
Candidate override:
- candidate id: v2
- variable: idempotency strategy
- setting: invoice-level processing lock
- do not change any other dimension
EOF

cat >> "$ROOT/prompts/v3.prompt.md" <<'EOF'
Candidate override:
- candidate id: v3
- variable: idempotency strategy
- setting: webhook processing ledger
- do not change any other dimension
EOF

for id in v1 v2 v3; do
  test -s "$ROOT/prompts/$id.prompt.md" && echo "$id prompt:OK"
done

For checkout UI integration, the same shape works:

snippet-20
variable: UI integration strategy
v1 setting: inline payment method selector in checkout form
v2 setting: account drawer payment method selector
v3 setting: review-step payment method selector

Keep database contracts, provider APIs, security constraints, and verification fixed unless those are the chosen variable.

Fan-Out

Use the same launch shape for every candidate.

Local background shape:

bash
GEN="codex"

for id in v1 v2 v3; do
  (
    cd "$PARENT/$id"
    "$GEN" "$(cat "$ROOT/prompts/$id.prompt.md")"
  ) > "$ROOT/logs/$id.generate.log" 2>&1 &
done

wait

Visible pane shape:

bash
cmux-run-surface "cd '$PARENT/v1' && codex \"$(cat '$ROOT/prompts/v1.prompt.md')\""
cmux-run-surface "cd '$PARENT/v2' && codex \"$(cat '$ROOT/prompts/v2.prompt.md')\""
cmux-run-surface "cd '$PARENT/v3' && codex \"$(cat '$ROOT/prompts/v3.prompt.md')\""

What to preserve:

snippet-23
same base prompt
same invariants
same launch shape
one isolated lane per candidate
one log per candidate

Verify Every Candidate

Run the same verification command for every candidate.

bash
VERIFY='pnpm test billing-webhook -- --runInBand && pnpm build'
: > "$ROOT/review/verification.tsv"

for id in v1 v2 v3; do
  (
    cd "$PARENT/$id"
    bash -lc "$VERIFY"
  ) > "$ROOT/logs/$id.verify.log" 2>&1
  status=$?
  printf '%s\t%s\t%s\n' "$id" "$status" "$VERIFY" \
    >> "$ROOT/review/verification.tsv"
done

column -t -s $'\t' "$ROOT/review/verification.tsv"

Save diff material for review:

bash
for id in v1 v2 v3; do
  git -C "$PARENT/$id" diff --stat \
    > "$ROOT/review/$id.diffstat.txt"
  git -C "$PARENT/$id" diff \
    > "$ROOT/review/$id.diff.patch"
done

For UI candidates, also save screenshots:

bash
mkdir -p "$ROOT/review/screenshots"
# Save:
# "$ROOT/review/screenshots/v1.png"
# "$ROOT/review/screenshots/v2.png"
# "$ROOT/review/screenshots/v3.png"

Review Surface

Use the lightest surface that makes comparison honest.

SurfaceUse WhenSave
simple matrixtext, plans, test results, diffsreview/comparison.md
diff reviewcode changes matter mostreview/v*.diff.patch
screenshotsUI integration or visual changesreview/screenshots/
test resultscorrectness is the main gatelogs/*.verify.log, review/verification.tsv
structured feedbackchoice needs to be preserved exactlydecisions/selection.md or saved JSON

Create comparison notes:

bash
cat > "$ROOT/review/comparison.md" <<EOF
# Variation Comparison

## Goal

Prevent duplicate processing of paid billing webhook events.

## Variable Under Test

Idempotency strategy.

## Verification

| Candidate | Status | Log |
| --- | --- | --- |
| v1 | see \`verification.tsv\` | \`$ROOT/logs/v1.verify.log\` |
| v2 | see \`verification.tsv\` | \`$ROOT/logs/v2.verify.log\` |
| v3 | see \`verification.tsv\` | \`$ROOT/logs/v3.verify.log\` |

## Diff Review

| Candidate | Diff Stat | Full Diff |
| --- | --- | --- |
| v1 | \`$ROOT/review/v1.diffstat.txt\` | \`$ROOT/review/v1.diff.patch\` |
| v2 | \`$ROOT/review/v2.diffstat.txt\` | \`$ROOT/review/v2.diff.patch\` |
| v3 | \`$ROOT/review/v3.diffstat.txt\` | \`$ROOT/review/v3.diff.patch\` |

## Review Criteria

- prevents duplicate processing
- preserves provider contract
- keeps rollback simple
- passes the same verification command
- minimizes unnecessary code movement

## Notes

| Candidate | Strengths | Risks | Keep / Drop |
| --- | --- | --- | --- |
| v1 | | | |
| v2 | | | |
| v3 | | | |
EOF

Decision Outcomes

End every review with one action.

OutcomeMeaningNext Step
Pickchoose one candidate as-ispolish and verify selected candidate
Combineuse one primary candidate plus named pieces from othersmerge only named pieces, then verify
Refinekeep one candidate and apply specific feedbackapply bounded edits, then verify
Expandrun another controlled round on a new dimensionwrite a new matrix
Rejectthrow away the setrewrite the prompt, matrix, or verification

Saved decision shape:

bash
cat > "$ROOT/decisions/selection.md" <<'EOF'
# Saved Decision

- outcome:
- primary candidate:
- secondary candidate ids:
- decision:
- why:
- rejected candidates:
- required follow-up:
- final verification command:

## Review Notes

- v1:
- v2:
- v3:

## Continue Prompt

Continue from this saved decision.
Do not reopen the entire search.
Preserve the selected candidate.
Apply only the named feedback.
Run final verification and save the result.
EOF

Use the saved decision as the next prompt material:

bash
cat "$ROOT/decisions/selection.md" \
  | p --stdin --print-prompt \
  "Use this saved decision to converge the variation set. Keep scope narrow and run the required verification." \
  > "$ROOT/prompts/converge.prompt.md"

test -s "$ROOT/prompts/converge.prompt.md" && echo "converge-prompt:OK"

Ask The Agent This

Create a variation matrix:

snippet-30
Create a controlled variation matrix for adding saved payment methods to checkout.

Use exactly one variable: UI integration strategy. Keep database schema,
provider API, auth behavior, and verification fixed. Propose three candidates,
the same verification command for each, isolation strategy, review criteria,
and the saved decision file shape.

Generate candidate prompts:

snippet-31
Using this base Prompt Builder prompt, create three candidate prompts.

Each candidate must preserve the same goal, invariants, verification command,
and review criteria. Only vary the idempotency strategy for the billing webhook:
provider event id guard, invoice-level processing lock, and webhook processing
ledger. Do not introduce a second variable.

Review a finished variation set:

snippet-32
Review this variation run.

Inspect the matrix, candidate prompts, verification logs, diffs, and comparison
notes. Decide Pick, Combine, Refine, Expand, or Reject. If Combine, choose one
primary candidate and name the exact pieces to take from the others.

Converge after selection:

snippet-33
Use this saved decision to converge the variation set.

Do not reopen the entire search. Preserve the selected candidate, apply only
the named feedback, run the final verification command, and save the result.

What To Confirm / How To Know It Worked

GoalConfirmation
base prompt savedtest -s "$ROOT/prompts/base.prompt.md"
launch shape savedtest -s "$ROOT/launch/base.launch.json"
matrix existstest -s "$ROOT/matrix.md"
one variable namedmatrix has one Variable Under Test
candidates isolatedeach candidate has a distinct branch/worktree/directory
same verificationverification.tsv shows the same command for every candidate
logs saved`find “$ROOT/logs” -type f
diff/review material saved`find “$ROOT/review” -type f
comparison completetest -s "$ROOT/review/comparison.md"
decision savedtest -s "$ROOT/decisions/selection.md"
convergence prompt savedtest -s "$ROOT/prompts/converge.prompt.md"

If It Goes Wrong

Candidates Changed Multiple Variables

Stop and rewrite the matrix.

snippet-34
If the candidate changes UI, database schema, and test strategy, it is not a controlled variation.

Recovery:

snippet-35
Keep one chosen variable.
Move every other change into invariants or out-of-scope.
Regenerate candidate prompts.

Verification Differs Per Candidate

The comparison is weak if every candidate gets a different proof command.

Recovery:

snippet-36
Pick one verification command before generation.
If a candidate needs a different proof, that is evidence for a new round.

Repo Was Dirty Before Fan-Out

Dirty state is not forbidden. Unknown dirty state is the problem.

Recovery:

bash
git status --short
git diff
git diff --staged

Then commit, stash, clean, or explicitly record what should carry into every candidate.

Best Idea Is Split Across Candidates

Use Combine, not vague merging.

snippet-38
primary candidate: v2
take from v1: rollback guard
take from v3: test fixture
forbidden: unrelated renames

Then run final verification after convergence.

All Candidates Fail

Do not pretend one failed candidate won.

Use:

snippet-39
Reject

or:

snippet-40
Expand with a narrower matrix

Save what the failures proved before starting the next round.

References And Resources