When the same mistake returns, turn the correction into a check that runs on each change. You will learn to choose its boundary and prove that it rejects the mistake while allowing valid work.
Observed failures define useful checks
Begin with work that already failed, because imagined dangers create noisy and expensive rules. Keep one failing example and one valid alternative before deciding how any check should behave. Hamel Husain and Shreya Shankar recommend building evaluators from discovered errors in their guidance. A recurrence usually justifies automation, while one severe failure can justify earlier protection. An explicit request for a standing guard also removes the need to wait.
Repeated comments should become executable rules
Repeated review comments show that memory and written guidance are no longer reliable controls. Boris Cherny says he would write a lint rule and automate that repeated feedback in his recommendation. Ryan Lopopolo describes promoting a missed written rule into code when documentation falls short in the described practice. Choose the smallest existing mechanism that recognizes the entire demonstrated class of mistakes. Use a static rule when the failure has a visible and stable code shape. Use a focused test when the failure appears only while the software runs.
The check must permit valid work
A useful check protects one boundary without rejecting every nearby and acceptable change. Save the forbidden example, then preserve one approved alternative beside it for comparison. Set the rule to block changes when both the boundary and remedy are certain. Use a warning while evidence remains incomplete, then remove or strengthen that warning. Narrow exceptions should name their reason, because broad escape routes hide future recurrences. Theo Browne says an entire recurring issue class can be automated permanently in his argument. That promise applies only inside the precise boundary the chosen mechanism can observe.
A small import rule proves the boundary
Suppose interface code repeatedly imports a database client instead of using the approved service layer. The following exercise creates one forbidden import, one permitted import, and one blocking rule. Type these commands inside an empty working directory on a development machine.
mkdir recurring-import-check
cd recurring-import-check
npm init -y >/dev/null
npm install --save-dev eslint@10.10.0
npm pkg set type=module
npm pkg set 'scripts.lint=eslint .'
mkdir -p src/db src/api src/ui
cat > src/db/client.js <<'EOF'
export const db = {};
EOF
cat > src/api/users.js <<'EOF'
import { db } from "../db/client.js";
export const users = db;
EOF
cat > src/ui/UserCard.js <<'EOF'
import { db } from "../db/client.js";
export const card = db;
EOF
cat > src/ui/UserList.js <<'EOF'
import { users } from "../api/users.js";
export const list = users;
EOF
cat > eslint.config.red.js <<'EOF'
export default [{ files: ["src/ui/**/*.js"], rules: {} }];
EOF
cat > eslint.config.js <<'EOF'
export default [
{ ignores: ["node_modules/**"] },
{
files: ["src/ui/**/*.js"],
rules: {
"no-restricted-imports": [
"error",
{
patterns: [{
group: ["**/db", "**/db/**", "@db/*"],
message: "UI code must import through the API layer."
}]
}
]
}
}
];
EOF
npx eslint --config eslint.config.red.js src
npx eslint src/ui/UserList.js
set +e
npm run lint
status=$?
set -e
test "$status" -eq 1
The unprotected validation succeeds, proving the recurring mistake remains invisible to routine checks. The protected validation rejects the direct import, while the approved service import remains accepted. The ordinary project command uses the same protection, so future changes cannot depend on memory.
Ordinary validation keeps protection active
A check run only by hand still depends on someone remembering when to use it. Place it inside the project's ordinary validation, using the same command locally and during shared review. Make every failure message name the approved alternative, so rejection includes the next correction. Mitchell Hashimoto describes changing the environment after failure so an agent cannot repeat that mistake in his account. 逆瀬川ちゃん recommends adding a test whenever an agent makes a mistake in the stated practice. Together, those practices move correction from a person's memory into the environment surrounding future work.
Every check needs limits and upkeep
The import rule sees direct imports matching configured locations, while other dependency routes remain outside. Dynamic loading, indirect re-exports, and unchecked folders require separate evidence before receiving additional protection. Record these limits beside the rule, then expand only after observing another missed class. Review exceptions and messages whenever architecture changes, because an outdated remedy can block valid work. Retire the rule when its boundary disappears or a broader control safely replaces it. Automation reduces repeated correction, while maintenance decides whether the long-term cost remains worthwhile.
Sources support evidence-led automation
Together, these sources support replacing repeated correction with an executable guard around an observed failure. They favor checks built from discovered errors and written rules that have already failed. They describe practitioner guidance and experience, rather than measurements proving universal outcomes. Each local check still requires direct proof that forbidden work fails and permitted work succeeds.
- Theo Browne, video, A Message for Passionate Devs, at 08:15. "that class of issue can be fully automated forever."
- Boris Cherny, video, Inside Claude Code with Boris Cherny, at 41:35. "I would write a lint rule for it. So just automate it"
- Mitchell Hashimoto, blog, My AI Adoption Journey, at I don't know if there is a. "engineer a solution such that the agent never makes that mistake again."
- Hamel Husain, Shreya Shankar, blog, AI Evals: Everything You Need to Know, at A better approach is to start with error. "Write evaluators for errors you discover, not errors you imagine."
- Ryan Lopopolo, blog post, Harness engineering: leveraging Codex in an agent-first world, at Human taste is fed back into the system. "When documentation falls short, we promote the rule into code"
- 逆瀬川ちゃん, blog post, Harness Engineering Best Practices for Claude Code / Codex Users, Explained Plainly, at Tests resist rot better than documentation. "Whenever an agent makes a mistake, add a test to prevent it."
Convert one recently repeated mistake into a blocking check with one permitted example.