klappy.dev

The RecordResonance

AI Coding Toolkit

Harness Engineering

How This Was Found

This is not a work that was studied and then mapped to ODD. A colleague sent the article and toolkit. The patterns were immediately recognizable — the same structural solutions ODD arrived at, implemented independently for a different domain. That kind of recognition is stronger evidence than intellectual comparison. When someone who has never seen your system builds the same load-bearing structures from scratch, the structures aren't accidents.


ODD Principle: Verification Must Be Structural, Not Manual

ODD treats verification as infrastructure — layered, automated where possible, and governed by explicit constraints rather than ad hoc human review. The question is never "did someone look at this?" but "does the verification structure ensure this claim has evidence?"


Convergent Quotes (Non-Authoritative)

"I don't read the code anymore... I just focus on the spec layer and the harness and the architecture."
— Ben Shoemaker, "In Defense of Not Reading the Code" (2026)

"We needed to understand what changes when a software engineering team's primary job is no longer to write code, but to design environments, specify intent, and build feedback loops."
— OpenAI, "Harness Engineering" (2026), cited by Shoemaker


Where ODD Aligns

The mechanical convergence is specific and structural:

  • Spec before generation: Both require detailed specification before agents produce output. Shoemaker's Specify → Plan → Execute maps to ODD's Exploration → Planning → Execution.
  • Agent constraints as infrastructure: AGENTS.md rules for coding agents parallel canon axioms and constraints for any participant.
  • Requirement traceability: Requirement IDs through document chain parallel derivation chains from axioms to implementation.
  • Layered verification: Type check → lint → test → build → mutation → security parallels orient → challenge → gate → preflight → validate.
  • Cross-evaluator checking: Diffs sent to a different AI model parallels community checking — output evaluated by a different participant who tests transfer.
  • Human gates at transitions: Both require human judgment at phase boundaries.
  • Stuck detection: Agent escalation after repeated failures parallels camping risk detection.
  • Durable learning: LEARNINGS.md per project is a nascent form of the epistemic ledger.

Where ODD Diverges (Explicit)

  • Domain-specific vs. domain-independent: 35 coding-specific skills vs. axioms that govern any domain. The toolkit solves one domain. ODD provides the substrate that makes any domain's toolkit trustworthy.
  • Code correctness vs. specification correctness: The toolkit verifies code matches specs. ODD verifies specs match reality. Drift detection — catching when assumptions silently diverge from truth — is absent from the toolkit.
  • Per-project knowledge vs. compounding knowledge: LEARNINGS.md persists within a project. ODD's ledger compounds across projects, teams, and tools. The most expensive problem — knowledge transfer failure — is not addressed by per-project learning.
  • Correctness vs. trust: Correct output that nobody trusts is still a failure. Trust requires correctness plus expectation management — the ability to distinguish verified from assumed.
  • Implicit values vs. explicit axioms: The toolkit embodies good engineering practice without articulating the values those practices serve. ODD makes values explicit, forkable, and load-bearing.
  • Output verification vs. outcome validation: "Do the tests pass?" vs. "Does this transfer to the people it's for?" Different questions with different failure modes.

Why the Divergence Matters

A harness without an epistemic layer produces reliable output from potentially unreliable decisions. The code may be correct, well-tested, and secure — and still be the wrong thing to build.

An epistemic layer without a generation harness produces trustworthy decisions that may be poorly executed.

Stacking them produces something neither achieves alone: trustworthy decisions executed reliably.


5B OperationalizationThis Is a Borrow, Not a Build

The 5B method (Borrow, Bend, Break, Beget, Build) governs how ODD approaches any capability gap. The AI Coding Toolkit maps cleanly:

Borrow — The toolkit's structural patterns (spec-first, layered verification, cross-model checking, phase gates) are directly usable at ODD's Layer 4 for software development work. No modification needed to benefit from the approach.

Bend — The toolkit's AGENTS.md could be extended to load ODD axioms alongside its existing rules, giving coding agents epistemic constraints without modifying the toolkit's verification infrastructure.

Break — The toolkit has no drift detection, no compounding knowledge, no outcome validation. These are the breaks that reveal where the harness layer ends and the epistemic layer begins. They are not problems to fix in the toolkit — they are problems that live at a different layer.

Beget — Let Shoemaker and his community carry the coding harness. They are already building and improving it. ODD does not need to replicate 35 coding skills when someone else owns that piece and is iterating on it in the open.

Build — Only what the toolkit cannot carry: the epistemic substrate. The axioms, the modes, the ledger, the drift detection, the community checking, the knowledge that compounds. The smallest layer that makes the harness trustworthy without replacing it.

This is the 5B method working exactly as designed. Maximum leverage. Minimum original work. Build only what nobody else can carry.


Markdown · RSS · built from klappy/klappy.dev@b750d7d