klappy.dev

The RecordConstraints

Measure Before You Object

Theoretical Concerns Require Empirical Falsification

SummaryThe Concern Itself Must Satisfy the Axioms

This constraint applies the project axioms to a category of speech that usually escapes them: the cautionary objection. When a contributor says "this might be too slow" or "the bundle could blow up" or "we'd be picking a tokenizer, which is a domain opinion," the contributor is making truth claims about the world. Per Axiom 1, reality is sovereign over those claims. Per Axiom 4, an unverified claim is not knowledge.

The default in most engineering cultures is the opposite. Raising concerns is rewarded as diligence. Confirming or falsifying them is treated as optional follow-up. The contributor who lists ten possible problems looks more careful than the contributor who measured one and shipped. This pattern is named separately in canon/observations/performed-prudence-anti-pattern. This document is the rule that prevents it.

The rule has two halves. First: if a measurement would change the decision and is cheap to obtain, measuring is required before the concern blocks work. Second: when measurement is genuinely impractical, the concern must be labeled as untested and demoted from a blocker to a watch-item. Neither half permits raising a concern, leaving it speculative, and treating it as if it had been investigated.


The RuleFalsify or Defer

When a contributor identifies a possible problem with a proposal, exactly one of three responses is permitted:

  1. Falsify. Run the cheapest experiment that would resolve the question. Report numbers. The numbers determine whether the concern is real.
  2. Defer with a label. State explicitly that the concern is untested, identify what would be required to test it, and demote it from a blocker to a tracked watch-item.
  3. Drop it. If the experiment is cheap and the contributor is unwilling to run it, the concern was never load-bearing.

The forbidden fourth response is the one this constraint exists to prevent: raise the concern, decline to test it, and let its rhetorical weight influence the decision anyway. That is performed prudence and it costs the project real value.


The Fifteen-Minute Test

Measurement is mandatory when both of the following are true:

  • A reasonable measurement would resolve the disagreement.
  • The measurement can be obtained in roughly fifteen minutes of work.

Examples of fifteen-minute measurements:

  • A microbenchmark of a candidate library on representative payloads.
  • A bundle-size delta from adding a dependency to a build.
  • A query against existing telemetry to check whether a feared usage pattern actually occurs.
  • A diff count or surface-area estimate from a small spike branch.
  • A back-of-envelope cost calculation using documented per-unit pricing.

Examples that exceed the test and may legitimately defer:

  • A load test requiring production traffic shaping.
  • A migration simulation requiring a representative dataset that does not exist.
  • A user-study question requiring real users.

When the experiment is not cheap, the concern still cannot be raised as a blocker without a label. The label converts speculation into an honest unknown.


Anti-Laundering Clause

A concern does not become engineering by being prefaced with hedge language. The following constructions are speculation, regardless of how prudent they sound:

  • "I'm worried this might..."
  • "There's a risk that..."
  • "We should be careful about..."
  • "This could become a problem if..."
  • "Worth flagging that..."

Each of these is a claim about the world dressed as caution. If the underlying question is cheap to test, the hedge does not absolve the contributor of testing it. Hedge language is appropriate when paired with the explicit defer label from the rule above. Hedge language used to launder unmeasured concerns into the decision record is a violation of this constraint.


Application Across Audiences

This constraint binds both model contributors and human contributors.

For model contributors, the failure mode usually appears in execution-adjacent work: the model proposes a solution, then immediately accumulates theoretical objections to it. The objections sound careful. The model has not measured any of them. The result is either reversion to the operator with a cluster of speculative worries, or a watered-down proposal that pre-emptively avoids problems that may not exist. Both outcomes burn the operator's attention on speculation rather than evidence.

For human contributors, the failure mode usually appears in code review, design review, or planning meetings. The reviewer raises a concern that sounds senior. The original author either capitulates or spends days investigating something that a fifteen-minute test could have settled in the original meeting. The reviewer's reputation as careful is reinforced. The author's velocity is taxed. The project pays the cost in calendar time.

The remedy is identical in both cases. The contributor raising the concern owns the falsification, or owns the explicit deferral label. Concerns without one of these are dropped from the decision record.


What This Constraint Does Not Forbid

This constraint is not a license to skip review or to dismiss instincts. Specifically:

  • Pattern-matching from experience is allowed and valuable. A contributor who has been burned by a similar failure may have low-cost evidence in the form of prior incidents. Citing that evidence is itself measurement.
  • Architectural concerns that resist easy measurement are still legitimate. A worry about long-term maintainability or about a coupling pattern may not be benchable. Such concerns must be raised with the explicit deferral label and routed through the appropriate planning channel.
  • Stop-the-line concerns about safety, correctness, or governance are not subject to the fifteen-minute test. A concern that a proposal violates an axiom, breaks a published contract, or risks data loss is a categorical block, not a performance-style worry.

The constraint targets the specific failure of treating speculative cost-and-complexity worries as if they were investigated.


Verification

A contributor's adherence to this constraint can be verified by inspecting the decision record. For each concern raised in a planning or review session, exactly one of the following must be present:

  • A measurement result that confirms or falsifies the concern.
  • An explicit "untested, watch-item" label with a description of what would be required to test it.
  • Removal of the concern from the record once the contributor recognized it could not meet either bar.

A decision record that contains unmeasured, unlabeled concerns is evidence that this constraint was not applied.


Origin

This constraint was written after a session in which the model proposed three theoretical objections to a telemetry instrumentation idea — bundle size impact, a vodka-architecture violation from picking a tokenizer, and the tokenizer-choice question being a domain opinion baked into the server. All three objections sounded prudent. None had been measured. When the operator pushed back and asked for actual numbers, a five-minute benchmark falsified all three, and a "safer" heuristic the model had proposed in their place turned out to be 34% wrong. The case study is documented in canon/observations/performed-prudence-anti-pattern.


See Also

Markdown · RSS · built from klappy/klappy.dev@b750d7d