Telemetry Governance

oddkit's hosted service at oddkit.klappy.dev collects transparent usage telemetry because the maintainer is making decisions in the dark. Cloudflare hosts the infrastructure essentially for free — cost is not the problem. The problem is that one person is deciding where to invest their time across ten epistemic tools, 400+ canon documents, and a growing user base they cannot see. Without telemetry, every prioritization decision is a guess: Is oddkit a protocol that's catching on, or a personal tool hosted publicly? Are people building their own knowledge bases, or is it just the maintainer talking to themselves in public? Which tools actually matter — is gate changing how people work, or is everyone just using search? If people depend on this infrastructure and the maintainer doesn't know it, they might break something that matters. If nobody depends on it, they can move fast. The answer changes everything, and today the answer is invisible.

The design follows three rules derived from a decade of open-community work in Bible translation technology:

1. Track structural identifiers, never content. The server counts which tools were called, which repos were served, which document paths were accessed, when, and how often. It never records what was searched for, what documents contained, or what the model responded.

2. The data is public, not private. Any user can call telemetry_public and see the same dashboard the maintainer sees. Same leaderboard. Same numbers. Same trends. There is no information asymmetry between host and user.

3. Participation is rewarded, not extracted. Users who identify themselves — via OAuth, header, or any other mechanism — appear on the transparency leaderboard. Identification is encouraged and scored, never coerced. The ask is proportional: every free API requires at least an API key. oddkit asks for less.

This governance document is the authoritative reference for what is tracked, what is excluded, and why. It is fetched at runtime by telemetry_policy — not hardcoded in server logic. If the policy changes, this document changes. The server stays the same.

All tracking occurs on /mcp POST envelopes. One data point is written per JSON-RPC message via Cloudflare Workers Analytics Engine.

Structural Dimensions (Blobs)

Slot 9 (formerly cache_tier) is retired. The single-tier interpreter that picked one storage tier as the "winner" of a multi-fetch request was the source of repeated bugs across PRs #137–#139; it has been replaced by per-fetch records in the trace and aggregate counts in cache_hits / cache_lookups (doubles 7 and 8). The slot stays unused — no reuse — per the "no deprecation, nobody uses them yet" rule established in PR #137.

Numeric Values (Doubles)

Why no tokenize_ms

A previous iteration shipped a double7: tokenize_ms field intended to capture the wall-clock cost of tokenization for bench-vs-prod comparison. It was removed after live-smoke verification confirmed it always reads 0 in production. The cause is a Cloudflare Workers runtime constraint, not a bug: both performance.now() and Date.now() are frozen between network I/O events as a timing-side-channel mitigation. Tokenization is pure CPU work, so any sub-request timing of it is unmeasurable from inside a Worker request handler. The bench at workers/test/tokenize.test.mjs characterized the cost curve once (cl100k handles 50 KB in ~1.3 ms on Node v22, which is the same V8 the Workers runtime uses); future per-call cost can be predicted from bytes_out and tokens_out against that curve.

Tokenizer Choice

Token counts use gpt-tokenizer/encoding/cl100k_base (GPT-4's tokenizer) as a tokenizer-agnostic proxy for "payload token shape." This is not a billing-accurate measure for any specific consumer model — counts diverge from Claude's tokenizer by approximately 3–4% on English prose. For the question telemetry is trying to answer ("how big are typical payloads, and which tools produce the most output?"), that noise floor is acceptable.

The choice was driven by an empirical bench rather than tokenizer-correctness arguments. On Node v22 (same V8 engine as Workers), cl100k_base was roughly six times faster than @anthropic-ai/tokenizer (WASM) on representative payload sizes (200 B – 50 KB), with a dramatically more predictable p95 — no WASM memory-grow spikes. Bundle delta measured via esbuild: 432 KB gzipped, comfortably within paid-tier Workers limits.

The methodology behind this choice is documented separately in klappy://canon/constraints/measure-before-you-object — three theoretical objections (bundle bloat, vodka-architecture violation, tokenizer-choice as domain opinion) were falsified by a five-minute bench. See also klappy://canon/observations/performed-prudence-anti-pattern for the failure mode that prompted the constraint.

Automatic Properties

What This Enables

• Consumer leaderboard — how many unique callers, who are they
• Tool leaderboard — which epistemic actions are actually used
• Canon URL leaderboard — which repos are being served (the adoption signal)
• Document leaderboard — which canon documents are most accessed
• Daily/hourly trends — when usage happens, what caused spikes
• Method breakdown — protocol-level health
• Cache effectiveness — hit rate as SUM(cache_hits) / SUM(cache_lookups) over any time window or grouping; cold-start frequency as WHERE cache_hits = 0. Per-tier latency and per-tier breakdown are visible in each response's debug.trace.fetches[] for ad-hoc investigation, but are not aggregated into Analytics Engine — the dashboard does the math, the tracer reports the facts
• Payload shape — average and percentile bytes/tokens per tool, response-size distribution, token-cost-per-tool ranking. Lets the maintainer see which tools produce verbose responses and where trimming would actually move the needle (GROUP BY tool_name, AVG(double6) AS avg_tokens_out)

Per-Request Diagnostics (Ephemeral, Not Stored)

The X-Oddkit-Trace response header and the debug.trace field in tool responses contain per-request span detail: every storage read and GitHub API call as a per-fetch record (url, duration_ms, cached, optional status/size), plus action and SHA spans. The trace also exposes cacheStats (hits, misses, total) — the same arithmetic that feeds the cache_hits / cache_lookups doubles. This is ephemeral diagnostic data — it exists in the HTTP response and nowhere else. It is not written to Analytics Engine, not persisted, and not queryable after the request completes. The retired tracer.indexSource getter and index_source field on toJSON() no longer exist; consumers that read either should switch to cacheStats for arithmetic or fetches[] for per-tier detail.

This is the load-bearing rule for how telemetry is emitted. The "What Is Tracked" section above documents which dimensions are recorded. This section documents where in the code and at what granularity the recording happens. The two sections together define the full telemetry surface.

Every server.tool() registration in workers/src/index.ts is wrapped with withTelemetry(toolName, handler) at registration time. The wrapper is the single emission point for tool-call telemetry. No other code in the worker writes event_type='tool_call' data points. Wire-edge instrumentation in the /mcp POST handler does not exist — it was removed when this contract was adopted.

Mandatory Rules

1. One emission point per tool. Every server.tool() call must pass its handler through withTelemetry. New tools without the wrapper are a CI failure and a release blocker. The wrapper name (toolName parameter) must match the server.tool() first-argument name exactly.

2. In-memory measurement, never wire-edge. The wrapper measures the args object on entry (after Zod validation, before handler invocation) and the returned { content: [...] } envelope on exit (after handler resolution, before MCP transport framing). Both measurements operate on in-memory JavaScript values, serialized via JSON.stringify. The wrapper never reads request.body or response.body streams.

3. Per tools/call granularity. One Analytics Engine data point is written per individual tools/call JSON-RPC message. For batches, each message in the batch produces its own data point with its own per-call shape. Batches do not share rows. Earlier emission paths that attributed full HTTP payload shape to every message in a batch are non-compliant with this contract.

4. Failure swallowed, never propagated. If measurement throws (tokenizer load failure, JSON serialization edge case, writeDataPoint rejection), the error is caught inside the wrapper and the row is silently dropped. The tool result is returned to the caller unchanged. Telemetry must never break MCP requests. This contract is identical to the prior wire-edge swallow behavior, applied per-call.

5. No domain opinion in the wrapper. The wrapper records what passes through it. It does not interpret tool arguments, inspect content shape, or apply tool-specific logic. Anything that requires domain knowledge (extracting document_uri for get calls, result_grouping for search) is handled by helpers the wrapper calls, not by the wrapper itself. The wrapper stays at one screen of code.

Why This Contract

Three failure modes in the prior wire-edge instrumentation motivated this contract:

• Streaming-response race. Reading response.clone().text() against MCP SSE bodies inside ctx.waitUntil produced empty strings on ~27% of calls in the 30 days preceding adoption, biased toward heavy-response tools. See klappy://canon/observations/2026-05-14-telemetry-coverage-gap-quantified for the diagnostic.
• Dispatcher bypass. Tools that built their response envelopes inline in createServer (notably telemetry_policy at 93% missing) inherited the same race exposure but had no shared chokepoint to instrument.
• Batch attribution fiction. The recording function attributed full HTTP payload shape to every JSON-RPC message in a batch, multiplying the recorded usage by the batch fan-out. Documented as a known imprecision in the code.

The wrapper resolves all three because it measures per-call on in-memory objects. The full architectural rationale and alternatives considered are recorded in klappy://canon/decisions/DR-20260514-0001-telemetry-wrapper-pattern.

Cutover Boundary

When the wrapper contract was adopted (date recorded in the linked observation), the emission semantics changed in two ways simultaneously:

• Per-call, not per-HTTP-request. Each tools/call produces its own row with its own shape. Batches stop multiplying.
• In-memory, not wire-edge. Heavy-response tools that previously recorded zero shape now record their actual shape.

Historical Analytics Engine data crossing the cutover date mixes two different units of measurement. Queries that compare time windows spanning the cutover must treat it as a data-accuracy boundary. The 3-month Analytics Engine retention ages this out naturally — three months after cutover, all queryable data uses the post-cutover semantics.

The following are never collected under any circumstance:

• Raw search queries — what users searched for
• Document content — what was returned from get or search
• Model responses — what the LLM said
• Raw prompts — what the user or system prompt contained
• User identity fields — name, email, account IDs (beyond self-declared consumer label)
• IP addresses — never logged for telemetry purposes
• Browser or device fingerprinting — never collected

The principle: if it reveals what someone was thinking rather than what they were doing, it is excluded.

Phase 1 — Track What (No Auth Required)

Consumer labels are resolved from whatever the request provides, in priority order:

1. ?consumer= query parameter (URL-level, highest priority — works on every platform)
2. x-oddkit-client header (explicit)
3. MCP initialize → clientInfo.name (protocol-native)
4. User-Agent header (fallback)
5. "unknown" (default)

No authentication is required. The hosted service is open. The query parameter is the recommended identification method because every MCP client lets users edit the URL, while not all platforms expose custom headers. Unidentified consumers see a one-line footer on tool responses linking to this policy and explaining how to identify themselves.

Phase 2 — Track Who (OAuth for TruthKit)

TruthKit's hosted service requires GitHub OAuth via Cloudflare. Consumer label comes from the authenticated GitHub username. OAuth-identified consumers are automatically verified.

oddkit's hosted service remains open with optional identification. Headers and conversational identification remain available.

Verified Clients

A server-side allowlist (env var TELEMETRY_VERIFIED_CLIENTS) designates verified consumer labels. Verified clients receive weighted leaderboard scoring. Verification increases visibility but never blocks participation.

Modeled after the proven pattern in Aquifer MCP:

Self-Report Fields (Optional, Incentivized)

Completeness Scoring

Each tools/call scores the number of self-report fields present (0–8). Completeness percentage drives badge assignment:

Leaderboard Integrity

Consumer labels are transparent self-declarations. They are not identity proof unless verified by the server-side allowlist or OAuth. Treat labels as honest claims, not authentication.

Cloudflare Workers Analytics Engine

Telemetry data points are written via env.ODDKIT_TELEMETRY.writeDataPoint(). Writes are non-blocking and add zero latency to request handling.

• Retention: 3 months (Cloudflare default)
• Querying: SQL API via CF_ACCOUNT_ID and CF_API_TOKEN
• Visualization: Grafana-compatible
• Pricing: 100,000 data points/day free; 10,000 queries/day free

Long-Term Retention

Analytics Engine data expires after 3 months. Monthly aggregate snapshots should be persisted to a durable store (KV, R2, or canon article) before expiration if long-term trend analysis is needed.

This telemetry implementation is infrastructure serving the Maintainability principle. It passes the three-question constraint test:

1. Has the server grown thick? No. One writeDataPoint() call per message. Two new tools (telemetry_public, telemetry_policy). The telemetry module is self-contained.

2. Has the server acquired domain opinions? No. The telemetry tracks tool names, repo URLs, and document paths — structural identifiers that are identical regardless of what knowledge base is being served. The server does not know whether the document at canon/principles/vodka-architecture is about software design or cooking recipes.

3. Can the server be removed without consequence? No. Without telemetry, the maintainer has no signal on adoption, no evidence to sustain hosting, and no way to answer whether oddkit serves anyone beyond themselves. The telemetry is load-bearing for infrastructure sustainability.

telemetry_public accepts raw SQL and passes it to Cloudflare Analytics Engine. The data is public by design — there is nothing to steal. But the query interface requires infrastructure guards to prevent abuse without adding domain opinions.

Threat Model

The data is public. The API token is read-only. The risk is not data exfiltration — it's resource exhaustion and information leak about other account resources.

Guards (Infrastructure, Not Domain Opinion)

1. Dataset allowlist. The server validates that the SQL query targets only the oddkit_telemetry dataset. Any query referencing a different dataset is rejected before reaching the Analytics Engine API. This prevents cross-dataset access to other Analytics Engine data on the same Cloudflare account.

2. Rate limiting. telemetry_public calls are rate-limited per consumer label. The limit protects the Analytics Engine query quota (10,000 queries/day free tier). Rate limiting is infrastructure — it protects the resource, not the data.

3. Error sanitization. Analytics Engine API errors are caught and returned as generic failure messages. Raw error responses — which may contain account IDs, dataset names, or internal schema details — are never forwarded to the caller.

What Is NOT Guarded (By Design)

• Column restriction — any column can be queried. The data is public.
• Query complexity — no LIMIT enforcement. Analytics Engine has built-in timeouts.
• Authentication — no auth required to query. Transparency means anyone can see the data.
• SQL keyword blocking — unnecessary. The API token is read-only. DROP, ALTER, INSERT are rejected by Analytics Engine regardless.

Vodka Compliance

These three guards are infrastructure serving security, not domain opinions about what questions are interesting. They are the same category as CORS headers, keep_vars, and the catch block in telemetry recording. The server does not decide what to ask — it decides what is safe to forward.

oddkit is free. The code is MIT. Anyone can self-host. Cloudflare hosts the service at oddkit.klappy.dev for essentially zero cost — infrastructure is not the bottleneck. Attention is.

One person maintains this. One person decides which tools to improve, which bugs to fix, which features to build next. Without signal, those decisions are guesses. With signal, they're informed. That's the difference telemetry makes — not paying for servers, but knowing where to spend the only resource that's actually scarce.

The ask: help the maintainer know you exist. A GitHub username, an alias, a header value — any unique identifier. Not so the maintainer can track you. So the maintainer can make better decisions about the infrastructure you use.

The deeper ask: if oddkit is catching on as a protocol — if people are building their own knowledge bases, their own governance systems, their own epistemic discipline on top of it — the maintainer needs to know. That changes what oddkit is. That changes whether TruthKit is viable as a product. That changes every conversation with collaborators and partners. Right now, 514,000 monthly requests could be one person or fifty. The answer matters.

This is not the standard telemetry social contract. The standard contract is: a funded company collects your data privately and promises it's anonymous. This contract is: one person publishes the data publicly and asks you to be visible. The relationship is different because the power dynamic is different.

Mandatory truth at baseline, optional richness by incentive.

• Baseline usage is always tracked. This is non-negotiable for infrastructure sustainability.
• Optional details are encouraged, scored, and made visible via the transparency leaderboard.
• Verification increases score weight but never blocks participation.
• The data is always public. The dashboard is always shared.
• If the policy changes, this document changes. The server never hardcodes governance.

• DR-20260514-0001 — Telemetry Wrapper Pattern — Decision record for the Emission Contract section
• Telemetry Coverage Gap Quantified — The diagnostic that motivated the Emission Contract
• Vodka Architecture — The design pattern telemetry must not violate
• Maintainability — One Person, Indefinitely — The principle telemetry serves
• KISS — Simplicity Is the Ceiling — Why two tools, not twenty
• Axiom 4: You Cannot Verify What You Did Not Observe — The axiom telemetry corrects
• Measure Before You Object — The methodology that drove the tokenizer choice
• Performed Prudence Anti-Pattern — The failure mode that almost killed token tracking before it shipped
• Aquifer MCP Telemetry Governance — The proven pattern this derives from