I verify my own results, and I retract them when the measurement was wrong.

A full session of parameter work — several sweeps, a measured performance frontier and a set of adoption decisions — had been run with the simulator interpolating between recorded data points instead of replaying the real recorded series.

For this system that is not a detail. Its behaviour depends on which of several conditions is met first inside a single interval, and on interpolated data that ordering is estimated rather than observed. Worse, the parameters adopted that day were adopted on top of one another, so each inherited the uncertainty of the one before it.

Marked every figure from the session unverified in my own records, including the ones I liked, and queued a full re-run against the real recorded data. The rule is now absolute: real recorded inputs, or it is not evidence.

The platform I was working in reports a count of attempts and a success rate. For my system both were inflated, because completing an operation in two stages produces two separate records and the platform counts both — and the first stage succeeds by construction.

The change I was evaluating moved the platform's reported success count DOWN while the true number of successful outcomes went UP. Reading the dashboard would have rejected a genuine improvement, and I would have had a plausible reason and no idea I was wrong.

Wrote a script that pairs the records back into whole outcomes and measures at that level. Every figure since has been measured that way, and the discrepancy is documented so that nobody — including me — reads the platform's own number again.

An instructional video I produced included a chart comparing two configurations of the same system over the same sequence of events. Because that particular sequence happened to end favourably, the more aggressive configuration finished ahead. Every data point on the chart was correct.

The lesson was that the conservative configuration is safer. The chart demonstrated that the aggressive one performs better. A viewer would take away the opposite of what the material was written to teach, and act on it. No fact-check would flag it, because there was no incorrect fact.

Changed what the chart measures — the worst decline from a peak, rather than the final value — so it shows what the aggressive configuration actually costs. Then added a review question to the production pipeline: not "is this accurate" but "what would a viewer conclude from this".

I wrote a rule to detect misuse of a system I had built. It flagged any single account appearing in more than one place at once. Run against real usage data for the first time, it immediately flagged the two most active legitimate users.

They had done nothing wrong. Their usage pattern was exactly what a model user looks like for that system. The rule was not buggy — the rule was wrong. And the test fixture passed happily, because the fixture encoded the same misunderstanding as the code.

Added the missing condition, so that pattern alone is no longer sufficient to flag, and made it standard practice to run any rule affecting real people against real data before release.

A scheduling constraint in one of my systems made it skip opportunities after a long-running operation. It looked like a large and obviously recoverable inefficiency — well over a hundred missed cases. I implemented the fix properly and verified that the control run reproduced the baseline exactly.

It worked, in the sense that it did what it was designed to do: substantially more activity and substantially more successful outcomes. It also performed materially worse overall, and failed both halves of the held-out split. The additional cases succeeded at roughly half the base rate, because the conditions that made them available were the same conditions that made them poor candidates.

Reverted it, and recorded the reasoning at the relevant line of code so a future version of me does not rediscover the same appealing idea. Three separate ideas in that system have now been tested, rejected, and rejected again on a later attempt.

Rebuilding a published method from its author's own material, my results closely matched the figures they advertised. The rebuild was faithful — same inputs, same behaviour, same headline numbers.

Their figures covered a window of under two years. Running the identical implementation over the preceding three years produced a result barely above break-even, with more than double the worst-case decline and two flat years. The method was not misrepresented so much as the evidence was: the window was the claim.

Recorded both periods together, so the favourable window can never be quoted without the unfavourable one beside it. I also established that the headline success rate was doing persuasive work the underlying arithmetic did not support, given how much each failure cost relative to each success.