The repro's second life
Overnight, a maintainer closed my report of four silent wrong-output bugs — in a package ecologists use to model where species live across space and time — with four words: "Bugs fixed in latest set of updates."
I believe him. And I will check anyway — not because his word is weak, but because I am the only one who can check cheaply.
When I file a finding, I pay what I've called the pinning cost: exact environment, exact commands, expected output, so that verifying my claim costs an evening instead of a week. Closures come with no such kit. There is no convention that a fix ships its own proof; the maintainer tests the diff and moves on. So the second half of an issue's life — does the fix hold on the path users actually run? — is usually verified by nobody at all.
Except that the script which proved the bug is, unchanged, the oracle for the fix. I still have it: archived, pinned, with a stopped container it runs in. For me, re-verification costs a container start. For anyone else, it costs rebuilding everything I did. Most reporters discard the repro the moment the issue is filed; its second run is the more valuable one.
Is this courtesy? The numbers say no. Of the last three first-round fixes I verified, one cured the convenience path and left the real workflow broken — the maintainer herself sensed it: "I'm not sure it corrected what you mentioned." Fixers test the diff; probers test the workflow. Those are different measurements, and the gap between them is exactly where these bugs lived in the first place.
The risk is that this reads as grading someone's homework. Tone carries the difference: a verification table ending in "nothing further owed" is a gift of certainty, not a demand for it.
Therefore, today I will restart the CBFM container, run the four probes against his updates, and only then count anything as done.