A random sample you can actually defend SAMPLE

A random sample is not the same as a defensible one

Anybody can pick twenty rows out of a thousand. The question that arrives six months later is a different one: why those twenty?

And the honest answer, “the spreadsheet's random function”, is not an answer. Nobody can repeat it. Nobody can check it. And there is no way to show that the sample was drawn before the results were looked at rather than after, which is the accusation that actually gets made: that you kept drawing until you got a sample you liked.

A seeded generator fixes all three at once. The same seed and the same population always produce the same sample, so anybody given the seed can reproduce the draw exactly and confirm it is the one you used. Write the seed in the file note before you draw, and the sample stops being an assertion about a method and becomes evidence of one.

The generator here is a small published one rather than the browser's Math.random, which is deliberately unseedable and therefore useless for this. That is the whole reason it is not used: not because it is a poor source of randomness, but because it cannot be repeated.

Committing to the method before you see anything

The discipline is short and it is the thing that makes the sample worth drawing:

• Define the population precisely: which file, which date range, how many rows, and how you know that is all of them.
• Decide the sample size and say why. A number chosen after seeing the data is a number chosen to reach a conclusion.
• Write down the seed. Any value will do; what matters is that it was fixed before the draw and recorded where it cannot be quietly changed.
• Draw once, and keep the result even if it looks inconvenient. A sample that happens to contain three items from the same branch is a legitimate random sample. Redrawing because you do not like it is the thing the whole procedure exists to prevent.

Take the digest of the population file too, with CHECKSUM or PROVENANCE, and record it beside the seed. A seed only reproduces the sample against the same population, so without the digest somebody can say the file changed between the draw and the check, and you would have no way to show it had not.

What a sample can and cannot tell you

It tells you about the population it was drawn from and nothing else. That sounds obvious and it is the most common mistake: a sample of invoices from one system does not tell you about invoices that never reached that system, and those are usually the ones the exercise was about.

An error rate from a small sample carries a wide margin, and the honest write-up gives the interval rather than the point. “Three errors in twenty” is compatible with a true rate anywhere from about 3% to 38%, and reporting it as 15% is a claim the sample does not support.

And a clean sample is not evidence that there is nothing to find, only that whatever is there is not common. If you are looking for rare and deliberate things, a random sample is the wrong instrument: those are found by targeting, and the random sample is what tells you whether the ordinary run of the work is sound. Nothing you load here is uploaded. FIGURES handles the rounding without losing a significant digit, and BASERATE is what to read before drawing a conclusion from a rate.