Stats is hard, and numbers can be Vast

(with bonus Martin Gardner puzzle)

The Nautilus article linked above captured this long, festering issue so very clearly: yes, even honest, well-meaning scientists publish statistically worthless data.

It's the unfortunate result of two almost independent causes. The first is that there is a huge push to publish. The second is that statistics is hard.

Last night, I read a little stats problem in a Martin Gardner book. Try it yourself:

Suppose, for instance, that you shuffle a packet of four cards — two red, two black — and deal them face down in a row. Two cards are picked at random, say by placing a penny on each. What is the probability that these two cards are the same color?

(Write your answer here in permanent ink: → → → )

I’m not a statistician, but I have a degree in mathematics that I keep bright, and I am generally OK at not making a fool of myself in these matters, so I must be at least better than average, right? but I got this one wrong when I was reading the book, and I had read this book before at least twice (decades ago, admittedly).

It’s not just me. Humans do not have a natural intuition for statistics.

Look at how badly your average person responds to a pretty simple problem in immunology — basically statistics with a dash of dynamical systems thrown in — like COVID-19. “One chance in a thousand? That’s the same as zero!” say a hundred million people, and hundreds of thousands needless deaths ensue.

Now that chess and go have more or less fallen to the machines, it’s easy to think of all games as broken, but it is telling that computers beat the game of backgammon a decade before Machine Learning hit the picture, just with simple programs written by people and not by complex AI models that no human understands.

The big difference? Chess and Go involve extremely long chains of hypothetical but certain logic. Mathematically, they are games of complete information, and chance in not involved. Humans are really good at this.

Evaluating a backgammon position involves adding together a lot of small conditional probabilities. Humans are really bad at this, particularly when multiple stages are involved.

Mathematically, the space of possible games of Go is so huge is it is Vast, as Daniel Dennett uses the word — there are very roughly 10³⁶⁰ possible games, a one followed by 360 zeros, a number so huge it is literally beyond human comprehension (though that isn’t to say we can’t reason about such numbers or investigate their properties).

Backgammon by contrast is a tiny little game with something like 10¹⁹ different board positions, a mere ten quintillion, or ten billion billion.

We should be so much better at backgammon than we are at Go, but we aren’t, because we’re logical geniuses and statistical dummies.

The solution in science would be to train more statisticians, and then pay them to support and peer review the scientific work of others, and give them credit. It would be “simple” if change were at all possible and funding adequate.

A diversion: how big is 10³⁶⁰ exactly?

10¹⁹ is big by everyday standards — there are less than 10¹⁴ dollars worth of money in total, for example — but not really that big.

There are at least ten million computers on the planet, excluding embedded devices, and let’s say they each perform a billion operations per second, and a year is very close to pi times ten million seconds, so each year humans perform pi ✕ 10⁷ ✕ 10⁹ ✕ 10⁷ = pi ✕ 10²³ operations, that is, ten thousand operations for each possible backgammon position.

10¹⁹ is ten billion billion. 10¹⁹ is so tiny that humans perform that many operations every hour.

But 10³⁶⁰ is one billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion billion. I wrote a one-line program just to write the name of that number.

If we needed to perform 10³⁶⁰ operations per hour, we might go all Rick-and-Morty on this intractable problem. Imagine for each individual operation of the ten billion billion individual operations the world performs in an hour, we have instead an entire nanoworld of computations.

Any time a computer anywhere in the world performs any individual instruction, we create a new nanoworld, which instantly performs a full world hour of computations and gives us the result back — in a nanosecond of course.

So for each clock cycle in each individual machine in our main world, we get a whole world computation hour of clock cycles, ten billion billion cycles. The machine you’re reading this on would create a billion nanoworlds every second, and each of these would give us ten billion billion cycles of computation!

Amazing, right? We must have made progress towards 10³⁶⁰!

Not really. This gets us to 10³⁸, which is only 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of 1% of the way there.

What if we do the whole process again? For each of the ten billion billion computations within each of the ten billion billion nanoworlds, we create a new attoworld, that does a full world hour of computation in an attosecond?

Sadly, that only gets us to 10⁵⁷. To get to 10³⁶⁰, you’d need to recursively transform each individual computation into an entire tiny world of computations nineteen times. Even Rick would get tired.

And if you think 10³⁶⁰ is big as big numbers go, you have another think coming. It even has a name, it’s an octodecillion centillion.

There are numbers like Graham’s number, not infinite but finite numbers that have come up as solution to actual problems, numbers that are so obscenely and bloatedly great that they cannot be represented in any direct fashion, and even attempting only to print the digits of the number would exhaust all of time and space and matter before making any progress on the task at all, unlike little old 10³⁶⁰, also known as 1,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000.