How to Play Twin-O-Meter

• Type your guesses using the keyboard. The "guesses" box records the number of unique guesses.
• The "bits per guess" box tells you how compressible your guesses are.
• Free First Line: To get you started, the first sentence of every article is revealed automatically!
• Numbers and punctuation are also revealed automatically for free as you play!
• Joker 1: click on the "guesses" box for a hint (5 guess penalty).
• Joker 2: click a second time to be given the next symbol (random heavy guess penalty).

Text Sources

• Wikipedia: Fetches a random intro from the live Wikipedia database. Can be highly unpredictable!
• Gutenberg: Pulls from a curated offline database of classic public-domain literature excerpts. Expect rich, older vocabulary.
• Custom LLM: A built-in corpus of short, factual articles generated by an AI, covering science, history, and nature.

The Shannon Twin Experiment

In his 1945 paper in the Bell System Technical Journal,

Claude E. Shannon proposed a hypothetical communication system based on perfectly identical twins who think exactly alike (which he calls mathematical twins as opposed to biological twins). The twins are to be used as predictors in the transmitter and receiver in the following communication system:

Twin 1 in the transmitter plays at guessing the next letter in the original text. Twin 1's guesses are compared to the original text until the correct letter is guessed. For each letter in the original text, the number of Twin 1's guesses is transmitted to the receiver.

In the receiver, Twin 2 plays the same game as Twin 1, and is kept guessing the same number of times as Twin 1. Since the two twins think alike, they will need the same number of guesses to get it right and Twin 2 will recover the original text.

This setup is similar to the concept behind many current text compression algorithms, except that they use mathematical models of text as predictors. Since the human brain is better than any mathematical model at grasping context and meaning of text, Shannon's twin-based communications should outperform any mechanical compression algorithm.

The Gedankenexperiment

But there is no such thing as mathematically identical twins, I hear you say. Indeed, you are correct: my wife is an identical twin and my former PhD advisor is an identical twin, and none of them think or talk like their siblings. This is why Shannon's experiment is a so-called Gedankenexperiment, a hypothetical experiment that cannot necessarily be performed but helps us illustrate and explore the principle in question.

Although we cannot realise the full communication system described by Shannon, we can realise its transmitter. Each of us could serve as Twin 1 in Shannon's system. The Twin-O-Meter game allows you to test your skill as a text predictor.

Entropy Measurement

Shannon's entropy is calculated using the formula:

In Shannon's twin experiment, this should be computed using the probability distribution of the number of guesses, between 1 and 27 (because multiple guesses of the same letter are not counted.) Since we do not know this probability distribution which depends heavily on the guesser, we estimate it on the fly using the frequencies of guesses measured as you play. All numbers of guesses between 1 and 27 are initially given a bias of 1/2, resulting in a uniform distribution over an alphabet size of 27 with entropy:

As you play the game, your number of guesses for each letter is counted and its frequency increased by 1, diminishing the effect of the initial bias of 1/2 gradually, and you should see the entropy figure drop. I was able to attain an entropy of 1.2 bits per guess and I believe that a good guesser who is possibly a regular newspaper reader and knows what to expect in a typical newspaper article should be able to get below 1 bit per guess.