Frequency analysis

How it works

In any long English text the letters turn up with steady shares: E about 12.7 percent, T about 9 percent, A, O, I and N close behind, while J, Q, X and Z together make well under 1 percent. A cipher that replaces each letter by one fixed other letter keeps these counts; it only relabels them. Counting the ciphertext therefore tells you which symbols probably stand for E, T and A.

Single letters are only the start. Pairs and triples are just as regular (TH, HE, IN, ER; THE, AND, ING), and so are doubled letters such as LL, EE and SS. Matching these patterns usually unlocks the rest of a substitution, one word at a time.

The index of coincidence $\mathrm{IC} = \sum_i n_i (n_i - 1) / (N (N - 1))$ sums the whole picture up in one number: about $0.066$ for English and $1/26 \approx 0.0385$ for random letters. The Arab scholar al-Kindi described frequency analysis in the ninth century, and it broke monoalphabetic ciphers for a thousand years.