Aiso experiment · 207,132 web pages

Why does ChatGPT keep picking 17 when you ask for a random number?

Benjamin Tannenbaum, Founder and CEO, Aiso
By Benjamin Tannenbaum · Founder and CEO, Aiso · LinkedIn
7 min read

First published .

I kept seeing the same small trick online: ask an AI for a random number between 1 and 30. It answers 17. So I wanted to test the obvious explanation.

Aiso experiment showing ChatGPT's preference for 17 and Common Crawl frequency estimates for 17, 23 and 29

Try this in a fresh chat: “Give me a random number from 1 to 30.”

Very often, the answer is 17. A September 2026 experiment across 12 current models found the same thing: 17 was the most frequent answer for the 1-to-30 prompt in all five languages tested. In that test, several models returned 17 on every one of 20 attempts.

That is not random. But why 17?

My first hypothesis: 17 looks random because it is rare

Humans do not experience every number equally. Prime numbers feel less structured than multiples of 2, 5 or 10. Seventeen is odd, prime and does not have an obvious visual pattern. So I wondered whether it also appears less often in the text models learn from.

If a number occurs less often on the web, perhaps it feels less familiar and therefore more “random.” It is a plausible story. It is also testable.

So we counted numbers on the web

We took a random sample of 10 WET shards from the August 2026 Common Crawl and parsed the extracted page text. The sample contained 207,132 web pages and 1.65 GB of plaintext. We counted standalone integer tokens from 1 through 30, excluding things like version17 or 17.5.

We then scaled the observed rates to Common Crawl's roughly 2.14 billion-page snapshot. These are estimates, not an exact census of the full web, but the ordering is clear:

PrimeSample countEstimated crawl count
11226,1612.34B
13150,2061.55B
17137,5681.42B
19136,2091.41B
2396,188994M
2976,838794M

If rarity were the explanation, 29 should beat 17. It appeared roughly 44% less often in our sample. Twenty-three should also beat 17. It did not.

17 is not the rarest. It is the most random-looking.

The better explanation is not raw frequency. It is association.

An LLM asked for a “random number” is normally predicting a plausible continuation of that instruction. Unless it routes to an actual random-number generator, it is not sampling uniformly from the mathematical set {1, …, 30}. It is producing the sort of number that appears in contexts where people talk about randomness.

And humans have biases here too. Research in numerical cognition has repeatedly found that people do not generate numbers uniformly when asked to be random. A 2014 Frontiers in Psychology paper, “Not all numbers are equal”, reviews and analyzes these systematic preferences. A 2025 paper on LLM random-number generation similarly found strong, repeatable biases that changed with the model, range, temperature and prompt language.

Why 17 rather than 11, 23 or 29?

Seventeen has a useful collection of properties at the same time. It is prime and odd. It sits near the middle of 1 to 30 without being the midpoint. It is not next to an endpoint. Its digits do not repeat. It is not a multiple of 3 or 5. And it contains 7, which is itself famously attractive when humans are asked to pick a random-looking digit.

Compare the alternatives. 11 is prime but visually repetitive. 13 carries a lot of cultural baggage. 19 sits right next to 20. 23 is a strong candidate, but farther from the center. 29 sits beside the upper boundary.

None of those properties mathematically makes 17 more random. That is the point. They make it look random to a human.

The model may be learning our idea of randomness

This is where the result becomes more interesting than a party trick. A 2025 arXiv study, “Deterministic or probabilistic? The psychology of LLMs as random number generators”, found that LLMs often produce highly predictable numerical preferences despite their stochastic generation process. The author argues that the pattern resembles human cognitive bias embedded in training data.

A separate 2026 test across 12 models found that the favorite number changes with the range. For 1 to 10, 7 dominated. For 1 to 30, 17 dominated. For 1 to 100, the leading choice was 47. That makes a fixed “17 token bias” much less convincing. The model appears to be constructing a number that fits the requested range and its learned notion of randomness.

Could the model simply have memorized this exact prompt?

A reader suggested a simpler possibility: perhaps sentences such as “give me a random number between 1 and 30” occur repeatedly in training data, often paired with 17, and the model is reproducing that association.

We tested six exact variants in the same 207,132-page Common Crawl sample, including “random number between 1 and 30,” “random number from 1 to 30,” “choose a random number between 1 and 30,” and both “give me” versions. None appeared even once. That does not prove the phrases are absent from frontier-model training data, but it argues against simple verbatim memorization being the whole explanation.

We also counted numbers near the word “random” and near phrases such as “pick a number” and “choose a number.” That contextual slice did not produce a clean 17 spike either. There were 5,640 random-word windows and 475 choose/pick-number windows, so it is too small to estimate the true conditional distribution precisely.

What about data poisoning?

Small numbers of repeated training examples really can matter, but the details are important. Anthropic, the UK AI Security Institute and the Alan Turing Institute found that about 250 malicious documents were enough to install a simple backdoor during pretraining in models from 600M to 13B parameters. A separate 2026 Anthropic experiment found that 32 poisoned examples could reliably backdoor a fine-tuned constitutional classifier.

Those results make the memorization idea worth taking seriously, but they do not show that 30 copies of a normal sentence are enough to make a frontier chat model answer 17. The poisoning studies used deliberately constructed triggers and target behaviors. Our exact-phrase test also found no copies of the prompt variants in this sample. For now, the evidence fits a distributed learned stereotype of randomness better than a single repeated sentence.

The practical lesson

Language models are probabilistic systems, but that does not mean every answer they produce is a uniform random draw from the space of valid answers. When the prompt asks for something that has a human stereotype, the model can reproduce the stereotype extremely consistently.

If you actually need randomness for an experiment, lottery, simulation or application, ask the system to use a random-number tool or generate the number in code. Do not ask the language model to “think of” a random number.

And for AI search, this is a useful reminder of something broader: models do not merely retrieve facts. They have learned distributions over what answers look right in a given context. Sometimes the easiest way to see that is to ask for a number and get 17.

Sources and methodology