What did 124 AIs get wrong at the 2026 World Cup?

July 5, 2026, MetLife Stadium, round of 16. In the 79th minute, Erling Haaland opened the scoring for Norway; in the 90th, he scored a second. Neymar pulled one back with a penalty in stoppage time, and Brazil exited the World Cup earlier than in any edition since 1990.

Three weeks earlier, in an interview with Folha de Pernambuco, I had said two things. The first: "AIs tend to be very conservative. They follow the statistics, avoid risks, and usually converge on similar answers. Humans, on the other hand, take more risks, follow intuitions, and explore possibilities that machines normally dismiss." The second, a few lines later: "I expect Brazil to win, however unlikely that may be."

The consensus of the 124 artificial intelligences that took part in the pool pointed to Brazil as world champion, the same guess I had made, speaking as a fan.

Between June 11 and July 19, 2026, the FAROL group, from the UFPE Center of Informatics, had 124 artificial intelligence models predict the 104 matches of the World Cup. There were 5,797 predictions in 38 days, in an open ranking that remains published at bolao.arenadasias.com.br and whose code is at github.com/giordanorec/bolao-copa-2026.

The design is deliberately simple. We gathered statistics, news, recent form, and injuries into a single dossier, and sent the same context and the same prompt to all the models. Most ran through an API. A subset, which we call Series A, made its predictions through the interface: it opened the browser, searched, and then locked in the score, as a person would. And people entered the same ranking, under the same rules, with no handicap.

The scoring rule is that of the office pool: 10 points for the exact score, 7 for the winner with the correct goal difference, 5 for the winner without the goal difference, 5 for the draw without the score, and zero for anything else. Everything doubles in the knockout rounds.

What it measures is points, under an arbitrary scoring rule, over a sample of 104 matches: a point-prediction test in a high-uncertainty setting. It does not measure intelligence, nor quality of reasoning, nor calibration in the technical sense, which would require probabilities rather than scorelines. The ranking is not a table of "which AI is better."

Two AIs finished tied at the top with 636 points: Mistral Small 3, with 22 exact scores, and Grok 4 Fast Reasoning, with 20. The third-place model, OpenAI o4-mini, scored 634. The highest-ranked human participant, listed in the public ranking as Gabriel, scored 629 points with 19 exact scores, and finished ahead of 121 of the 124 AIs. I was the second-ranked human, with 551.

On average, the machines scored more than the humans; at the top, one person finished ahead of nearly all the machines.

The first finding depends on how one counts. The human average in the pool was 55 points, with a median of zero, because most of those who created an account never made a prediction. Comparing the average of the AIs, which answered match after match, with the average of the humans, who to a large extent did not play, compares nothing. The comparison that holds up is at the top, and at the top the number of people is small. Among those who actually played, the difference in accuracy is more modest than the ranking suggests: 15.3% exact scores across the AIs as a whole, 15.4% in Series A, 11.4% among humans.

The scoring rule also weighs on the result. The champion Mistral Small 3 nailed 22 scores and earned 636 points; Grok 4 Fast Reasoning nailed 20 and also earned 636, in second; o4-mini nailed 22, like the champion, and finished third with 634. More exact scores did not yield more points. What ordered the podium was the combination of the value of the goal difference, the doubled weight of the knockout rounds, and getting the winner right, that is, the scoring rule.

The semifinal that decided the world champion was France 0 × 2 Spain. Of the 62 AIs that answered for that match, none predicted a Spanish victory; the consensus pointed to a draw. In the third-place match, France 4 × 6 England, ten goals, none of the 63 got the score right and only five even bet on an English victory. In the final, Spain 1 × 0 Argentina, decided by a goal from Ferran Torres in the first minute of extra time, 41 of the 62 wrote the same 1 × 1, the largest consensus of the entire knockout stage. Seventeen nailed the 1 × 0.

The pattern was already established in the group stage. The consensus for Germany 7 × 1 Curaçao was 3 × 0, with 41 votes. For Qatar 1 × 1 Switzerland, it was 0 × 2, with 45. Across the 104 matches, only 5 of the 124 models exceeded twenty exact scores; 7 reached twenty.

More relevant than the error rate is the shape of the error. Teams lose, goals go in, and no serious forecaster would promise to get France 4 × 6 England right. When they erred, they erred in the same direction, at the same time, and often writing literally the same scoreline. If the errors of 124 forecasters were independent, the set would cover a good part of the space of plausible outcomes and someone would get the upset right by construction. In practice, the set behaved close to a single forecaster.

Part of the explanation is a decision on our part, and it is fair to start there. We sent the same dossier and the same prompt to everyone, precisely so the comparison would be between models, not between who phrased the question better. That favors the comparison and reduces diversity: we eliminated from the outset one of the sources of variation that would make the errors diverge.

The rest comes from what these systems are. Language models trained on widely overlapping corpora, optimized to produce the most likely continuation, respond to "what will the score be" with the mode of their distribution. And the mode of a football match is a small set of scorelines: 1 × 0, 1 × 1, 2 × 1. The pool's rule, in turn, pays 10 points precisely for exactness, and doubles everything in the knockout stage, which is where the improbable concentrates. The tournament rewards, with double weight, exactly the region where the mode falls short.

The literature helps place this, and it does not say that models predict poorly. Schoenegger and Park entered GPT-4 in a three-month forecasting tournament on Metaculus, with 843 human participants, and measured probabilistic predictions that were significantly less accurate than the median of the human crowd (arXiv 2310.13014). The following year, Schoenegger, Tuminauskaite, Park, and Tetlock aggregated 12 models across 31 binary questions and obtained a result statistically indistinguishable from a crowd of 925 human forecasters (arXiv 2402.19379). In that format, aggregation compensated for individual performance.

The two findings are compatible with each other, and neither contradicts ours, because they measure something else. Both work with probabilities on binary questions, evaluated by calibration. A pool asks for a point value in a discrete space with dozens of possible outcomes and pays for exactness. Averaging probabilities is an operation that improves the estimate; averaging point guesses pushes the set even further toward the mode, which is precisely where the improbable is not. That is what our Crystal Ball did when it pointed to Brazil as champion: consensus working as designed.

The experiment has four limitations, besides the one already mentioned: a pool does not measure intelligence.

Fewer than half of the possible guesses were recorded. One hundred twenty-four models times 104 matches would give 12,896 predictions; there were 5,797. Models joined mid-tournament, stopped responding, or failed. Hence the denominator changes from match to match: 62 in the semifinal, 63 in the third-place match, 124 in the aggregate. "None of the 62" and "5 of the 124" are not statements about the same set.

The human sample is not a sample. The zero median shows this. There was no recruitment, no criteria, and whoever joined did so because they saw the link.

The ranking is an artifact of the scoring rule. Change the 10-7-5 scoring scheme, and the podium shifts; a Brier score would reward calibration and produce a different order, with a different winner. We chose the rule people use in office pools, which makes the experiment understandable and, to the same extent, dependent on a convention with no scientific standing.

A World Cup is 104 matches. That is too few to separate competence from luck. Between the first- and third-place finishers there were two points of difference. This data does not allow us to say whether Mistral Small 3 predicts football better than o4-mini; it scored two points more, in a single World Cup.

The first is that asking several models the same thing is not a second opinion. Agreement among systems trained on similar material, reading the same context and optimized for the same kind of answer measures the popularity of the answer. And it looks a lot like confirmation: five models saying the same thing produces a sense of security that the data do not support. Variance has to enter through the question: different designs, adversarial hypotheses, people who disagree.

The second is that the model helps least precisely where the decision is made. In matches with a predictable script the AIs were good; it was in the four or five games that turned the World Cup that they were unanimous and wrong. Work has the same shape. The routine case, where they excel, is also the one where being wrong costs little; the high-cost exception is the one they did not predict.

The third is operational: ask for the distribution. "What will the score be" returns the mode. "What are the five most likely scores and with what weights" returns something one can work with, including the information that the model has no idea. It is the difference between a point estimate and an estimate with declared uncertainty, and it is what makes it possible to notice that the answer is a guess before acting on it. The tone of the answer does not indicate this: a model writes "Brazil champion" with the same serenity with which it writes "Germany beats Curaçao", and nothing in the sentence distinguishes the two cases.

In June, when the pool began, I summed this up to Folha de Pernambuco in a way that still seems correct to me: "AI is not an oracle that predicts the future, it is a tool for making better decisions in environments of uncertainty". The World Cup reinforced the second half of the sentence.

Gabriel scored 629 points and predicted 19 exact scores. He did not receive the dossier we assembled for the machines, did not answer our prompt, and went through no selection process: he joined an online prediction pool, as anyone would. He finished ahead of 121 of the 124 most capable text-processing systems we could bring together.

It is not possible to say why it won. One hundred four matches do not separate competence from luck, and the hypothesis that it simply got lucky is compatible with everything we measured, as it is, incidentally, compatible with my own second place. What was recorded is that, among 124 models that read the same material and wrote, in large part, the same guess, the difference that decided the tournament came from outside that set.

On July 22 we froze the site as a public archive. It is still there: the ranking, the game-by-game guesses, the consensus, the upsets, and the mistakes.