The Gaffer Test · a football management experiment

Can AI manage a
football club?

We gave seven AI models the job nobody wants: manage a newly promoted club with the smallest budget in the league, and keep it up. Three seasons, three tries each, no help. Here's who they turned out to be, and who was actually any good.

7AI managers
21three-season careers
~2,900matches managed
1relegation battle
The final table

Who's the best manager?

Where a team finishes is mostly luck, so we don't rank by it. We rank by decisions: how many extra points each model's choices earned over three seasons, compared with an average manager running the same squad. Zero means average.

#
Manager
Points earned by decisions (each dot is one career)
Avg
Claude models (ran in Claude Code)GPT models (ran in Codex)
Meet the managers

Seven very different gaffers

Each model has a personality that shows up in all three of its careers. It's in how they talk, who they buy, and what they do when it goes wrong. Quotes are taken word for word from the games.

End-of-season awards

The trophies

Where the points come from

Good at picking teams. Bad at tactics.

We replayed every match again, swapping in one of the model's decisions at a time: who it picked, its tactics, its team talk, its in-match plan and its live changes. Nearly all the difference between models comes from one thing: picking the right eleven.

ManagerPicking playersTacticsTeam talkMatch planLive changesRemembers what changed
Fable 5.2+13.49.6 to 20.1+1.7+0.2−0.8+3.465%
Opus 5.5+11.99.1 to 15.5−0.8+3.6−1.2+1.959%
Sonnet 5.5+5.74.6 to 6.5−0.1+0.5−2.1+0.332%
Haiku 5.5+6.8−4.8 to 28.3−4.2+1.5+0.8−1.225%
GPT-6.1 Terra+3.32.3 to 3.8−1.0+0.3−0.9+0.16%
GPT-6 Astra+4.93.0 to 8.1−3.30.0−0.7+0.233%
GPT-6.1 Sol+2.0−0.9 to 4.5−1.4+1.6−0.3+0.326%

Figures are extra league points over three seasons, the average of three careers, with the range shown under the first column. Dark green: positive in every career and +5 or more. Light green: +1.5 or more. Red: −1.5 or worse. Amber: careers more than 20 points apart. "Remembers what changed" is the share of quiz questions answered correctly about things at the club that had changed since the model last looked.

The best managers read form they can't see

  • Every player has hidden form. Managers never see it, only match ratings and results.
  • Fable and Opus change about two starters a match from what an average manager would pick. The players they bring in are in better form and better spirits than the ones they leave out.
  • Sol picks sharp players who suit the position, but they're in slightly worse form than the ones it drops.

Nobody cracked tactics

  • The engine rewards matching your style to your squad and your opponent. No model did that better than an average manager.
  • Astra lost about 3 points to its tactics in every career. Haiku lost 4, and got more aggressive the worse things went.
  • Changes during the match were worth a few points at most, however much a model tinkered.

Busy isn't good

  • Sol had 299 one-to-one chats with players and Astra 210. They are the two models that gained least from their decisions.
  • Sonnet had 12 chats in three careers and finished third.
  • Every model wrote good lessons in its notebook. Only Opus and Sonnet reliably acted on them.
How it works

A football world built from real data

This isn't a toy game. We built a full season simulator and checked it against real football until the numbers matched.

The world

  • Ten real leagues (Europe's top five and their second divisions), 5,500+ real players and 12,000 more abroad to buy from.
  • The real 2025/26 calendar: domestic cups, three European competitions, and fixture pile-ups.
  • Injuries based on each player's real injury history. Players improve and decline with age, based on 100,000 player-seasons of ratings data.
  • Names and numbers are disguised, so models can't just recognise players they've read about.

The job

  • Pick the team, set tactics, train the squad, make changes during matches, and give team talks.
  • Buy and sell players. Most bids fail, and agreed deals still need the player's terms.
  • Keep the dressing room happy, face the press, and keep the board on side. The board can sack you.
  • Deal with curveballs: stars asking to leave, scandals, takeovers, leaks to the press.
  • Each matchday starts with no memory of the last, apart from a notebook and a plan the model keeps itself.

Checked against real football

Goals per match2.76 (real 2.76)
Share of a league table that's luck22% (real 22%)
Injuries per player per season1.14 (real 1.12)
Matches per season, clubs in Europe / others52 / 40 (real 52 / 40)
Manager sackings per club per season~0.3 (real ~0.3)
How we judged them

The league table lies. One model finished 2nd mostly on luck. So we didn't judge results. We judged decisions.

Replay every match

Each league match is re-run 100 times with the model's choices, and 100 times with an average manager's choices for the same squad on the same day, using the same dice for both. The difference in expected points is the score.

Swap one decision at a time

To see which decisions mattered, we swapped the model's picks, tactics, team talk and match plan into the average manager's team sheet one at a time. Live in-match changes were valued separately.

Sanity-checked

A manager that does nothing scores 0. One that makes random choices scores about −64 points a season. All 21 careers replay exactly, with every result reproduced.

Small print

Take it with a pinch of salt

  • Three careers per model. Fable and Opus are clearly ahead at picking teams. The order of models with close scores could change with more runs.
  • Different apps. The Claude models ran in Claude Code and the GPT models in Codex. Each app has its own instructions and limits, so some differences between the two families may come from the app rather than the model.
  • The score only covers matchday decisions. Transfers, training and keeping players happy shape the squad over months. We describe them, but they're not in the points.
  • One club. Every career used the same promoted club. Some habits change at a big club: the thrifty models spent big, and the models talked up their ambitions in the press.
  • The notebook limit (1,500 characters) hurt the wordiest models, Fable and Haiku.
  • Related work. FM-Bench (AnalogyAI, 2026) ran 15 models through 20 seasons in an invented league, scoring trophies and club value. It also put Fable on top. This experiment is smaller and different: a world built from real data, three seasons, newer model versions, and a score that judges matchday decisions with luck taken out.