AI & Machine Learning / Case study
FPLBench
A Fantasy Premier League forecast that is committed before every deadline and scored in public
In short
An open FPL forecasting benchmark: leakage-safe features, a forecast committed before each deadline, and weekly scoring against FPL's own forecast once results are final.
- Status
- Live
- State
- Live, scored weekly
- Holdout MAE
- 0.876
- Last-five baseline
- 1.053
- Rows, as of gameweek 5
- 139,029
- Built with
- Python / LightGBM
Inside the project
01 / 01FPLBench
The brief
The problem
Fantasy football models are easy to evaluate badly. Statistics from the same gameweek can leak the answer, double gameweeks can contaminate each other's history, and a live score can still change while FPL reviews it. Most published models are graded on data they could not have had at the deadline.
What we did
Our approach
The data pipeline shifts every performance feature so a row only sees what was known before its first fixture, including in double gameweeks. FPL's post-match expected points are kept as a baseline and never used for training. LightGBM models predict minutes and points, and a calibrated classifier predicts the new defensive-contribution bonus. The weekly forecast is committed to git before the deadline, and a guard rejects later overwrites. Scores are written only after FPL marks the gameweek finished and data-checked. A transfer planner picks moves for the model's own FPL team. Injured, suspended or doubtful players (flagged below 75%) can never start or be bought. The bench is ordered by simulating 4,000 scenarios under FPL's autosub rules, because the bench scored 9, 17, 14, 25 and 16 points in the first five gameweeks of 2026-27.
Features
What it does
- 01
Forecasts committed before each deadline, with overwrites rejected
- 02
Scored only after FPL finalises the gameweek
- 03
Minutes, points and defensive contributions modelled separately
- 04
Transfer planner that never starts or buys injured or doubtful players
- 05
Bench order chosen by simulating FPL's autosub rules
- 06
Public dataset, board and model-run FPL team
Results
What happened
- Holdout MAE of 0.876 points on 29,417 player-fixtures from 2025-26, against 1.053 for a last-five-games average
- Beat FPL's own pre-deadline forecast in gameweeks 1 to 3 of 2026-27 (MAE 1.569 against 1.690 in GW1)
- Lost to FPL's forecast in gameweeks 4 and 5. In GW5 the model's MAE was 1.306 against 1.156 for FPL
- Published dataset of 139,029 rows and 165 columns as of gameweek 5, with row counts and hashes in its manifest
- The model's own FPL team had 318 points as of gameweek 5, 102 of them in gameweek 2
Services
What this work shows we do
AI automation
Service pageModels wired into the tools your team already uses, with logs and limits.
Related work
Built with the same services
Developer ToolsJarvisMCP
JarvisMCP gives AI agents one governed door into our company systems: CRM, search intelligence, infrastructure, memory and coordination. Agents find the capability they need, act through it, and leave a record. They never hold the keys.
Read the case study
Business SystemsButlerCRM
ButlerCRM keeps contacts, deals, email, calendar and tasks in one workspace that updates live for everyone in it. We rebuilt it on Elixir/Phoenix so changes appear for the whole team as they happen.
Read the case study
How we work
How a change reaches production
The same five gates for our own products and for yours. Nothing reaches your customers on the strength of a demo.
- 01 · Isolate
Its own branch and workspace
Every change starts in a separate copy of the code, so work in progress never touches what is live.
git worktree add .worktrees/your-change - 02 · Build
Engineers with AI agents
Agents write and test code beside us. They hold no passwords: every call they make goes through our gateway, which checks it and logs it.
agent → gateway → policy · audit · call - 03 · Prove
Evidence, then review
Tests, before-and-after screenshots of the running app, then independent reviewers. It does not move on until no serious finding is left.
review: 0 high · 0 medium → pass - 04 · Ship
Deployed and checked live
The pipeline builds and deploys. Then we check the running service itself, not just a green build.
replicas 1/1 · image sha matches · health 200 - 05 · Watch
Checked every night
Each integration we run is tested against its provider every night, so a changed API is caught before your team notices.
nightly: each integration vs its provider