F1 Strategist — LLM Race Engineer

WEATHER ROULETTE · SPA · 12 LAPS

DECISION LOG

envopenenv-core 0.2.3

policyQwen3-4B + GRPO trained

scenarioweather_roulette · Spa · 12 laps

score—

scroll

ARCHITECTURE

Three-phase training pipeline

An LLM agent learns to be a race engineer — reading hidden state, calling the pit, managing tyres across a 12-lap sprint with six independently-scored reward dimensions.

01

OpenEnv Environment

FastAPI server wrapping a deterministic F1 physics model. Each step = one strategic decision. 25+ racetrack CSVs, 6 scenario families, 5 rule-based opponents, real weather events.

Tyre degradation & compound rules
Fuel burn model per track
Safety car / VSC events
Hidden opponent strategies

POST /reset · POST /step · GET /state

02

SFT + GRPO Training

Qwen3-4B warm-started with SFT on 4,900 expert turns (enriched obs, thinking-off render), then GRPO-refined for 200 steps with a fully-deterministic reward function. No LLM judge — every reward is a Python check.

SFT v3 → GRPO starts at 0.82
200 steps · RTX 5090 · LoRA r=16 · num_generations=8
6-dim weighted scalar reward
Peak 0.93 @ step ~30 (then plateau)

train.py --base-checkpoint sft/merged --backend trl

03

Held-out Evaluation

Six scenario families × 5 seeds. Real LLM forward pass through the env (not the scripted-fallback bug we caught) — the trained model averages 0.62, +0.20 over untrained Qwen3-4B. On weather it (0.97) actually edges the rule-based expert (0.95).

Weather roulette: 0.41 → 0.97 (beats expert)
Late safety car: 0.53 → 0.65
Championship decider: 0.27 → 0.56
Dry strategy sprint: 0.51 → 0.52 (PIT-rate undertrained)

evaluate.py --model grpo_v2/merged --n-seeds 5

6-DIMENSION REWARD (weighted, deterministic, no LLM judge)

35%Race result

20%Strategic decisions

15%Tyre management

10%Fuel management

10%Comms quality

10%Operational efficiency

ENVIRONMENT

6 scenario families across 25+ circuits

Each circuit has a natural scenario archetype. The agent must generalise across very different strategic challenges — not just memorise one sequence.

TRY IT LIVE

Run a race simulation in your browser

Select a scenario, seed, and policy, then hit Run Race. The environment runs a full episode server-side and streams the lap-by-lap result back — no command line needed. The trained policy is the SFT+GRPO Qwen3-4B model (grpo_v2).

SCENARIO

SEED

POLICY

Python (openenv client)

# pip install openenv
from f1_strategist import F1Action, F1Env

with F1Env.from_env("Deltasthic/f1-strategist") as env:
    obs = await env.reset(task="weather_roulette")
    while not obs.done:
        action = my_agent(obs)
        obs = await env.step(F1Action(message=action))
    print(obs.score)

Available actions (21 commands)

# Investigation
INSPECT_TYRE_DEGRADATION
CHECK_OPPONENT_STRATEGY <num>
REQUEST_FORECAST
ASSESS_UNDERCUT_WINDOW
INSPECT_FUEL_MARGIN

# Strategy
PIT_NOW <soft|medium|hard|inter|wet>
SET_MODE <push|conserve|race|defend>
RADIO_DRIVER <"message">
DEFEND_POSITION · HOLD_GAP · DONE

HTTP API (curl)

# 1. Reset
curl -X POST .../reset \
  -d '{"task":"weather_roulette","seed":7}'

# 2. Step
curl -X POST .../step \
  -d '{"action":"PIT_NOW inter"}'

# 3. Run full episode (returns trajectory)
curl -X POST .../simulate \
  -d '{"task":"weather_roulette","seed":7}'

▶ Live Simulator ⎋ GitHub ⚡ API Health 📓 Colab

TRAINING EVIDENCE

Real run results — RTX 5090 · SFT v3 → GRPO v2 (200 steps) · Qwen3-4B LoRA

Every plot comes from the actual grpo_v2 checkpoint run. No cherry-picking — 5 seeds per task family, averaged. The training reward curve is generated from grpo_v2/checkpoint-200/trainer_state.json — every point is a real logging step.

KEY DECISION — LAP 7 · SPA · RAIN PEAK

BEFORE — untrained baseline

L07 STAY_OUT rain=0.45, mediums at 66%
L10 PIT_NOW soft wrong compound in damp
L12 DONE P6 finish — score 0.378

→

AFTER — SFT+GRPO trained (grpo_v2)

L07 RADIO_DRIVER "box for inters"
L08 PIT_NOW inter optimal window
L12 DONE P4 finish — score 0.787 (+0.409)

SCORE BY SCENARIO (5-seed average)

Scenario	Random	Untrained	Trained ↑	Expert
Dry sprint	—	—	—	—
Weather roulette	—	—	—	—
Late safety car	—	—	—	—
Championship	—	—	—	—
VSC window	—	—	—	—
Tyre cliff	—	—	—	—

■ random ■ untrained ■ GRPO trained ■ expert

GRPO V2 TRAINING REWARD CURVE

— smoothed reward - - expert 0.94 — untrained 0.42

200 steps · Qwen3-4B LoRA · RTX 5090 · peak 0.93 · SFT warm-start → GRPO refine

RACE STORY · TRAINING CURVE · TRACK GRID

GRPO v2 training reward curve — 200 steps from real trainer_state.json — GRPO v2 reward curve — 200 steps from SFT v3 base · real points from `trainer_state.json` · peak 0.93

Race story: untrained vs trained — position, tyre health, 6-dim score breakdown — Race story — Weather Roulette · Spa · seed 7 · untrained (left) vs GRPO-trained (right)

Track grid — 8 circuits with expert policy — Expert policy across 8 F1 circuits — scene generalization

MODEL'S JOURNEY

From 0.32 random to 0.62 trained — and the bugs we caught along the way

We didn't get to the final number on the first try. Five distinct bugs hid the model's true performance — including one that proved the originally-reported 0.79 was a hand-coded scripted policy, not the LLM. The chart below is every iteration we ran. Honest numbers. Real model.

Iteration journey: random → untrained → SFT v3 → RFT v1 → GRPO v2 → expert — Average `weighted_final` across 4–6 scenarios at each stage. Star is our shipping checkpoint.

Bug 1 · The mirage

Reported "0.79 trained" was actually a scripted-policy fallback. Verified bit-exact by running the rule policy with no model loaded — std=0.000 across 5 seeds was the smoking gun.

Bug 2 · Thinking-mode trap

Qwen3 has reasoning mode on by default. With max_new_tokens=64 the model never finished thinking → unparseable rambles → STAY_OUT default every step.

Bug 3 · Format asymmetry

Trained with thinking-on, eval'd with thinking-off. The chat-template prefix the model never saw at training tanked the score by another 0.10.

Bug 4 · Scenario blindness

format_obs stripped the scenario briefing/hint. The model couldn't tell late_safety_car from dry_strategy_sprint at lap 0 → applied the wrong playbook everywhere.

Bug 5 · GRPO collapse

Cold GRPO from base Qwen3 hit reward-variance collapse by step 200. Fix: SFT warm-start first, then GRPO. Each layer added what the other couldn't — the breakthrough.

Where we'd take this next

Concrete starting points to push past 0.62

Closing the 0.30 gap to expert isn't a single fix — it's a tier list of changes. Ranked by expected ROI per hour of work, the things we'd start with:

Tier 1 · high ROI · days

Larger base model — Qwen2.5-7B / Qwen3-14B. Likely +0.10 from raw capacity alone. 3× wall-clock.
PIT-rate reward shaping — explicit GRPO penalty when an episode finishes without satisfying the FIA two-compound rule. Directs gradient toward correct pit timing. +0.05–0.08 on dry/champ.
Multi-round RFT — DeepSeek-Math style: 3–5 rounds of best-of-16 sampling + fine-tune. Compounds gradually. +0.03 per round.

Tier 2 · moderate · ~1 week each

Curriculum learning — train on dry sprint to convergence, layer in harder families. +0.05 on champ.
CoT-with-data rebuild — regenerate SFT data with explicit reasoning chains in the assistant turn so the model learns to reason before committing.
Self-play opponents — replace rule-based opponents with a weaker copy of the training model so the env keeps providing signal as capability grows.
Verifier-style auxiliary reward — "did this action match what the expert would do?" as a per-step shaping signal alongside the existing 6-dim reward.

Tier 3 · research-grade · weeks-months

Procedural scenario diversity — sample across thousands of generated seeds during GRPO instead of 100 hand-authored ones.
Policy distillation — train Qwen3-4B with the full pipeline, distil to Qwen2.5-1.5B. ~80% of the score, 4× faster inference.
True multi-turn dialogue training — full rollouts where the model conditions on its own action history, with a careful curriculum (we tried it once and tanked champ — needs nuance).
Replace Unsloth or upgrade TRL — currently pinned to TRL 0.18.2 because Unsloth's GRPO assumes ≥0.22. Either fork the patch or migrate the stack.

Full tier list with rationale lives in blog.md under "What's next — concrete starting points". This is the document we'd hand to the person picking this up next.

Full writeup with code: blog.md · iteration data: results/

MATERIALS

Blog, notebook & demo video

All judge-required materials in one place. The Colab notebook is self-contained and re-runnable. The blog post is the full writeup on HF. The video walks through the live environment.

HF BLOG POST

Teaching an LLM to be an F1 Race Engineer

Full writeup: environment design, GRPO training setup, results analysis, and key learnings. Pushed as blog.md to the HF Space repo.

Read blog →

DEMO VIDEO

2-minute environment walkthrough

Shows the environment in action, the trained vs untrained comparison on Weather Roulette, and the key pit-call decision at Lap 7. Link in README.

See README for link →

COLAB NOTEBOOK

Re-runnable training notebook

Full environment smoke test, SFT warm-start, GRPO training with TRL + Unsloth, evaluation, and result plots. Verified runnable on T4/A100.

Open in Colab →