BuilderBuilder

From code, to simulation, to physical machines.

Independent AI benchmark lab. Same task, frozen prompt, every failure kept.

· Latest test · Four Flash models

Qwen 3.8 Flash vs DeepSeek V4.1 Flash vs GLM 5.3 Flash vs Gemini 3.8 Flash

No universal winner. The cheap models were brutally task-specific. Qwen led several game and voxel rounds. DeepSeek produced the better CAD submission. GLM was the only model to pass the apple-stem grasp. Gemini led several robot tasks.

See the comparison Watch the reviewInspect all public resultsExplore benchmark projects and reproduction paths
Simulation capture

Experiments

All experiments →
  1. Qwen 3.8 Flash vs DeepSeek V4.1 Flash vs GLM 5.3 Flash vs Gemini 3.8 FlashThere was no universal winner. Flash-model performance was brutally task-specific.Games · Graphics · CAD/3D · Robot simulation · 14 tests
  2. Claude Opus 5 across nine game-building testsOpus 5 delivered the strongest detail in several rounds. It also made the efficiency argument against itself.Games · Interactive worlds · 9 tests
  3. I Gave Qwen3.8 Max, Kimi K3, GPT-5.6 and Fable 5 the Same 9 Game TestsKimi K3 was more consistent; Qwen's visual-agent workflow was not reliable enough to win overall.Games · Interactive worlds · 9 tests
  4. Kimi K3 vs GPT-5.6 SOL vs Claude Fable 5: 9 Game TestsFable 5 sets the visual ceiling. Kimi K3 wins on value. GPT-5.6 Sol Ultra is the practical middle when technical coherence matters without paying Fable’s full cost.Games · Interactive worlds · 9 tests
  5. GPT-5.6 vs Claude Fable 5: Same Prompts, 7 AI Game-Dev TestsFable sets the visual ceiling. Sol Ultra wins the value argument.Games · Interactive worlds · 7 tests

Models

All results →
ModelProviderRunsAvg costAvg time
GLM 5.3 FlashOpenCode Go34$1.23 · includes provider estimates1h 14m
GPT-5.6 LunaOpenAI Codex27$2.0323m 26s
GPT-5.6 SolOpenAI Codex26$8.4539m 15s
Grok 4.6Grok CLI subscription26$4.85 · includes provider estimates33m 25s
DeepSeek V4 FlashOpenCode Go25$0.261h 56m
Claude Opus 5.5Anthropic24$52.8030m 29s
GPT-6 AstraOpenAI24$31.941h 12m
Claude Opus 5Anthropic23≥$68.522h 31m

Method

Methodology →
  1. Same taskEvery model gets the same frozen target and prompt.
  2. One attemptNo human follow-up or rescue pass after submission.
  3. Failures retainedFailed and partial runs stay in the record.
  4. No run, no scoreUntested systems stay untested — including physical machines.