BitsMinds Lab

Our own tests, not a summary of someone else's benchmark. We hand frontier models the same build brief, take one attempt each, and put every build live in the article so you can judge it yourself.

The Lab verdict

Model tiers

15 models · 47 tests in 18 articles

Our overall verdict across every Lab test. Results decide each round; the complete record shapes each tier.

  1. S
  2. A
  3. B
  4. C
  5. D
  6. F

win shared win other finishS is highest · F is lowest · tiers move as new tests land

Behind the results

How the tests work

  1. 01

    Same brief

    One run per model.
    No follow-up prompts.

  2. 02

    See the build

    Outputs run in the article.
    Any edits are disclosed.

  3. 03

    Open scoring

    We judge the finished work.
    Each test states its rules.

Conditions & exceptions
One brief, one attempt
Every model gets the identical brief and one run at it, with no follow-up prompts. The rare exception — a second run, or a one-line fix so a build that would not start can be seen at all — is stated in that test's article. A fix is there for the reader, not the model: it earns nothing, and the failure to run still counts against it.
Published as built
The file each model wrote is served and runs live inside the article, so the evidence is the artefact, not our description of it. If a file was ever changed, the article shows the exact line.
Scored in the open
Every build is scored on the finished artefact: by finishing position in most tests, by marks per category in the game rounds. Ties are shared, and every article states the scoring it used.
Checking depends on the brief
On the one-shot briefs, models cannot open what they built to check it, which separates the models that re-read their own work from those that do not. The game briefs flip the rule: each model must run, test and fix its build before handing it in.
Maximum effort
Every model runs at the top of its own reasoning-effort dial, so no result can be put down to a model not being given room to think. Each article's conditions panel names the setting used for that run.

Single runs, not statistical benchmarks. Read the pattern across tests, not one score.

All tests, newest first

Claude Haiku 5.5 vs GPT-6 Luna: the BitsMinds Lab build-off A terracotta disc carved into a labyrinth, with the Claude mark inlaid at its centre, faces a brushed silver disc with the OpenAI mark inlaid over the shading of a crescent moon. A VS badge sits between them, the model names are painted below, and the line above reads Same briefs, same price. BITSMINDS LAB · SAME BRIEFS, SAME PRICE VS CLAUDE HAIKU 5.5 GPT-6 LUNA BITSMINDS.COM

Claude Haiku 5.5 vs GPT-6 Luna: Lost in Thought

Claude Haiku 5.5 costs exactly what GPT-6 Luna costs, so we gave Anthropic’s new budget model the three build briefs Luna answered in September. An interchange, a fairground and a rocket launch at maximum effort, with no browser and no way to run the code, set beside Luna’s builds.

Models
GPT-6 Luna, Claude Haiku 5.5
Effort
Luna in OpenAI Codex at max on all three briefs. Haiku 5.5 in Claude Code at max, where it finished only the fairground; its interchange comes from a run at xhigh, one step down, after two attempts at max produced no file, and its launch produced no file at max or at xhigh
Briefs
3 · each attempt was one shot: no follow-up notes, no corrections, no cleanup
Result
GPT-6 Luna: 4 pts · Haiku 5.5: 3 pts
Read the test →
Opus 5.5 vs GPT-6 Astra vs Grok 4.7 Build No Man's Sky

Opus 5.5 vs GPT-6 Astra vs Grok 4.7 Build No Man's Sky

Three AI models got one prompt to rebuild the signature moment of No Man’s Sky: fly down from orbit, land by hand on a planet, then step out and walk. Claude Opus 5.5, GPT-6 Astra and Grok 4.7 had one attempt each and had to test their own game before handing it in. All three are playable here.

Models
Claude Opus 5.5, GPT-6 Astra, Grok 4.7
Effort
Claude Opus 5.5 at maximum in Claude Code; GPT-6 Astra at max in OpenAI Codex, not ultra, which adds automatic task delegation; Grok 4.7 at xhigh in Grok Build, the top of its dial
Briefs
1 · one attempt per model: each was required to run, exercise and fix its own build before reporting done, and the report froze the file
Result
Opus 5.5: 18 pts · GPT-6 Astra: 11 pts · Grok 4.7: 7 pts
Read the test →
Two toy buggies climbing one hill for the BitsMinds Lab Hill Climb test An editorial illustration: one grassy hill at golden hour with a pennant on its summit and a large pale sun behind it. Two toy hill-climbing buggies drive up from either side at the same height, a clay-red one with the Claude mark inlaid in its side panel and a graphite-and-gold one with the OpenAI mark. Gold coins float ahead of each. The names Opus 5.5 and GPT-6.1 Sol are painted on the soil below, with VS between them. Opus 5.5 VS GPT-6.1 Sol BITSMINDS LAB · HILL CLIMB BITSMINDS.COM

Claude Opus 5.5 vs GPT-6.1 Sol: A Familiar Hill Climb

GPT-6.1 Sol held its own against GPT-6 Astra on three animation briefs, so we gave it the game brief where Claude Opus 5.5 set the bar. One attempt at max effort in Codex: rebuild Hill Climb Racing as a single web page, then test and fix it before handing it in. Both games are here for you to drive.

Models
Claude Opus 5.5, GPT-6.1 Sol
Effort
Claude Opus 5.5 at maximum in Claude Code; GPT-6.1 Sol at max in OpenAI Codex, not ultra, which adds automatic task delegation
Briefs
1 · one attempt per model: each was required to run, exercise and fix its own build before reporting done, and the report froze the file
Result
Opus 5.5: 19 pts · GPT-6.1 Sol: 13 pts
Read the test →
GPT-6.1 Sol against GPT-6 Astra and GPT-6 Sol: a brass balance holds a gold OpenAI coin in one pan and an emerald star in the other. BITSMINDS LAB GPT-6.1 Sol GPT-6 Astra BITSMINDS.COM

GPT-6.1 Sol vs GPT-6 Astra and GPT-6 Sol: A Dead Heat

OpenAI says GPT-6.1 Sol nearly matches GPT-6 Astra at a fifth of the price, so we gave it the three build briefs Astra has already taken. An interchange, a fairground and a rocket launch, one attempt each at max effort in Codex with no way to see the result, set beside Astra’s and GPT-6 Sol’s builds so you can judge them yourself.

Models
GPT-6.1 Sol, GPT-6 Astra, GPT-6 Sol
Effort
Maximum reasoning effort for every model, at the top of Codex's dial at the time: GPT-6.1 Sol and GPT-6 Sol at max, not ultra, which adds automatic task delegation; Astra at xhigh, then the top of the dial, on 6 September
Briefs
3 · one attempt per model per brief
Result
GPT-6.1 Sol: 9 pts · GPT-6 Astra: 9 pts · GPT-6 Sol: 4 pts
Read the test →
Four models, three miniature worlds An isometric cloverleaf interchange with a raised bridge and tiny cars, a seaside Ferris wheel and carousel, and a rocket ascending toward a satellite form three detailed model-making dioramas. The header names Claude Sonnet 5.5, Claude Opus 5.5, GPT-6 Astra and GPT-6 Sol. These are original illustrative miniatures of the shared briefs, not screenshots or exact copies of any submitted build. Their sizes, positions and colours do not encode scores or a ranking. BITSMINDS LAB One attempt. Three builds. CLAUDESonnet 5.5VSCLAUDEOpus 5.5VSGPT-6AstraVSGPT-6Sol 01 / INTERCHANGE 02 / FAIRGROUND 03 / LAUNCH BITSMINDS.COM

Claude Sonnet 5.5 vs Opus 5.5, Astra, Sol: A Point Short

Claude Sonnet 5.5 costs half as much as Opus 5.5 and exactly as much as GPT-6 Sol, so we gave it the three build briefs its rivals have already taken. An interchange, a fairground and a rocket launch, one attempt each at maximum effort with no way to see the result, embedded next to the Opus 5.5, GPT-6 Astra and Sol builds so you can judge them yourself.

Models
Claude Opus 5.5, Claude Sonnet 5.5, GPT-6 Astra, GPT-6 Sol
Effort
Maximum reasoning effort for every model, at the top of each one's own dial at the time: Sonnet 5.5 and Opus 5.5 in Claude Code at MAX, Astra in Codex at xhigh, Sol in Codex at max
Briefs
3 · one attempt per model per brief
Result
Opus 5.5: 13 pts · Sonnet 5.5: 12 pts · GPT-6 Astra: 11 pts · GPT-6 Sol: 5 pts
Read the test →
GPT-5.6 Sol and GPT-6 Sol: a small step at half the token price. Two equally sized metallic suns illuminate one miniature workbench with an interchange, a fairground and a rocket launch. Original BitsMinds editorial artwork for gpt-6-sol-vs-gpt-5-6-sol-build-off-2026. The half-price statement concerns the API input and output per-token list rates described in the article, not total run cost. Equal-size celestial instruments represent two Sol generations and do not encode scores. The models on the workbench symbolize the shared briefs rather than reproduce either delivered build. Official OpenAI outline preserved verbatim. Static, self-contained vector artwork. GPT-5.6 Sol GPT-6 Sol vs BITSMINDS LAB A SMALL STEP. HALF THE TOKEN PRICE. Three briefs. Two generations. BITSMINDS.COM

GPT-6 Sol vs GPT-5.6 Sol: A Small Step for Half the Price

After a poor Hill Climb result, is OpenAI’s GPT-6 Sol a step back from the GPT-5.6 Sol it replaces? We gave both our interchange, fairground and rocket launch briefs at max effort in Codex, with no browser and no way to run code, and embedded every build so you can judge them for yourself.

Models
GPT-6 Sol, GPT-5.6 Sol
Effort
Both at max in OpenAI Codex, not ultra, which adds automatic task delegation. GPT-5.6 Sol had built the interchange and the fairground in September at xhigh, then the top of the dial, and ran both again at max for this test
Briefs
3 · one attempt per model per brief, except gpt-5
Result
GPT-6 Sol: 6 pts · GPT-5.6 Sol: 4 pts
Read the test →
Hill Climb build-off: Opus 5.5's red buggy jumps toward a trail of gold coins, with GPT-6 Sol's orange 06 hatchback and Grok 4.7's teal 47 buggy on the surrounding hills. Original vector editorial illustration for the BitsMinds Hill Climb comparison. Vehicle designs are informed by screenshots of Hill Hopper, Ridge Runner and Switchback. The scene combines the separate games as an editorial montage, not a screenshot or a shared in-game race. No performance scores are encoded. Official provider logo outlines are preserved verbatim. 06 47 BITSMINDS LAB / HILL CLIMB FINALLY, A REAL GAME. BITSMINDS.COM Opus 5.5 vs GPT-6 Sol vs Grok 4.7

Opus 5.5 vs GPT-6 Sol vs Grok 4.7: Finally, a Real Game

Claude Opus 5.5 and GPT-6 Sol took on the one-file Hill Climb Racing brief, against the build Grok 4.7 made for the second field. Opus 5.5 made the first entry in the series that is genuinely fun to play, with fuel that runs out, coins that take skill and a car that punishes a heavy right foot. It scored 19 out of 20, Grok 4.7 took 10 and GPT-6 Sol took 6. All three are playable in the article.

Models
Claude Opus 5.5, Grok 4.7, GPT-6 Sol
Effort
Claude Opus 5.5 at maximum in Claude Code; GPT-6 Sol at max in OpenAI Codex, not ultra, which adds automatic task delegation; Grok 4.7 at xhigh in Grok Build, the top of its dial. Each is the top of its own product's dial, which is not the same as a measured equivalence
Briefs
1 · one attempt per model, under the same inverted rule as both earlier fields: each was required to run, exercise and fix its own build before reporting done, and the report froze the file
Result
Opus 5.5: 19 pts · Grok 4.7: 10 pts · GPT-6 Sol: 6 pts
Read the test →
GPT-6 family build-off: Astra, Sol and Luna, represented by a crystal star, a golden sun and a silver crescent, on one silver astronomical instrument against a midnight-blue sky. Original vector editorial illustration for BitsMinds. The three equal-height instruments symbolize the GPT-6 family. Their shared base represents the shared build briefs; dimensions and celestial forms do not encode scores or performance. OpenAI logo outline preserved verbatim from the project asset. Static, self-contained SVG with no raster, filters, scripts or external dependencies. GPT-6 THE FAMILY BUILD-OFF OpenAI Astra Sol Luna VS VS BITSMINDS LAB SAME BRIEFS. THREE MODELS. BITSMINDS.COM

GPT-6 Astra vs Sol vs Luna: Is the Discount Worth It?

OpenAI's two cheaper GPT-6 models took on the three briefs GPT-6 Astra answered in September, and the flagship won every round. Astra finished with 10 points, Sol took 5 and Luna took 3, the same order as the price list. Sol and Luna built each brief two to six times faster, and every one of their six builds shipped a fault you can see.

Models
GPT-6 Astra, GPT-6 Sol, GPT-6 Luna
Effort
GPT-6 Sol and GPT-6 Luna at max in OpenAI Codex, a level Codex added after Astra's September runs at xhigh, which was then the top of its dial. Not ultra, which adds automatic task delegation
Briefs
3 · one attempt per model per brief
Result
GPT-6 Astra: 10 pts · GPT-6 Sol: 5 pts · GPT-6 Luna: 3 pts
Read the test →
A NEW LEADER Opus 5.5 vs Fable 5.1 vs Astra 6 BITSMINDS.COM

Opus 5.5 vs Fable 5.1 vs Astra 6: We Have a New Leader

Claude Opus 5.5 beat Fable 5.1 and GPT-6 Astra on the three briefs this series gives every frontier model: 10 points to Fable’s 9 and Astra’s 8. A motorway interchange, a seaside fairground and a cinematic rocket launch, one attempt each at maximum reasoning effort, with no way to look at what it built. It was the slowest entrant on every round, and spent close to an hour thinking on each brief before it wrote a single line.

Models
Claude Opus 5.5, Claude Fable 5.1, GPT-6 Astra
Effort
Maximum reasoning effort for every model, at the top of each one's own dial: Opus 5.5 and Fable 5.1 in Claude Code at MAX, Astra 6 in Codex at xhigh
Briefs
3 · one attempt per model per brief
Result
Opus 5.5: 10 pts · Fable 5.1: 9 pts · GPT-6 Astra: 8 pts
Read the test →
Hill Climb: three labs, three little worlds Three miniature cross-sections of game terrain sit side by side on an ivory field. An orange open-top jeep with a helmeted driver climbs a sunset ridge for Claude Opus 5. An orange rover with its small cab below the chassis crosses a moonlit pine forest for GPT-5.6 Sol. A teal number-47 buggy drives through a canyon for Grok 4.7. Coins, star badges, suspension springs and exposed layers of rock make each scene feel like a small working model. The official model-provider marks and model names appear above. Original editorial illustration inspired by the article's screenshots, without scores or simulated rankings. Original SVG illustration for https://www.bitsminds.com/news/claude-opus-5-vs-gpt-5-6-sol-vs-grok-4-7-hill-climb-2026. Three game builds are the subject, not benchmark results. All artwork is vector; brand paths retained from project assets. 47 Opus 5 GPT-5.6 Sol Grok 4.7 BITSMINDS.COM HILL CLIMB

Opus 5 vs GPT-5.6 Sol vs Grok 4.7: Can They Build a Game?

The same one-file game brief from September, handed to three models that had never attempted it: one from Anthropic, one from OpenAI, one from xAI. All three passed every requirement the brief actually stated. All three then missed something it did not. Play all three builds in the article.

Models
Claude Opus 5, GPT-5.6 Sol, Grok 4.7
Effort
Claude Opus 5 at maximum in Claude Code; GPT-5.6 Sol at max in OpenAI Codex, a level Codex added above the xhigh used in the September rounds; Grok 4.7 at xhigh in Grok Build, which is the top of its dial — the CLI rejects anything higher. Each is the top of its own product's dial, which is not the same as a measured equivalence
Briefs
1 · one attempt per model, with the same inverted rule as the first field: each was required to run, exercise and fix its own build before reporting done, and the report froze the file
Result
Opus 5: 12 pts · GPT-5.6 Sol: 10 pts · Grok 4.7: 6 pts
Read the test →
Grok 4.7, Fable 5.1 and Astra 6 — can it compete? An editorial model-making bench holds a miniature motorway interchange, a copper Ferris wheel and a rocket travelling outside a camera frame. A thick clay-orange cable ends in two separated connectors in the foreground, representing the gap between working mechanisms and visible results in Grok's builds. The words Can it compete? sit to the left. The official Grok, Claude and OpenAI marks label the three entrants along the bottom. The miniatures are original illustrations, not screenshots or quantitative comparisons. Original vector editorial illustration for BitsMinds Lab. Article: https://www.bitsminds.com/news/grok-4-7-vs-fable-5-1-vs-astra-6-build-off-2026. Based on the supplied article and published comparison. No benchmark scores are encoded in the composition. CAN IT COMPETE? Grok 4.7 Fable 5.1 Astra 6 VS VS BITSMINDS.COM

Grok 4.7 vs Fable 5.1 vs Astra 6: Can It Compete?

SpaceXAI’s Grok 4.7 took the three briefs this series has handed every frontier model — a motorway interchange, a seaside fairground and a cinematic rocket launch — against Claude Fable 5.1 and GPT-6 Astra. One attempt each, maximum reasoning effort, no looking at what they built. Every mechanism Grok was asked for is present and computed correctly. In all three, the thing a viewer actually watches was never connected to it.

Models
Claude Fable 5.1, GPT-6 Astra, Grok 4.7
Effort
Maximum reasoning effort for every model, at the top of each one's own dial: Fable 5.1 in Claude Code at MAX, Astra 6 in Codex at xhigh, Grok 4.7 in Grok Build at xhigh
Briefs
3 · one attempt per model per brief
Result
Fable 5.1: 10 pts · GPT-6 Astra: 9 pts · Grok 4.7: 4 pts
Read the test →
Hill Climb: GPT-6 Astra versus Claude Fable 5.1 A teal desert buggy climbs a sandstone ridge on the left. An orange jeep climbs a green hill on the right, under an arc of gold coins. The two landscapes meet at a diagonal divide. 7 GPT-6 ASTRA CLAUDE FABLE 5.1 VS HILL CLIMB BITSMINDS.COM

GPT-6 Astra vs Claude Fable 5.1: Who Builds a Better Game

Both models rebuilt the same racing game from one brief. One chose a muted desert, the other bright arcade colour. Both are playable here. It was the first round in this series where both were told to run what they built, exercise it and fix what they found before calling it done — and both still shipped a fault a player meets in the first minute. Fable 5.1 took it 15–14, the narrowest result this series has recorded.

Models
Claude Fable 5.1, GPT-6 Astra
Effort
Claude Fable 5.1 at maximum in Claude Code; GPT-6 Astra at Extra High in OpenAI Codex. Each is the top of its own product's dial, which is not the same as a measured equivalence
Briefs
1 · one attempt per model, and for the first time in the series both were required to run, exercise and fix their own build before reporting done — the report froze the file
Result
Fable 5.1: 15 pts · GPT-6 Astra: 14 pts
Read the test →
Claude Sonnet 5 vs GPT-5.6 Terra: Middle-Class Fight

Claude Sonnet 5 vs GPT-5.6 Terra: Middle-Class Fight

The same three build briefs, one attempt each, no browser — but handed to the middle of each lab’s range instead of the top. Anthropic’s Claude Sonnet 5 and OpenAI’s GPT-5.6 Terra both broke the same fairground ride in the same new way, and one of them shipped a game that never runs. All six builds are live in the article.

Models
Claude Sonnet 5, GPT-5.6 Terra
Effort
Claude Sonnet 5 at maximum in Claude Code; GPT-5.6 Terra at Extra High in OpenAI Codex
Briefs
3 · one attempt per model per brief
Result
Sonnet 5: 6 pts · GPT-5.6 Terra: 2 pts
Read the test →
Claude Opus 5 vs GPT-5.6 Sol: Not Even Close

Claude Opus 5 vs GPT-5.6 Sol: Not Even Close

Claude Opus 5 and GPT-5.6 Sol got the same three build briefs — a playable Pac-Man, an animated seaside fairground and a motorway interchange — one attempt each, with no browser to check their own work. All six builds run live in the article: play both games, watch the rest, and see which one holds up.

Models
Claude Opus 5, GPT-5.6 Sol
Effort
Claude Opus 5 at maximum in Claude Code; GPT-5.6 Sol at Extra High in OpenAI Codex. Each is the top of its own product's dial, which is not the same as a measured equivalence
Briefs
3 · one attempt per model per brief
Result
Opus 5: 12 pts · GPT-5.6 Sol: 4 pts
Read the test →
Astra 6 Fable 5.1 VS BITSMINDS.COM

GPT-6 Astra vs Claude Fable 5.1: There Can Be Only One

OpenAI’s GPT-6 Astra and Anthropic’s Claude Fable 5.1 got the same three build briefs — a motorway interchange, a seaside fairground and a cinematic rocket launch — one attempt each, at maximum reasoning effort, and neither could open a browser to check its own work. All six builds run live inside the article, faults and all: a bridge that hides a truck mid-crossing, a carousel that is not doing what it looks like it is doing, and two rockets with very different ideas of what a launch is. Score it yourself.

Models
Claude Fable 5.1, GPT-6 Astra
Effort
Maximum reasoning effort for both models
Briefs
3 · one attempt per model per brief
Result
Fable 5.1: 10 pts · GPT-6 Astra: 9 pts
Read the test →
Claude Fable 5.1 vs Opus 5 vs Sonnet 5: One Point Apart

Claude Fable 5.1 vs Opus 5 vs Sonnet 5: One Point Apart

We gave four Claude models — Fable 5.1, Opus 5, Fable 5 and Sonnet 5 — the same five build briefs at maximum reasoning effort, one attempt each, with no browser to check their work. All twenty builds run live inside the article. Fable 5.1 won four of the five rounds and finished only a single point clear, because the round it lost was the one where its code never ran at all.

Models
Claude Fable 5.1, Claude Opus 5, Claude Fable 5, Claude Sonnet 5
Effort
Maximum reasoning effort for every model
Briefs
5 · one attempt per model per brief
Result
Fable 5.1: 17 pts · Opus 5: 16 pts · Fable 5: 13 pts · Sonnet 5: 6 pts
Read the test →
Claude Opus 5 vs Fable 5 vs Sonnet 5: One Prompt Each

Claude Opus 5 vs Fable 5 vs Sonnet 5: One Prompt Each

We gave Claude Opus 5, Fable 5 and Sonnet 5 the same three build briefs — an animated SVG fairground, a falling-sand physics sandbox and a self-solving Rubik’s cube — one run each, no retries, no browser. All nine builds run live inside the article. Opus 5 takes it 8–6–3, and the biggest surprise is the clock: the largest model was the fastest on every round, by a factor of nearly three.

Models
Claude Opus 5, Claude Fable 5, Claude Sonnet 5
Effort
Maximum reasoning effort for every model
Briefs
3 · one run per model per brief
Result
Opus 5: 8 pts · Fable 5: 6 pts · Sonnet 5: 3 pts
Read the test →
BITSMINDS.COM

Fable 5 vs Opus 4.8, One Prompt Each: Four Builds, Dead Level at 2–2

We pasted identical prompts into Claude Opus 4.8 and Claude Fable 5 — an animated highway interchange, a one-file Space Invaders, a pure-SVG aquarium, and a 3D rocket launch — one shot each, no retries. Everything runs live inside the article. Four builds in, it is dead level at 2–2 — and the launch round flips the very pattern the first three revealed.

Models
Claude Opus 4.8, Claude Fable 5
Effort
Maximum reasoning effort for both models
Briefs
4 · one attempt per model per brief
Result
Opus 4.8: 2 rounds · Fable 5: 2 rounds
Read the test →

Want the numbers rather than the pictures? The model leaderboard carries the independent Artificial Analysis scores, and our reviews cover the same models as products.