Median agent cost per JavaScript framework ran $2.67 to $4.97

Developers who have a coding agent build web apps can see here whether the framework named in the prompt changed the cost, the code and the speed of the result. In Nolan Lawson’s benchmark, Claude Code built the same Pokédex app 5 times in each of 12 JavaScript frameworks, 60 apps in all. Nothing was run for this post: the sources were read on 2026-11-02, and repository timestamps are UTC.

How each agent run was set up

The benchmark README says each agent gets the same spec (a Pokédex backed by a local copy of PokeAPI), the same tests, the same tools and the same one-line prompt. Each agent runs alone in a fresh Podman container, and pokeapi.co resolves to localhost. The prompt is:

Implement the app described in SPEC.md using **{framework}**. Use the latest stable version. Work autonomously until `npm test` passes.

Every iteration used Claude Code 2.1.310 with the model claude-opus-5-5 at high effort, according to the results report. The spec leaves libraries, rendering strategy, routing, data fetching and project structure to the agent. It requires all data to come from the API at request time, and the tests check this by swapping in slightly different data.

Official agent tooling was installed, but agents were never told to use it

The benchmark treats each framework’s agent tooling as part of the result. The README says each agent got what the framework’s official docs recommend, installed up front, and was never told to use it. Nuxt got its MCP server, Astro its Docs MCP, Angular its CLI MCP plus agent skills, and SvelteKit the Svelte Claude Code plugin. SolidStart, Qwik, Gracile + Lit, the vanilla build and TanStack Start got nothing; TanStack’s skill setup needs an interactive terminal.

How the apps were tested and measured

Each app had to pass a conformance suite, then its production build was measured with Lighthouse. The post says the suite checks the accessibility tree rather than the raw DOM, because one framework renders into shadow DOM. Lighthouse ran with mobile settings on a 2025 Framework 13 laptop under Ubuntu 24.04, with 4x CPU and “Slow 4G” throttling. Each figure is a median of 5 runs per framework, 3 for some Angular metrics. Tokens are Claude Code’s own reporting, and time is wall-clock time.

Cost needs a caveat: the author writes that the figures are API prices quoted by Claude Code, not money he spent, since he was on a $100/month Max plan. Lines of code, per the README, are non-blank source lines, not counting JSON, lockfiles or tests.

All 60 apps passed, and two Angular apps could not be measured

The results report shows every iteration passing its final tests, with angular-3 and angular-4 stopping at the measurement step. In the angular-3 files, the agent finished in about 10 minutes with a final test exit code of 0, then Lighthouse got HTTP status 400 from the server.

The post explains that in those 2 runs the agent built a server-rendered app that rejected non-localhost requests, although the spec said a reverse proxy would sit in front of the server. Angular’s cost figures include all 5 runs, while its code size and performance figures use 3. An update note on the post says an earlier version overstated the Angular agents’ inability to build a working app.

Median cost, JavaScript and code size by framework

These medians appear in both the results report and the post. JavaScript is the amount transferred to the browser and TBT is Total Blocking Time, both from Lighthouse. Angular’s last three columns use 3 apps.

In the results report, Astro’s median time was 8.2 minutes and Angular’s 17.2. Nuxt’s median was 3.85 million tokens and Angular’s 9.76 million, and cache reads make up most of each total: 3.66 million of Nuxt’s 3.85 million.

Run-to-run spread overlaps many of the gaps

A median hides how far single runs moved. The post gives its own example: React Router’s cheapest run cost $2.82, less than Hono + htmx’s most expensive at $3.19. The results repository leaves out the Claude transcripts, so these ranges cannot be checked against its data, and the post gives no statistical test.

In the next-2 measurements, home-page TBT across 5 Lighthouse runs ranged from 56 to 151.5 ms. That 95.5 ms spread within one app is wider than the 71 ms gap between the Astro and Next.js medians. The README says it directly: “One iteration is an anecdote.”

Why three frameworks cost more, in the author’s reading

SvelteKit, Gracile + Lit and Angular have the three highest median costs in the table. The post says this may be due to what the agents did: they read the SvelteKit v3 migration guide in every run, and for Gracile they cloned its repository or read its compiled source from node_modules. For Angular, the author says cost seems to track total code size, and Angular has the largest median line count in the table.

Official tooling was used unevenly, by the author’s count

Installed tooling is not the same as used tooling. The post reports that agents never used Nuxt’s MCP server, and used the SvelteKit MCP in 5 of 5 runs, Angular’s in 3 of 5 and Astro’s in 2 of 5. The results report lists the Svelte MCP as “pending” among the MCP servers in the session for all five SvelteKit runs, while Nuxt’s shows “connected”. The transcripts behind the counts are left out of the results repository.

Limits

Nothing was run for this post, and of the 58 measured apps, one per-app Lighthouse results file (next-2) was checked against the post’s ranges. The README lists its own limits: one content-heavy app that favors server-rendered multi-page apps, page loads measured and nothing after them, and no artificial API latency. One agent, one model and one effort level were tested, and no run compared tooling against no tooling. Frameworks and their AI tooling were deliberately not pinned, so the next release may change the numbers. The README gives the command used for the post, which needs Linux, Podman and Claude credentials:

node bench/run.mjs --frameworks=all --iterations=5 --effort=high

It also warns: “Use a key with a spending limit.”