All news

AlphaCode shows code models win by sampling and testing at scale

A steel sieve full of pale grey pebbles rests over a white porcelain bowl where a single cobalt pebble lies alone

DeepMind's AlphaCode matched the median human contestant by writing a million programs per problem and testing them. Verification becomes the moat.

A million attempts per problem have put a machine level with the median human competitor in open programming contests. DeepMind, the research lab owned by Alphabet, unveiled AlphaCode on 2 February, a system that writes complete programs for competitive programming problems from their plain-language statements. In ten recent contests on the Codeforces platform, each with more than 5,000 participants, it ranked on average within the top 54.3%, the first time an AI code generation system has reached a competitive level in these contests.

How it got there matters more than the rank. AlphaCode moves the economics of code generation from training toward inference, and it shows that the layer which verifies output automatically, through tests, compilers and sandboxes, matters as much as the model that writes it.

The method is generation guided by tests. Transformer models pre-trained on public GitHub code and fine-tuned on contest problems produce up to a million candidate programs per problem; the system runs them against the example tests, groups the survivors by behaviour and submits ten. Unlike the race to train ever larger language models, a large share of the result comes from sampling at scale and filtering by execution. The cost per solved problem therefore depends on how many candidates the system must generate and check. That cost falls with cheaper chips and smarter filters, so code tools can improve without a new model each time.

The commercial reach goes well beyond autocomplete, because a machine that solves unseen problems at the level of a median contestant widens the market for developer tools. DeepMind also released its contest dataset openly.

Startups should pick domains where a machine can check its own answer. Code, spreadsheets, data pipelines and configuration all have executable tests, and there a generate-then-verify loop turns compute into quality. The asset worth owning is the test harness and the evaluation data, since curated problems with reliable tests are the scarce input, not raw code. Products built on today's code models should plan for many samples per task, track the cost of running them and price the result rather than the call.

One lab's ranking still has to survive other hands. Independent groups need to reproduce it, and the work has to hold up in peer review. For the money, the number to follow is how fast the samples needed per solved problem fall, which sets the price of such systems in production. Willingness to pay gets its clearest test when code assistants now in technical preview move to paid plans for companies.

Code is a large professional market where AI output can be graded automatically, and that makes progress measurable and investable. Fanaura, a family office that backs AI-first companies over a horizon of years, looks there for teams that own the verification layer, in line with our thesis on software that builds software. In code, the grader is worth as much as the writer.

For founders

Building tools that test and verify AI-written code? Send us the project.

Fanaura invests its own capital and the capital of its limited partners (LPs), usually through a separate SPV for each deal. If we see a fit, we get in touch.

Apply in five minutes
Subscribe

Get the news by email

One short note each weekday. No marketing, and you can unsubscribe at any time.