By Evan Vega
A viral post quotes Andrej Karpathy saying today’s AI models waste a ton of capacity remembering things that don’t
matter, and that a one-billion-parameter model can be much more effective than most of them. Most of that is his words:
on the No Priors podcast, published 5 September 2024, he said current models are “wasting a ton of capacity remembering
stuff that doesn’t matter” and “I think even a billion suffices. We’ll probably get to that point.” Novel Cognition found
that the post’s “90%” and its comparison with today’s models are not in that clip, and that Karpathy framed the idea as a
prediction resting on distillation from much bigger models.
The post held up webAI’s new open-weights TwIL-8B as the idea made real. Novel Cognition checked its numbers, read its
licence and ran it on a Mac Studio. The full analysis is at
cogcore.novcog.us.com.
A viral post’s Karpathy quote is mostly his words from the No Priors podcast (Sept 2024); the “90%” and the comparison with today’s models are not. webAI’s TwIL-8B, held up as proof, reports 93/92/90
Stout’s post passed 640,000 views in two days. Novel Cognition could not find a formal independent evaluation of
TwIL-8B in that time; a YouTube home test on a MacBook Air reported strong coding and weak factual recall. NullTX called the
claim of beating bigger models “better described as unproven than wrong”. On Karpathy’s idea itself, Zvi Mowshowitz wrote in
October 2025 that he predicts “the cognitive core hypothesis is wrong”. Earlier small-beats-big results followed the same
shape: DeepSeek’s 1.5-billion-parameter R1 distill beat GPT-4o on contest math in 2025 and lost on science and on
LiveCodeBench (reactions).
webAI chief executive David Stout’s launch post on 4 November 2026 said TwIL-8B reaches 93% of frontier performance on
coding, 92% on instruction following and 90% on contest mathematics. All three are real arithmetic from webAI’s own
model card: 81.6 against 87.8 on LiveCodeBench, 74.4 against
81.3 on IFBench, 86.7 against 96.5 on HMMT November 2025. The card also records the catches. The coding and instruction
rows were measured on a wordier “verbose” variant rather than the default, and the LiveCodeBench edition was not recorded.
HMMT November 2025 is the one contest set the card says was not in its decontamination check. On the hardest set, HMMT
February 2026, the model reaches 69% of the frontier, and on Humanity’s Last Exam 26%. The card states: “It is not a
knowledge model.” The comparators are GPT-5.5, Claude Opus 4.8 and Grok 4.3; the card notes the newest releases no
longer report these tests. Every TwIL-8B score is self-reported. Detail:
the numbers.
The model is released under the webAI Non-Commercial License 1.0, which limits use to “non-commercial research and
educational purposes only”. IBM’s Granite 4.2 8B, which TwIL-8B is built on, is Apache 2.0
(the licence).
Novel Cognition ran TwIL-8B, for evaluation only, against Qwen3-8B and DeepSeek V4 Flash on a Mac Studio M3 Ultra with
256 GB of memory: eight machine-scored tasks (a code fix, a function written to a spec, a seven-rule writing brief, two
counting puzzles, two trivia quizzes and a data extraction), three runs each. TwIL-8B passed 164 of 168 checks, Qwen3-8B
153 and DeepSeek V4 Flash all 168. TwIL-8B generated text about 3.6 times faster than DeepSeek but averaged 50 seconds per
answer against 22, because it wrote about seven times as much, most of it reasoning. DeepSeek V4 Flash is a 156 GB model;
TwIL-8B fits on a laptop. The quizzes were too easy to reproduce the knowledge gap webAI’s card reports. It is a spot
check, not a benchmark (our test,
method).
Novel Cognition’s reading: Karpathy’s idea comes out of this better than the post that borrowed it. A small model that
reasons well and knows less is what showed up, and it clearly beat the older model its size, but it is not 90% of the
frontier across the board and it cannot be put to commercial work. Check the licence before building on a small model,
give it the narrow jobs, and count the whole job rather than the speed. The analysis is also an episode of the
Frontier Watch podcast.
Novel Cognition tests vendor claims like these for companies as part-time AI leadership
(novcog.com/contact). The checks behind every published number are at
hiddenstatedrift.com/verify. Novel Cognition is not affiliated with webAI,
IBM, Andrej Karpathy or No Priors.
Part of the Frontier Watch Series: Read the previous investigation
More Coverage:
→ Read this investigation on North Denver Tribune
→ Coverage from Daily Colorado News
