The Cheapening of Intelligence: A 67-Cent Model Shakes the AGI Narrative
AI-generated image · US National Wire
While the industry chases trillion-parameter gods, a lone researcher is proving that high-level reasoning can be squeezed out of a small transformer for less than the cost of a vending machine snack.
The prevailing narrative in the AI arms race is one of scale: more compute, more data, more billions of dollars. But the real story isn't the ascent of the god-like AGI; it's the terrifying efficiency of low-cost benchmarks proving that 'intelligence' is becoming a cheap commodity.
**Opinion:** We are being sold a vision of AGI that requires the energy of a small city, yet the most interesting breakthroughs are happening in the basement. When a researcher can achieve competitive reasoning scores for pennies, the 'moat' promised by Big Tech starts to look like a puddle.
According to a technical report published on Hacker News, as mvakde first reported, the researcher has developed a small transformer model that scored 44% on the ARC-AGI-1 benchmark. The most jarring detail? The training cost was just 67 cents. The model was trained from scratch in 1.5 hours using a 5090 GPU.
As reported by Hacker News, the researcher's goal was to test the limits of sample efficiency—the ability of a model to learn from very few examples—using the ARC benchmark. ARC is designed as a metalearning test where each puzzle follows a different rule, making it difficult for models that rely on massive synthetic datasets or human inductive biases. The researcher notes that the benchmark remains unsaturated for those focusing on data efficiency.
Technical details provided in the Hacker News report reveal that the model utilizes a supervised approach, training autoregressively on input-output pairs converted to tokens. Key architectural choices contributing to the performance include:
* **Representations:** The use of 3D RoPE (Rotary Positional Embeddings) and separate additive embeddings for each puzzle. * **Modern Architecture:** A shift to SwiGlu instead of GELU and RMSnorm instead of layernorm. * **Optimization:** The implementation of NorMuon, which the researcher notes solved convergence issues found with vanilla Muon and outperformed AdamW. * **Scaling:** Increasing the model to 8 layers from 4.
The researcher claims this model beats many Large Language Models (LLMs) and matches the scores of TRM/HRM. The work has already garnered attention from prominent researchers including Lucas Beyer, Jeremy Howard, and Rohan Anil.
Beyond the score, the researcher highlights a paradoxical finding: by removing training on input tokens (making the approach supervised), the test loss actually worsened, yet the final score improved to 44% from 40%. This suggests a failure mode in the current industry trend of optimizing for the lowest validation loss on small datasets.
To further refine the model, the researcher incorporated non-overlapping tasks from ARC-2, carefully filtering out 773 repeated puzzles to prevent data leakage. Without this extra data, the model still scores approximately 40%, though it requires double the compute. The researcher has open-sourced the code on GitHub, challenging others to reach a 65% score, noting that a union of solved tasks from multiple runs already reached 55%.