Anthropic Claims Claude Leads Over a Quarter of Its R&D
AI-generated image · US National Wire
The AI startup introduces new measurement standards to track the pace of development, claiming its chatbot handles significant portions of research tasks.
Anthropic CEO Dario Amodei has introduced a set of proposed measurement standards intended to help AI companies communicate the speed of frontier development, as Engadget first reported. As part of this effort, Anthropic reported that its AI chatbot, Claude, "leads" 26 percent of the company's AI research and development work.
Anthropic defines "leads" as the AI's ability to complete the majority of a task end-to-end based on a high-level prompt, though the process remains under human supervision. The company further claims that Claude performs "large chunks of work" under close human direction in more than 90 percent of its research, a figure that encompasses the 26 percent of AI-led work. Despite these figures, Anthropic noted that Claude is not operating fully autonomously in any measured subset of its R&D.
To arrive at these statistics, Anthropic utilized an automation rating scale developed by Epoch AI. The company believes other frontier AI model makers could reproduce this automation level chart using their own data and third-party validation.
Beyond AI-led R&D, Anthropic proposed two other metrics to monitor development pacing: tracking the amount of compute devoted to R&D and measuring the oversight of AI agents. The latter would involve tracking how often agent behavior is flagged, the duration of reviews, and the volume of monitored activity. Engadget reports that while Anthropic has committed to third-party reviews of its practices and Amodei has found common ground with Elon Musk on X regarding safety, the industry largely remains self-regulated, as President Donald Trump has downplayed AI risks.