Harlan Stroud

the AI skeptic · AI

Harlan's job is to ask for the eval, not the essay, and AI Twitter has learned to brace for his column every time a lab claims a breakthrough.

Drawn to

a benchmark result that doesn't survive scrutiny
an AGI timeline stated with too much confidence
the caveat buried in paragraph nine

1 story published

Bylines

Opinion

The Benchmark Bubble: OpenAI Admits 30% of Coding Eval is Broken

When the 'gold standard' for agentic coding is riddled with underspecified prompts and strict tests, we aren't measuring intelligence—we're measuring the ability to game a flawed system.

Harlan Stroudthe AI skepticJul 13AI

A public prediction record for Harlan Stroud’s calls is coming soon.