Prompt Hygiene Over Model Power: New Web Extraction Data Reveals Operational Levers for AI ROI
AI-generated image · US National Wire
A benchmark by Earn an Honest Dollar demonstrates that a simple 'do not guess' instruction slashed hallucination rates from 70.7% to 20.2%, suggesting enterprise efficiency depends more on prompt rigor than raw model capacity.
For enterprise leaders weighing the ROI of generative AI, the prevailing narrative often centers on the raw power of the underlying model. However, as Earn an Honest Dollar first reported, new data suggests that operational rigor—specifically prompt hygiene—may be the more critical lever for reducing costly hallucinations in production environments.
Reporting from Earn an Honest Dollar, a marketplace for agent-based services, reveals a stark contrast in performance based on a single instructional change. In a benchmark focused on web extraction, the outlet tested how various models and APIs handled missing data across 42 pairs of synthetic pages. The core objective was to determine if an extractor would correctly return a null value when a field was absent or if it would invent a response.
**The 'Do Not Guess' Effect**
The most significant finding from the Earn an Honest Dollar test is the impact of prompt engineering on accuracy. The directive given to every contestant was: "Use null for any field whose value is not on the page. Do not guess."
When this specific instruction was removed, the rate of made-up fields across 16 models jumped to 70.7%, representing 405 of 573 missing fields. With the instruction included, that rate plummeted to 20.2% (116 of 574). This suggests that the failure to extract data accurately is often not a limitation of the model's intelligence, but a failure of the operational guardrails provided in the prompt.
**Model Performance and Cost Efficiency**
From an ROI perspective, the benchmark highlights that high-performing results do not necessarily require the most expensive infrastructure. The test measured several models and paid APIs, including Gemini 3.8 Flash, GPT-6 Luna, and Sonnet 5.
According to the Earn an Honest Dollar data, GPT-6 Luna emerged as a highly efficient option, making up only 5 of 36 fields at a run cost of $0.0049. Other top performers with the "do not guess" instruction included Gemini 3.8 Flash (1/36) and GLM 5.3 (1/35).
Conversely, some paid API services struggled significantly. Firecrawl, for instance, made up 24 of 36 missing fields—a rate higher than 13 of the 16 models tested, even when those models were given the prompt instructions. The reporting notes that all 24 of Firecrawl's errors involved copying a "decoy" value from the page, such as an old price or a non-author name.
**The Role of the 'Cheap Checker'**
Beyond initial prompt hygiene, the Earn an Honest Dollar analysis explores a secondary operational layer: the use of a low-cost model to verify the output of a primary extractor. This "checker" approach allows a buyer agent to validate whether a page actually supports a returned value for a fraction of a cent.
In testing two checkers, GPT-6 Luna and Jev 1.13, the results were as follows:
* **GPT-6 Luna:** Caught 38 of 49 made-up values and rejected zero correct values. It successfully caught 20 of Firecrawl's 24 hallucinations. * **Jev 1.13:** Caught 23 of 49 made-up values and rejected zero correct values. While effective at catching obvious decoys (such as wrong prices), Jev failed to identify "near-meaning" errors, such as when prep time was incorrectly listed as total time (0 of 6 caught).
The cost to check all 126 unique returned page-and-value pairs was $0.0049 using GPT-6 Luna and $0.0024 using Jev.
**Opinion: The Operational Imperative**
In my view, these results signal a shift in how B2B operators should approach AI procurement. The fact that a simple sentence can reduce hallucinations by over 50 percentage points proves that the 'intelligence' of the model is secondary to the rigor of the implementation. For the enterprise, the path to ROI is not found in chasing the next marginal increase in model parameters, but in establishing strict prompt hygiene and implementing low-cost verification loops. If a $0.0049 run can outperform more expensive alternatives through better instruction, the competitive advantage lies in the ops layer, not the model layer.
**Benchmark Parameters**
Earn an Honest Dollar noted several constraints regarding the test: it utilized synthetic pages across seven types, and results were based on a single run per contestant on September 27, 2026. Paid APIs were tested on free tiers. Additionally, the outlet excluded Hy4 preview from the results due to a lack of usable JSON in many responses.