Diagnostic Gap Challenges Frontier LLM Superiority
AI-generated image · US National Wire
A dermatologist's test shows a decade-old convolutional neural network outperforming generalist AI models in skin cancer detection.
A diagnostic test conducted by dermatologist Dr. Magnus Lynch, as first reported by Lynchlab, suggests that frontier generalist AI models may struggle to match the accuracy of specialized, older neural networks in medical imaging.
According to reporting from Lynchlab, Dr. Lynch tested GPT-6 Astra and Claude Fable 5.1—two of the most capable models available as of October 2026—against a fine-tuned version of Inception v3, a convolutional neural network (CNN) originally published by Google in 2015. Using 200 biopsy-proven lesions from the MILK10k public dataset, the CNN correctly identified the diagnosis in 125 cases (62%). In comparison, GPT-6 Astra and Claude Fable 5.1 were less accurate, identifying the correct diagnosis in 84 and 85 cases respectively (both 42%).
Dr. Lynch noted specific failures in the frontier models, such as misidentifying an invasive melanoma as a harmless seborrhoeic keratosis and suggesting the patient be discharged. The CNN, which was fine-tuned on 68,011 images from the ISIC Archive using a single H100 GPU, proved superior in recognizing skin cancer despite being a ten-year-old architecture.
While the generalist models showed varying strengths—GPT-6 Astra correctly identified all 25 basal cell carcinomas—the specialized CNN maintained a higher overall success rate across the test set. The results indicate a significant performance gap between broad-scale LLMs and narrow, vertical AI applications tailored for specific diagnostic tasks.