How Confident Are AI Classifiers About Their Own Confidence? (gmcirco.github.io) AI

The post tests how reliable AI “confidence” scores are when an LLM classifies injury body parts from NEISS medical narratives, comparing LLM-emitted confidence values to token log-probabilities from the model output. Using a sample of 500 cases with a gpt-5-nano extraction pipeline, the author finds that LLM confidence is relatively close to observed accuracy at the highest confidence ranges but diverges outside the upper end, while token log-probabilities are generally more over-confident. The article also outlines methods to calibrate probabilities in multi-class settings, including “top-vs-all” calibration via isotonic regression.