Analyzing the Relationship Between Model Diagnostics and Predictive Uncertainty in Statistical Inference

Posted: Sep 16, 2018

• Evelyn Gray
• Gabriel Perry
• Gavin Russell

Abstract

This research presents a novel framework for understanding the intricate relationship between traditional model diagnostics and predictive uncertainty quantification in statistical inference. While both areas have developed independently within the statistical literature, their interplay remains underexplored despite having profound implications for model reliability and decision-making under uncertainty. We introduce the Diagnostic-Uncertainty Nexus (DUN) framework, which establishes formal connections between common diagnostic measures—including residual analysis, goodness-of-fit tests, and influence diagnostics—and various uncertainty quantification methods such as prediction intervals, credible regions, and conformal prediction sets. Through extensive simulation studies across diverse data generating processes, we demonstrate that conventional diagnostics often fail to capture important aspects of predictive uncertainty, particularly in the presence of model misspecification, heteroscedasticity, and non-stationarity. Our results reveal that standard diagnostic thresholds correspond to predictable patterns in uncertainty calibration, enabling practitioners to anticipate when traditional models may produce misleading uncertainty estimates. We further develop a diagnostic-weighted uncertainty adjustment procedure that improves predictive reliability by 23-47