Evaluating the Role of Missing Data Mechanisms in Biasing Statistical Estimates and Reducing Model Reliability

Posted: Nov 18, 2018

• Scarlett Jones
• John Jones
• Levi Johnson

Abstract

Missing data represents one of the most persistent and challenging problems in statistical analysis and machine learning applications across diverse domains. The conventional taxonomy of missing data mechanisms, first formalized by Rubin (1976), distinguishes between Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) scenarios. While this framework has provided valuable conceptual guidance for several decades, its practical application often fails to capture the complex realities of missing data in contemporary datasets. The fundamental assumption underlying most statistical procedures—that analyzed data represent complete and unbiased samples from the population of interest—becomes untenable when missingness exhibits systematic patterns that correlate with both observed and unobserved variables. The consequences of improperly handled missing data extend beyond mere reduction in statistical power. When missingness mechanisms operate in ways that violate the assumptions of standard analytical approaches, parameter estimates can become severely biased, confidence intervals may fail to achieve nominal coverage rates, and hypothesis tests can exhibit distorted Type I and Type II error rates. More concerningly, in predictive modeling contexts, the reliability of trained models may be compromised when deployment environments differ from training conditions in ways related to the missingness mechanisms. This research addresses critical gaps in the current understanding of how different missing data mechanisms influence statistical estimates and model performance. We move beyond the traditional tripartite classification to develop a more nuanced framework that characterizes missingness along multiple continuous dimensions.