Analyzing the Role of Information Criteria in Model Selection and Complexity Control in Statistical Modeling

Posted: Oct 27, 2019

• James Perez
• Zoey Green
• Levi Scott

Abstract

The selection of appropriate statistical models represents a fundamental challenge in data analysis, with profound implications for inference, prediction, and scientific discovery. Information criteria have emerged as essential tools for navigating the delicate balance between model complexity and generalizability, providing formal mechanisms for comparing competing statistical representations of data. Since their introduction by Akaike and subsequent development by Schwarz and others, information criteria have become ubiquitous in statistical practice, influencing model selection across diverse domains from ecology to econometrics. Despite their widespread adoption, the theoretical properties and practical performance of these criteria in complex, high-dimensional settings remain incompletely understood. This research addresses critical gaps in our understanding of how information criteria function as complexity control mechanisms, particularly in scenarios that challenge their underlying assumptions. Traditional derivations of information criteria typically rely on regularity conditions and asymptotic arguments that may not hold in finite samples or complex modeling environments. Moreover, the increasing prevalence of high-dimensional data and sophisticated modeling techniques has exposed limitations in conventional information-theoretic approaches to model selection. Our investigation systematically examines the behavior of major information criteria across a spectrum of modeling scenarios, with particular attention to their sensitivity to sample characteristics, model misspecification, and structural complexity.