A project with Phil Waggoner (University of Chicago)

A common problem in political research is numerous, competing, and theoretically-derived explanations of a single phenomenon, each claiming superiority. In such overly-complex cases, we suggest researchers employ regularization, which is a statistically efficient method for feature (or “variable”) selection. A second order, yet no less valuable benefit of regularization is the ability to sort between theoretical claims, ultimately reducing model complexity. Regularization, which is widely used in many fields engaged in predictive analysis, has been mostly absent from political science. Thus, upon introducing regularization and three popular penalized regressions (ridge, LASSO, and elastic-net), we demonstrate statistical efficiency by revisiting the question of state Medicaid expansion. In addition to discovering that only a few features matter in predicting state Medicaid expansion, we show that many past explanations do not seem to matter in this question, despite prior assertions in the literature. We therefore suggest any researchers engaged in predictive modeling adopt regularization as a regular part of their research process.

Project image