Type I error inflation in DIF identification with Mantel-Haenszel: an explanation and a solution

It is known that sum score-based methods for the identification of differential item functioning (DIF), such as the Mantel-Haenszel (MH) approach, can be affected by Type I error inflation in the absence of any DIF effect. This may happen when the items differ in discrimination and when there is item impact. On the other hand, outlier DIF methods have been developed that are robust against this Type I error inflation, while they are still based on the MH DIF statistic. The present paper gives an explanation for why the common MH method is indeed vulnerable to the inflation effect while the outlier DIF versions are not. In a simulation study we were able to produce the Type I error inflation by inducing item impact and item differences in discrimination. At the same time and in parallel with the Type I error inflation the dispersion of the DIF statistic across items was increased. As expected, the outlier DIF methods did not seem sensitive to impact and differences in item discrimination.