Reproducibility of Implicit Association Test (IAT) -- Case study of meta-analysis of racial bias research claims

Abstract

The Implicit Association Test, IAT, is widely used to measure hidden (subconscious) human biases, implicit bias, of many topics: race, gender, age, ethnicity, religion stereotypes. There is a need to understand the reliability of these measures as they are being used in many decisions in society today. A case study was undertaken to independently test the reliability of (ability to reproduce) racial bias research claims of Black White relations based on IAT (implicit bias) and explicit bias measurements using statistical p value plots. These claims were for IAT, real world behavior correlations and explicit bias, real world behavior correlations of Black White relations. The p value plots were constructed using data sets from published literature and the plots exhibited considerable randomness for all correlations examined. This randomness supports a lack of correlation between IAT, implicit bias, and explicit bias measurements with real world behaviors of Whites towards Blacks. These findings were for microbehaviors (measures of nonverbal and subtle verbal behavior) and person perception judgments (explicit judgments about others). Findings of the p value plots were consistent with the case study research claim that the IAT provides little insight into who will discriminate against whom. It was also observed that the amount of real world variance explained by the IAT and explicit bias measurements was small, less than 5 percent. Others have noted that the poor performance of both the IAT and explicit bias measurements are mostly consistent with a (flawed instruments explanation) problems in theories that motivated development and use of these instruments.