Understanding and Addressing Common-Method Bias in International Large-Scale Assessments The Example of Reading Opportunities-To-Learn in PISA 2009

Response style bias linked to Likert scales jeopardizes the validity and cross-cultural equivalence of constructs in comparative large-scale assessments. In our study, within-item multidimensional CFA and IRT models were implemented in order to disentangle common-method bias from the target opportunity-to-learn constructs in PISA 2009. This approach revealed promising results: evidence was found that the within-item multidimensional models improved the validity of the measurement of opportunity-to- learn (OTL) in the PISA context. With this model, stronger positive correlations between target OTL and reading achievement were observed at the student- and school-levels, whereas the common method dimension on which all OTL items loaded had robust negative correlations with reading achievement on both levels in all of the countries except Finland. Most of the variation in the common method dimension appeared at the student level, but some variation was also observed at the school and country levels. Country-level variation in the common method dimension appears to mainly reflect country differences in the tendency to acquiescence; country-level correlations with an acquiescence index computed on broader sets of PISA items were strong while those with an extreme response style index were rather weak. Further, negative correlations with socioeconomic status (SES) and reading ability were observed at the country level, showing that countries scoring high on the common method dimension are the less affluent and the lowest performing ones in reading.