Construct validation of the reading subskills of the Boğaziçi University English Proficiency Test

Abstract

The purpose of this study is to present construct validity evidence for the reading module of the Boğaziçi University English Language Proficiency Test (the BUEPT). Following the suggestions of Messick's (1989a) validity framework, the study provides evidence for content, substantive, structural, generalisability and external aspects of construct validity of the BUEPT reading module. Initially, a theoretically sound and practically applicable reading framework (Urquhart and Weir, 1998) that would ensure content relevance and representativeness was chosen and test specifications were developed based on that framework. The tests were piloted and the analysis of score distributions and item performance through classical test theory helped improve the technical quality of the tests minimising the construct irrelevant test variance. Expert judgement was taken using an analysis scheme based on Bachman et al.(1995) and verbal protocols of the test takers were analysed in order to investigate whether or not each item reflects the content defined by each dimension of the reading construct as defined in the framework. The factor structures of the tests were analysed using the Principal Component Analysis and the BUEPT reading module was compared to the IELTS reading test both in terms of content congruence and the correlation between them. The findings from these investigations provided substantial support for the validity of the score interpretations based on the BUEPT reading test. The study generally supports the soundness and applicability of the Urquhart and Weir's (1998) framework.