Assessment of textual level EFL reading comprehension: A cognitive and contextual investigation

Abstract

The purpose of the study is to investigate whether test takers can understand a text at whole text level upon the completion of a commonly used assessment technique, the multiple choice test. Attaining comprehension at textual level and the formation of macrostructure is critical in interpreting test takers’ competence in real life or academic contexts. However, if the multiple choice test technique is ineffective in directing test takers towards tackling reading texts in a way that leads to whole text comprehension, accurate interpretations concerning test takers’ future performance cannot be made. To fit the purpose of the study, a test with two reading tasks were created: a multiple choice task and a summary task. Thirty-two undergraduate students from a state university based in İstanbul, Turkey took the test and their performance in macrostructure formation after completing both tasks were quantitatively analyzed. During the completion of both tasks, retrospective verbal protocols were carried out with all participants and their responses during the protocols were categorized to probe into the reading processes participants went through while dealing with both tasks. The findings from these analyses provided substantial evidence towards the inefficacy of the multiple choice test technique in reflecting the ability of test takers’ macrostructure formation and thus pointed at a serious threat towards validity of the test technique. The study also put forward distinctive processes that test takers tended to carry out during multiple choice test completion, which were explanatory in the occurrence of such a deficiency of the technique.