APPLYING AN ARGUMENT-BASED APPROACH FOR VALIDATING LANGUAGE PROFICIENCY ASSESSMENTS IN SECOND LANGUAGE ACQUISITION RESEARCH: THE ELICITED IMITATION TEST FOR RUSSIAN

Abstract

This dissertation was motivated by the need for practical suggestions as to what can be done to improve L2 proficiency assessment practices in SLA research. In response, it is proposed that an argument-based approach to validity (Kane, 2006) can be applied to improve SLA test validation practices. To exemplify this approach, a new assessment was developed and interpretive arguments associated with its distinct uses were crafted. The associated inferential claims were then tested empirically. The two intended uses for the new Russian Elicited Imitation Test (EIT) were: controlling participants' initial levels of Russian oracy (study 1) and selecting participants into a study based on their holistic level of speaking and listening proficiency (study 2).

To evaluate the accuracy of the score interpretation for the first use, the EIT was administered to 97 Russian learners in Germany and the USA along with a background questionnaire. To evaluate the accuracy of the score interpretation for the second use, the EIT was administered to 67 Russian learners in the US along with the Russian Speaking Test, a listening comprehension test, and a C-test. Multiple descriptive, graphical, and inferential statistical techniques were employed in the data analysis. Of particular utility in study 1 use were descriptive statistics, item response theory analysis, differential item functioning analysis, inter-rater reliability analysis, and correlational analysis. Key to understanding the effectiveness of the EIT for predicting learners' speaking and listening proficiency in study 2 were descriptive statistics, correlational analysis, ANOVA and Kruskal-Wallis one-way analysis of variance, and 95% confidence intervals.

The Russian EIT was found to constitute a reliable instrument that functions well with Russian learners in the US and Germany. It showed the ability to differentiate between a wide range of oracy skills, and proposed cut scores were identified to enable the prediction of four distinct levels of the speaking proficiency and to predict a threshold of intermediate listening skills. Findings are discussed in terms of their implications for using the argument-based approach for validating assessments in SLA research, and for the use of and future needed research on the Russian EIT and EITs in general.