Measuring Heritage Language Learners’ Proficiency for Research Purposes: An Argument-based Validity Study of the Korean C-test
Abstract
Heritage language learners (HLLs) have increasingly become a focus of interest in applied linguistics research (Kagan & Dillon, 2012), but the lack of consistent conceptualization of HL proficiency has hindered the systematic accumulation of research knowledge about HLLs (Son, 2017). Shortcut proficiency measures may be a way to address these shortcomings (Norris, 2018). C-tests have been found to be particularly promising in providing a quick measurement of language learners’ global proficiency (Eckes & Grotjahn, 2006), yet validation studies on this shortcut measure have focused on Foreign Language Learners (FLLs).
To address these critical gaps, this study developed a validity argument (Kane 2006, 2011, 2013) to evaluate the use of an innovative Korean C-test (Son et al., 2018) to assess Korean HLLs and FLLs for applied linguistics research purposes. Ninety-three Korean language learners, 41 HLLs and 52 FLLs, were assessed using five instruments: the Korean C-test, an Elicited Imitation Test (EIT, Kim et al., 2016), ACTFL Oral Proficiency Interview-computer (OPIc), ACTFL Writing Proficiency test (WPT), and a self-assessment questionnaire. The resulting data were then examined for five inferences—theoretical grounds, evaluation, generalization, explanation, and extrapolation—in terms of warrants, assumptions, backing, and rebuttals.
C-test items accurately and reliably distributed Korean learners into a wide range of proficiency levels (IRT person separation index=5.50 and Cronbach’s α=.94). Although C-test performance was closely related to the literacy-based WPT proficiency indicator (ρ=.87), it also correlated strongly with the oracy-based OPIc and EIT proficiency measures (ρ=.81 or higher). Multiple regression analyses showed that both speaking and writing proficiency could explain 80% of the C-test score variance with 𝑅²=.80, 𝐹(2,90)=181.83, 𝑝=.000. Nevertheless, writing was more important than speaking proficiency in predicting C-test performance for HLLs, whereas both language skill predictors were equally important for FLLs. Furthermore, a hierarchical cluster analysis provided a bottom-up categorization of learners into HLLs and FLLs and questioned the expectation that HLLs are always linguistically different to FLLs. Overall, the evidence supported the use of the Korean C-test to assess both HLLs and FLLs across a range of proficiency levels for applied linguistic research purposes. The study concludes with suggestions for future research.
Description
Ph.D.
Permanent Link
http://hdl.handle.net/10822/1053074Date Published
2018Subject
Type
Publisher
Georgetown University
Extent
364 leaves
Collections
Metadata
Show full item recordRelated items
Showing items related by title, author, creator and subject.
-
APPLYING AN ARGUMENT-BASED APPROACH FOR VALIDATING LANGUAGE PROFICIENCY ASSESSMENTS IN SECOND LANGUAGE ACQUISITION RESEARCH: THE ELICITED IMITATION TEST FOR RUSSIAN
Mozgalina, Anastasia (Georgetown University, 2015)This dissertation was motivated by the need for practical suggestions as to what can be done to improve L2 proficiency assessment practices in SLA research. In response, it is proposed that an argument-based approach to ...