The Development of Frequency-Based Assessments of Vocabulary Breadth and Depth for L2 Arabic

Abstract

Lexical knowledge is an essential component of language knowledge. Vocabulary size has consistently been found to be correlated with other measures of language proficiency and to predict functional language ability. Assessment formats designed to measure lexical knowledge have been influenced by the rich description of lexical patterns made possible by recent advances in corpus linguistics; in particular, these assessments have incorporated word frequency statistics as an organizing structural principle. This dissertation details the development of three frequency-based, web-delivered assessments, modelled on established English as a Foreign Language formats, designed to measure the vocabulary knowledge of non-native learners of Arabic. Word frequency data for these assessments was taken from Buckwalter and Parkinson’s Frequency Dictionary of Arabic. This study represents the first empirical, frequency-based investigation of learner vocabulary knowledge carried out for Arabic.

This dissertation discusses the test development process and identifies challenges inherent to the development of Arabic pseudowords and limitations of both space-delimited strings and raw lemmas as Arabic word units. Following test piloting, 161 non-native learners of Arabic of varying proficiency and experience levels completed the assessments and a learner history questionnaire. Reliability and item performance metrics indicate that these assessments are able to successfully distinguish between learners with varying amounts of vocabulary knowledge.

Other questions investigated in the study include (1) the effect of word frequency ranking on item performance, (2) the ability of learner independent variables to predict assessment performance, and (3) the internal correlation of the three assessment measures. Results indicate that word frequency was significantly correlated with item facility ratings. Learner years of study and self-reported proficiency were both effective predictors of performance on the assessments. Results on the two measures of vocabulary size were highly correlated with each other and with the results of the third format, designed to measure depth of vocabulary knowledge. Aggregate group performance by years of study indicates an acquisition rate of 800 to 1000 words per year, although some caution may be necessary in generalizing the results of this population to the larger population of L2 Arabic learners.