Post by Admin on Jan 15, 2014 22:38:37 GMT -6
Mike Hogan recently sent this astonishing and somewhat disappointing article to our group about the use of the Culture-Language Interpretive Matrix. I was planning to use it on an adolescent girl whose first language is Low German. She missed the first 5 years of schooling in Bolivia and came to Canada to get an education. She is struggling greatly in high school now, with modifications to her curricula, despite a sincere effort and extra support. I was hoping this software tool could provide some insight into her situation. See below:
Diagnostic Utility of the Culture-Language Interpretive Matrix for the Wechsler Intelligence Scales for Children—Fourth Edition Among Referred Students
Kara M. Styck & Marley W. Watkins (2013). School Psychology Review, vol 42, issue 4, pp. 367-382.
Abstract. The Culture-Language Interpretive Matrix (C-LIM) was developed by Flanagan, Ortiz, and Alfonso (2013) to evaluate the extent to which developmental language proficiency and acculturative learning opportunity affect the validity of standardized test scores for individual students. According to this approach, validity may be compromised for children with cultural and language experiences, such as learning English as a second language, that differ from the population on which the test was normed. This study employed diagnostic utility statistics to test whether the C-LIM for the Wechsler Intelligence Scales for Children—Fourth Edition (WISC-IV) could accurately distinguish between students from a referred sample of English language learners (n = 86) and monolingual students without disabilities from the WISC-IV normative sample (n = 2,033). Results indicated that the C-LIM identified the English language learner children at chance levels. Evidence from previous studies as well as the current negative results does not support use of the C-LIM for making decisions about individual students [for determining whether or not WISC-IV results for an ELL student accurately represent their true intellectual ability].
MY OPINION: I recently read the original article cited above, and the authors make some interesting points for further research, but concluding that the C-LIM should not be used to make decisions about individual students has not been decided yet in my opinion. First, althought the ELL sample in their research did not have any primary language disorders, the amount of English teaching and exposure was not reported. Moreover, they did not measure the ELL sample's English proficiency with standardized instruments. Therefore, lack of differences between the ELL students and the monolingual English students using the C-LIM could have been do to the similarity in English proficiency between them. In addition, their 86 subjects had unspecified learning disabilities (not your typical ELL student), with Below Average auditory working memory scores, which could have affected how well they fit the expected declining score pattern in the C-LIM. The authors also mention other limitations to their study: a small sample size that could have inflated either Type I or II errors, the absence of matching criteria on characteristics that may have influenced score variation (e.g., age, gender, FSIQ, etc.), and no control to separate the effects of culture and language. In addition, they did not assess the difference between ELL students with and without primary language disorders using the C-LIM, which is another way we use the software. The authors do have an important caveat however about the lack of empirically-derived methods to determine appropriate Cultural Similarity/English proficiency weights for the various subtests categorized in their matrix. Flanagan, Ortiz, and Alfonzo used only expert opinion to assign subtests to various cells in their matrix. More research is needed to determine how valid their clinical judgment was.
Dr. Dawn Flanagan generously made the following comment:
Identification of ELLs is not the purpose of the C-LIM. Why would you need a tool to identify an ELL? If English is not the student's primary language and he or she is learning English, then the student is an ELL. We have some tools to help us determine levels of BICS and CALP in the ELL population....but, that also is not the purpose of the C-LIM. We test ELLs all the time with the WISC-IV and conclude that the results "are considered reliable and valid". Why? because there was no reason for the evaluator to believe that the student's culture and language background adversely affected performance. But what about a situation where we suspect that the results of an evaluation may not be a valid indication of the student's abilities b/c the student's culture and language background is quite different from student's we typically assess? This is where the C-LIM is useful. It provides a systematic means of evaluating patterns in cognitive assessment data to determine whether results suggest a systematic and predictable (based on research) decline as a function of increasing language and culture demands of the test. If an ELL does not have a cultural background that differs significantly from the mainstream; if he has parents who have developed CALP in their primary language and who also speak English; good parental support with regard to homework; student puts forth good effort and has been receiving instruction in English for several years, then it's unlikely that he would demonstrate a pattern on the C-LIM that suggests that the results of WISC-IV are invalid (yet he is still an ELL). Conversely, if a student came to the US three years ago from Pakistan, entered the 3rd grade speaking only Punjabi, but has just tested out of ESL, lives in a Pakistani community in a large city, parents do not speak English, and the family maintains its distinct Islamic identity....if given the WISC-IV, an evaluator may suspect that the results may not be valid. To assist in answering that question, the C-LIM may be used. If this student's performance systematically declines as a function of increasing culture and language test demands based on the C-LIM results, then the validity of the WISC-IV results is called into question. ELLs are not a homogeneous group; they are heterogeneous....if the researchers understood the purpose of the C-LIM, then they would not have set out to determine if the C-LIM could distinguish ELLs from non-ELLs - that was never the intention of the tool and to use it in that way makes no sense.
Dawn P. Flanagan, Ph.D.
Professor of Psychology
School Psychology Program
Department of Psychology
St. John's University
8000 Utopia Parkway
Jamaica, NY 11439