Accommodations for English Language Learner Students: The Effect of Linguistic Modification of Math Test Item Sets

Primary Researchers: Edynn Sato, Stanley Rabinowitz, Carole Gallagher, Kevin (Chun-Wei) Huang

Publication Date: June 2010


Summary and findings. This study on middle school math assessment accommodations explored whether linguistic modifications to the way math test items are presented could increase English language learners’ access to the mathematics being tested and subsequently increase the validity of the test. The study examined students’ performance on two sets of math items (original and linguistically modified) that varied across the three subgroups of students: English learner (EL), non-English language arts proficient (NEP), and English language arts proficient (EP) students.   

The results of the study indicated:

  • Linguistically modifying the language of mathematics test items, as implemented in the study, did not alter the targeted math construct—math understanding—being assessed.
  • The effect of linguistic modification on students’ math performance varied between the three student subgroups (EL, NEP, EP) analyzed and depended on the scoring approach that was used. (i.e., how scores for each student were calculated or estimated).
  • For each of the four scoring approaches analyzed, the effect of linguistic modification was greatest for EL students, followed by NEP and EP students.
  • For all three student subgroups, one dominant factor—math understanding—was found to underlie performance on both item sets.

Regional need and study purpose. The complexity of the language in mathematics test items may interfere with the ability of English language learner students to demonstrate their understanding of mathematics concepts on achievement tests. This study examines whether linguistic modification of such test items affects middle school English learner students' ability to show what they know and can do in mathematics.

Intervention description. Linguistic modification purposefully alters the language of test items, directions, and response options—by reducing sentence length and complexity and using common or familiar words and concrete language—to clarify and simplify the text without simplifying or significantly altering the construct (concepts, knowledge, skills) tested. Experts developed two item sets to measure the effects of linguistic modification on student access to test content: the treatment item set, containing 25 linguistically modified mathematics items, and the control item set, with the original, unmodified items.

Study design and period.
Using a 2-by-3 fully crossed design, the study administered the linguistically modified and unmodified item sets to 4,600 students in grades 7 and 8 in the spring or summer of 2008. The factors were item sets (linguistically modified or original) and student population; grade level (7 or 8) served as a blocking factor. Student population comprised English learner (EL), non-English language arts proficient (NEP), and English language arts proficient (EP) students.

Data collection approach. Data collection involved six sources of data: expert review, cognitive interviews, item tryouts, operational administration of the item sets, student language background survey, and student achievement history data.

Short summaries of this study for distribution are forthcoming. 

  • Published: June 2010
  • Methodologies: Experimental
  • Contact info:
    Neal Finkelstein 415.615.3171


Attachments:


How can we help you?

Do you want to know the latest research or where to turn for information? This free collaborative service from the 10 RELs provides references, referrals, and brief responses to education questions. To ask a question, visit the national portal and select REL West -- or directly.

Search By State