The Salt Lake Tribune
November 14, 2011
An Analysis of Utah's K–3 Reading Improvement Program
Publication Date: June 2007
Utah's K–3 Reading Improvement Program provides state matching funds to help districts and charter schools achieve the state's goal of having third graders read at or above grade level. All 40 Utah school districts took part in both Year 1 (2004/05) and Year 2 (2005/06) of the program, and the number of participating charter schools increased from 10 (of 12) in Year 1 to 17 (of 20) in Year 2.
With the aim of informing Utah legislators and education officials on program activities, use of funds, and outcomes, Utah's Superintendent of Public Instruction, Patti Harrington, asked WestEd's Regional Educational Laboratory West to review district and charter school participation during the first two years of the program. The Year 1 analysis was presented in early 2006, while the results of this Year 2 analysis was published in June 2007.
To download a 2-page summary click here.
This study addressed the following research questions:
- How have participating school districts and charter schools carried out the provisions of the state legislation?
- What outcomes are evident after two years of program implementation?
Data analysis relied on two sources of data: the year-end Annual Reading Proficiency Reports submitted to the Utah State Office of Education (USOE) in the fall of 2006 by participating districts and charter schools, and English Language Arts Criterion-Referenced Test (CRT) data files provided by the USOE.
The Annual Reading Proficiency Reports contained self-reported data on proficiency goals and use of funds, and the CRT scores were used to obtain aggregated proficiency rates for each program participant in 2004, 2005, and 2006. Researchers reviewed both the self-reported data and the CRT data to get a broad view of reading and literacy levels in participating districts and charter schools. Because the researchers made no site visits and were unable to observe local implementation of any reported practices, this study could not address the quality of local efforts.
Program funds most commonly supported key elements of the state literacy framework, including tiered literacy instruction (a targeted reading intervention model), small-group literacy instruction, and focused professional development. More than two-thirds of program participants also reported funding literacy coaches or reading specialists and training for staff to administer and interpret reading assessments to monitor student progress and guide instruction.
More participating districts and charters reported meeting their self-established goals in Year 2, and statewide grade-level CRT proficiency rates increased slightly for grades 1 and 3 but remained about the same for grade 2. However, it is difficult to assess the program's influence on these changes, as there is no control group and researchers did not observe the local efforts under way in schools.
Limitations of the Study and Implications for Further Research
After the implementing legislation was passed in 2004, the K–3 Reading Improvement Program was immediately embraced, with all 40 of the state's school districts signing on. As a result, no control group exists for comparison, and it is impossible to contrast participants and non-participants. In addition, the WestEd research team did not conduct site visits to observe local implementation of reported activities, so it was not possible to evaluate the quality of the literacy efforts under way in schools and classrooms.
Utah should continue its efforts to see that districts and charter schools have the capacity to ensure that school staff implement research-based literacy strategies correctly and with technical fidelity. The state may also want to consider supporting more research on program implementation, including site visits, to evaluate the quality of the local efforts and the extent to which they meet the spirit of the law. Finally, to allow for more insightful evaluation in the future, as Utah policymakers consider legislating any new program, they may also want to consider mandating a detailed evaluation procedure in the statute or specifying the allocation of program funds based on specific criteria.
- Published: June 2007
- Research Type: Issues & Answers
- Methodologies: Case Study
- Contact info:
Reino Makkonen 415.615.3356