Edison school test scores do not reflect claims
Feb. 22, 2001
KALAMAZOO -- When it comes to measuring student achievement, Edison schools perform at levels similar to their host districts but fail to make the educational gains Edison administrators claim.
[Go to the WMU Evaluation Center Web site for full copy of the report or for a 28-page executive summary, available in both html and as a pdf file.]
That is the finding of one of the most comprehensive evaluations of student achievement data from students enrolled in schools operated by Edison Schools Inc., the nation's largest private educational management firm. The study was recently completed by two Western Michigan University researchers, Dr. Gary Miron and Dr. Brooks Applegate, who are both part of WMU's Evaluation Center. They analyzed data from across the nation to determine whether the Edison model for education improves student performance on standardized tests.
Their study focused on 10 of Edison's oldest schools---all operating for at least four years---and compared student test data from the Edison schools with data on schools in the surrounding school districts as well as with state and national norms on standardized tests. The researchers sought to determine if Edison students are making gains larger than expected or larger than the district or other relative comparison groups. Edison's annual reports have repeatedly indicated that students enrolled in the firm's schools are making large and substantial achievement gains.
The WMU report--"An Evaluation of Student Achievement in Edison Schools Opened in 1995 and 1996"--examines the track record of schools established during the first two years the company, then the Edison Project, was operating schools. Since then, Edison has become the largest such firm operating schools in the United States, with 113 schools serving 57,000 students in 21 states and the District of Columbia. Approximately half of the schools Edison operates are charter schools. The remaining schools are run under direct contract with local school districts.
The WMU study focused on schools in Colorado Springs, Colo.; Miami-Dade County, Fla.; Wichita, Kan.; Boston, Mass.; Worchester, Mass.; Mt. Clemens, Mich.; Lansing, Mich.; and Sherman, Texas.
"It is clear from our findings that across all schools we studied, Edison students did not perform as well as Edison claims," Miron says. "We looked at these 10 schools because they were the Edison schools with the most years of data available for study. We evaluated student achievement in terms of gains Edison students made relative to comparison groups, as opposed to Edison's preference for evaluating gains made by schools relative to themselves."
Miron says Edison schools do show improvement from year to year in norm-referenced tests--tests that measure gains in student knowledge over time. Examples of such tests include the Iowa Test of Basic Skills and the Metropolitan Achievement Test. Unfortunately, the results from the norm-referenced tests were limited by the small number of students who could be traced from year to year in the data provided by Edison.
"But," says Miron, "on criterion-referenced tests--those that measure whether or not students meet prescribed state standards--Edison students' gains or losses mirror those of students in the comparison groups examined, which included students from surrounding public school districts." An example of a criterion-referenced test is the Michigan Educational Assessment Program, or MEAP, tests.
Using criterion-referenced analyses, the WMU researchers measured achievement trends in 49 different categories across the 10 schools. They found:
Edison student performance often lags behind district performance and almost always is below state performance levels.
In 23 trends measured, Edison students were making smaller gains on criterion-referenced tests than comparison groups.
In 21 of the 49 trends, Edison students' gains or changes in test results were similar to the comparison groups.
In five of the 49 trends, Edison students made larger gains than the comparison groups.
"While students in Edison schools often start out at levels below the national norms and district averages, they progress at rates comparable to students in other district schools," Applegate says. "Unfortunately, this conclusion does not meet Edison's goal, which is to have achievement performance levels that exceed the levels at comparable schools."
Many of Edison's own claims for positive trends in student performance at the schools examined are based on a relatively small number of students, Miron notes. In Edison's own reports on student achievement, the numbers of students reflected in the trends they present are not reported. By contrast, Miron says his team made a conscious effort to report the actual number for all trends reported and to identify the sources of data so that others could replicate the analyses they conducted.
The study was funded by the National Education Association, which sought to assure the autonomy of the evaluators by giving them full control over publication and dissemination of their findings. Both the NEA and Edison Schools Inc. were given advance copies of the report to review and both organizations were provided with feedback on WMU's findings while the study was progressing and the final report was being written.
A 28-page executive summary of the report (in both pdf and html) as well as the full 300-page report (pdf only) are available on the WMU Evaluation Center Web site, <www.wmich.edu/evalctr>.
Media contact: Cheryl Roland, 616 387-8400, cheryl.roland@wmich.edu