An Update of the Rural Systemic Initiatives Evaluation Study by The Evaluation Center at Western Michigan University

Vol. 1, No. 1 October 2000

Awards to Conduct the Study

In December 1998, the Division of Research, Evaluation and Communication of the National Science Foundation awarded of $577,297 to The Evaluation Center at Western Michigan University to conduct an evaluation study of the Rural Systemic Initiatives Program. Scheduled to be completed by November 30, 2001, and under the direction of Dr. Jerry Horn, the study was designed to accomplish four objectives:

To develop a system of indicators around each of the identified six drivers of educational system reform
To determine the perceived relative importance and value of each of the drivers and indicators for reform in RSI schools in selected communities
To determine the status of innovation and reform within selected communities with respect to factors thought to support or serve as barriers to innovation and education reform
To determine the ways and the extent to which the perceived importance and value of the drivers and the characteristics of the community impact on systemic reform efforts and student achievement in mathematics, science, and technology

The three RSIs selected to be the focus of the study were the Appalachian RSI, the Delta RSI, and the UCAN RSI.

After one year of operation and in discussions with NSF officials, it was decided to expand the study by adding three more RSIs (the Texas RSI, the Coastal RSI, and the Michigan RSI) and to extend the completion date to May 31, 2003. An additional financial commitment of $358,162 was made to the study by NSF, which brings the total amount of the award to $935,459.

Additional Questions to be Addressed in the Expanded Study

What new or different forms of student assessment and teacher effectiveness have been developed and used as a result of curriculum transformation and alignment with state or national standards in science and mathematics, and how were these developed and used?

What contextual factors (within and across collaboratives) serve to support reforms that result in or are associated with the development and implementation of standards-based curricula?

What processes and conditions are essential for effective partnerships within large-scale collaboratives that are characterized by large geographic distances between entities and multiple/potentially contrasting governance arrangements?

What considerations have been given to the use of technology for enhancing the accessibility and effectiveness of math and science instruction and communication within and among collaborative members?

Resource Advisory Team (RAT)

The Resource Advisory Team is a key component of the study. Jerry Horn, as project director, and Craig Russon, as project manager, are the only senior professional personnel in The Evaluation Center with responsibilities for this project; therefore, we call on members from the Resource Advisory Team for expertise and knowledge as needed. This group of ten nationally known professionals provides an array of expertise and experience that is likely unmatched on any project of this kind. This staffing concept is rather unique, but it promises to be a model for other large-scale evaluation studies of this type. In addition to the wealth of expertise, there is an added element of credibility that will likely prove useful as the findings of the study begin to emerge and be discussed among RSI and NSF personnel. With the expansion of the study, we plan to increase the RAT by two or three members. The current membership of the RAT, their geographic location or professional affiliation, and primary area of expertise for this project are found below.

Steve Oliver-U. of Georgia-science education

Mary Harris-U. of North Texas-education reform and standards/curriculum

Gene Hall-U. of Nevada, Las Vegas-science education and education reform

Joseph (Joe) Newlin-National Rural Education Association and Colorado State U.-rural education and school administration

James (Jim) Jess-CAL (IA) Community Schools-rural education, school administration, and education reform

William (Bill) Webster-Dallas Independent School District-testing/achievement assessment

Daniel (Dan) Stufflebeam-Western Michigan U.-standards/curricula and testing/ achievement assessment

Paul Nachitgal-formerly with Annenberg Rural Challenge project, Granby, CO-rural education and education reform

John (Jack) Sanders-SERVE (Greensboro, NC)-rural education and education reform

Larry Enochs-Oregon State U.-science education, standards/curriculum, and education reform

Others who have worked with the project during the first year and a half have been consultants Barbara Havlicek (Manhattan, KS) and Brian Lotven (Stillwater, OK).

Professionals like these have played and will continue to play a valuable role in our continuing studies of the RSI initiative.

ARSI, UCAN, and Delta RSI Case Studies

Based on 3-5 day on-site study visits to each of two communities in each RSI, six case study reports have been completed. The study sites were Rockcastle County (KY), Cocke County (TN), Humphreys County (MS), East Feliciana Parish (LA), Wagon Mound Community (NM), and the Gila River Indian Community (AZ).

As expected, there was a large range of environmental, societal, and educational factors across these communities. Yet, they are clear examples of the diversity in rural America and a reflection of the dedication of professional educators to reform and improve the educational opportunities for their students in science and math. At the same time, we observed some of the limitations, barriers, and problems that the schools and communities have faced in the past and many that they are addressing through the RSI program.

Drafts of the case study reports were submitted to the designated contact person in each community for their feedback and suggestions for corrections or improvements. Copies of the reports were submitted as a part of the annual report to NSF, and they will be available for distribution upon approval by NSF. We think the case studies will prove to be valuable pieces of information to help inform others about rural communities, including the historical development of the area and the educational system. Also, the case study reports include a summary of each case study team's assessment of the extent to which the drivers have been implemented or observable in each community.

Communications and Distribution of Initial Findings

In late October 2000, the project director, Jerry Horn, will make presentations on the project at the annual meeting of the National Rural Education Association in Charleston, South Carolina, and the 20^th Annual Meeting of the KSU Rural and Small Schools Conference at Manhattan, Kansas. Both of these meetings attract a variety of personnel, including teachers, administrators, board members, government officials, parents, and other stakeholders interested in rural America.

Craig Russon, project manager, is updating and expanding a background paper on systemic education reform and NSF's approach via the RSI program. This document, in final form, will become a part of the Occasional Paper Series at The Evaluation Center at Western Michigan University. Lynde Paule from Portland, Oregon, is collaborating with Craig in this effort.

The results of a survey that was administered to a designated contact in each of the ARSI, UCAN, and Delta school district participants have been summarized by Craig Russon and Louisa Stark of Salt Lake City, Utah. The report on this internal study will be designated for dissemination in the near future.

Louisa and Lynde were participants in the NSF- supported MTS Project. This project, now in its 5^th year at The Evaluation Center, is designed to provide training and internship opportunities for persons interested in conducting evaluations of NSF-type programs.

A web site for this project is under development, and it will be operational in late October 2000. We plan to use the web site as a means to make interim findings and plans readily available to interested parties. Also, formal reports, after approval by the National Science Foundation, will be available through the Web.

Visitation and Orientation to the Newly Designated RSIs for the Evaluation Study

Jerry Horn has conducted introductory visits to the Texas RSI, headquartered in Canyon, Texas, and to the developing Coastal RSI, to be centered in North Carolina. Horn made other visits to a leadership meeting of school district personnel in Austin, Texas, in late spring and more recently (September 2000) to a meeting of superintendents in the Virginia component of the Coastal RSI.

In all of these visits and meetings, he and plans for the evaluation study were well received, which has been the case in all visits and discussions with RSI project personnel and participating schools/communities.

For information or questions about the Rural Systemic Initiatives Evaluation Study, contact:

Dr. Jerry G. Horn
RSI Evaluation Study
The Evaluation Center
Western Michigan University
Kalamazoo, MI 49008-5237

(405) 707-7143 / (405) 707-7148 (fax)

email: jhorneval@AOL.com