The Relationship



Between the



Drivers of Educational Reform



and the



Rural Systemic Initiatives in Science, Mathematics, and Technology Education Program



of the



National Science Foundation



by



Craig Russon, Lynde Paule, & Jerry Horn

The Evaluation Center

Western Michigan University

Kalamazoo, MI 49008-5237





May 2001





This paper was written as part of The Evaluation Center's Rural Systemic Initiatives Study, with funding from the National Science Foundation (NSF). The opinions expressed are those of the authors, and no official endorsement by NSF is intended or should be inferred. The authors wish to thank Gene Hall of the University of Nevada at Las Vegas and James Jess of the CAL (Iowa) Community School district for their comments on an earlier draft of the manuscript.





The Relationship Between the Drivers of Educational Reform and the

Rural Systemic Initiatives in Science, Mathematics, and Technology Education Program

of the National Science Foundation





In 1994, funding of the Rural Systemic Initiatives in Mathematics and Technology Education Program (RSI), the third in a set of systemic reform initiatives sponsored by the National Science Foundation (NSF), was offered on a competitive basis. The RSI program was created to enhance math, science, and technology education in economically disadvantaged, rural areas through community development activities and instructional and policy reform. RSIs are expected to address policy, leadership, and work force issues by involving communities in creating a comprehensive and sustainable system of math, science, and technology education that reflects current advancements in these areas. Instruction and performance standards are expected to play a major role for districts involved in these programs.

In this paper, we discuss how systemic reform can help rural schools overcome some of their special challenges. The paper draws upon the literature from rural education, comprehensive school reform, and educational change, as well as from the effective schools and teacher effects research initiated in the 1970s. Summaries of six case studies conducted as part of The Evaluation Center's study of the RSI program are presented. In the conclusions, we used the theoretical literature as a basis for describing the relationship of the supposed drivers of educational reform and participation by six rural schools in the National Science Foundation's RSI program.

Background

Over the last 30 years much attention has been given to improving students' academic performance. Researchers have identified, catalogued, and disseminated information about hundreds of effective schooling practices that have been found to improve student learning. Schools have made decisions about which practices to implement based on locally determined goals; state standards; student performance on achievement tests; and other factors, including untested but popular movements touted as quick fixes to persistent problems. Typically, schools have selected the educational practices associated with the greatest gains in student performance.

However, the effective practices identified in the research were, by and large, discrete entities (e.g., academically heterogeneous class assignments; flexible in-class grouping); and little was known about their interactions with other practices or their effects over time. Without knowing precisely which combination of practices would provide the best fit with a school's existing set of conditions, educators had to hope that the practices they chose would provide the greatest promise for improving student learning. Further complicating this method of selection was the way in which the practices typically were implemented: stepwise, with little attention to integrating them with current programs or practices. Because decisions about practices did not always consider the school's whole fabric--the contextual, organizational, and instructional factors that impact student learning--hoped-for effects resulting from implementation were not always realized.

School reform efforts implemented in this manner looked like a patchwork quilt, with new sets of practices added at different times by different teachers and "failed" practices (those not achieving their desired results in a given period of time) deleted from the reform package over time. The difficulty with this approach to school reform is that it became impossible to know the reason why any practice succeeded or failed. For example, did a reform succeed because it was implemented as intended or because it interacted positively with other factors? Did a failed reform actually fall short of meeting its expected result because of improper implementation, or did it flounder because it interacted poorly with another reform strategy and its expected effect was thereby reduced? Frustration with this piecemeal approach and its uneven results led educators and researchers to look at a more comprehensive and systemic approach to school reform.

Today, educators and researchers recognize that genuine and lasting school improvement is unlikely to be achieved in a piecemeal manner. Reform that begins with the whole school, not targeted grade levels, or clusters of classes, or programs within the school, is the key to enduring improvement. This holistic approach is known as comprehensive school reform, and it looks at schools as systems wherein effective practices cannot be maximized without attention to all of the system's contextual variables. Comprehensive school reform also attempts to address the interaction of practices that may affect the impact of any given practice, as well as the effects of those practices as they accumulate over time. Because a reform package is tailored to a specific school's needs and unique set of conditions, its contents may, and likely will, look very different from reform efforts at another school. Because of these factors, implementing effective practices in an integrated, sustained, schoolwide fashion holds much greater potential than a piecemeal approach or as a "one model fits all" approach.

Comprehensive school reform. Since single-tactic approaches have not proven efficacious, comprehensive school reform is now widely used to address the multiple elements that comprise the operation of successful schools in an integrated way. Some of the more important dimensions of comprehensive school reform are a carefully aligned curriculum linked to state standards and assessments, consistently and appropriately matched instruction to the needs of students, ongoing professional development tailored to the needs of staff, parent involvement, and school and district administrator support. Further, Datnow and Stringfield, as cited in the October 2000 issue of Research Brief, a publication of The National Clearinghouse for Comprehensive School Reform, listed the following conclusions:

Because of the systemic nature of comprehensive school reform, teachers and school administrators need to be involved from the earliest stages of evaluating their programs and identifying areas of concern to the later stages of implementing and evaluating the reform's effectiveness in meeting targeted outcomes. School and central office leadership must be visible and actively supportive of reform implementation. Outside support (e.g., professional development activities provided by consultants) must be provided as an integral part of the reform effort, not as a one- or two-day training event at the beginning of the effort. Collaborative decision making and discussion about the features of the reform package and their implementation need to take place among school staff regularly and consistently.

For these critical processes to occur, early buy-in from faculty members followed by a groundswell of support is essential. Comprehensive school reform simply cannot be the pipe dream of one or two staff members or the platform of a new superintendent. Although not everyone may be on board in the beginning, identifying areas of concern and the resulting design of the reform package must come from information that is gathered systemically by the people who will be involved in its implementation. Ownership of a reform effort needs to happen from within the school for it to be successful. As a matter of importance, comprehensive school reform takes time--often a lot of time--patience to see it through, and willingness among educators to modify and adjust practices and strategies as different needs arise and findings from ongoing evaluations emerge.

An integral part of any successful systemic reform effort is the inclusion of faculty members at each stage of decision making and planning--from analyzing data and reviewing the literature through designing the program and implementing the new practices and/or programs to evaluating the reform effort. Systemic reform occurs when all essential features of a school are considered and the parts are operating in concert. To enable this to occur, appropriate and adequate resources--time, money, and space--must be factored into the reform effort (Grissmer, Flanagan, Kawata, and Williamson, 2000). By its very definition reform implies that a change of some magnitude will occur, and change requires that adequate and appropriate resources are available for use by those involved in the reform effort.

 

Educational change. The purpose of any reform effort is to help schools accomplish their goals more effectively by replacing some structures, programs, and/or practices with better ones. Change for the sake of change will not help. New programs may make no difference, help improve the situation, or make the situation worse. The failure of educational change may be related just as much to the fact that new programs and/or practices are not truly implemented (real change never occurred) as to the fact that exigent forces in the school's larger environment inhibited change within the school (Fullan, 1991).

The process of educational change is complex and is enhanced by the involvement of individuals who will be affected by the changes: teachers, school and district administrators, program specialists, parents, and students. Also, it requires an awareness of and understanding about how the larger environment in which a school operates impacts what goes on in a school. Successful systemic educational change necessitates a change in beliefs about how to maximize student learning by those involved in the reform effort. Finally, the proposed changes need to be highly valued by faculty members for real change to occur, and staff must share a commitment to change current practices and/or programs (Rowan, 1996).

One example will illustrate the complexity and evolution of comprehensive systemic reform. An elementary school found that its students performed poorly in certain areas of math skills and knowledge on the state assessment test. Student achievement in math had been declining since the state had implemented standards and realigned the state test to match its math standards. Teachers continued to try different teaching methods, but students' scores did not improve. At this point, several teachers decided to take a careful look at the alignment between the state's standards and assessment system with the school's math curriculum. They discovered that the areas where alignment was weak were also the areas where students performed the poorest. After thinking through the implications of a misaligned curriculum, a committee of teachers comprised of representatives from each grade level decided to align their curriculum with the state standards and assessments.

Further examination of the state test revealed that students were expected to demonstrate their learning by showing how they reached their answers to math problems. Students could illustrate their thinking process or explain it in narrative form. Another committee of teachers decided to research different teaching strategies that could be used to teach these skills. The newly aligned curriculum and the research on teaching strategies resulted in the creation of more flexible in-class grouping practices and the provision of clear and focused instruction in problem-solving strategies to smaller numbers of students. This required restructuring the school day to allow each teacher to have fewer students during the math period. In addition, school and district administrators had to be involved in these decisions because additional space and support for in-service activities were needed--even the bus schedule had to be altered. All of these changes required critical resources--time, money, and space--as well as the full support of administrators and parents in the community.

The events that resulted in the changes described in the example took resources and a collaborative culture that included and involved individuals from within and outside the school. It was both top down and bottom up. It depended on a change in beliefs about how to maximize student learning and a shared commitment among all parties involved.

Although the brief description of a school's change process appears sequential, a change process is nonlinear. Various actors may weave in and out of the process at different times, students move to different grade levels and have different needs, new students may enter the school with a variety of special needs, educational priorities of teachers and administrators may change, and school district and state policy may change. Each of these factors by itself or in combination may alter or change the course of the school reform effort. Indeed, Fullan (1999) aptly describes change as a journey with many unpredictable turns. Generally, it is recognized that comprehensive school reform is far from an orderly process and often rather difficult to describe due to the context of the effort.

Because the process is loaded with uncertainty, a sense of chaos frequently accompanies the early stages of implementation when some practices may succeed in reaching their desired outcomes and others may not. Flexibility and adaptability are critical. The reform package must be flexible enough to encompass and accommodate the varying conditions and problems that schools encounter. Indeed, the greatest impediments to comprehensive school reform are misunderstanding and confusion about the design of the reform effort, rules and regulations that must be hurdled, instability in school or district leadership, and preexisting turmoil among faculty members (Grissmer, Flanagan, Kawata, & Williamson, 2000).

For a number of years, Gene Hall, Shirley Hord, and associates have studied change and the predictability of various phases and behaviors. In a recent publication, they (Hall & Hord, 2001) identified 12 principles of change.

Change Principle 1: Change is a process, not an event.

Change Principle 2: There are significant differences in what is entailed in development and implementation of an innovation.

Change Principle 3: An organization does not change until the individuals within it change.

Change Principle 4: Innovations come in different sizes.

Change Principle 5: Interventions are the actions and events that are key to the success of the change process.

Change Principle 6: Although both top-down and bottom-up change can work, a horizontal perspective is best.

Change Principle 7: Administrator leadership is essential to long-term change success.

Change Principle 8: Mandates can work.

Change Principle 9: The school is the primary unit for change.

Change Principle 10: Facilitating change is a team effort.

Change Principle 11: Appropriate interventions reduce the challenges of change.

Change Principle 12: The context of the school influences the process of change.

Continuing with reference to the work of Hall and Hord, they identified seven stages of concern (about an innovation) (p. 63). These stages with brief definitions are listed below in the order of a quasi-developmental order of stages:


Level

Stage

Brief Definition

0 Awareness Little concern about or involvement with the innovation is indicated.
1 Informational A general awareness of the innovation and interest in learning more detail about it is indicated.
2 Personal Individual is uncertain about the demands of the innovation, his/her inadequacy to meet those demands, and his/her role with the innovation.
3 Management Attention is focused on the processes and tasks of using the innovation and the best use of information and resources.
4 Consequence Attention focuses on impact of the innovation on clients in his or her immediate sphere of influence.
5 Collaboration The focus is on coordination and cooperation with others regarding use of the innovation.
6 Refocusing The focus is on the exploration of more universal benefits from the innovation, including the possibility of major changes or replacement with a more powerful alternative.

They go on to emphasize that individuals within an organization will likely be at different stages of concern and that the order of concern, i.e., from one level to another, may not be direct or lineal for a number of reasons. The systemic reform effort of the National Science Foundation, which is described later, includes planned interventions; and each Rural Systemic Initiative (RSI) collaborative employs various interventions and does not adhere to the same time or event schedule. Thus, it is important to understand the complexities of the context in which the innovations/-change/reform is occurring and the type and intensity of the concern being experienced by teachers and other stakeholders.

National Science Foundation Rural Systemic Initiatives in Science, Mathematics, and Technology Education

The National Science Foundation promotes and advances scientific progress in the United States by awarding competitive grants for research and education in the sciences, mathematics, and engineering. The NSF Rural Systemic Initiatives program was developed to promote systemic reform of science and mathematics education in schools and districts in rural, economically disadvantaged areas. The RSI program encourages the development of strategies that will result in sustainable, adaptable, and systemic improvements in science, mathematics, and technology. RSI also emphasizes the importance of developing a broad constituency and brings together education, economic, and community leaders as partners to allow the development of a comprehensive plan for community development.

The RSI grant application guidelines state that a successful proposal will include a well-developed plan that contains six essential elements that NSF has found to result in successful systemic reform:

The need or basis for this program is reflected in this statement in the NSF publication.

National tests in science and mathematics achievement indicate a performance gap across ethnic groups and regions of the Nation. This gap has been attributed to a number of factors, but is strongly linked with the level of economic poverty of students and the regions in which they reside. Course-taking patterns are strong predictors of the likelihood that students will enroll and succeed in advanced science and mathematics courses or in programs that lead to scientific and mathematics courses or in programs that lead to scientific or technical degrees. Therefore, the lack of students' access to appropriate science and mathematics courses in school severely limits their educational career choices and their ability to be competitive in the increasingly technical workplace. Data show that students in extreme rural or disadvantaged urban areas receive the least exposure to science and mathematics courses. (p. 1)

The Systemic Reform Drivers flesh out NSF's six essential elements and are used by prospective grantees to develop proposals as well as to evaluate an RSI's effectiveness. With the exception of the program's specific emphasis on broad-based support from the larger community of parents, institutions of higher education, business and industry, and foundations, the Systemic Reform Drivers echo the essential elements of comprehensive school reform discussed earlier in the paper. However, despite their being developed for implementation in a rural setting, some of the drivers conflict with existing conditions in rural areas. Indeed, the values and beliefs of some rural residents run contrary to the tenets and assumptions of systemic reform. In the following section we discuss the Systemic Reform Drivers and provide examples from RSIs that are a part of the larger WMU study of the RSI initiative for the National Science Foundation.

RSI Systemic Reform Drivers

Driver 1. Implementation of comprehensive, standards-based curricula as represented in instructional practice, including student assessment in every classroom, laboratory, and other learning experience provided through the system and its partners

Smith and O'Day (1991) and other systemic reform advocates have called for the establishment of state curriculum frameworks (based on national standards) that define what students should know and be able to do. These frameworks would provide direction and vision for significantly upgrading the quality of the content and instruction within all schools in the state. Unfortunately, some rural schools may have difficulties bringing their curricula into compliance with the applicable framework. Traditionally, schools in small, rural communities have had trouble maintaining curricula with the depth and breadth to meet all their students' interests and needs. Sometimes rural schools cannot offer fundamental courses such as algebra or physics, let alone more specialized courses that are favored by systemic reform advocates (National Network of Regional Education Laboratories [NNREL], 1999). Among the reasons for this are those listed below:

As one can see, the implementation of a comprehensive, standards-based curriculum in every (rural) classroom may actually have less to do with curriculum reform and more to do with economic and human resources and information access constraints to the system. Past attempts to reform curriculum may not have taken these constraints into consideration. For this reason any attempts at curriculum reform have realized limited success.

Advocates of systemic reform say that eliminating constraints to curriculum reform should promote increased success. Curriculum reform that avoids an overly rigid application of standards, such as that offered by the National Science Education Standards (National Research Council, 1995) and the Curriculum and Evaluation Standards for School Mathematics (National Council of Teachers of Mathematics, 1989) may actually play to the strengths of rural education.

One theme that is pervasive throughout the standards is the importance of hands-on, experiential learning. Students in rural areas often have access to agriculture and natural habitats in which to make observations and conduct science experiments. It would be an error to consider that all rural America is related to an agriculture base, but living in a rural environment and in a community that values education provides a unique and powerful context for experiential science instruction (NNREL, 1999). For example, when one of the authors lived in rural Iowa, his children's school classes would walk across the street to a cornfield to observe the developmental phases of the plants. Field trips to the local dairy were a special occasion. Other opportunities are available through the Cooperative Extension Service (including 4-H) and the FFA (formerly known as Future Farmers of America).

Opportunities also exist for hands-on, experiential learning of mathematics in rural schools. At a basic level, real-life experiences and applications of mathematics are used in calculating volumes of grain storage units, yield of crops per acre, application rates for fertilizer and irrigation water, etc. Many statistical concepts that are commonly used today were first developed as a result of research in the field of agronomy. At an advanced level, students might replicate agronomy experiments to learn about means, variance, standard deviation, and analysis of variance.

RSIs may face some curriculum reform challenges that other systemic initiatives do not. For example, a review conducted by the Council of Chief State School Officers (1998) showed that all 50 states have in place a set of content standards or a curriculum framework. Many of these appear to be based on the national math and science standards mentioned above. However, all of the RSI projects cross state boundaries. The question of which set of content standards and/or curriculum framework should be applied to the project may arise and complicate the operations of the project or require adjustments from state to state. But, it is not an issue that the RSIs will or attempt to resolve or for that matter even debate. Authority for K-12 education rests with the states, and not the federal government. In the main, schools are required to adopt state standards and may voluntarily accept or apply national standards. However, the RSIs can play a huge role in facilitating the understanding and implementation of recognized standards or curricular guidelines.

Driver 2. Development of a coherent, consistent set of policies that supports provision of high quality mathematics and science education for each student, excellent preparation, continuing education, and support for each mathematics and science teacher (including all elementary teachers), and administrative support for all persons who work to dramatically improve achievement among all students served by the system

One reform critic thinks that current education policy is characterized by contradiction and ambiguity because of the segmented organizational structure, emphasis on elections, policy overload, and specialization (Fuhrman, 1993). O'Day and Smith (1993) argue that alignment of state education policies would provide a coherent structure to support schools in developing effective strategies for teaching the content of standards and/or curriculum to their students. Other advocates of systemic reform favor the implementation of a centralized/decentralized reform process in which schools have a substantial amount of freedom in interpreting the centrally developed curriculum framework (Fuhrman, 1993).

The decentralized part of the reform process that Fuhrman describes would probably not require a big adjustment for rural schools since they have always enjoyed a high degree of autonomy. The type of centralization that would support improvement in rural schools includes policies that provide a high degree of autonomy at the building level; foster collaboration among schools and teachers, schools and communities, and teachers and administrators; and create effective channels for communication and staff development (NNREL, 1999).

Policy that supports building-level autonomy is important because state education policy is sometimes made with little regard for rural schools. Rural schools have sometimes found it necessary to depart from district or state policies and regulations to improve instruction. Building-level autonomy gives schools the flexibility to do what is in the best interest of their students. The emerging presence of charter schools and other forms of public schools of choice may provide some interesting opportunities for creating schools for which many legislative/state agency rules and regulations are waived.

Policy that supports collaboration among schools and teachers, schools and communities, and teachers and administrators is important. Rural schools cannot always count on the support of outside stakeholders such as institutions of higher education, foundations, and policymakers. Therefore, it is especially important that policies be developed to foster collaboration among local stakeholders. A self-reliant group of rural teachers, administrators, and community leaders can go far in helping rural schools deal with unique challenges.

Policies that support effective channels for communication are important because they can help rural school personnel overcome the sense of isolation they feel at times. Policies that support staff development are important because rural schools often have trouble recruiting and retaining qualified staff. It is important that rural schools maximize the human resources that they do have through staff development.

Driver 3. Convergence of the usage of all resources that are designed for or that reasonably could be used to support science and mathematics education-- fiscal, intellectual, material, curricular, and extracurricular--into a focused and unitary program to constantly upgrade, review, and improve the educational program in mathematics and science for all students

Inadequate fiscal resources are a widespread problem for schools (NNREL, 1999). When a state legislature cuts the education budget, the department of education often responds by cutting back on programs, delaying funding, or mandating programs without providing funding. It then falls to residents of local districts to try to make up for the lost funding, a particularly difficult problem for rural schools.

Studies have shown that young people often leave rural areas to seek employment opportunities in urban and suburban areas. The older people who remain behind often live on fixed incomes and are less enthusiastic about approving school bond referendums because it means higher taxes. Budget cutbacks, along with stops and starts in reform initiatives, result in nonimplementation or incremental or cheapened versions of reform programs, as well as educators' loss of trust in the system (NNREL, 1999).

While most schools share the problem of inadequate fiscal resources, human resources pose a challenge that is unique to rural schools. Many factors work against rural schools' abilities to obtain and maintain a high level of human resource support for their programs and students. Rural geographic and cultural isolation often stand in the way of hiring or retaining high quality staff, as do the relatively low pay of rural educators and the stress they experience (NNREL, 1999). This problem is amplified in small schools that attempt to offer advanced and highly specialized courses in math and especially the sciences. Issues of teacher qualifications, appropriately credentialed teachers, and the lack of professional opportunities for advanced education are regularly faced by small schools in rural areas.

Driver 4. Broad-based support from parents, policymakers, institutions of higher education, business and industry, foundations, and other segments of the community for goals and collective value of the program, based on rich presentations of the ideas behind the program, the evidence gathered about its successes and its failures, and critical discussions of its efforts

Rural schools can usually count on strong support from parents. Whenever an event was held in rural Iowa, where one author lived, the building was always packed with parents. When volunteers were needed to install new playground equipment, twice as many parents showed up as were needed. Athletic and other scheduled school events were social affairs for the entire community, not simply events for students and their parents. School events seem to bring together multiple communities within a consolidated district, even though previously they may each have had their own K-12 schools and had been long-term competitors. Certainly, there are exceptions to these observations, as numerous school consolidation efforts have divided friends and families alike.

Schools in rural areas can usually count on strong support from community leaders, businesses, and industries. There may be an economic component to this support. A substantial amount of research chronicles the demise of towns after a rural school is closed (NNREL, 1999). Business leaders may see support for the school as a strategy for maintaining a prosperous economic environment. However, the authors suspect that much of the reason for this support is that schools are usually the focus of community identity. Most rural people take pride in where they live and naturally want to support their schools.

Unfortunately, rural schools have not always been able to count on the support of institutions of higher learning. Isolation and distance have combined with inconvenient scheduling of professional development programs to make it difficult for rural educators to take advantage of opportunities offered by colleges, universities, and intermediate service agencies. Even when available (as in-service or preservice offerings), most of these programs have a general emphasis and tend not to emphasize the rural context, making them less useful than they could be. One way that rural schools are overcoming this barrier is through the use of telecommunications and information technology (NNREL, 1999).

In addition, rural schools have not always been able to count on the support of foundations and other funding agencies. This is partly because administrators in rural schools wear many hats. In addition to administrative duties, administrators sometimes teach, coach, counsel, and teach drivers education. When fulfilling all their primary duties, rural school administrators usually have no time to write competitive proposals to attract the funding from major foundations that is needed for pursuing attainment of the national education goals. As a result, competition for available grant monies often tends to be skewed in favor of larger urban and suburban school districts (NNREL, 1999). When a grant is obtained, the amount of funds sometimes is based on the number of eligible students or teacher beneficiaries or has other eligibility requirements that a small or rural school simply cannot meet. In small schools the amount of money that can be received is hardly worth the time and effort to apply for, much less to administer and account for programmatic activities and expenditures.

Lastly, rural schools have not always been able to count on the support of policymakers. In 1918, one-half of all U.S. residents called rural America home. Sixty years later, only one-fourth of the population was rural. Today, in only 15 states does 50 percent or more of the population live in nonmetropolitan areas. The interests of governmental agencies has shifted to urban and suburban issues at both the state and federal levels (NNREL, 1999).

Driver 5. Accumulation of a broad and deep array of evidence that the program is enhancing student achievement, through a set of indices that might include achievement test scores, higher level courses passed, college admission rates, college majors, advanced placement tests taken, portfolio assessment, and ratings from summer employers that demonstrate that students are generally achieving at a significantly higher level in science and mathematics

According to Gullickson (2000), sound assessment of student achievement is an essential ingredient to strong educational programs and is probably the most common and pervasive aspect of student instruction. It is a tool for guiding student development, crossing all academic disciplines. Certainly, assessments of student achievement occur in all classrooms and regularly confront students and educators with a wide variety of decision situations that affect their educational development. Such decisions include matriculation, admissions, grading, tracking, instructional decisions for individual students, discipline, and merit awards.

Gullickson (2000) further states that when assessments of student achievement are not sound, educational programs suffer. Poor student assessments victimize and harm students. When questionable assessment practices are employed in high-stakes situations, the negative consequences are likely to draw attention. In addressing the effects of poorly conducted large-scale assessment activities, several authors have noted their concerns about the increase in the number of high-stakes decisions based on limited student information (Brandt, 1989; Campbell & Greenberg, 1993; National Forum on Assessment, 1995; Pollard, 1993; Shepard, 1989).

While high-stakes assessments of student achievement capture high profile attention, far more assessments touch individual students on a daily basis. These more obscure assessments of student achievement can be equally devastating for individual children when conducted poorly. For example, students, their parents, and others use information from assessments to set goals and expectations. If the information is incorrect or unsound, individuals' decisions are likely to be academically, economically, and socially harmful to the student's growth.

For these reasons, NSF advocates the mixed method approach to assessment. The Foundation realizes that educators need to be certain they are conducting appropriate assessments for each student and that the results given to the student and others are accurate. Mixed method approaches are recognized as having greater validity and reliability than single methods approaches. Additionally, using different sources and methods of assessment can strengthen each type of data collection and minimize the weaknesses of any single approach (Frechtling & Sharp, 1997).

Driver 6. Improvement in the achievement of all students, including those historically underserved

At the center of the content-based systemic reform is the tenet that all children should have access to the new challenging content and should be expected to learn this content to a high standard of performance. The assumption that underpins this tenet is that academic content, complex thinking, and problem solving are necessary for responsible citizenship in a diverse modern society. A second assumption is that all children can acquire these skills (O'Day & Smith, 1993).

These assumptions are at odds with the prevailing values and beliefs held by some rural people. Royster (1994) identified the following values and beliefs that run contrary to the tenets and assumptions that underpin systemic reform: "A . . . low value placed on education, low self-esteem perpetuated by a welfare system, low expectations for students' educational achievements due to parents' life experiences and regional values, . . . lack of awareness of role of education in student future, . . . [and] willingness to accept different academic performances and standards for different groups" (p. 70).

It would be easy to dismiss the values and beliefs described above. However, research has found that the change process can be more successful if the values and beliefs of participants are taken into consideration (Hall & Hord, 1987). Then interventions may be adjusted to be synchronized with the viewpoint of the target group. Facilitators might also work with rural people to raise their levels of concern and to make them more receptive to systemic reform.

Case Studies


The preceding section of the monograph focuses on the rural education reform literature that was published prior to the outset of the RSI evaluation study. The Evaluation Center's commitment to the National Science Foundation included visits of three to five days to six selected communities. During the site visits a profile of each community was developed based on the results of an exploratory survey. Data-gathering activities during site visits consisted of reviewing documents, conducting interviews, and observing community interactions and events. Project staff and at least one expert in rural education/communities and/or science-math education conducted the site visits. The results of the site visits were written up in a series of case studies, which are an important addition to the professional literature. They provide insights into why some RSIs have enjoyed degrees of success and others have struggled in their attempts to implement significant reform and improve science and math student achievement across all school that have chosen to be a part of this initiative. This section contains summaries of the case studies. Each summary has a narrative description of the case followed by an assessment of the extent to which the six NSF drivers were apparent in each school/community. The case study summaries are followed by a cross-case analysis.

Title: A Case Study of Rockcastle County (Kentucky) School District and Its Role as a Partner in the NSF-Supported Appalachian Rural Systemic Initiative

Author: Jerry Horn

Rockcastle County is located in east central Kentucky and covers a land area of 318 square miles with an estimated population of 15,744 in 1997. The topography of the area would appropriately be described as picturesque, with wooded hills and valleys. Of the early explorers to the area, the most famous is probably Daniel Boone. Settlers followed Boone down the Ohio River and through the Cumberland Gap. Many early efforts to establish settlements failed because of skirmishes with Indians of the area or for other reasons. Kentucky became a state in 1793, and Rockcastle was approved as a county in 1810.

During the same year that Rockcastle was established as a county, an educational academy/seminary was established. In 1837-38, the Kentucky legislature provided a state school fund to support free public schools. In 1908, the state dictated that each county would comprise one school district except for cities or towns that might maintain a separate district. The ensuing consolidation lives on today. There are five schools in the county: Brodhead Elementary, Mt. Vernon Elementary, Roundstone Elementary, Rockcastle Middle School, and Rockcastle High School.

Kentucky has engaged in major statewide educational reform for almost 10 years. In response to court decisions about the inequities of educational opportunities, the state legislature passed the Kentucky Education Reform Act (KERA). KERA helped reduce the influence of "politics" in funding and regulations for school operation and required the development of organized planning efforts. In 1990, local plans were integrated with KERA requirements. As a result of state and local efforts, major changes occurred in education, not the least of which was an equalization of resources across the state.

The positive experience that the district had with previous education reform efforts probably influenced its decision to participate in the Rural Systemic Initiative. Rockcastle is 1 of 44 counties in 6 states that participate as a part of the Appalachian Rural Systemic Initiative (ARSI). "ARSI builds on local efforts and coordinates people and resources to the fullest advantage of the students. Keeping local educators in the driver's seat, ARSI adds value to reform efforts through its Resource Collaboratives, Community Engagement, and Resource Awareness activities as well as on-going partnerships with local and national mathematics and science initiatives" (Royster & Smith, 1999).

Rockcastle School District receives several services through ARSI. The district receives support for the part-time release of a Teacher Partner who is charged with the task of acquiring more in-depth knowledge about math and science, planning and implementing research-based instruction, providing hands-on learning opportunities for students, serving as a mentor for other teachers, and making resources available to colleagues.

Other forms of assistance and support include use of national and local mathematics and science experts to provide Teacher Partners with localized professional development. Teachers have almost unlimited access to standards-based curriculum materials through the World Wide Web. Teacher Partners can obtain assistance in developing classroom lessons for their unique circumstances. ARSI staff can provide specific strategies for adopting instructional materials, aligning curricula, and analyzing students' needs based on math and science assessment results.

ARSI also coordinates math and science program audits, helping districts develop educational plans that identify strengths and weaknesses in each school. As a result of the audits, recommendations are made for improvement. Opportunities are provided for teacher networks that promote the exchange of innovative ideas among communities throughout the region. In addition, assistance with local school level web sites is available. Catalyst schools in each district are established as model schools. In the Rockcastle School District, the middle school and high school were designated as catalyst schools.

Evidence of the presence or progress toward fulfilling the intent of the six drivers for educational system reform, as disseminated by the National Science Foundation, was a major focal point of the visitation team's work. In the following section, these findings are summarized.

Driver 1: Standards-Based Curriculum, Instruction, and Assessment. The researchers concluded that there are clear and concerted efforts to coordinate a standards-based, high quality curriculum. Data suggested that the elementary schools are most active in the ARSI-led reform. All teachers report an increase in in-service or professional development activities. There is some question as to which group is responsible for the increase: ARSI, some other group within the district or state or, more likely a combination of a number of improvement efforts now and in previous years.

Driver 2: Coherent, Consistent, and Supportive Policies. The central administration of Rockcastle School District shows particular strength in implementing policies that improve professional development, encourage support, and provide opportunities for all students. In response to ARSI program audits, elementary, middle, and high school principals have participated in professional development activities. The principals are seen as instructional leaders in their schools. Although teachers credit ARSI with improving professional development, some think there are too many school programs competing for time and attention.

Driver 3: Convergence of Resources. The district has a long and ongoing involvement with a number of funding entities. As a result, it is difficult to specify percentages dedicated to RSI efforts. (ARSI does contribute $15,000 toward the resource teacher's salary.) Anecdotal reports from district and building level administrators seem to reflect a strong and laudable integration of the various initiatives. This integration has resulted in computers in every classroom, Internet access, graphing calculators, satellite capabilities, etc. However, there does not appear to be a lot of utilization of the technology.

Driver 4: Broad-Based Support. Parents with low levels of education have contributed to the lack of value for education among many of the students. To counter this problem, schools have tried to encourage broad-based support for program goals through a variety of means. School site councils are given broad policy and budgetary powers. Close communication is maintained with parents and parent groups. (One elementary school principal reported the creation of a Parent Network that meets once a month "just to talk about school.") Collaborative efforts are undertaken with area colleges and universities. No direct evidence of major involvement of business and industry in math and science education was observed.

Driver 5: Enhanced Student Achievement. Kentucky is widely known for its high stakes testing program. Rockcastle School District has even gone above the state requirements for student testing. Results show that, in the early grades, students in the county have Comprehensive Test of Basic Skills and Terra Nova scores equivalent to those in the rest of the nation. Throughout elementary and middle school there is a puzzling drop in test scores. The site visitation team concluded that the effort to improve math and science is there, but it doesn't show in the test scores.

Driver 6: Achievement of Underrepresented and Underserved Groups. While few members of minority races or ethnic groups live in the community, there appears to be considerable variation in socioeconomic status. Sixty-two percent of the district's students are eligible for USDA's free or reduced lunch program. This is a historically underserved population. Unfortunately, test data and other measures of achievement were not disaggregated by economic level, so there is no way to know how academic achievement for this group has improved.

Title: A Case Study of the Cocke County (Tennessee) School System and Its Role as a Partner in the NSF-Supported Appalachian Rural Systemic Initiative

Authors: Jerry Horn, Steve Oliver, and Daniel Stufflebeam

Cocke County is located directly against the mountain range that became the western boundary of North Carolina and the edge of the East Tennessee Valley. It lies almost halfway between Chicago, Illinois, and Tampa, Florida, and is becoming an attractive area for retirement. Spurred by the fame of country singer Dolly Parton and the development of "Dollywood," the area has also become a major entertainment center. The development provides employment for many Cocke Country residents in seasonal tourist-related jobs as well as in other jobs that provide goods and services in support of the industry.

Cocke County was once home to the Cherokee Indians before their removal to Oklahoma in 1838, along the infamous "Trail of Tears." In Oklahoma, the Cherokees became known as one of the "five civilized tribes" and have become one of the most respected and powerful tribes. The Cherokees who chose not to accept the "offer" to leave the area were allowed to remain, and many eventually gained legal ownership of land now known as the Cherokee Indian Reservation located near Cherokee, North Carolina.

Many of the early settlers in Cocke County were of Scotch-Irish descent, and they were known as roamers and adventurers. These settlers were poor; with little more than the clothes on their backs, a rifle, an ax, gunpowder, and cooking pots, they began to carve out homes in some of the most rugged and isolated lands possible. But they found individual freedom and a place of their own in the process. They cleared trees from rich lands to be farmed later, and they cut trees, constructed log cabins, built churches and schools, and developed towns.

Schools were built in virtually every community. Some of the first teachers came over the mountains (from North Carolina) to teach in missions or in the churches that dotted the landscape. Oftentimes, the teachers rotated their residence among the families of the children, and this became one way in which many of the adults learned to read and write.

During the Civil War, the people of the area suffered physically and emotionally. To some, it became known as "The Brothers' War," and it split families along ideological, religious, and economic lines. It is from the Methodists of East Tennessee, who took a stand against slavery, and the support and influence of the Quaker and Presbyterian faiths that the "volunteers" of Tennessee became a reality for the Union. After the war ended, families put back together their lives, lands, and homes that had been figuratively and literally pillaged by the war efforts of both sides.

Over the years, from the time of new settlements with individual schools (often 1-room, multiple level elementary schools) to the time of several separate school systems (1 per any town of a reasonable size) to the eventual consolidation of these small town school systems to a county system, there has been a predictable evolution of a comprehensive school system for Cocke County. Today, the Cocke County school system has approximately 4,500 students enrolled in 8 K-8 schools, one 9-12 school, and 1 K-12 school. In addition to these schools, the Newport Grammar School serves students in grades K-8. Started in 1897, this is one of the oldest continuing schools in the area, and it has an enrollment of more than 700 students.

It is interesting that there are no middle schools in this system. While some educators have pressed for a middle school, there is a turf issue. Each school wants to keep its own school intact, its own board member, and its athletic teams. These desires would be in jeopardy if the system were to develop a large middle school and move all the grade six through grade eight students to that middle school. Interscholastic competition is clearly present among the several elementary (K-8) schools, and athletic trophies and team mascot images are proudly displayed. In essence, the school teams represent communities that have traditionally taken great pride and ownership of the school athletic teams and other student groups.

The Cocke County schools' 1998-99 Report Card provided the basic characteristics about the system. The school system had about 4,638 students, of which 96.4 percent were white, 2.1 percent black, 0.3 percent Asian, 1 percent Hispanic, and 0.3 percent American Indian. The overall attendance rate that year was 94.8 percent. Nearly 4 percent of the students dropped out, while 14.5 percent of the entering ninth grade students had dropped out by the end of the 1998-99 school year. We were told that the dropout rate has been improving, but this was not borne out by the data we reviewed. Many students do not go to college because of lack of resources. It was also reported that high-achieving students avail themselves of scholarship opportunities. It was said that the counselors do a good job of informing the students of these opportunities and helping them make strong applications.

Cocke County School District is one of 44 counties in 6 states that participates as a part of the Appalachian Rural Systemic Initiative (ARSI). ARSI's main thrust is through its resource collaboratives. These collaboratives, located at five partner institutions (Clinch Valley College at the University of Virginia, Marshall University, Ohio University, University of Kentucky, and the University of Tennessee) provide information and services to the school district. The primary point of contact with the district is the resource teacher, and ARSI pays for approximately half of this salary. The resource teacher's job description includes the following objectives:

In Cocke County, the resource teacher set up a network of teacher partners who meet on a regular basis. Most importantly, these teacher partners serve as the conduit for the dissemination of ARSI materials to the teachers in the schools. ARSI also pays for attendance at monthly meetings, with the county school system supplementing required travel funds as necessary. ARSI brings excellent professional development opportunities to teachers and others in the system. Administrators say that workshop participants have brought back tubs of useful materials to teachers. Staff members cited the ARSI-associated curriculum audits as helpful. It was said that ARSI gives them a lot of information.

Other forms of assistance and support through ARSI's resource collaborative include use of national and local mathematics and science experts to provide teacher partners with localized professional development opportunities; almost unlimited access to standards-based curriculum activities and educational materials through World Wide Web sites; help for teacher partners to develop classroom lessons for their unique circumstances; and specific strategies for adopting instructional materials, aligning curricula, and analyzing students' needs based on mathematics and science assessment results. In ARSI description materials, it said that "Teacher Partners, in turn, broaden ARSI's impact by sharing lessons learned through hands-on experiential opportunities with other teachers in their district."

This takes the form of coordinating mathematics and science program improvement reviews. The reviews help districts develop educational improvement plans that identify strengths and weaknesses in each school along with recommendations for improvement. Opportunities are provided for communities throughout the region to exchange innovative ideas and network among peers. In addition, assistance with local school level web sites is available.

Driver 1: Standards-Based Curriculum, Instruction, and Assessment. A new accountability system has been put in place in which teachers are judged based on the test scores of their students. As a result, the curriculum is being modified across the grades to emphasize those things that are on the exam and to deemphasize those things that are not. This act overshadows any curriculum reform or desire to teach inquiry. At one of our meetings, one teacher said, "If it's not on the test, I won't teach it." The teachers are doing this with the full knowledge of the principals. All persons involved are very concerned that the exams are driving the curriculum. For the most part, this is seen as negative, but some teachers and principals disagree.

Driver 2: Coherent, Consistent, and Supportive Policies. The greatest policy impact occurring in the Cocke County schools is related to the statewide assessment that is being put in place. There is almost unanimous resistance throughout the system to the notion of judging teachers based on the test scores of their students. Most of the system personnel we interviewed neither understood nor trusted the value-added testing system being used throughout Tennessee. A variety of school system staff emphasized that it is not fair to evaluate teachers based on student test scores because there are simply too many factors that go into a student's test performance and the testing period takes place in a concentrated period during only one small segment of the long school year.

Driver 3: Convergence of Resources. While there is not a systematic plan to converge resources on science, math, and technology, there appears to be an informal understanding that math is a primary focus of the district followed by science and technology. As a result, science and math probably receive more than their proportional share of resources and administrative attention. Further, various title programs--i.e., I and VI--are used when possible to support professional development efforts within the district. While not necessarily a policy or declared long-term practice, leftover funds from other accounts have been used to make one-time purchases of specific science and math materials, including microscopes. In the educational media/resource center, the director explained that science and math were areas for which there were a lot of requests for materials; therefore, she tried to give priority status to these areas.

Driver 4: Broad-based Support. The most visible efforts of attempts to involve stakeholders are related to the work of the Community Engagement Team as it provides guidance and direction for the resource teacher and her work with the network of school-based Teacher Partners. The team is composed of key personalities in the community who have clout and visibility. However, most parents of students in this school system are not directly involved in the reform planning or activation of the effort. Whether this is due to the relatively low educational level of many parents or whether this is consistent with similar activities in the county is not clear. The board members are elected by the citizens and, as such, have considerable influence in decision making that directly affects personnel and resource decisions.

Driver 5: Enhanced Student Achievement. Various reports and manuals were given to us to help us see the kinds of achievement trends in science and mathematics over the past several years. Generally, these provided indications of the extent to which recent test scores have been better than those in previous years and also contrast with similar mean scores or mean gains at the state and national levels. The broad array of information reflects Tennessee's longstanding, systematic, and comprehensive approach to student achievement evaluation that was pushed by the previous governor, Lamar Alexander. While the testing program is systematic and comprehensive, it is also confusing to many educators in Cocke County. In response to a request for information on the schools' testing program, a principal provided a document entitled Using and Interpreting Tennessee's Value-Added Assessment System (Bratton, Horn, & Wright, 1999). This manual seems to be technically defensible. However, it also appears to be almost incomprehensible to generalist educators. In response to questions about this manual, respondents said, "We don't understand it," "We doubt if almost anybody in Tennessee understands it," or "I have never seen such a manual." The principal who loaned the manual to us said she must have it back, because it is the only one in the school. She may have been the only person in the school who had seen the manual.

Driver 6: Achievement of Underrepresented and Underserved Groups. The achievement data we saw suggested that over the last five years there has been a gain in the writing performance of students, but no clear trend was evident in other subject matter areas. (The gains in writing are so great that one suspects a different overall basis for determining competence.) The achievement data we saw in math, science, and other areas seem to be comparable to data for the rest of the state.

Title: A Case Study of Humphreys Country (Mississippi) School District and Its Role as a Partner in the NSF-Supported Delta Rural Systemic Initiative (RSI)

Author: Jerry Horn

Humphreys County (pronounced "Umphreys" with a silent H) covers 412 miles in the Mississippi delta. Its history can be traced back 20,000 years to when the Mound Builders (so named because of the earthen mounds they left behind) inhabited the region. From this prehistoric tribe came the "Five Civilized Tribes": the Choctaws, Chickasaws, Creeks, Cherokees, and Seminoles. In 1789, the United States acquired control of the territory through the Treaty of San Lorenzo. After Andrew Jackson's election to the presidency in 1829, large tracts of the land were sold to the wealthy to establish cotton plantations. Without mechanized farming, much of the labor was done by slaves. Humphreys County played a minimal part in the Civil War.

Little information could be found about early educational opportunities in the county. The first school was a room in the home of the Knott family. Sometime before 1895, this school moved to a one-room building. The Humphreys County School District was established in 1918, with schools located in every town in the county. In 1970, the schools were consolidated, forming one of the largest consolidated school districts in the state. There are three schools: Ida Green Elementary School, O. M. McNair Upper Elementary School, and the Humphreys County High School. All schools are located in Belzoni, the county seat.

A court order to desegregate in 1972 led to the creation of the Humphreys Academy. This private school, with an enrollment of nearly 300, is 100 percent white. The Humphreys Country School District, with an enrollment of 2,500 is nearly 100 percent African American. In effect, there are two racially separated school districts in the county.

Humphreys County School District is one of 106 school districts participating in the Delta Rural Systemic Initiative. Because of multiple funding sources and a lack of activities explicitly defined as RSI-related, it was difficult to identify how the district was involved with the Delta RSI. Several district administrators have attended one or two leadership institutes sponsored by the Delta RSI.

The Delta RSI developed a Regional Advisory Council to fulfill its goal to create a regional infrastructure, and a member of the Humphreys County district is on the council. Council members are expected to (1) be available to help the field coordinator promote the goals/mission of the RSI, (2) promote systemic reform, and (3) attend major Delta RSI events. In addition to a family math and science night, the district has also instituted town meetings in each community in the district in order to increase parent and community involvement.

The idea of "teacher leaders," a practice in some other RSIs, was rejected for this project due to the high turnover rate among teachers. Also, two of the three states served by the Delta RSI (Arkansas and Louisiana) have NSF-supported State Systemic Initiative (SSI) projects. Efforts are made to coordinate activities between the RSI and the SSI in each state. Program level coordination is to be accomplished by field coordinators.

In their classroom observations, the visitation team saw little hands-on work by students and no instructional time in which students used computers in the classroom. In the biology room at the high school, we saw a well-designed laboratory including very modern fixtures. However, it looked as though it had been some time since most of the fixtures had been used for their intended purpose. For example, the lab stations were being used for drying art projects, and the exhaust hood was being used as storage space.

Driver 1: Standards-Based Curriculum, Instruction, and Assessment. There is weak evidence of the implementation of standards-based curricula. It appears that much of the curriculum is selected and controlled by individual teachers. In essence the curricula is instructor driven.

Driver 2: Coherent, Consistent, and Supportive Policies. Policies addressing graduation requirements and mandated student assessments are being developed. Few policies deal with the quality and appropriate training of teaching personnel. There seems to be considerable community pressure to maintain employment for some instructional personnel because of their history with the school or the need for their families to have an income.

Driver 3: Convergence of Resources. Overall, the site visitation team judged this driver to be weakly implemented, but that is not to say that math and science are not receiving resources. The district is a sought- after participant in a number of externally funded projects--Star Schools and others--because of the high eligibility levels of the schools, i.e., poor and with high minority enrollments. The availability of external grants may reduce the financial burden on the local community. However, it may also cause confusion and competition if programs are not compatible.

Driver 4: Broad-Based Support. Humphreys County has approximately 117 catfish farms with more than 35,000 acres of farm-raised catfish, making it the "Catfish Capital of the World." There is a problem attracting new industries to the area because of a lack of a well-educated labor force. This is a self-perpetuating problem. There is no incentive to gain an advanced education to prepare for jobs that don't exist. Some insist that there is an unspoken intent in the community to maintain a low level of education in the labor force because this ensures a pool of persons to work in the catfish industry who can demand only low wages.

Driver 5: Enhanced Student Achievement. The Humphreys County School District currently requires a variety of standardized tests. These include the Iowa Tests of Basic Skills (ITBS), exit exams in the fourth and seventh grades, and the McGraw-Hill exam that replaced the ITBS. At this time, standardized tests are the most notable forms of accountability for student achievement. However, there is some question about the seriousness with which some students respond to achievement tests for which there are few, if any, individual consequences.

Driver 6: Achievement of Underrepresented and Underserved Groups. Given the fact that all of the students in the Humphreys Country School District are historically underserved, this driver is not applicable to the case. Clearly, since virtually all of the students are African American, the issues regarding minorities are rendered moot.

Title: A Case Study of East Feliciana Parish (Louisiana) School District and its Role as a Partner in the NSF-Supported Delta Rural Systemic Initiative

Author: Jerry Horn

East Feliciana Parish is located in the heart of English Louisiana's plantation country where cotton was once king. It is strategically located in the southeastern part of the state, immediately north of Baton Rouge, the capital of Louisiana. The parish occupies a land area of 453.4 square miles of which 2.3 square miles are covered by water. At one time large tracts of land were sold to persons of English and German ancestry who established plantations that were run with slave labor. The Civil War had a major impact on this parish and the people residing here. Some of these impacts are still felt today.

The parish to the immediate west (West Feliciana Parish) is one of the wealthiest because of a nuclear power plant within its borders that provides a large tax base. East Feliciana, with its five state correctional institutions and a state hospital for the mentally ill, is one of the poorest. The presence of such a high number of state facilities in a sparsely populated county/parish created an abnormal balance between tax-generating industries and state-owned/nontaxable properties. This clearly created a sense of "haves" and the "have-nots" between the parishes. This condition is exacerbated in East Feliciana Parish by the existing long-time poverty, particularly among the black population, and the wealth and power among those associated with the "old money."

As a result, residents of East Feliciana Parish are poor, and they have attained less education and are generally lower than the state on most socioeconomic indicators. Teen pregnancy in the parish was reported to be 33 percent in 1996, compared with 18.9 percent for the state and 12.9 percent for the nation. The rate of single parenthood, based on 1990 census data, was 22.7 percent in the parish and 19.1 and 14.8 percent respectively for state and nation. The unemployment rate is high, and the reported aspiration of many young adults is to find employment in the state-operated mental health and correctional institutions that have provided employment for their parents and families for generations. This would not call for substantial changes in the schools' educational programs in science and math.

The parish has 2 segregated educational systems and 2 state-approved nonpublic schools: a small Christian Bible school and the much larger Silliman Institute. Silliman serves an almost totally white population of 650 students. It began operating as a private K-12 school in 1972, which corresponds to the date of a court desegregation order. The East Feliciana Parish school district consists of 2 high schools, 2 middle schools, and 3 elementary schools. The district has an enrollment of about 2,900 students, of which there is about a 4:1 (black to white) racial mix. In the public school district, most white teachers and central office staff with school-age children sent them to Silliman or some other private school.

Schools in the parishes adjoining East Feliciana are reported to have teacher salaries from $6,000-$8,000 above what is paid in the district. This may be a factor contributing to a very high turnover of teachers in the district, up to 70 percent in some years. Few teachers who are assigned to teach math and science are certified, which is apparently a problem all over the state but is particularly evident in this school district.

The East Feliciana Parish school district is one of 106 districts in Arkansas, Louisiana, and Mississippi participating in the Delta Rural Systemic Initiative (Delta RSI). The Delta RSI established a Regional Advisory Council to fulfill its goal of creating a regional infrastructure. A member of the school district is on the council, whose members are expected to (1) be available to help the field coordinator promote the goals/mission of the RSI, (2) promote systemic reform, and (3) attend major Delta RSI events.

The idea of "teacher leaders," a practice in some other RSIs, was rejected for this project due to the high turnover rate among teachers. However, the district hired a specialist in science and two in mathematics to serve as resource persons/coordinators for those two subject areas. This was viewed as a strategy for overcoming the lack of qualified teachers in science and math.

At the local level, the RSI project is closely aligned with Project SMILE (Structuring Mathematics Initiatives with Leaders in Education). SMILE was written in response to a Delta RSI-issued request for proposals. It is a K-9 mathematics project that included 16 workshop days during the summer and ongoing professional development during the year. The project enabled teachers to create activities that introduced manipulatives and other hands-on approaches to mathematics instruction and served as a catalyst for change and improvement of mathematics education.

A young white man from a wealthier family in the county who, as a student, attended Silliman Academy, serves as science facilitator for the entire parish school district. His role is to develop and formalize the science curriculum so that it is aligned with the state standards and benchmarks, to generally work for improvement of the science curriculum, and to provide assistance to individual teachers whenever needed and practical. A notable activity in which he has engaged is the creation of a "nature walk" on the grounds of two of the elementary schools.

Most of the monies from the Delta RSI go into sponsorship or cosponsorship of workshops and professional development activities for teachers. For example, funds were used to support summer leadership institutes, which included two weeks of staff development for selected teachers followed by two weeks with students to practice the use of new content and instructional strategies. RSI monies also paid for half of one math facilitator position.

Driver 1: Standards-Based Curriculum, Instruction, and Assessment. East Feliciana Parish first chose to concentrate on math (because it is an immediate testing area by the state agencies) and to add emphasis in science at a later date. Local efforts to develop a written curriculum that is based on state standards and to align appropriate instructional techniques to these standards and expectations are in process. The extremely high turnover of teachers (50-70 percent per year) and the necessity to use uncertified and likely unqualified teachers signal an even greater need for an organized and publicly accessible curriculum.

Driver 2: Coherent, Consistent, and Supportive Policies. There is little evidence of development of a coherent, consistent set of policies that supports any aspect of high-quality mathematics and science education.

Driver 3: Convergence of Resources. Virtually no funds are available from the regular operating budget due to the limited ability to raise funds locally. However, the parish has made a concerted effort to gather discretionary resources to support the K-8 mathematics improvement initiative, Project SMILE. These resources include federal funds (Title I, II, VII) and state resources made available through the Delta RSI and the LaSIP program, both sponsored by NSF.

Driver 4: Broad-Based Support. Evidence to support the presence of this driver is limited. Efforts to garner support from parents included the preparation of "refrigerator skills," reminders of math skills that can be reinforced at home, and a parents' "math night" designed to inform and engage parents in math activities designed to meet state standards.

Driver 5: Enhanced Student Achievement. The district implements a criterion-referenced test as a part of the Louisiana Educational Assessment Program, the Iowa Tests of Basic Skills, and the ACT. High stakes testing that is being implemented as a part of the state's assessment and accountability program has had a definite impact on the instruction in these schools. Teachers openly admit that they take instructional time to prepare students for testing by explaining the importance of the test and completing work sheets with math problems similar to those on the test. Questions have been raised about whether less is actually being taught as a result of the testing program, and there seem to be some legitimate concerns about this.

Driver 6: Achievement of Underrepresented and Underserved Groups. Eighty percent of the students in the East Feliciana Parish School district are African American. In the main, this fact mitigates the intent of the driver. All discussions regarding the school and the students are de facto discussions about historically underserved students. All of the aforementioned successes, failures, and challenges involve this historically identified underserved component of school children and youth.

Title: A Case Study of the Gila River Indian Community (Arizona) and its Role as a Partner in the NSF-Supported UCAN Rural Systemic Initiative

Authors: Craig Russon, Jerry Horn, and Steve Oliver

Gila River Indian Community occupies 372,000 acres of land located adjacent to the metropolitan area of Phoenix. The population is hard to determine because the reservation does not correspond with traditional census tracts. The terrain not under irrigation is typically desert, with little evidence of grazing or other agricultural value. Although some would find the landscape and the area to be stark and somewhat forlorn, others would find it to be picturesque and even beautiful, especially during sunrise and sunset.

The Gila River Indian Community traces its roots to the Hohokam, prehistoric Indians who migrated to the Gila River region around 300 B.C. A thriving civilization farmed the desert using canals for irrigation until around 1200 A.D. when they disappeared without a trace. The Pima and Papago tribes are thought to be descendants of the Hohokam. Sometime in the 1700s, the Maricopas moved into the region and the two tribes formed an alliance.

In 1694, the first Christian missionaries came to the area. This opened the door to Mexican and European immigration. In 1854, the Gadsden Purchase made southern Arizona a U.S. territory. This further promoted immigration. The Pimas, known for their farming ability, provided the newcomers with food and water. Congress created a reservation for the Pimas in 1859. For nearly 70 years, the tribe was governed by the Bureau of Indian Affairs (BIA). The loss of independence is thought to have contributed to the substance abuse and violence that exist on the reservation even today.

With the passage of the Wheeler-Howard Act in 1934, tribes were once again able to rule themselves after years of forced dependence on the white man. Many tribes instituted constitutions and governments. The Gila River Indian Community is headed by a governor and a tribal council that is made up of elected representatives from each of the seven districts on the reservation. Tribal administrative offices and departments were established in Sacaton.

In 1937, the U.S. government completed work on the Coolidge Dam on the upper Gila River to create the San Carlos Reservoir. This stopped the flow of water to the Gila River Indian Community. The loss of water caused a profound negative change in the Pima way of life. The tribe went from being self-sufficient farmers to being dependent on government surplus foods. The drastic change in diet led to severe problems with obesity and diabetes. The Pima have a higher per capita incidence of diabetes than any other group in the world.

The Gila River Indian Community is slowly beginning to recover from the harm that resulted from dependence, unemployment, drastic change in diet, alcoholism, poor schooling, and disrupted lifestyle. The community opened 3 industrial parks and expanded its commercial agricultural operations. In September 1994, it opened an 8,500 square foot casino near the heavily populated area of south Phoenix and Chandler. Education has come to be seen as a way to prepare tribe members to fill the job opportunities created by these revenue-generating projects.

There are seven elementary/junior high schools and one high school (a tribal charter high school with a very low enrollment at the time of the site visit) in the Gila River Indian Community. Among the schools in the community are BIA grant schools, parochial schools, public schools, state charter schools, tribal schools, and tribal charter schools. Each school on the reservation is independently controlled through its own local board of trustees. These boards appear to have limited accountability to the tribal council.

The tribal council is not responsible to the state and federal government on matters of education because the Gila River Indian Community is a sovereign nation. However, it is clear that the council provides resources to the school if it is demonstrating success or proposing to do something that is deemed to be especially beneficial. Like any political organization, there is a general feeling that personal connections and family ties influence some decisions in the distribution of discretionary funding to individual schools.

Students are generally free to attend any school they wish, and many make multiple transfers each year. There is a mechanism in place to track students. A number of students begin the year by attending off-reservation high schools. Bus transportation is provided free to students to attend these schools, including off-reservation public high schools. Approximately 1,250 students who reside on the reservation attend public or private schools off the reservation. The transition from reservation primary schooling to off-reservation secondary school has been identified as one of the points where students are most at risk of dropping out. The high school graduation rate for students residing on the reservation is reported to be 12 percent.

All of the reservation schools belong to the Arizona Tribal Coalition (ATC) of the Utah, Colorado, Arizona, and New Mexico Rural Systemic Initiative (UCAN RSI). UCAN's mission is to provide support to selected schools for comprehensive programs that broaden the impact, accelerate the pace, and increase the effectiveness of improvements in standards-based science, mathematics, and appropriate applications of technology.

In its first two years, the ATC was not as effective as it has been in subsequent years. Once the tribal council lent its support to the initiative, problems began to be resolved. One such problem confronting teachers in the Gila River Indian Community was a sense of isolation. Even though the community is a relatively small geographic area, teachers typically did not have contact with colleagues from other schools. They lacked the opportunity to share ideas and obtain support. There appeared to be little if any interaction of any consequence between school personnel in the reservation schools and their counterparts in the nearby areas of Phoenix and Chandler.

The ATC's outstanding accomplishment is the creation of the Gila River Community Schools Coalition. Through the coalition, teachers from most schools meet once a month to discuss issues relevant to math and science education. The teachers then return to their schools and share the information they received with other teachers. The ATC also provides professional development workshops and summer institutes on standards-based curriculum and assessment through a partnership with WestEd.

With its limited resources (only 2 FTEs for 42 schools with one-tenth of the UCAN budget) and multiple jurisdictions (local board, district, tribe, county, state, federal), one might think that the ATC has little chance of accomplishing much. However, somehow ATC has been able to make math and science education a priority for charismatic school administrators in the community. These dynamic individuals are able to mobilize the resources and the support necessary to change policy and implement standards-based curriculum.

Though not directly involved, ATC has benefited from a partnership between the Intel Corporation and the Gila River Indian Community. Through the Gila River Education Network, six technology education centers were established in 1996. The centers have networked the entire reservation, providing the tribe with access to the Internet and World Wide Web. Through 1996, Intel's total donation to the partners added up to more than $1.2 million in cash and equipment grants, plus in-kind and volunteer time.

With technology education centers forming the foundation of the partnership, another component is the school-to-community alliance. In an effort to engage and keep Indian students involved in science and math, alliances between the Gila River Department of Education and reservation public schools have been formed. Through the alliances, teachers are developing a culturally relevant math and science curriculum. This effort is linked with an innovative community-based education model under development at the Santa Fe Indian School (New Mexico), which uses real community issues and problems to teach the basics of math and science. The goal is to make the curriculum relevant to the needs of the Native American community.

Driver 1: Standards-Based Curriculum, Instruction, and Assessment. Some schools in the Gila River Indian Community seem to have embraced a community-based education model. For example, Gila Crossing School has developed a curriculum around the Pima's historical vocation of farming. The centerpiece of the effort is the school's garden. Around this centerpiece, the teachers indicate that they have constructed a curriculum that is not dependent on outside materials, but is supplemented by them. The total school effort is designed to make a curriculum for students that is "totally inclusive." For example, all students at Gila Crossing School participate in the science fair (several of the students have gone to competition at the national level).

In addition to being culturally relevant, the curriculum is also standards-based. BIA grant schools are given the choice of using the BIA curriculum or teaching to the Arizona Academic Standards. The principals at most schools in the community chose to use the state standards. Students take the Arizona Instrument to Measure Standards (AIMS), but scores are not sent to the state.

Driver 2: Coherent, Consistent, and Supportive Policies. The schools in the Gila River Indian Community are governed by multiple jurisdictions--community, district, tribe, state, and nation. In addition to the formal governance structures, it is likely that influential informal governance structures also exist. Each jurisdiction has its own policies and it seems unlikely that they will ever be totally aligned, because there is not a direct line of authority. No evidence was found of policies that support preservice education of teachers, recognize and reward excellence in teaching, support the system's capacity to collect and use data for continuous program improvement, and encourage qualified teachers to remain in rural, small schools.

Driver 3: Convergence of Resources. The schools in the Gila River Indian Community are quite entrepreneurial when it comes to seeking resources. For example, Sister Martha of the St. Peter's Mission School has obtained resources from the Phoenix Cardinals, Williams Field Air Force Base, the host of NBC's Today show, New York Times correspondents, Intel Corporation, and the Tribal Council. Perhaps the reason for the entrepreneurism is that schools in the community must still fight for what they get. For example, the site visit team was told that when some schools on the reservation were chartered by the state of Arizona, they received state money in addition to BIA money. There was an outcry because people thought the schools were double-dipping. The additional money, however, was just enough to bring the schools up to the median funding level for the state.

Driver 4: Broad-Based Support. Support for science and math programs in the Gila River Indian Community is fragmented. This is probably a function of the multijurisdictional entities in the community. Project staff informed us that in this culture it is important to have all stakeholders on board before undertaking activities. Sometimes that is slow to happen. It is estimated that it took as long as two years for important stakeholders to lend their support to the ATC. Once this support was forthcoming, the way was cleared for the ATC to begin to effectively coordinate science and math education activities for the schools in the community.

Driver 5: Enhanced Student Achievement. A central issue for the RSI is the assessment of student gains in science and math. Several schools administer the AIMS test, which measures achievement on math standards but not science. In addition, some schools administer the Stanford 9. An examination of the Stanford 9 results showed that students in Gila River schools are generally below the thirtieth percentile in math in grades 3 through 8. Some administrators are not convinced of the usefulness of standardized testing. They think testing results from the various coalitions are not comparable because each state has a different system of measurement that seems to change often. Alternative methods of assessment, such as student portfolios, have been explored. According to administrators, however, the nonreservation high schools to which Gila River students transferred threw the portfolios away when they received them. The high schools wanted three pieces of information: letter grades, standardized test scores, and attendance/behavior.

Driver 6: Achievement of Underrepresented and Underserved Groups. Some schools in the Gila River Indian Community are "full inclusion" schools, which means that all students have opportunities to participate in all activities, such as the science fair and school gardens. At other schools in the community, only students in the gifted class receive those privileges. The teachers and administrators from the full inclusion schools proudly reported that some of the students who succeeded at the highest levels are identified as special education students. The teachers and administrators firmly believe that the students are a cohesive group who recognize that no one will be left out in the educational efforts.

Title: A Case Study of Wagon Mound (New Mexico) School District and Its Role as a Partner in the NSF-Supported UCAN Rural Systemic Initiative

Authors: Craig Russon and Brian Lotven

Wagon Mound is the sole incorporated village of Mora County in the northeastern part of New Mexico. The county covers 1,931 square miles of picturesque countryside including mountains and grassland. In 1996, the population was 4,798 for a population density of 2.5 persons per square mile. The major employers are the highway department, the school district, and cattle ranches. The median family income for 1996 was $20,600. Approximately 84 percent of the county's population is of Hispanic descent, with the families having resided in the area for generations.

The history of the Wagon Mound community is integrally linked with the Santa Fe Trail. The first white person believed to have ridden the future trail was a Frenchman named Pedro Vial. He left Santa Fe in May 1792 and reached St. Louis in early October. Others soon followed in his footsteps. The Spanish authorities of the area were hostile to Americans who used the Santa Fe Trail and arrested many traders and confiscated their goods. This changed in 1821, when Mexico became independent from Spain. Mexico opened its doors to anyone who wanted to trade. By fall of that year, William Becknell from Missouri had reached Santa Fe. Becknell is generally acknowledged as the "father" of the Santa Fe Trail.

Senator Thomas Hart Benton from Missouri was the person most responsible for getting the government to survey the trail. He believed that it would benefit all parties--Mexicans, Native Americans, and Americans. President Monroe authorized the survey on March 3, 1825, and $10,000 was allocated for expenses. Joseph Brown, a Missouri senator, was chosen as the surveyor. The survey team headed out from Fort Osage on July 17, 1825, with 7 baggage wagons, 57 horses and mules, and 40 men. In January 1827, a second expedition was sent to make corrections to the original survey. After filing their report with Washington, a flood of traders started using the trail.

The community of Wagon Mound was founded during this time period. The community received its unusual name because it is located at the base of an enormous rock formation that resembles a Conestoga wagon. This prominent landmark, which can be seen from many miles away, guided travelers along the trail. The Santa Fe trail flourished until the arrival of the railroad in July 1879. Wagon Mound probably continued to prosper after that date because the rail went through the community.

New Mexico became a state on January 6, 1912--the forty-seventh state to join the union. Shortly thereafter, permanent settlers began to come to the Wagon Mound area. A large percentage of the early settlers were young men who had served in World War I and were taking advantage of the Homestead Act to create a new life for themselves. Some sought the high altitude, dry climate, and clean air and sunshine to regain their health--mostly from tuberculosis (Daniels, 1991). The early settlers were conscious of the need for education and tried to educate their children the best way possible (Hanosh, 1967). Their efforts in many ways parallel the educational methods that are used today: home schooling, parochial schools and, later, public schools.

According to Hanosh (1967), schools were not plentiful in New Mexico before statehood. Wealthy families often sent their oldest child to school in the East. However, most homesteaders were not wealthy, so they tried to teach their children essential things such as figuring and reading and writing. This was taught in the Spanish language. At times, teaching was done from the Bible, which also served as instruction in religion. Classes of this nature were usually taught by one of the elders of the village who was more acquainted with the Spanish language. In many cases, the father of the family would gather his children around him in the evening, read to them, and ask them questions.

In the early 1900s, public schools began to appear in the valley (Daniels, 1991). These began as little one-room schoolhouses. The country schools usually did not start until the first of the year because the children had to help with the cotton picking. School lasted only 3 months and was over in April. A community might have a school with 15 or 20 pupils enrolled. All the boys and girls, whatever their age or grade, attended 1 of these schools and received their schooling from 1 teacher. The only subjects taught were the basics of reading, writing, spelling, and arithmetic. The schoolhouse was the scene of many gatherings such as school programs, church, picnics, and revivals.

In 1922, several of the one-room schools in the valley were consolidated. This was followed by a steady decline in enrollment (Daniels, 1991). Several factors may have caused this decline. Land availability decreased. That caused a decrease in the flow of homesteaders to the area. Many people went broke staying on the land the required length of time, so they were glad to get the title and go back home. These problems were compounded by a drought. The school building closed and reverted back to the original landowner who had moved away.

Today, the Wagon Mound school district is K-12 with an enrollment of approximately 130 students and 13 teachers. On the main campus there is a complex of buildings. One building houses the elementary school and another houses the middle school and high school. The administrative office is housed between the 2 buildings. The schools are very clean and in good repair. The elementary school has 1 class per grade. The students in the middle school and high school have different schedules to keep younger students from associating with the older students.

In addition to the main campus in Wagon Mound, the school district also provides educational services to students placed at Rancho Valmora, a nonprofit residential treatment facility that was established within the school district in January 1994. The facility provides long-term treatment for seriously emotionally disturbed adolescents. The school district provides all educational services, including special education and related services. Valmora High School is located on the facility grounds and serves approximately 50 students.

The Wagon Mound school has become allied with a number of initiatives that enable small, rural schools to work together in a common effort, both professional and financial, that will give them a better chance of achieving their goals. The Utah, Colorado, Arizona, New Mexico Rural Systemic Initiative (UCAN RSI) is one of the programs that is providing much-needed assistance.

UCAN RSI coordinates activities in five local coalitions, situated in four states, and several reservations and tribes. These coalitions provide catalytic support to selected rural school districts for changes in policy, organization, government, and community involvement in the support of science and math education.

Wagon Mound school district has made several major improvements in the areas of science, math, and technology. Though not directly involved, New Mexico County Coalition (NMCC) has benefited from the upgrade of the high school computer lab. This brings the total number of computers available for student use to 60. In a school district with an enrollment of 130, that is an impressive accomplishment. In addition, the school district had taken advantage of federal programs that provide funds to rural schools so that they can hook up to the Internet.

The district is a participant in the state bilingual program that provides two hours of instruction daily for students in K-6 and one hour for high school. In grades K-6, the district implements a multiage grouping approach to teach the Spanish language skills. One of the hours uses a team teaching approach to teach math and utilizes both the English and Spanish languages in its instruction. While on site, the evaluators observed one of these classes. The teachers used Spanish to teach a lesson on Roman numerals that the children had been taught earlier in English. The skill with which these veteran teachers involved the children in the lesson was most impressive.

The elementary teachers implemented a science program theme dealing with solid waste issues. The theme was selected because a landfill is located nearby. A district team consisting of the superintendent, four elementary teachers, and one administrator and teacher from Valmora (the residential facility for troubled youth in the Wagon Mound district) attended an NMCC-sponsored workshop at Ghost Ranch during which an elementary science program theme was developed that aligns the curriculum with the content standards.

The NMCC also sponsors monthly leadership workshops, which are attended by Wagon Mound school district administrators. During the workshops, administrators have been trained in the use of disaggregated data as a means to improve student performance. In addition, the NMCC sponsored two conferences for school board members.

Driver 1: Standards-Based Curriculum, Instruction, and Assessment. Systemic reform is taking place at the elementary level. District administrators and coalition leaders recognize that it will be much more difficult to implement reform at the high school level. The challenges that the district faces at this level include too many preparations at the secondary level and lack of curricular breadth due to size of school and faculty. In order to address these problems, the superintendent is working toward a distance education program provided by colleges and universities.

Driver 2: Coherent, Consistent, and Supportive Policies. The New Mexico County Coalition appears to have had some influence on the development of a coherent, consistent set of policies at the local board level. The Coalition has sponsored two conferences during which members of the Wagon Mound school board received instruction on how to better perform their policymaking functions. In addition, the Coalition has trained administrators in data collection and analysis. As a result, the school board knows how to do its job better and has good information that can be used in decision making.

Driver 3: Convergence of Resources. It is clear that there is a convergence of fiscal resources in the Wagon Mound school district. In 1988, Wagon Mound hired a consultant who helped the district pass a bond referendum. Wagon Mound has also been successful at attracting outside funds. The district received a Two Languages grant, Star Schools grant, and federal and state grants. The district plans to hire a grant writer next year. UCAN appears to have a small role in procuring financial resources. It plays a much more prominent role in the development of the district's human resources. Money from UCAN is channeled through NMCC to the Regional Cooperative Centers or Regional Education Centers to activities that benefit all districts in the collaborative.

Driver 4: Broad-Based Support. Support of systemic reform of science and math education in Wagon Mound is fragmented. There is very little business and industry in the area from which support could be solicited. Community support is probably influenced by local politics since the district is the largest employer in the county. Faculty support is limited by the location of the teachers' residences. Only two of nine teachers live within the district.

Driver 5: Enhanced Student Achievement. All students in grades 3 through 9 take part in the New Mexico Achievement Assessment Program in mathematics, science, and other content areas. The norm-referenced test that is used is the CTB McGraw Hill TerraNova Survey Plus. TerraNova covers reading/language arts, mathematics, science, and social studies. A customized, supplemental test booklet accompanies this exam in order to assess student progress toward the New Mexico content standards and benchmarks. Together, both tests provide performance level data showing student progress toward the standards.

Driver 6: Achievement of Underrepresented and Underserved Groups. The racial mix in the Wagon Mound school district is about 80 percent Hispanic and 20 percent white. There is no clear evidence that students' achievement test scores are improving in this school. However, there is a bilingual program in place, which provides 2 hours of bilingual instruction daily to K-6 students. In addition, all high school students are required to participate in the science fair (not just honor students).

Summary

The first section of this paper drew upon the literature from rural education, comprehensive school reform, and educational change, as well as from the effective schools and teacher effects research initiated in the 1970s. Summaries of six case studies conducted as part of The Evaluation Center's evaluation of the RSI program were then presented. In this section, the NSF Drivers are compared with the empirical findings from the case studies to see how the two match.

Driver 1--Implementation of comprehensive, standards-based curricula as represented in instructional practice, including student assessment in every classroom, laboratory, and other learning experience provided through the system and its partners

Most states have developed curriculum frameworks or standards and benchmarks. In some cases, the frameworks drive the curriculum and instruction. For example, the Appalachian Rural Systemic Initiative (ARSI) maintains a web site with standards-based curriculum materials, provides professional development on the topic, and conducts program audits that identify discrepancies between the curriculum and the relevant framework. In some Gila River Indian Community (Arizona) schools, the curriculum is not only standards-based, it is also culturally relevant. For example, Gila Crossing School has developed a curriculum around the Pima's historical vocation of farming. The centerpiece of the curriculum is the school's garden.

In other cases, the real drivers of curriculum and instruction were other than the frameworks and standards. In the Humphreys County (MS) school district, the curriculum was teacher-driven because turnover was so high it was impossible to retain teachers trained in the delivery of standards-based curriculum. The East Feliciana (LA) and Cocke County (TN) districts are both located in states that have high stakes testing programs. In both districts, the curriculum was driven by the standardized tests that schools were required to administer.

In addition, implementation of a comprehensive, standards-based curriculum in every classroom throughout the school systems did not always occur. The case study data suggested that elementary schools were the most active in systemic reform. It appears to be more difficult to implement systemic reform at the secondary school level.

Driver 2--Development of a coherent, consistent set of policies that supports provision of high quality mathematics and science education for each student, excellent preparation, continuing education, and support for each mathematics and science teacher (including all elementary teachers), and administrative support for all persons who work to dramatically improve achievement among all students served by the system

The RSIs have had little impact on the alignment of educational policies, and there was little prospect that they ever will. RSIs typically cross multiple geopolitical boundaries. The most extreme example was found in the Gila River Indian Community, where schools are subject to community, district, tribe, county, state, and national jurisdictions. Each jurisdiction had its own education policies and procedures. It is unlikely that the education policies of all jurisdictions will ever be totally aligned because there is no direct line of authority or requirement for coordination. Though not as extreme, the other RSIs experienced the same fundamental problem. ARSI worked with 44 counties in 6 states. UCAN worked with 4 large western states. Delta was a tristate effort serving 32 counties in Mississippi, 11 counties in Arkansas, and 21 parishes in Louisiana.

Any alignment of policies that RSIs have been able to accomplish was done at the district or school level. For example, the Rockcastle School District (KY) is governed by a five-member board with a chair and an appointed superintendent of schools. The board is responsible for setting district policy. In addition, each building has a five-member, site-based decision-making council that is composed of three teachers and two parents. These councils have considerable power and authority over school policy and the curriculum.

Driver 3--Convergence of the usage of all resources that are designed for or that reasonably could be used to support science and mathematics education-- fiscal, intellectual, material, curricular, and extracurricular-- into a focused and unitary program to constantly upgrade, review, and improve the educational program in mathematics and science for all students

RSIs mostly worked to promote the convergence of human resources. This was primarily accomplished through professional development activities and directed assistance in utilizing state and federal grant funds to support a common effort to improve science and/or math education. ARSI brought in national and local math and science experts to provide Teacher Partners with professional development. The Delta RSI provided 16 days of professional development during the summer and ongoing professional development during the year. UCAN channeled money through Regional Cooperative Centers or Regional Education Centers for activities that were intended to benefit all districts in the collaborative.

In some cases, RSIs were indirectly able to help school districts obtain external resources. For example, the Arizona Tribal Commission was able to make math and science education a priority for charismatic school administrators in the Gila River Indian Community. These dynamic leaders were able to mobilize the resources and the support necessary to change policy and implement standards-based curriculum. Further, local monies generated by tribal-owned casinos are being used to build a strong infrastructure on the reservation to provide essential and specialized services for its people. Not only are some of these monies being funneled to schools (public, private, and parochial), they are being used to improve health care and social services that have the potential for indirectly benefiting the educational efforts of the schools. For example, children who are well nourished and living in a safe and strong family environment are more likely to attend school on a regular basis and to gain/learn more from the school experience.

Clearly, experienced and effective administrators in small, rural schools are "masters" at leveraging resources. When grants or other entitlement programs are based on enrollment, there are hardly enough monies to support the administrative management of a program intended to provide student services; therefore, school leaders have had to supplement targeted funding with other monies that provide some discretion in how they may be spent. For example, some technology grants only allow for the purchase of equipment without professional development resources for teachers to learn how to use it in the classroom. At the same time, another program provides support for professional development, but not funds for equipment. Thus, effective administrators merge these two efforts into a common effort with monies for equipment coming from one source and monies for professional development coming from another. Specific to science, "Eisenhower" monies and RSI resources are often combined to facilitate common efforts in science and math at the local level. At the same time, "Goals 2000" monies were used to help align curricula with state standards.

Driver 4--Broad-based support from parents, policymakers, institutions of higher education, business and industry, foundations, and other segments of the community for the goals and collective value of the program, based on rich presentations of the ideas behind the program, the evidence gathered about its successes and its failures, and critical discussions of its efforts

Some schools received an abundance of support from parents. Math and science nights were common across the RSIs, and in some cases they were well attended. However, in other cases the parents were far more likely to support the football or basketball team than to support math and science education reform. It is not clear if there are appropriate forums for local school personnel and parents/community members to discuss creative and effective means to identify mechanisms to develop broad-based support from parents in ways different than attendance at school events. At the same time, it is recognized some RSIs, such as ARSI, have tried to provide direction through the development and dissemination of guidebooks on community involvement.

Some RSIs successfully promoted collaboration between school districts and institutions of higher education. For example, ARSI established resource collaboratives at five partner institutions (Clinch Valley College at the University of Virginia, Marshall University, Ohio University, University of Kentucky, and the University of Tennessee) to provide information and services to the districts. Other RSIs were unable to link participating school districts with institutions of higher education, usually because these districts were too isolated and/or geographically dispersed. Wagon Mound (NM), Gila River Indian Community (AZ), Humphreys County (MS), and East Feliciana Parish (LA) were good examples of sites in which there has been limited involvement of higher education in the actual reform effort. However, four-year institutions are reasonably close to these four sites/communities.

There was not a lot of evidence indicating that business and industry supported systemic reform. There were some notable exceptions, such as the grant that the Gila River Indian Community received from the Intel Corporation. It might be expected that these stakeholders would support systemic reform in the interest of creating a highly skilled labor force. In fact, just the opposite was suspected by some in Humphreys County (MS). Some thought the catfish industry had a vested interest in maintaining a poorly educated work force that could demand only low wages.

For the most part, RSIs did not help participating school districts identify funding opportunities. However, some districts proved to be quite successful at obtaining funding from grant-making institutions on their own. For example, the Rockcastle School District was a recent recipient of a grant for the Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) through the U.S. Department of Education.

Driver 5--Accumulation of a broad and deep array of evidence that the program is enhancing student achievement, through a set of indices that might include achievement test scores, higher level courses passed, college admission rates, college majors, advanced placement tests taken, portfolio assessment, and ratings from summer employers, that demonstrate that students are generally achieving at the significantly higher level in science and mathematics

The multiple sources/methods approach for assessing student achievement suggested by NSF was not generally found in public reports of achievement. Standardized achievement tests, often mandated by the applicable states, were used to the virtual exclusion of all other reported accounts of assessment (other than normal classroom tests developed by teachers). Continuing to focus on this single and perhaps inappropriate indicator of student achievement in math and science caused some problems. First, there is no clear evidence that standardized tests prepared for mass administration across all school districts in all states are related to the missions and goals of the schools, the focus of the schools' curricula, or the classroom instruction students receive. Failure to meet the standard for any one of these three conditions would invalidate the results as being a fair assessment of student achievement or even instructional/school effectiveness.

Secondly, most states are engaged in independent efforts to identify standards and benchmarks and to facilitate the alignment of local curricula to these standards. Presumably, state tests that reflect the newly aligned curricula and the expected local instructional response will be used. However, most RSIs extend across state boundaries and each state has its own standards, tests, pass rates, result reporting elements, etc. This makes it difficult to compare student achievement within a single RSI, let alone among the RSIs. To compound the problem, there is not a continuous (year after year) administration of most mandated state tests. This precludes efforts to identify trends for schools or to track the progress of individual students. Further, many would argue that achievement test scores reflect only one component of learning and student achievement.

In spite of these glaring problems, society continues to demand that schools and programs implemented to improve learning for students be assessed on the basis of standardized scores. Because of regulations or requirements, often tied to funding or efforts to remain in good standing with state departments of education and federal funding agencies, schools have responded accordingly.

In our review of RSI reports, we seldom saw data disaggregated on the basis of factors known to be associated with student achievement, i.e., socioeconomic status, race/ethnicity, educational level of parents, disabilities, etc. Also, school personnel offered a variety of interpretations and explanations about the same set of student achievement test data. We do not know whether this is the result of poorly published reports to accompany summaries of test results, inaccurate interpretations, or simply selective use of results. However, it does complicate the ability of evaluators to assess the impact of programs designed to increase achievement. Certainly, a value-added, longitudinal approach based on individual student scores would be a better way to measure achievement over time. In a few states, this is becoming a reality although it, too, is open to varying perceptions and interpretations.

Driver 6--Improvement in achievement of all students, including those historically underserved

As to the question of whether achievement of all students, and particularly the historically underserved, is improving, at best there is mixed evidence. One is likely to find as many declines as there are improvements across all students. However, all results must be carefully examined because the tests and/or passing standards may have changed, different students are being examined, many rural schools have small numbers of students that can vary tremendously from one year to the next, and the RSI program is only one of several school improvement interventions that is being experienced by students. Because of the nature of the criteria for the RSIs--i.e., isolated rural environment and high qualification rate for free or reduced lunch--the variability among students is reduced and student bodies are relatively homogeneous as to race and ethnic origin, be it all white, black, Hispanic, or American Indian. At our case study sites, we found an almost 100 percent single racial/ethnic mix of students in the schools, i.e., all white, all Native American, all African American, etc. At two case study sites, most white students attended private academies, which left the public schools with an almost totally African-American student body and, by definition, the Gila River Indian Community is a reservation for persons of American Indian heritage.


Conclusions

A review of the evidence contained in the case studies and the summary leads one to conclude that education reform is a difficult task. The RSI program has achieved successes in improving education for students in rural environments. However, it seems that the successes may not always have been as intended by educational reformers (e.g., policy alignment). A major challenge for the RSIs and any other school improvement/reform movement in communities like those we studied is to address the negative elements in the context of the community. Generally, this would include economically poor and minimally educated parents, lack of a history of parental involvement for many of the children with the greatest needs, and a feeling of uncertainty among students that they are capable and able to further their education or find employment that requires much education or background/knowledge in science or math. In some instances, a tremendous challenge is trying to find and retain well-prepared and qualified teachers of science and math. While some schools in larger districts have made major progress in aligning the curricula with recognized (mostly state originated) standards and implementing instructional methods that are compatible and show promise for success, reform efforts, as we found them, are not generally across all school levels or universally being implemented in all schools within a school district. This situation raises the obvious question and begs for an acceptable operational definition of "systemic."

In regard to the development of a "coherent, consistent set of policies," as set forth in the second driver, we find evidence of supportive practices that might be governed by policy in larger and more bureaucratic organizations. However, in the rural context, policies seem to be more generally developed to accommodate practical interpretation at the school or even classroom level. Importantly, we did not find evidence of policies that would auger against school reform, as generally defined by actions of the RSIs, nor did we find outward resistance to practices that provide some rather broad interpretation of selected policies.

There are efforts to converge and leverage resources to support the RSI goals. At the same time, the magnitude of these resources, including those allocated or made available to the local schools/districts by the RSIs, is small in comparison with the tremendous need and geographic area included in the collaboratives.

Participants in RSI efforts feel good about their experience, and they recognize it as a rare opportunity to engage in activities that have promise for long-term, broad-based impact in their school district and local school district. Participation in RSI for most teachers means attending professional development programs with the corresponding use of some materials and equipment. With the large turnover of teachers in some schools, it is clear that awareness and initial orientation, with the hope of further development of individuals over time, simply is not a reality. Either ongoing training for new teachers and/or planned support in the form of in-school/district consultants will be necessary. One district has come to the realization that qualified teachers in science and math are simply not available to it, and it has chosen to develop "coordinator" positions and to staff them with experienced teachers. These coordinators will be responsible for providing individual assistance and guidance to classroom teachers and for modeling effective teaching. The development of the "teacher partner" concept appears to be a viable approach, but the effectiveness of any of these specialized positions depends on the personality and skills of the individual and the time allocated to provide these services.

Substantive involvement of parents in the reform effort seems to be minimal in most cases, other than attendance at special school nights that focus on science or math. However, interviews and focus group meetings with parents and other community members during case study site visits revealed interest but not much understanding of how to effectively participate in the process. At the individual student level, it is difficult for parents who might not have graduated from high school to engage in discussions on issues for which they have never been exposed, i.e., advanced levels of math and science.

Where there are entrepreneurial leaders, one finds some involvement with the private sector in the form of some resources, recognition services, and/or participation on planning committees. At the same time, we did not observe much evidence of coordinated program activities for students or teachers between the RSI schools and the private sector. Higher education personnel have provided some professional development work for the RSIs, but this seems to be primarily on an individual basis as opposed to an institutional commitment.

The evidence that student achievement is being enhanced by RSI activities is clouded by the various tests and procedures associated with statewide testing practices. Basically, there is not a good set of benchmarks or procedures to consistently measure science and math achievement for individual students over time. In this study, 12 states are involved, and each has its own basis for determining what content is to be taught and assessed. To complicate the problem, most states are in the process of changing tests and/or procedures, which does not provide an opportunity to compare one year with another. Importantly, teachers and schools think they are improving, and they can cite specific evidence: more students taking advanced science and math classes, science fair participation and success, greater enthusiasm, etc. We feel strongly enough about the richness of this area that the WMU RSI study has developed a separate thrust to pursue these informal types of evidence of student achievement.

By definition of the RSI program, the schools and the students eligible for this program are among the underserved. Thus, the last driver that includes this element is not fully applicable, since rural and economically poor have been identified as two conditions for underachievement in the program guidelines. However, in the broader context and even though most of the schools in the RSIs are rather homogeneous in terms of race and ethnicity, we have been sensitive to the issue and have found no evidence that individuals or groups of individuals are being excluded from full participation in the RSIs that we have studied.

With regard to the RSI work and the more general area of school reform, the NSF RSI effort is hampered by its inability to include all subject areas in this effort. In K-12 schools and especially elementary schools, science and math are not separate entities for teachers or students. And while we recognize that many schools are aligning their science and math programs with state standards, they are also doing or shortly hereafter will be doing the same thing with all of their curricula. Will the experience in science and math serve as a good model for the other areas, and will science and math be set aside while that is being done? The school reform literature calls for the inclusion of all faculty/teachers in the process. Is it possible for only the teachers of science and math to implement systemic reform, i.e., across the whole school and the school district, or does it require some involvement of the other teachers? While voluntary participation has its advantages, it also provides an opportunity for individuals to criticize an innovation without really knowing what is involved or intended. In some RSIs, individual schools have been selected as "first efforts" and have been designated as "catalyst schools." The literature is somewhat moot on whether this strategy will work or not, i.e., whether the reform can expand beyond a single school in a district.

The extent to which many of the RSI participating schools have been involved in planning the RSI itself is not clear, but there are opportunities for input during the operational period. Further, the appointment of persons with coordination responsibilities across multiple districts is one way in which input is received and activities are jointly planned.

We identified a deep-felt perception that there are elements in some communities that really don't want to see a better educated populace, since this would eliminate or at least reduce the size of the labor pool without power to demand higher wages. Even if this is a false perception, it is a discouraging finding in the study. At the same time, there are strong family ties and an overt reluctance to go outside the community for advanced education or employment. Obviously, this makes it even more difficult to "sell" the need for advanced science and math training or even high school diplomas, much less college degrees. In discussions with school personnel, there was a strong perception that "lack of parental value for education" is the number one obstacle to school improvement. Certainly, there are exceptions among both the "haves" and "have-nots," but there seem to be divisions that are created and maintained on the basis of something more than just money. There is an ingrained set of circumstances in many rural communities that leads to disparity for many and as means to power and control for others.

Cynthia Duncan describes this phenomenon in a recent book, Worlds Apart (1999), where she relates the results of her study of three anonymous rural communities (a farming community in the delta of Mississippi, a coal mining town in Appalachia, and a mill town in Maine). Employment, bank loans and approvals for credit, political favors, invitations to participate in civic as well as social activities, and candidacy for appointed or elected positions in the schools and city governments are often controlled by a small but powerful element within the community. Duncan suggests that in some cases the power is not always held by persons in visible positions, but is maintained by those who had them elected or appointed to these positions. This influence continues from one generation to the next, which generally results in a greater gap between the haves and have-nots, and the power within a community is maintained by a small minority of the residents.

While we did not attempt to ascertain the various Stages of Concern, as per the work of Hall and Hord, in this phase of our study, it was clear that we saw situations and heard remarks that clearly support their findings. In part, individuals seem to behave in fairly predictable patterns with regard to their concerns about an innovation or change that will have substantial impact on their lives. A "safe" expression of concern is related to consequences (Level 4) and, with the voluntary "buy-in" of teachers to participate, Level 2 (Personal); and this is probably where we would find many of the persons who have been engaged in RSI work at the local level for three or more years. Those with fewer years of experience are most likely at levels 0 (Awareness) and 1 (Informational).

We have found that change, particularly of the potential magnitude of systemic reform, is a slow and deliberate process for most schools and many teachers and administrators. RSI directors and other project personnel are eager to see results and impact, but there is a need to be patient. In fact, the long-term results of participating in an RSI project may provide greater benefits than will be realized within even the five years of a project's initial period of operation. Further, fast-paced innovators are often relieved of their responsibilities in rural communities or move on to other challenges in relatively short order. But still, we observed the importance of having strong administrative support in these reform efforts. RSI support for effective leadership at the local level is a critical element in the overall reform movement.

In closing, it is important to note that each RSI that we studied and all six communities in which we conducted case studies have put a tremendous amount of effort into the project. There are areas in which they and others see substantial impact, but this may be but a "tip of the iceberg" for what might be achieved over time. Obviously, there are limitations, monetary and otherwise, to the RSI approach. However, the RSI effort coincides well with state efforts to align curricula, to develop sound and defensible assessment techniques, and other ongoing reform/school improvement efforts at the local, regional, or state levels. In many ways, our study seems to validate the work and writings of Datnow and Stringfield (2000), Hall and Hord (2001), and others who have spent years studying school reforms and change processes. The RSI initiative is probably overly ambitious and has greater expectations than can be realized within the time frame of one cycle (five years) of a project. In our study of rural communities, we recognize that geographic and economic isolation are only parts of the total picture and, although schools have evolved from one-room country schools to a number of schools organized as a countywide district, the local schools in individual communities are still quite important to residents. Recent literature emphasizes what almost everyone already knew, i.e., small schools with opportunities for much parental involvement are both desired and show promise for a superior school setting for many students. Parents who themselves are not well educated and are members of a powerless social class could very well feel more comfortable in settings among neighbors and where they feel their voices are heard. We want to think that reform and school improvement are both possible and desirable in schools serving poor, isolated rural communities. While the efforts of the RSIs seem to be following paths that are compatible with the research literature on reform, innovations, and change, some unique aspects of these rural communities test conventional wisdom about the starting point and expectations of the program.

References

Brandt, R. (1989). On misuse of testing: A conversation with George Madaus. Educational Leadership, 46(7), 26-29.

Bratton, S. E., Horn, S. P., & Wright, S. P. (1999). Using and interpreting Tennessee's value-added assessment system: A primer for teachers and principals. Knoxville, TN: University of Tennessee, Value-Added Research and Assessment Center.

Campbell, P., & Greenberg, S. (1993). Equity issues in educational research methods. In S. K. Biklen & D. S. Pollard (Eds.), Gender and education. Ninety-second yearbook of the National Society for the Study of Education, Part 1. Chicago: The University of Chicago Press.

Council of Chief State School Officers. (1998). State mathematics and science standards, frameworks, and student assessments [On-line]. Available: www.ccsso.org/frmwkweb.html

Daniels, D. (1991). The best 84 years of my life. Wagon Mound, NM: Santa Fe Trail Publishing Company.

Datnow, & Stringfield, (2000). Working together for reliable school reform. Journal of Education for Students Placed at Risk, 5(1, 2).

Duncan, C. M. (1999). Worlds apart. New Haven, CT: Yale University Press.

Frechtling, J., & Sharp, L. (1997). User-friendly handbook for mixed method evaluations. Washington, DC: National Science Foundation.

Fuhrman, S. (1993). The politics of coherence. In S. Fuhrman (Ed.), Designing coherent education policy. San Francisco: Jossey-Bass.

Fullan, M. (1999). Change forces: The sequel. Philadelphia: Falmer Press.

Fullan, M. (1991). The new meaning of education change. New York: The Teachers College Press.

Grissmer, D. W., Flanagan, A., Kawata, J., & Williamson, S. (2000). Improving student achievement: What state NAEP test scores tell us. Santa Monica, CA: Rand (MR-924-EDU).

Gullickson, A. (2000). The need for student evaluation standards [On-line]. Available: www.wmich.edu/evalctr/jc

Hall, G., & Hord, S. (1987). Change in schools. Albany: State University of New York Press.

Hall, G., & Hord, S. (2001). Implementing change: Patterns, principles and potholes. Boston: Allyn and Bacon.

Hanosh, E. (1967). A history of Mora. Unpublished master's thesis, New Mexico Highlands University, Las Vegas, NM.

National Council of Teachers of Mathematics. (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.

National Forum on Assessment. (1995). Principles and indicators for student assessment systems. Washington, DC: Author.

National Network of Regional Education Laboratories. (1999). Pulling together R&D resources for rural schools [On-line]. Available: www.ael.org/eric/pulling/

National Research Council. (1995). National science education standards. Washington, DC: National Academy Press.

National Science Foundation. (1994). Program solicitation, information, and guidelines for the rural systemic initiatives in science, mathematics and technology education. Arlington, VA: Author.

National Science Foundation. (Undated). Rural systemic initiatives in science, mathematics and technology education: Program solicitation, information, and guidelines [On-line]. Available: www.nsf.gov/pubs/2000/nsf0047/nsf0047.htm

O'Day, J., & Smith, M. (1993). Systemic reform and educational opportunity. In S. Fuhrman (Ed.), Designing coherent education policy. San Francisco: Jossey-Bass.

Pollard, D. (1993). Gender and achievement. In S. K. Biklen & D.S. Pollard (Eds.), Gender and education. Ninety-second yearbook of the National Society for the Study of Education, Part 1. Chicago: The University of Chicago Press.

Rowan, B. (1996). Standards as incentives for instructional reform. In S. Fuhrman & J. A. O'Day (Eds.) Rewards and reform: Creating educational incentives that work. San Francisco: Jossey-Bass.

Royster, W. (1994, February). Issues in rural education and the rural systemic initiative. Paper presented at Building the System: Making Science Education Work, Washington, DC.

Royster, W., & Smith, K. C. (1999). Appalachian Rural Systemic Initiative, program effectiveness review. Lexington, KY: Appalachian Rural Systemic Initiative Consortium.

Shepard, L. (1989). Why we need better assessments. Educational Leadership, 46(7), 4-9.

Smith, M., & O'Day, J. (1991). Systemic school reform. In S. Fuhrman & B. Malen (Eds.), The politics of curriculum and testing. Bristol, PA: Falmer.