Capabilities and Related Experiences of the Evaluation Team

The Evaluation Center at
Western Michigan University

The Evaluation Center at Western Michigan University is an internationally known research and development center that provides leadership for advancing the theory and practice of evaluation as applied to education and human services. The Center's activities include (1) state-of-the-art research to study current evaluation practices and to formulate and test theoretical propositions about evaluation; (2) development to produce data-gathering instruments, reporting formats, and evaluation procedures; (3) dissemination to inform interested persons about relevant evaluation issues and the Center's contributions; (4) service to help a selected group of clients--including school districts, nonprofit agencies, government agencies, foundations, and colleges--to evaluate their programs; (5) instruction to provide evaluation training to students at Western and other interested parties; and (6) leadership to help develop evaluation as a field of professional practice. The Center, which is in the division of research at WMU, was originally founded at Ohio State University in 1963 and moved to WMU in 1973. Dr. Daniel Stufflebeam has continuously served as its director.

Evaluation Center personnel, keeping with the Center’s mission to advance the theory and practice of evaluation, are highly skilled professionals who are dedicated to providing sound, useful, and efficient evaluation services. The coming sections provide further information about The Evaluation Center, staff, and affiliated consultants who will work on this evaluation, related experience, evaluation resources, and data collection and management capabilities.

Over the past two years, The Evaluation Center has been an active observer and learner in the area of charter schools. Two of the evaluation team members, Dr. Jerry Horn and Dr. Gary Miron have been engaged in studies related to the development and operation of charter schools. The Center is particularly interested in working on projects that have potential for major impact on educational practice and for which evaluation is seen as a viable tool for improvement and accountability.


[top] Research Staff Skills The Center is particularly strong in the areas of educational program evaluation and personnel evaluation. Members of the staff are prolific writers about the theory and practice of evaluation, and they are sought after for their services and to lecture internationally on their area of specialty, most recently in the Philippines, Malaysia, Spain, and Taiwan. They have served in a variety of university, state, and national leadership roles; have had many journal articles, monographs, books, and chapters published; and have participated in evaluation work in over 20 countries outside the United States. As appropriate, the Center engages consultants and collaborators on a worldwide basis as their expertise is needed. Over the years, this has included such persons as Carl Candoli, Joy Frechtling, Gene Glass, John Hattie, Edward Iwanicki, Richard Jaeger, Jason Millman, Rita O’Sullivan, William Sanders, Michael Scriven, Anthony Shinkfield, James Stronge, William Webster, and Patricia Wheeler, among many others. [top] Data Collection and Management Capabilities The Evaluation Center is located at WMU in Kalamazoo, Michigan. It occupies about 9,000 square feet of office, library, data processing, and conference space. Generally, the Center has 20-30 employees at one time, with a consistent core of senior researchers. The Center is home to the national Joint Committee on Standards for Educational Evaluation. The Evaluation Center is equipped with the latest and most effective statistical, database, and word processing packages and also has access to the University’s computing resources. The Center maintains, in house, the equipment (NCS Opscan 5) and expertise for scanning large data sets. The Center also utilizes the Survey Tracker Plus package. This survey management software facilitates the production and distribution of surveys via mail-based paper surveys as well as e-mail and web-based surveys. The software package allows for data entry from computer scannable forms and manual data entry as well as electronic forms. The various sources of data are automatically integrated into the same database. The Center’s support staff provides word processing, editing, clerical, accounting, and data encoding/processing services. Center staff are experienced and adept at managing multiple, large-scale projects. [top] Related Evaluation Experience Evaluation Center staff have been involved in developing and evaluating numerous evaluation systems at the local, state, and national levels. The Center has provided evaluation services to a number of state departments of education, including Michigan, Hawaii, Alaska, Ohio, Texas, and Kentucky, as well as the U.S. Department of Education. From 1991-96, the federally funded research and development Center for Research on Educational Accountability and Teacher Evaluation (CREATE) was operated by The Evaluation Center. From that effort, a new professional organization, the Consortium for Research on Educational Accountability and Teacher Evaluation (CREATE) was formed to continue the work of the R & D Center; it will conduct the 9th annual National Evaluation Institute in Traverse City, Michigan, in July 1999. The Evaluation Center has a long history of providing the types of services relevant to this project. The first three projects described deal directly with charter schools. Each of the other selected projects demonstrates the capability of The Evaluation Center to conduct the activities detailed in the proposal.


[top] The Initial Study of Pennsylvania Charter Schools. The Evaluation Center was contracted by the Pennsylvania Department of Education (PDE) to provide an initial formative evaluation of the state's charter school law (Act 22). Evaluation questions included charter school start-up; finance; student, staff, and parent characteristics; teacher professional opportunities; innovations in curriculum, instruction, and assessment; student achievement; and customer satisfaction.


Michigan Public School Academies (Charter Schools) of Michigan (1996-1997) . The Center was contracted to develop an evaluation plan for the charter schools that took into account the needs and concerns of the state of Michigan, the Michigan Department of Education, the charter schools, and the communities involved. Site visits to the individual charter schools and the use of focus groups with the various stakeholder groups were an important part of this work. [top] Connecticut Charter School Evaluation. The Evaluation Center is evaluating individual charter schools and the charter school initiative in Connecticut for the Connecticut Department of Education (CDE). The charter school legislation in Connecticut was in direct response to the perceived need to provide alternative and diverse educational programs for the ultimate goal of improving student academic achievement;; The Center is conducting the evaluation over 4.5 years by assisting each charter school to identify indicators unique to its specific goals and mission; providing technical assistance to the charter schools in developing a system for annually collecting and reporting data pertinent to their local evaluation model; and serving as a resource for measurement, evaluation, and decision-making issues. The bulk of the work, including technical assistance is provided during the first 2 years of the project. Because new, innovative programs (e.g., charter schools) are of considerable interest to the public, these programs/schools are often bombarded with requests for information and to serve as subjects for a variety of studies. Center staff are making every effort to use existing data. Both quantitative and qualitative data are being collected through surveys; interviews; document review; focus groups; direct observation; case studies; and review of work samples and portfolios. The Program Evaluation Standards (Joint Committee on Standards for Educational Evaluation, 1994) are being followed in the conduct and operations of this study. A web site has been developed for posting technical assistance bulletins, updates about the project, and facilitating communication with all stakeholders. Annual reports are submitted that contain results to date. An annual report has been prepared and posted on the web site. Subsequent reports and the final report will also be prepared for web publishing after approval from CSDE.. [top] Michigan Public School Academy/Charter School Evaluation (9/1/97-12/23/98) . Michigan has played a leadership role in shaping the legislation and developing the infrastructure for the development of charter schools. The Evaluation Center was one of two contractors selected to conduct independent evaluations of the charter school initiative in Michigan. The design work for this evaluation was performed by The Evaluation Center under an earlier contract with the Michigan Department of Education. The purpose of this evaluation is to determine if charter schools are fulfilling their missions. More than 51 charter schools are being visited, and the effectiveness of the authorizing process, the role of private service providers in the schools, the role of the Department of Education, and the schools' impact on their communities are being studied. Information was collected through document reviews; direct observation; interviews; surveys; focus group meetings; reviews of student work samples; test scores; and reviews of diaries, logs, and portfolios, if available. The services of part-time Traveling Observers (TO) participate directly in the collection of information from identified sources, provide on-site assistance and consultation to the schools regarding the collection of data, and act as the eyes and ears; of the core evaluation team. The TO concept has been used extensively by The Evaluation Center on a variety of projects. Results from the project will provide the legislature with information to determine if the charter school movement is meeting its statutory objectives. Charter schools will receive guidance regarding effective processes being used in other charter schools, and students and parents will learn about how well their charter school is doing relative to other charter schools and traditional public schools. In addition to these projects conducted by The Evaluation Center, a description of two related project is included below. These studies were led by Gary Miron, proposed project director, while working at Stockholm University. [top] Restructuring of Education in Europe. This study examined changes taking place in national education systems in Europe due to political, economic, and social forces, as well as the increasing role of the European Union. Four countries were studied in depth. The study included document and literature review, interviews of policy makers, analysis of national data sets, and case studies of four countries. The countries that were studied in depth were the Czech Republic, Denmark, Germany, and Sweden. [top] Choice and the Use of Market Forces in Schooling. An investigation was conducted for the National Agency for Education on the reforms in Sweden dealing with decentralization, choice, and the use of market forces as a means to restructure the education system (1992/93). The study involved document and literature review, interviews of policymakers and school directors, site visits to independent schools that received public money in the form of vouchers, and the establishment and analysis of a database with descriptive and statistical indicators on the enrollments and costs of independent schools. [top of page] [Cleveland Community Schools Home]