An Administrator's Perspective of Evaluation*

by

Jack P. Taylor, Ph.D.

Superintendent

Saginaw Public School

October 1974

*Based on an address to a Faculty-Student Seminar Series on Evaluation at Western Michigan University sponsored by the Evaluation Center.

Let me begin this paper on evaluation with a hypothetical situation, one that a superintendent of schools is apt to fact at any time.

Imagine this scene if you will: A board of education meeting crowded with community residents-mostly parents-who have just heard the superintendent deliver a glowing report on the "quality education" their children are receiving through the public schools. The question and answer session is going smoothly with the superintendent handling the questions with a flawless mixture of reassuring platitudes and technical jargon. The audience seems content.

Suddenly, in the midst of this calm exchange, an angry parent rises up with a pointed question: "OK, we've listened to you say a lot of good things about this new reading program and about how successful this pre-school program has been. You say both of these are helping to provide quality education and are worth every penny they're costing us. Just how do we, the parents and taxpayers, know hoe good these programs are? What proof do you have that they are successful?"

A question like this typically causes one or two reactions:

1. The superintendent who has little or no hard data to support his claims usually tries to deflect the question with some more generalities or defers it to one of his assistants-who more than likely will be hard pressed to answer it satisfactorily. At this point, the crowd becomes more insistent, demanding that the superintendent present actual proof to justify his claims. The superintendent may survive this meeting but you can bet it won't be the last time he's confronted by the public. They will keep after him until he has some solid evidence about the effectiveness of his schools.

2. The superintendent who has an evaluation staff within his administration handles the situation in a much different manner. There is no need for him to deflect specific questions with clichés or generalities-or to pass the buck to an assistant who doesn't have the answers either. This superintendent has the information at hand. He can relate hard data and other evaluative information about student achievement levels, cost-benefit ratios and the like. His answers should provide the public with the information they are seeking and help them to better understand how the school system is educating their children.

This brief illustration, while somewhat oversimplified, points out how the public's attitude toward education has changed in the last decade. Parents today are much less passive about the education of their children; they no longer automatically defer to teachers and administrators in matters concerning their schools. More and more parents-as well as non-parents who contribute tax dollars to the schools-are demanding proof of performance in education. They want to know, among other things:

1. What $1000 per year per pupil will buy that $750 or $500 won't buy?

2. What actual outcomes have resulted from the new reading program and are they worth the cost?

3. Why are test results at some schools always higher than those at other schools?

4. Are our children meeting the performance goals set by the school system? If not, why not and how far do they have to go?

5. Why is the dropout rate in the junior and senior high schools so high? And why are these people dropping out?

6. What is the employment rate of our graduates? How many go to college? How does this compare with other districts?

These are just some of the questions our constituents want answered. In short, they want to know what educational benefits their children are receiving, and if these outcomes warrant the cost. They want to know if new, supposedly innovative programs are really working or if they are just brightly burning stars that cause a brief moment of excitement before fading quickly into obscurity.

As school administrators, we have the responsibility to provide the public with answers to these questions. The day has long passed when we educators can expect the public to rely on our infinite wisdom; no longer can we say: "Trust us, we know best." The public is better educated and more sophisticated today and they will not be bought off or pacified with slogans and clichés. For the public to take us seriously, we must have rational basis for making policy decisions, we must have real information to support our claims.

As Dr. Mary Alice White of Columbia University expressed so well several years ago:

The day has almost passed when our constituents are going to accept another innovation just because it is the current fad or gimmick. With all the vast problems it will bring, the revolution is toward rational educational management. This involves the ability to state educational goals clearly, to institute systematic ways of measuring progress toward them, for public reporting of that progress, and reconsideration of goals and strategies based upon the data. ⁽¹⁾

One process by which a system of rational educational management can be achieved is, as you might have guessed, through program evaluation. Before getting into the specifics of evaluation and how it is used in the Saginaw Public Schools, I would like to put this issue in some kind of historical context.

Over the years, the concept of educational evaluation has been in a constant state of evolution. It has responded to the demands of a changing society, new educational theories, and technical advances within evaluation itself.

Although the origins of educational evaluation stretch back to the mid-19th century, the modern catalyst for program evaluation was the passage of the 1965 Elementary and Secondary Education Act. Up until this time, there were few paid professional evaluators and little information available about educational outcomes. In short, evaluation had little effect on educational practices.

The entire thrust of evaluation was changed with the SEA legislation. It was a landmark bill both in terms of educational funding and educational evaluation. Under Title I, it provided for the first time massive federal aid to local school districts with high concentrations of disadvantaged children. But this money-some one billion dollars-was not given without some provisions. Both Title I and Title III required the local district receiving funds to evaluate the effectiveness of its programs to insure that this federal money was accomplishing its intended purposes, that it was actually leading to improvements within the system.

. . . effective procedures including provisions for objective measurements of educational achievement will be adopted for evaluating at least annually the effectiveness of the programs in meeting the special education needs of educationally deprived children in that local agency. ⁽²⁾

A major force behind the effort to attach an evaluation requirement to the SEA legislation was Senator Robert Kennedy. He was concerned that too often local districts receiving federal funds were not held accountable for the way those funds were spent. ⁽³⁾ This time he wanted local school districts to account for monies they were allocated under SEA and to file a yearly evaluation report indicating the outcomes of each program.

It almost goes without saying that this legislation threw most school administrators into a panic. This was the first time in history that educators were mandated by law to evaluate their own classroom programs and, not surprisingly, most were caught unprepared. The results were often chaotic at best. After all, few educators had any expertise in evaluation techniques. And the small number of evaluation consultants who were available could not meet the growing demand. Besides being few in number, most of these consultants were from universities and were trained in experimental techniques that were limited to laboratory settings. Their methods just weren't applicable to a public school setting.

Saginaw, like most other school districts around the country, was not prepared for the evaluation requirements of Title I. As a result, the district's early response to evaluation was to comply with the minimum requirements of the law.

When I came to Saginaw as superintendent in 1967, during the early years of mandated evaluation, there was just one person working on all the district's evaluation activities. By the fall of 1968, a staff of five full-time professionals had been assembled.

In the years following passage of the ESEA legislation, evaluation models and theories began to appear on the scene-much to the relief of many administrators who were groping for workable models on which to base our operations.

In 1968, our Department of Evaluation contracted with The Ohio State University Evaluation Center to assist the district in training its local evaluators and to help install the CIPP model of evaluation. The CIPP model-developed at the Center under the direction of Dr. Daniel Stufflebeam-is considered by our evaluators to be the most comprehensive of the current evaluation mechanisms.

For those of you who have not yet been introduced to the CIPP model, the acronym stands for Context, Input, Process, and Product evaluation. While time does not permit a lengthy explanation of the model here, its basic purpose is to provide useful information to educators for use in decision-making. Through the use of these four types of evaluation, a decision-maker is able to answer four basic questions:

1. What should we do? This involves collecting and analyzing needs assessment data to determine goals, priorities and objectives. For example, a context evaluation of a reading program might involve an analysis of the existing objectives of the reading program, reading achievement est scores, staff concerns, reports of reading conferences, and community concerns.

2. How should we do it? This involves steps to best meet the new goals and objectives and might include identifying successful outside programs and materials as well as gathering information from literature on the strengths and weaknesses of relevant programs.

3. The third question to be answered is: Are we doing it as planned? This provides decision-makers with information about how well the program is being implemented. By continuously monitoring the program, we learn such things as how well it is following the guidelines, conflicts in organization, staff support and morale, strengths and weaknesses of materials, budgeting problems.

4. The final question to be answered is: Did the program work? By measuring the actual outcomes and comparing them to the anticipated outcomes, we are better able to decide if the program should be continued, modified, or dropped altogether. This is the essence of product evaluation.

That is a very brief overview of how the CIPP evaluation model is supposed to work. Figure One displays the relationship between the type of evaluation, the decision served and the question answered. I would like at this point to leave the theoretical concepts of the CIPP model to the experts and begin discussion how evaluation is used in the Saginaw Schools and what my role as superintendent is in the process.

TYPE OF EVALUATION TYPE OF DECISION TYPE OF QUESTION ANSWERED

Context Evaluation Planning decisions What should we do?

Input Evaluation Structuring decisions How should we do it?

Process Evaluation Implementing decisions Are we doing it as

planned?

Product Evaluation Recycling decisions Did it work?

Figure 1.

THE CIPP MODEL OF EVALUATION

The main purpose of our evaluation department is to support the decision-making process in the district by supplying valid and relevant information to decision-makers at all levels. We depend on this evaluative information to help us reach sound, rational judgments about our educational program. Reducing the guesswork in decision-making, I might add, makes my job a little easier.

Briefly, our department of evaluation, which now has seven full-time staff members, is organized around three interrelated divisions, Figure Two is a graphic display of the organization of the unit.

The first is Program Evaluation, which is responsible for designing and implementing all program evaluations for the district. This year, for instance, the department is involved in evaluating four separate Title I projects: the states Chapter 3 compensatory education program; the migrant and bilingual programs; an adult education program; and an early childhood program for children with learning disabilities.

The second division is Assessment Services which, as the name implies, has the task of developing and supervising the district's norm-referenced and objective-references testing program.

The third division is Research Services which is responsible for gathering needs assessment information for the district,

[INSERT FIGURE 2 HERE.]

primarily through institutional and survey research studies. The research division conducts a yearly system-wide dropout study, and periodically, published elementary or secondary school profiles. During the past year, two special projects have taken up a good share of the research unit's time. One is a follow-up study of former graduates and the second is a junior and senior high student opinion survey, the results of which are being released this month.

My relationship with the evaluation department is one of mutual dependence. I rely on them to provide the kind of information necessary to make rational decisions and they in turn depend on me, as superintendent, to create a climate of acceptance for evaluation procedures among the professional staff and especially among the administrative decision-makers.

Before I get any further into the role of evaluation in the Saginaw Schools, it would no doubt be helpful if you know something about our school district. Saginaw is Michigan's sixth largest school district. There are over 20,000 students in our public schools. The city has 26 elementary schools, 5 junior highs, 2 high schools, and a Career Opportunities Center. Of our total school population, about half are minority students, most of whom are black and Chicano. Eleven of our buildings have Title I programs operating in them. The problems that are plaguing other urban areas in Michigan and around the country are evident in Saginaw-low academic achievement by many of our students, a declining tax base within the city, an increasing difficulty in getting millage requests passed at the polls, a white exodus to suburbs and a residential housing pattern that has kept our schools for the most part segregated. In short, we are experiencing problems very similar to larger urban districts. The only real difference seems to be numbers.

Now that you have at least a brief sketch of the Saginaw Public School District, I would like to discuss, in greater detail, the several roles played by evaluation in our schools, as well as some of the problems inherent in the process.

Evaluation has several important functions within the Saginaw Public Schools. Obviously, a major responsibility is to meet the federal requirements for evaluation of Title I, Title III, and ohter external programs with mandatory evaluation. This is literally a twelve-month chore-from proposal development, to pretesting in the fall, to mid-year process evaluations, to spring post-testing to, finally, report-writing.

A second important priority for the evaluation department is to provide decision-makers with comprehensive needs assessment data. This information lets us know the area in which we need to focus our attention. A couple of examples will help illustrate this point. Several years ago in Saginaw, we went through a difficult period of student disturbances at one of our high schools. Students were dissatisfied with many aspects of their education and voiced their disapproval and proposals for change to the administration. It was a difficult period for everyone involved. But it was also a constructive period. Students and educators had an opportunity to come together, to talk over problems and come up with alternatives to the status quo. We became more aware of how students felt, where problem areas existed. If, at the time, we had had a mechanism for systematically monitoring student attitudes, much if not all of the turmoil might have been avoided. We would have been aware of many of the problems in advance and could have moved to solve them before they reached the crisis stage. We now have such a means of monitoring student opinion. The evaluation department has started an annual student opinion survey in our junior and senior high schools. The results of this survey will provide us with valuable needs assessment data than can be useful in detecting potential problems before they reach the crisis stage.

A more recent example of needs assessment activity involved the department's work with the district's Task Force on Hunger. This month the department conducted a survey of children in two elementary schools in order to learn about the eating habits of those pupils who live in Saginaw's Title I neighborhoods. Specifically, the Board of Education wanted to know how many children ate breakfast and lunch on school days. These survey findings will help us determine two things: 1) if there is a need to begin a hot lunch program in our elementary schools, and 2) how well our pilot breakfast program is working.

Collecting this kind of needs assessment data is imperative, especially in a large pluralistic, urban setting. Our children tend to be poorer. Their educational needs are more complex and demanding. And through this approach we are better able to define their needs and develop alternative programs to respond to their needs.

Most of the evaluation department's projects are mapped out before the school year begins. An appendix to the paper gives a sample of the annual work plan of the evaluation department. However, even the best schedules are subject to change. Occasionally during the year, the department is called upon to either evaluate local programs or to lend its expertise to an unanticipated problem. Over the last couple of years, such non-mandated or ad hoc evaluations have included an evaluation of a work experience program run by the system's Career Opportunities Center, and evaluation of counseling services at one high school, and evaluation of the AAAS Science Program, and an evaluation of a new General Motors' education project in the junior highs.

The evaluation department is also an active participant in program designing and proposal writing. Not only do the evaluators develop evaluation designs for each externally-funded project but they also use their expertise to help critique and refine the proposals. Over the years, the reputations gained by our evaluation department has contributed significantly to the success our federal programs department has had in securing competitive federal grants.

These four areas-mandated evaluation, needs assessment, non-mandated evaluation, and proposal development-are the major but by no means the only activities in which the evaluation department is involved.

I need people on my staff who are skilled and experienced in the many new educational techniques we have begun to employ in the Saginaw Schools. Specifically, I need staff members:

1. Who can conduct cost-benefit analysis of programs to find out if our outcomes justify our expenditures.

2. Who have both the expertise and the resources to stay current with the latest state and federal program validation projects-so that we will know what programs have been found to be successful around the country.

3. Who are skilled in the development, interpretation and utilization of objective-referenced tests.

4. Who understand how to conduct systems analyses and operations research of ongoing programs.

5. Who are knowledgeable about the whole complicated issue of performance contracting and know how to institute such a process in our district (as we must do with the state's Chapter 3 program).

6. Who can insure the proper and accurate use of Michigan Assessment data.

7. Who have the resources to keep up to date on the latest information from the various Research and Development Laboratories around the country.

8. Who know how to go about bringing change to the school system.

In each of the eight areas I have just mentioned, I rely heavily on the evaluation department. An increasing amount of their time and effort is spent on these areas.

In addition, the department is involved in a number of other projects that provide a continuous source of information to decision-makers. The evaluation staff regularly assembles elementary and secondary school profiles which list staff, pupil, and community data for each school and for the district as a whole. They also publish a research newsletter that reports on the latest trends, innovations, research findings and issues in education and is sent to all professional staff members. The department also maintains a professional library which contains, at present, well over 1,000 books and over 30 professional journals. As a result of these many and varied projects, our decision-makers rarely suffer from a lack of information.

Over the years, we administrators have been conditioned to make many of our decisions in a vacuum, outside the public's view and without much useful information about program outcomes at our disposal. That has begun to change in recent years. My staff and I rely heavily on evaluative information to keep us current on the many programs in the district. Given the size of our district and the number of programs we have running, it would be literally impossible to keep us with what was happening in each one without this information.

Technically, the role of the evaluator is to provide the decision-maker with sound and useful information. Likewise, it is the decision-maker's role to weigh the evidence provided to him by the evaluator and others and then make judgments concerning the situations confronting him. Because the decision-maker has neither the time nor, in many instances, the technical skill necessary to gather all the relevant information needed for making these decisions, the evaluation staff assists him in providing information about why action must be taken and what possible courses of action are available. Such decisions can range from modifying a delivery system, to adopting a validated project, to terminating an habitually poor program.

This is where a professional evaluation staff becomes invaluable to an administrator. The evaluator must know enough about how any one decision will likely be made, and enough about information used by an educator in reading that decision, to identify the scientifically sound and useful information needed by a decision-maker to reach a defensible decision.

Ideally, then, the administrator and evaluator should work cooperatively to produce a set of rational judgments.

What I have succeeded in doing up to this point, I feel, is to paint a picture-a picture of the Utopian ideal of evaluation. Anyone who has spent even the briefest of time working in a school system-whether in a large urban center of a bucolic, rural setting-knows that things seldom, if ever, work as smoothly in practice as in abstract theory.

You may be saying to yourselves at this point that all this mutual dependence and interaction sounds good in theory, but what happens when these theories are transformed into reality. When politics, personalities, organizational structures, and personal philosophies enter the picture, how do these evaluation theories really work? You may also be wondering how objective an evaluation department can be when it is part of the superintendent's staff.

Two points need to be made in this regard. First, the evaluation staff is directly responsible to me and not subordinate to those whose programs are being evaluated. This means that all evaluative findings are made directly available to me and do not have to be "challenged" to me through program directors. So, in this sense the data I receive can be completely candid. The second check to safeguard objective data is the external audit, in which an independent third party periodically reviews our evaluative findings.

Still, in all honesty, the precess does not always work as it is diagramed. Few things ever do, for that matter.

Even though evaluation rarely works as smoothly in the practical setting of an urban school district as it does in the abstract, it has helped create a climate for change within education. It is a valuable tool in that it creates an awareness of problem areas. For example, a federally funded program for disadvantaged children can look great on paper-it might have substantial funding, a tight, well-conceived delivery system, a competent and dedicated staff and a top-notch administrator. On the surface, you would assume that the program would be a success-every indicator is pointing in that direction. Under the old-or traditional-way of doing things, the superintendent and other decision-makers would look at the structure of the program, see that it "looked" good and label it a success. This method could then be repeated year after year. And that could be disastrous. Let's say, for instance, that there was poor communication between the project director and the staff; or that the staff had little understanding of the delivery system and the intended goals of the program; or that the wrong children were chosen to participate; or, ultimately, that the program-for all its sophistication-was merely a "paper tiger", that it was not, in fact doing what it was supposed to do. It is these kinds of problems that a good evaluation can help discover.

It often seems that the evaluation department is bringing me an unending string of problems. Like the ancient Greek messengers who reported from the battlefield, evaluators often seem to be the purveyors of bad news. This is the department's job It was set up to be a kind of "devil's advocate" within the system, providing objective data and honest views about what is happening as well as recommendations for change.

While the evaluation department necessarily creates some inevitable conflict along the way, it is this type of activity that can lead to change. One of the major pitfalls of the traditional way of doing things in education was that problems often went undetected for long periods of time. It was not so much that they were ignored but that they were never detected. These problems were allowed to build up gradually over the years until they reached crisis conditions. Many times there was little warning that such problem areas even existed. Although evaluation certainly has by no means eliminated these problems, it has served as a kind of early warning device, discovering and reporting many of them early enough to take steps to improve the program and head off a major crisis that might have developed otherwise. I don't really know how many potential problems our evaluative feedback has helped detect and how many crises have been avoided as a result, but I do know that I wouldn't want to go back to the old ways and find out.

Earlier in this paper when discussing the Saginaw school system, I touched briefly on the multitude of problems facing urban school districts today; the major dimensions of which are familiar to everyone here today. In very broad terms, these problems are: 1) the deepening financial crisis of the urban schools; 2) the urban environment of the students; 3) the impoverished urban student; and 4) the whole system of urban education. These problems did not occur suddenly, they have been allowed to exist and multiply over the years. Many of the problems facing urban schools are beyond the reach of its schools; there is little we can do directly to improve the urban environment, or bring back lost tax revenues, or relieve the immediate poverty of our students and their families.

However, we can work to improve the system of urban education. Our challenge is to design and implement a curriculum, an entire instructional program that is relevant to the needs of our students-and to do this within the constraints imposed on us by our precarious financial situation. This requires, in many cases, fundamental change. In confronting these problems and in affecting change, evaluation is absolutely essential. The whole thrust of the evaluation process is to help bring about change by first creating an awareness of problems, then by critiquing the strengths and weaknesses of alternate strategies, and finally, by monitoring the processes and outcomes of the new programs. Evaluation helps us discover whether new social action programs, many of which are experimental, are meeting their goals, why they are successful or, conversely, why they are failures.

Evaluation is also essential because of the sheer size of urban districts. With more children, a large staff, more special programs, and larger budgets it is difficult, almost impossible, for urban school administrators without evaluative material to stay on top of everything that is happening in the district.

I would be remiss if I didn't talk, at least briefly, about some of the types of problems regarding evaluation that I have witnessed over the past few years. These are areas, as I mentioned earlier, in which evaluation has-and sometimes does-become compromised: politics, personalities of the people involved, and the basic organizational structure of the school system.

I think I'm safe in saying that these problems are by no means unique to Saginaw.

1. From time to time conflicts between the evaluation staff and program directors arise over program management and planning. This is especially true in the earlier stages of a program. Oftentimes, the evaluators are better equipped to implement management and planning techniques than are the directors, many of whom are not skilled in program management, systems analysis and methods to bring about change. The directors, in some instances, begin to question who is actually running the program. The goal of evaluation, in cases like this, is to work with these people on a concentrated basis to help them acquire these skills and apply them to their programs. During this period, however, there can be friction between the two on how much influence evaluation should have on program decisions.

2. Project directors and principals who are not data oriented sometimes do not know how to use evaluative information. When this happens, it makes it difficult for the evaluation department to influence any substantive changes in a program over the short term. Over the long term, these problems can usually be worked out.

3. Even when all parties involved agree that the evaluative findings point toward major changes or outright termination of a program, it is not always that simple to make the necessary changes. Other factors like teacher contracts (particularly transfer and job assignment provisions) and tenure laws prohibit courses of action.

4. Another significant problem faced by our evaluation department is in developing program proposals. Our evaluators, whenever appropriate, want every facet of a program spelled out in detail--time-lines, objectives, delivery systems, criteria for success. During the early stages of a program, problems sometimes occur with program planners who are relatively inexperienced in developing guidelines. In most instances, these problems can be overcome as the two parties work together. When these conflicts continue, however, they can seriously limit a program's effectiveness and leave the evaluators with little or no useful data on which to base its evaluation.

5. Despite the fact that evaluation in our district is used for judging the effectiveness of programs and not personnel, there remain some people who still feel threatened by it, who view a critical program evaluation as a personal attack. Too often a reaction like this leads them to ignore, or dismiss or rationalize the findings and recommendations instead of acting upon them. When this happens, the program, and ultimately the children, suffer the consequences.

6. The dissemination of evaluation findings sometimes presents a problem. Like any organization, the public school are eager to tell the public about their successes and reluctant to share their failures. Obviously we have no difficulty making our successes public. The difficulty comes in telling the public about those programs that, for whatever reasons, have not been successful. It's the old problem of balancing the public's right to know with our obligation to conduct the most open and far-reaching evaluations possible. The way we have handled this delicate situation is to announce simultaneously our program shortcomings and our plan of action to correct these shortcomings. I think that this type of external dissemination, focusing on positive efforts, has helped to increase public confidence in our educational programs.

Perhaps the most important factor in guaranteeing effective evaluation, and ultimately effective educational programs, is internal dissemination. Although the federal government requires evaluation of the programs it funds, it is virtually impossible for the U. S. Office of Education or the State Department to effectively monitor every one of the thousands of programs that are currently operating. So the actual responsibility for program improvement rests with the local districts. If programs are to be improved we must do it. If evaluation is to have any effect on program improvement, the evaluation findings must be disseminated to all those who are connected with that particular program--from the superintendent to the teaching staff. The best way to negate any evaluation finding--and to stop any meaningful changes from taking place--is to keep that information private. Our evaluation staff makes an effort to see that its evaluative findings--both process and product--are disseminated to all relevant audiences.

As the above examples point out, things are not ideal concerning evaluation in the Saginaw Public Schools. But evaluation has played a significant role in moving the district away from purely traditional activities. We are trying new methods, learning what works and what doesn't, and keeping what works and reworking or discarding those programs that don't. If this process could be viewed on a continuum, with "what was" at one end and "what should be" at the other, right now we wold fall somewhere in the middle. However, the district is moving forward and not backward. This, as I see it, is evaluation's main contribution to education so far; it has forced us to examine many of our traditional practices objectively and the result has been a movement toward practical innovation. By practical innovation I mean new ideas, concepts, and theories that actually work; that make schools a better place to be; that increase the effectiveness of teaching and administration; and that help children to acquire a truly quality education. In this way, evaluation can help districts, urban districts in particular, do a more effective job of systematic long-range planning. All too often, we are guilty of moving from crisis to crisis, reacting only to immediate situations.

One other aspect of evaluation that I want to mention again is that it helps the schools be more accountable to their constituents. Evaluation provides information not only to decision-makers within the school system but also to the community. This information--whether the results of anew program, a report on student attitudes, or a study of dropouts--has the effect of bringing the community closer to its schools. It helps keep the school system's activities, particularly those related to curriculum, from becoming shrouded in secrecy.

Looking ahead, I believe that the use of evaluation within our district--and throughout the country--will continue to expand in both scope and effectiveness. Today, required program evaluation comprises the major portion of our department's work. But during the next few years, I see evaluation in Saginaw becoming involved in many new areas. Most importantly, I see evaluation services eventually expanding to include the whole general education curriculum. We need to apply the same techniques to all our programs that we have to our federal programs. Possibly as our federally funded programs mature, we can shift more of our evaluation emphasis away from them toward our "general fund" programs.

In conclusion, let me stress that while evaluation holds great promise, I do not view it as a panacea. Even in its most ideal and judicious application, it will not cure all the ills of our educational system. What it can do is help us change our approach to educational decision-making. Like any firmly entrenched bureaucracy, education has over the years resisted pressure for change from inside the system and from outside. There has been no tradition of planned, systematic change. We tend to go from crisis to crisis. Predictably, there has been little dependable information concerning our educational practices, programs, and products--and many of us were not trained to use such information even when it existed.

This is why, potentially, evaluation can have such a significant impact on education. Evaluation is providing educators, at long last, with badly needed information which can be used to improve the education of our children. The ultimate success of evaluation will, however, rest with those of us who must make decisions based on this evaluative information.

As I said earlier, evaluation holds no magic formula, no miracle ingredient that will immediately transform the quality of education in our society. The problems of our educational system are too complex and much too deeply rooted to be cured by a single tool. Evaluation is simply one of the most promising tools we have to help move us in a direction away from the failures and inadequacies of the traditional methods of deciding what is best for our schools and our children. Now it is up to us educators at all levels to give evaluation a chance to work.

BIBLIOGRAPHY

1. White, Mary A. "Memo to a Future Superintendent". Phi Delta

Kappan, June 1969. pp. 595-597.

2. Sec. 205 (A) (6), in part, ESEA Title I Act.

3. Hearing before the Subcommittee on Education of the United

States Senate, Eighty-Ninth Congress, First Session on S. 370. "A Bill to Strengthen and Improve Educational Opportunities in the Nation's Elementary and Secondary Schools." Washington, D.C.: U. S. Government Printing Office, 1965.

APPENDIX

TO: Jack P. Taylor

FROM: Jerry R. Baker

RE: Result of Annual Focus

As a result of our meeting last week, the following services will be provided by the Evaluation Department during the 1974-75 school year:

I. Program Evaluation

1. Evaluation of E.S.E.A Title I Programs

a. P.E.R.T.

b. Pre-School

c. Continuation School

d. Non-Public School Projects

--Sacred Heart

--Holy Rosary

--Trinity Lutheran

2. Evaluation of the Chapter 3 Program

a. 39A Projects

3. Evaluation of the Adult Basic Education Program

4. Evaluation of the Migrant Education Program

a. academic year

b. summer

5. Evaluation of the Early Childhood Education Project

6. Evaluation of the Bi-lingual Education Program

7. In addition, the department will provide assistance in the evaluation efforts associated with the Career Opportunities Program and the United Services for Exceptional Children (U.S.E.C.) Project

II. Assessment Services

8. Provide coordination and supervision of the school district's assessment program:

Test Grade(s)

a. Saginaw Objective Referenced Test Pre-K-6

b. Iowa Tests of Basic Skills 3-5-8

c. Differential Aptitude Test 9

d. Ohio Vocational Interest Inventory 10

e. Michigan Assessment 1-4-7

f. General Educational Development Test Adults

g. Supplemental Testing K-12

9. Provide an inservice education program for professional staff members in regards to the interpretation and utilization of all assessment data.

10. Develop and maintain the department's Instrument Bank.

III. Research Services

11. Conduct a dropout study in each secondary school.

12. Prepare a written report of the student attitude survey conducted in grades 8 and 11.

13. Prepare a written report of the follow-up study of 1973 graduates.

14. Develop and maintain the department's:

a. Innovative Practices Bank

b. Professional Library

15. Develop a mechanism for disseminating current research findings to the professional staff.

16. Implement the student mobility monitoring system in each elementary building.

17. Conduct a context evaluation (needs assessment) at the district and building level.

IV. Other Services

18. Assist in the development of proposals for external funding:

a. provide needs assessment data

b. develop research/evaluation designs

19. Coordinate the development and implementation of the Instructional Classroom Management Project with the Lansing, Grand Rapids, and Pontiac school districts.

20. Assist in the school district's efforts to implement the State Accountability Model:

a. classroom level

b. building level

c. district level

21. Assist in the preparation of various state and federal reports.

22. Serve as needed on the following school district committee(s)/council(s):

a. Joint Curriculum Committee

b. Professional Study Committee

c. Compensatory Education Advisory Council

d. Administrative Council

23. Represent the school district on the following committee(s)/council(s):

a. Advisory Council for Research Evaluation and Assessment Services, Michigan Department of Education

b. Committee on Evaluation and Information Systems, Council of Chief State School Officers

24. Serve in other areas as directed by the Superintendent.

9/74

1. White, Mary A. "Memo to a Future Superintendent" Phi Delta Kappan, June 1969, pp. 595-597.

2. Sec. 205 (A) (6), in part, SEA Title I Act.

3. Hearing before the Subcommittee on Education of the United States Senate, Eight-Ninth Congress, First Session on S. 370, "A Bill to Strengthen and Improve Educational Opportunities in the Nation's Elementary and Secondary Schools." Washington, D.C.: U. S. Government Printing Office, 1965.