ANNUAL MEETING OF THE

JOINT COMMITTEE ON STANDARDS

FOR EDUCATIONAL EVALUATION

Kalamazoo, MichiganOctober 4-6, 1996

Attendees:

Joint Committee Members:

Dr. Edith Beatty (ASCD)

Dr. Mark Davison, (APA)

Dr. Arlen Gullickson (Member-at-large)

Ms. Gwen Keith (CES)

Mr. William Mays, Jr. (NAESP)

Dr. Dianna Newman (AEA)

Dr. Kathleen Sullivan (NLPES)

Dr. James R. Sanders, Chair

Dr. William Webster (AERA)

Dr. Robert Wilson (CSSE)

Dr. Donald Yarbrough (NCME)

Joint Committee Staff:

Ms. Mary Ramlow, Secretary

JOINT COMMITTEE ON STANDARDS FOR EDUCATIONAL EVALUATION

MEETING MINUTES

OCTOBER 4-6, 1996

KALAMAZOO, MICHIGAN

Friday, October 4, 1996

Meeting Objectives

1. To conduct the business of The Joint Committee on Standards for Educational Evaluation

2. To identify revisions to be made in The Personnel Evaluation Standards

3. To schedule presentations and training for 1996-1997

4. To review and update dissemination strategies for 1996-1997

Dr. Sanders welcomed everyone to the 22nd annual meeting of The Joint Committee. He stated that instead of reiterating the history of The Joint Committee, which in the past has been a regular part of opening the meeting, that information had been transcribed and sent in written form to the members.

Introductions were made. Kathleen Sullivan gave background information about the National Legislative Program Evaluation Society (NLPES) as a new member of the Joint Committee.

Chair's Report

Dr. Sanders addressed several important items that have occurred over the past year, including:

• NLPES has accepted The Joint Committee's invitation to become a sponsoring organization, and Kathleen Sullivan is attending as their representative

• Proposal revisions for the project to develop a third set of standards, standards for evaluations of student performance. A copy of the proposed revisions was sent to the members prior to the meeting. The Kellogg Foundation is still considering the proposed revisions, even though they were submitted in April. Dr. Sanders has been informed that the Kellogg Foundation was not able to fund the project this year, but may consider it in the future. Dr. Sanders said he wishes to continue to seek funding, and feels it is a very important project.

A question was raised as to whether the objectives for student evaluation could be presented in the context of the popular social issues, e.g., at risk students or special needs populations and assessment and measurement done in those areas.

Dr. Sanders stated that he is seeing greater use of The Program Evaluation Standards internationally. He referred to a project out of the University of North Carolina where they are doing training in 14 countries, including Africa, Asia and Latin America in reproductive health areas. They are using The Program Evaluation Standards to teach people how to evaluate their reproductive health systems programs. He is seeing many uses of The Program Evaluation Standards for things that The Joint Committee members had never considered. Last year Dr. Sanders received information that people in Latin America and Germany were translating the Standards too.

There has been some discussion in Eval Talk, which is the listserve out of AEA, on the use of the Standards in other cultures, especially Australia and Asia, saying they are appropriate and work well there. In the U.S. legislative programs, such as where Kathleen Sullivan works, are becoming more involved in using The Program Evaluation Standards. The state of Florida has adopted The Program Evaluation Standards to evaluate legislated programs. Mississippi is also making heavy use of it. The state of Hawaii has adopted The Personnel Evaluation Standards as the basis for evaluating their educational personnel systems. They have put The Personnel Evaluation Standards in every school. Louisiana continues to use The Program Evaluation Standards as the requirement for any consultants to the state who are doing evaluation. They have to go through a workshop where they learn The Program Evaluation Standards and apply them in their work.

Dr. Yarbrough stated that in Iowa there is a state evaluation association, the Iowa Educational Evaluation and Research Association, the equivalent of AERA, but the emphasis is on evaluation. That would seem to be a natural place to introduce the standards.

Dr. Sanders said that in Michigan there is a new association, called the Michigan Association of Evaluators, which just began in May 1996. It is an affiliate with AEA, and has adopted The Program Evaluation Standards as part of its mission statement, saying that all evaluators in Michigan should be using The Program Evaluation Standards. One of their goals is to get the word out to people doing evaluation in Michigan, through workshops, and annual meetings, at least as a beginning.

Dr. Yarbrough inquired about the availability of workshop materials to use in a workshop at the annual conference in Iowa.

Dr. Sanders said The Joint Committee does have some resource material. He referred to a National Science Foundation grant to develop overheads for presentations on The Program Evaluation Standards and offered to provide a copy of the overheads. Presentations have been done at AEA, AERA, NCME and workshops have been done in Florida before the Auditor General's office staff, and those materials are available.

Dr. Gullickson suggested that in order to best promote the standards, The Joint Committee should begin at the state education level. He described experiences with Hawaii and Alaska state education boards, and noted how eager they were for this material and how little information was previously available to them. Dr. Gullickson suggested that the state level educational agencies are eager enough for the information to pay for travel and workshops. He points out that The Joint Committee has been strong in the area of producing the information and less effective in marketing or promoting. Dr. Gullickson further suggested approaching states for funding, i.e. $2,000 to $4,000 each and also go to major foundations, such as Lilly, to ask for additional funding coupled to state money, to improve educational evaluation in their state. He believes that states want this information enough to fund it.

Dr. Sanders reported that the Validation Panel report has been revised and approved by The Joint Committee. Bruce Gould has submitted the report to the ERIC clearing house system on Assessment and Evaluation and also to Evaluation Practice for publication. He has not heard back yet from Evaluation Practice.

Dr. Sanders said there are more dissertations that are using the Standards; he aware of 4 more in the past year that used the Standards to do analyses or look at some issues raised by the Standards, so the Standards continue to be a stimulus.

Dr. Sanders reported that there was a training session at last year's AEA on The Program Evaluation Standards. It was sponsored by AEA and CES together, with Dianna Newman and Gwen Keith doing the training with Jim. Also there was a session at AEA on the experience in Louisiana of using the Standards as a requirement for evaluation consultants in that state. There was also a session at APA in August of this year on the Standards that received substantial positive feedback, including questions of why they had not heard about the Standards before.

Discussion was held around past dissemination efforts, with suggestions that it may need to be expanded beyond education arenas. It was suggested that the publishers, Corwin and Sage, may need to expand their promotion efforts to a wider market, including mental health workers, policy and practitioner groups, i.e., National School Board Association, Elementary and Secondary Principals, the NEA and AFT union people, and accreditation groups. Dr. Sanders noted that The Joint Committee has member associations for each of the audiences on which they rely to take responsibility for getting the word out. A suggestion was made to advertise in periodicals and publications for the associations. There was general agreement that there is a potentially large and enthusiastic market for information in evaluation.

Dr. Gullickson and Dr. Sanders talked about how the Marine Corps adopted The Personnel Evaluation Standards.

Dr. Sanders said that The Joint Committee has signed a contract with The Evaluation Center to continue providing support to The Joint Committee, in terms of housing the archives and providing clerical support as well as an address for The Joint Committee through December 1999. There is no cost to The Joint Committee for doing this other than any costs that we incur for correspondence or using clerical help for typing and duplicating materials.

Dr. Sanders stated that an election must be held to elect a person to the Executive Committee to replace Bruce Thompson (NCME) who has been replaced on the Joint Committee. Dr. Gullickson agreed to serve as Chair of the nominations process to identify nominees for an election to be held Saturday. Dr. Sanders invited Joint Committee members to let Dr. Gullickson know if they were interested in serving on the Executive Committee. Dr. Sanders reminded The Joint Committee members that the Executive Committee is a group which operates during the year between meeting if there is business to be taken care of. The Executive Committee currently consists of Dr. Sanders as Chair, and Dr. Gullickson, as Vice-Chair of the Committee, and three elected members from The Joint Committee. The elected members of the Executive Committee include: Bill Mays and Edith Beatty, both in their first year of a three-year term.

A question was raised about sanctions for groups that consistently fail to send a representative to The Joint Committee meeting. Dr. Sanders responded by summarizing The Joint Committee Policies and Procedures on this issue. He noted that this year there are 6 organizations that are not represented, and for 3 there were last minute extenuating circumstances. Dr. Sanders agreed to write to those organizations to encourage their participation. A question was raised about the PTA, and Dr. Sanders reported that it had withdrawn from The Joint Committee due to being over committed to various organizations. It was suggested that The Joint Committee recontact the PTA, due to the focus on student evaluation. Dr. Sanders indicated that last year it was decided that funding would be pursued before recontacting them.

A question was raised about National School Board Association (NSBA) which is a member but has not sent a representative recently. It was suggested that the NSBA would be a good dissemination source.

Dr. Gullickson suggested that to save expenses a local representative of a particular organization could attend The Joint Committee meeting and submit an annual report to their board. This would remind the organizations why they need to be a part of The Joint Committee. Dr. Sanders stressed the importance of having direct contact with presidents and executive directors and boards about what The Joint Committee is doing.

Financial Report and Sales Report

Ms. Ramlow referred The Joint Committee members to a written report provided to all members. She said that last year The Joint Committee switched from Old Kent to School Employees' Credit Union for a higher interest rate, which is reflected. Revenue for sale of Step Guides and training materials is shown at $800. Ms. Ramlow explained expenses which leaves The Joint Committee with a balance of $13,724.48.

JOINT COMMITTEE

BANK ACCOUNT STATEMENT AS OF SEPTEMBER 30, 1996

Dr. Sanders said that the balance has always been within the range of $12,000 to $15,000, so this year's balance is within that range.

Ms. Ramlow presented the sales report. The Program Evaluation Standards report begins with McGraw-Hill Publishing Company and switches to Corwin Press, showing total numbers and consistency of sales. The sales through 12/31/95 are as follows:

The Personnel Evaluation Standards: 9,381

The Program Standards: 5,010

Arrangements for 1997 Annual Meeting

Dr. Sanders stated that The Joint Committee typically meets in the last week in September or the first week in October. All members present agreed to reserve October 2, 3, and 4, 1997 for the next annual meeting. Dr. Sanders stated that if funding is obtained for the student evaluation project, it will require a Thursday through Saturday commitment. Dr. Sanders opened discussion on location for the next meeting. It was suggested that if funding were obtained during the year, it might be beneficial to hold the meeting in Washington, DC with a reception to generate participation by the sponsoring associations. Discussion was held around maintaining visibility of The Joint Committee, and holding the meeting in a more accessible place, such as Washington or Chicago. Dr. Gullickson suggested including a briefing and press conference if the meeting were to be held in Washington DC. Dr. Sanders suggested that the executive committee could make a final decision on the location.

Dissemination

Dr. Sanders opened discussion on dissemination strategy for the coming year, scheduling presentations and training.

Dr. Sanders reminded the members that a newsletter release will be sent to each member after the meeting, and it should be included in the sponsoring organizations' newsletters as soon as possible.

Dr. Gullickson asked if the news release was just for sponsoring organizations, or if it could be sent and distributed through other organizations as well, e.g., CREATE. There was discussion around sending the news release to other organizations such as state departments of education and local state school board organizations.

Dr. Sanders stated that The Joint Committee has presessions this year at AEA in the beginning of November and at The Canadian Evaluation Society meeting in May. Submissions for AERA and NCME for next August were discussed. It was suggested that at the next AEA conference The Joint Committee could plan sessions or symposia addressing the need for student evaluation standards. It was suggested that for NCME there should be a clear layout of how the standards will be different from the existing test standards. There was a discussion around how student evaluation could be a territorial issue for NCME.

A question was raised about major conferences that have an emphasis on student evaluation, where there could be papers or a panel to spark the rationale about why this is so critically needed. Dr. Gullickson suggested a conference he is organizing for CREATE in Indianapolis next July. Conferences for elementary school principals and secondary school principals were also suggested as well as NSBA. It was suggested that it may be important to distinguish the student evaluation standards as a tool which helps sort through all the other layers of evaluation versus adding another layer on top of them, which practitioners often resist. There may be more credibility if a teacher or principal were to lead the session and translate the material into practical concrete information. Dr. Sanders asked for names of persons who might be capable and willing to do workshops. It was suggested that The Joint Committee draw from experience in Florida, Mississippi, Louisiana and Hawaii, and get examples from principals and teachers in those states that have used the standards. It was suggested that other packets of materials need to be developed to disseminate. It was suggested that presentations at NEA and AFT conventions could present the idea that evaluation is going to happen, and this is a way to benefit from and manage that process. It was also suggested that The Joint Committee work from the top down by gaining endorsement from NGA (National Governor's Association). The NLPES training meeting is usually in October in Mississippi next year, and The Program Evaluation Standards would be of interest for their own internal work, in addition to the other standards for teachers and administrator assessment. A panel discussion or workshop at NLPES was suggested.

Dr. Gullickson stated that step-by-step detailed procedures need to be developed to help apply the standards. There have been positive responses to The Program Evaluation Standards and The Personnel Evaluation Standards, but users are often confused about how to make the transition from theory to practice. The Alaska group is actively pursuing that process, and they have suggested about 10 steps to get the standards into practical use. Their message is to talk about the standards and also package them in ways that are more user-friendly. Dr. Gullickson mentioned the CD-ROM on teacher evaluation and the CREATE books.

The question was raised whether there has been an effort to incorporate The Personnel Evaluation Standards into state laws on teacher assessment. Dr. Sanders said that a previous Joint Committee member, Dianna Pullin, was unsuccessful in efforts to have the Massachusetts legislature adopt the standards. There was discussion of various states that have include the standards in legislation and accreditation. It was suggested that regulations are often ineffective and are not followed, and the most effective way to see that the standards are used is to go directly to the people who will use them, rather than governing bodies.

A proposal for next year's ASCD conference is being planned. It is such a large organization that it has been impossible to get evaluation on its annual meeting agenda, but they have an assessment task force, which may be an entity to build on. It was suggested that The Joint Committee should get information into ASCD's publication, Educational Leadership.

Dissemination through APA was discussed. Mark Davison stated that program time is not allowed for the same subject year after year. It would be better to follow up with boards and committees with the exception of a program in Division 14 on The Personnel Evaluation Standards. To keep the APA informed, Davison suggested that someone contact 2 committees in the APA, one on psychological testing and assessment and the other is The Joint Committee on Testing Practices. CTPA would be the main one, and they meet again in March, and the contact is Diane Brown. Davison also suggested that someone ask Bruce Thompson to make a submission to Division 14 on The Personnel Evaluation Standards.

Dr. Sanders asked for suggestions for CSSE. Bob Wilson stated that the next meeting is in St. Johns, Newfoundland, and may not be well attended, but there is a newsletter in which information about the standards will be published. Wilson stated there are 2 Canadian events that may be of interest. Teacher accreditation, training and evaluation is turned over to the College of Teachers in Toronto, and Margaret Wilson (no relation to Bob) is heading up that organization and should be contacted. Second is a meeting of people who teach in the area of student assessment, and there will be one next year. Wilson agreed to get details for a presentation there. There is also a Council of Ministers of Education of Canada, which has sponsored nationwide testing, headquartered in Toronto, and may not be aware of the standards. Dr. Wilson offered to contact that organization as well.

It was suggested that Sage be contacted to make certain that there is an ample supply of books at for sale at conventions. Dr. Sanders asked for a schedule of meetings to submit to Sage and Corwin. There was discussion about Sage and Corwin's lack of coordination and the necessity of distributing the books together.

Dr. Sanders asked Kathleen Sullivan to explain her idea for a developing a casebook for The Program Evaluation Standards, applying the standards to nonschool situations. Ms. Sullivan stated that the purpose of doing some sort of casebook would be to make the standards more accessible for a training medium to people outside of education and to evaluators in many different settings. The legislative oversight setting is the most important, but this type of publication could go way beyond legislative evaluators. The standards would be kept exactly as they are, and casebook developers would come up with case examples very similar to what are already in the red book, just having the setting of those cases outside of schools, so that people understand that these very same standards apply.

It was stated that from a training and evaluator's point of view, there is a small but obvious audience for a casebook, because the Standards are being used in classes, so if there were cases that could go along with it, it would make it easier to use the Standards. It would also make it easier to do workshops on the Standards.

It was noted that if there were a casebook that had separate cases outside of education, we can show there are other applications for training beyond education. The only education part of the standards is the illustrative cases; the standards themselves are not specific to education.

The question was raised around validation efforts for new cases. Dr. Sanders said that several alternatives are possible. One would be for The Joint Committee to decide to make it another project for The Joint Committee to develop, and follow a developmental process much like the development of the standards. That would require some funding. Another possibility would be for The Joint Committee to say that we endorse that idea, and we would encourage NLPES or any other organization who sees value in the casebook to go off and do it. The Joint Committee would be happy to review it if they want our endorsement. A third option is that another organization might do the work and the final product could come back to The Joint Committee for an in-depth review before publication. Dr. Sanders mentioned that AEA has expressed an interest in having more generalizable products.

Kathleen Sullivan agreed to talk with the NLPES executive committee, propose it to them and ask for their commitment. There would need to be involvement from other groups, however. Dr. Newman suggested that she could ask AEA to participate. Gwen Keith said that CES would support that, as their mandate is far larger than education, and she will take it back to them. It was suggested that a task force be organized to generate cases for review.

Dr. Sanders suggested that there might be volunteers if it was put out on Eval Talk saying we are looking for volunteers to generate cases, and they would be listed as contributing authors of this effort. If it was put out to the entire membership of any organization, there is likely to be 10 to 20 volunteers.

It was suggested that there may be some way to use the Internet in a case review, a way beyond e-mail. Guidelines for developing cases was discussed.

There was discussion around the representation at AEA of the health sciences or health services, and it was generally agreed that they are the largest growth areas.

It was suggested that a simple one-page explanation of the standards should be developed for principals, teachers, and other appropriate journals. This would be an attempt to reach the market of people who don't understand that this applies to them.

There was discussion about publicity of the standards: (1) the publishers who market the standards and (2) the association's newsletters and the release on the Joint Committee meeting each year. When the standards were published, there was a news release sent to all The Joint Committee member associations. It was generally agreed that there is generally a lack of awareness about the standards.

Dr. Gullickson stated that he and Dr. Horn are developing a World Wide Web site for evaluation for the National Science Foundation. The Joint Committee would be well served by a similar sort of WWW page, either on its own or in conjunction with this. The WWW has a way of identifying locations, and people who don't know anything about AERA, AEA and NEA, but want to know something about it will be able to find information. CREATE and The Evaluation Center are contacted often this way, and he urged the Joint Committee to find a way to put their information onto the WWW in such a way that people would want to contribute cases, meeting notices, other things for use. The trouble with it is that it requires maintenance, updating, and an overhead, but it is a way to make it available to the world. If information is available on the WWW about evaluation, The Joint Committee will get hits and people will start ordering the books also. Discussion was held around cost and availability of a server, and Dr. Gullickson indicated the only cost would be for maintenance of the Web page, as the university provides the server.

Dr. Sanders stated that the standards are on line with WWW sites for AEA, NCME, and the ERIC clearing house, so there are places already where the standards are available. He suggests we get as much as possible in the NSF project, and he will talk with Sage about what The Joint Committee can do there.

Dr. Gullickson suggested that if Sage would be willing include a list of professional meetings where information about the standards is provided, that is likely to stimulate some interest in the books.

It was suggested that for the student evaluation standards, the associations of standardized test publishers should be contacted, as they have an interest in promoting the proper use of tests.

It was suggested that dissemination should be a top-down process, beginning with informing senators and their aides, getting them written into federal legislation and state legislation so agency heads, schools and evaluators will attend and adhere to the standards. Dr. Sanders stated that strategies are needed.

Dr. Gullickson suggested developing "rules-of-thumb" to send to agencies and ask if they have addressed these matters in funding, saying if you want your projects evaluated, do you use the standards?

Gwen Keith stated that development of materials directly related to application of the standards is the important task, and addresses the need base rather than just an information base. Dr. Sanders stated that over the next year materials to help translate the standards can be developed for workshops and presession presentations, and then bring them back to The Joint Committee annual meeting to share.

A suggestion was made to talk to someone, like Richard Riley, who handles regulations for grant programs to educate them that we have the national standards, this is not a political issue, and this is the right way to do evaluation. Discussion was held around state and federal regulations, and how to get the standards included.

Dr. Sanders stated that when OERI was being reauthorized, The Joint Committee attempted to have a reference to The Joint Committee standards put into the legislation, and were not successful.

A "gratis" mailing of the Standards was discussed. Dr. Sanders stated that when the Personnel Standards were first published, the NEA bought 1,000 copies and distributed them. Discussion was held around further mailing to key people, such as governors and commissioners, with a cover letter explaining the importance of the standards. Dr.Yarbrough agreed to draft a short letter to go with a mailing of complimentary Standards.

It was stated that previous efforts to make people aware of the standards did not include steps for application, and without that the standards don't get used.

There was discussion around presenting workshops on evaluation for state legislators and whether there would be interest and financial support for that. Kathleen Sullivan said that there is an education group within the National Conference of State Legislators made up of legislators and staff members who are education chairpersons in various places around the country, and they have annual meetings, which are sometimes held in conjunction with the big annual national conference of state legislatures. NCSL, National Conference of State Legislatures, is the parent group of NLPES and there is a sibling group that deals with education issues. Her suggestion is to contact the chairpersons and staff rather than the legislators. The staff people should not be overlooked, as they often stay longer than the chairperson or legislators.

Dr. Sanders asked that The Joint Committee members in the coming year to make direct contact with the president, the board, and the executive director of their organizations to keep them informed about what The Joint Committee is struggling with, as well as what we are doing. It was suggested that a letter or phone call, separate from the news release, be done.

Review for revision of The Personnel Evaluation Standards

The Personnel Evaluation Standards books and manuscripts were distributed to 3 groups:

Propriety Standards: Dianna Newman, Bill Mays, Bob Wilson, Jim Sanders

Utility and Feasibility Standards: Edith Beatty, Don Yarbrough, Gwen Keith

Accuracy Standards: Bill Webster, Kathleen Sullivan, Mark Davison, Arlen Gullickson

Dr. Sanders stated that there has been a recommendation to add a new standard called "Due Process." The concept was discussed, and Dr. Sanders asked all members to consider the recommendation so that we can come back to it later. There was further discussion about due process and legal issues which vary from state to state making it impossible to write a single standard. It was also suggested that in the case review, it should be noted how many cases refer to due process. It was suggested that contact be made with NOLPE, the National Organization on Legal Problems in Education.

There was discussion on how extensive the changes should be on The Personnel Evaluation Standards. It was noted that a new copyright date is necessary to show that the text is current for future dissemination efforts.

Dr. Sanders also stated that The Personnel Evaluation Standards, if submitted now to ANSI for approval, may be rejected, as the references are all prior to 1988. The review process will bring it up to date for submission to ANSI.

Dr. Gullickson raised the issue of a standard that addresses rating inflation in evaluation, i.e., the natural process of beginning with a mid-point rating and after a few years, everyone is at the top of the scale. Dr. Sanders suggested one way to handle both "due process" and "rating inflation is to make headings in the functional table of contents, and identify those standards that apply to them. Also a guideline might be included on inflation and due process in existing standards.

We will need to request suggestions for changes from users of the standards. Dr. Sanders suggested that he would like to have a first draft ready by the end of February to send to reviewers as a typed manuscript. Between now and end of February, he asks members talk to the presidents and boards of their organizations about this revision, and ask for at least 5 reviewers or a task force to be identified by the end February. Then they will have the month of March to review proposed changes. Then the board of the organization would have April and May to meet, discuss, and get additional comments back to the member. Then in June, members would send their copy to Dr. Sanders, who will incorporate them with suggestions received from the field into a second draft and have it ready for the meeting next October. It will also be sent out ahead of time for members to review the changes. Then next year, the majority of the meeting would be reviewing the changes, assuming there are no other projects. If revisions turn out to be substantial, we may have to rethink the objective of publishing an updated edition in 1998.

The Joint Committee then broke into small groups.

The large group then reconvened, and the small groups reported on their review and recommendations for updating The Personnel Evaluation Standards.

Saturday, October 5

Dr. Gullickson presented one nomination for the Executive Committee position, Gwen Keith, which was approved unanimously.

Dr. Sanders stated that the 1997 meeting will be in Washington, D.C. with a reception for the associations if funding for a new project is received. Dr. Sanders asked that the executive committee, with Dr. Gullickson as chair, make proposals to him by January 1 about agenda and format, and invitations for that reception.

Dr. Sanders distributed the newsletter release for review.

Dr. Sanders described a list of procedures and deadlines for the review of The Personnel Evaluation Standards. He initiated a discussion on philosophy behind the standards to generate a clear understanding of the changes to be made. Some of the issues were top down vs. collaborative approach to personnel evaluation. Another issue was between having discrete personnel evaluations, something that happens every so many years vs. continuous personnel evaluation. Third is having a single evaluator being the way that evaluations get done vs. having multiple evaluators. Also the issue of having only one way of doing evaluation vs. the process being dependent on the organizational context, i.e., the kind of personnel and the difficulty of the case. The Joint Committee wishes to avoid saying there is only one way to do evaluation.

There was discussion around philosophy, concern about The Personnel Evaluation Standards following popular trends yet being able to adapt to changes in personnel practices of shared decision making, quality focus, and empowerment of individuals. A suggestion was made to discuss changes in personnel practices, so that the standards are applicable to both hierarchal and shared-decision-making environments and should accommodate both quantitative and qualitative approaches. It was agreed that an inclusive approach is needed.

It was stated that the Utility section does not include a collaborative approach nor any indication that the standards can be used for self-analysis, growth, or reflection of their own practice. It was suggested that this approach may need to be added to all the sections. It may not be enough to refer to plurality in the front part of the book, and ignore it in the rest of the sections. This indicates that the book needs to be edited from start to finish by someone who has a collaborative focus, so that will be represented.

It was stated that The Personnel Evaluation Standards is a summative work, and a question was raised as to whether The Joint Committee wishes to expand that to include formative evaluation. It was noted that schools expend most of their resources for evaluation in the summative area. However, very few people are fired, and the way schools or other systems can be improved is through increased formative evaluation and improvement of the people already in the system.

It was also suggested that if the standards apply only to hiring and firing, then a summative focus is appropriate. But if The Joint Committee wishes to address improvement, formative processes must be included.

There was discussion about the distinction between collaborative and self-assessment practices.

Dr. Sanders summarized that there is concern about bias in the book, but not in the standards themselves. The bias is toward a hierarchal, authority based, summative approach. The guidelines, the content under the standards, and the cases, need to be revised to reflect current practice.

There was agreement that formative evaluation needs to be included in the revision process. Dr. Gullickson urged that distinctions be made between the formative and summative processes in the book, as there are significant differences, and they can be at cross purposes. There was discussion about the need for an improvement plan (formative evaluation) with any summative evaluation. Both are required for due process.

Dr. Gullickson pointed out that all members agreed that The Personnel Evaluation Standards as currently written do not address the formative piece of evaluation, which would be a major revision and would require several years and additional funding.

Dr. Sanders suggested that the standards may not need that much revision, but only some editing of the content of certain guidelines and cases. We should give it a try and see how much editing is really needed.

There was disagreement about some of the standards, e.g., P-5, that they are written from a hierarchal power-based view rather than a partnership or empowerment view.

Dr. Sanders stated that the needed change may not be changing what is there, but instead adding support for an alternative view. He asked the members to attempt an editing process and decide, after some work has been done, whether a major revision is needed. Dr. Sanders agreed to circulate an attempt at addressing concerns by the end of January.

Small group reports were then discussed extensively.

Following the small group reports, Dr. Sanders suggested the following for the updating process:

1) Aim for January for a revised draft of the standards for personnel evaluations. Dr. Sanders will do as much as possible to incorporate the small group suggestions and new literature. He would like to have the Joint Committee look at it before we go any further in terms of getting outside reviewers. Then by the 28th of January we will be ready for outside reviewers.

2) Dr. Sanders asked that each representative ask the president of their association to appoint a review panel of at least five people to review the draft and ask the president to inform the board of this review and request the boards advice on review, thereby engaging board members. The review panel should be identified by January 1st and the list sent to the Joint Committee representative from the president and then sent to Mary Ramlow. If we decide in January that we do not want to proceed any further, then we will have to contact the reviewers, president, and the boards and say that we are not ready. We will put this on hold until we are ready. Otherwise, we should be ready to send it out for review by February 28th. We will also make a general call for suggestions from members of the associations and other users with a March 31 deadline.

3) The reviewers will have the month of March to do their reviews.

4) The reviewers then send their critiques to the Joint Committee to summarize and make a report to your board by May 30th. If reviewers get their suggestions to you by April 1st, then that would give you 2 months to summarize and make a report to your board.

5) Your board needs to discuss this and send you any further discussion. You then send the review discussion to Dr. Sanders by June 15^th. He will use that feedback then to revise the draft and send it to you by September 1st. You will have a month to look at it before our 1997 annual meeting in Washington. In Washington we will do a final review and approval of revisions at that meeting. After that we will send a final manuscript to Corwin Press and ANSI.

After some discussion the procedure was accepted. Dr. Sanders then asked for further business. None appearing, he thanked the Joint Committee and staff for their hard work and the meeting was adjourned.

96JCAnnualMeeting\MINUTES-REV.96.wpd\wp