Using the
Student Evaluation Standards:
A Field Guide
for K-12 Math & Science Teachers
Edith E. Beatty
Maura O’Brien Carlson
The
Arlen Gullickson, Director
To Start: Some
Questions and Answers
What are the Student Evaluation Standards?
They are the product of a
collaborative effort to provide teachers, others who evaluate student learning,
and those who use evaluation information with clear, useful, research-based
principles and guidelines for developing, delivering, and improving evaluations
of student learning.
The Student Evaluation Standards
were adopted in 2003 by the Joint Committee on Standards for Educational
Evaluation (JCSEE) and published as The
Student Evaluation Standards (Corwin, 2003). Created in 1975 and now
chaired by Arlen Gullickson, the Joint Committee includes representatives of
There are 28 standards. For a summary
of them, please see page 36. For a
simplified set of “Prompts to the Standards,” please see page 6.
Why do we need another set of standards?
Teachers already have content
standards, state frameworks of standards, and other sets to which they are
urged or required to align their teaching and classroom practice. All these
standards have grown from the nationwide consensus that we must achieve a
higher level of learning in our K-12 schools. Content standards reflect
broad-based efforts, based on research and professional experience, to say what
we want our students to know and be able to do.
In order to know how well we are
achieving our goal of higher levels of learning for all students in our
schools, it is vital that we employ student evaluation and assessment practices
and procedures that are also based on
research and the best professional experience. The Student Evaluation Standards
are a broad-based effort to guide all assessment practices in our classrooms
and schools.
We use the word “evaluation” more
than “assessment” to reflect the consensus that the most useful, meaningful
evaluations of student learning involve much more than some kind of test at the
end of a unit or a year. To evaluate a student’s learning well is to find a
rich array of ways to collect and assess outcomes throughout the teaching and
learning process.
Who should use these standards? In what settings?
The Student Evaluation
Standards—and this Field Guide to
using them—are designed for teachers, instructional leaders, professional
developers, school leaders, and education policymakers. They are useful in
classrooms, schools, and districts; in professional development settings; in
task force and committee work; and in local, state, and national policy
development.
How should I use these standards? Do I need to be familiar
with all of them?
We suggest that you start by
reading this Field Guide. It is
concise, simple, and written to help.
As with all standards, it is often
best to start small with a specific project or challenge that speaks to your
professional needs and priorities. Beginning on page 11, this Field Guide presents several case
studies of how educators might use the Student Evaluation Standards to help and
guide them in meeting realistic challenges in informing, measuring, and
reporting on student learning. These brief stories provide and invite insight
into how specific standards can play that useful role.
Before or after reviewing the case
studies, please read the Introduction and Overview to this Field Guide, which begins on the next page. You may also want to
review the simplified “Prompts to the Standards” on page 5. These are concise,
question-based “windows” into the standards themselves.
Finally, please also see the
Student Evaluation Standards themselves, which begin on page 36.
Introduction and Overview
This Field Guide has been designed to accompany The Student Evaluation Standards
(SES) and the SES
Facilitator’s Guide. We have sought to create
user-friendly materials that are applicable to work-embedded, classroom-based
professional development for math and science educators who are committed to
improving their student evaluation capacity.
This Field Guide has these specific purposes:
• To create an inviting “window” into the Student
Evaluation Standards
• To make using the SES book easier, by creating tools for
navigating the book
• To enhance the direct relevance of the standards to
teachers
Audience
This guide has three primary
audiences: teachers, instructional leaders, and professional developers. Policymakers—in
particular, school board members—will also benefit from this work.
To clarify, instructional leaders (such
as principals, district administrators, special education or curriculum
directors) supervise and evaluate teachers and may plan or direct instruction,
assessment, and evaluation at the building or district level. Professional
developers lead or provide learning opportunities for educators to enhance
their instructional skills. School board members make policies directed toward
instruction, curriculum, evaluation, and professional development, which
teachers have primary responsibility to implement.
Context and Need for this Work
Good teachers love to teach. They
want their students to learn, achieve at high levels, meet high standards,
succeed at what they want to know, create what matters most to them, and enjoy
learning in the process.
Pressures and demands on teachers’
time and priorities have increased markedly over the past several years. This
has culminated with the passage of federal and state legislation, such as the
No Child Left Behind Act, with its heightened demands for accountability of
public schools—including assessments and high-stakes testing at almost every
grade level.
In 2003, the Joint Committee on
Standards for Educational Evaluation (Gullickson, chair, et al.) published The Student Evaluation Standards: How to
Improve Evaluations of Students. A year later, an
The Facilitator’s Guide introduces the Student Evaluation Standards and
paves the way for professional learning about using them—but it only begins to
provide what teachers, teacher leaders, instructional leaders, administrators,
and professional developers need to learn about the depth, breadth, and
richness of this work. Professionals also need to know how these evaluation
standards align with the other sets of standards they need to know and use.
These include standards for content, content pedagogy, and professional
practice; frameworks of state or local standards; grade expectations and
benchmarks; scope and sequence matrices; and the list goes on.
Most teachers have a limited
number of days for professional development during each school year. Often
these days are used for meetings or work sessions that may not actually qualify
as professional development or for work targeted to individual teachers’
instructional needs for professional learning.
When we combine the scarce time
that teachers have with the tremendous pressures on them to become masters at
instruction and assessment, plus the content they are required to teach, plus
these new Student Evaluation Standards and related resources, it is
clear that much professional development needs to occur. The best—and perhaps
only—way to get this done is to design professional development that is
embedded, as much as possible, into the work that teachers are already doing.
That is the approach we have tried
to take with this Field Guide to
using The Student Evaluation
Standards—to make it as useful as possible to teachers and others in the
context of their jobs. Rather than adding a new layer of work, we hope this
brief publication will help teachers and others see with ease how they can use
the Student Evaluation Standards to help students learn and to measure that
learning more effectively.
Of course, taking concentrated
time to explore and practice the Student Evaluation Standards, using this
guide, during off-site workshops is the optimal approach. But the
focus for this guide is the real-time context of classroom planning for
evaluations that both serve and assess student learning.
Students will be better served if
use of this field guide promotes increased implementation of The Student Evaluation Standards.
Given the high need for math and science work in instruction, assessment and
testing, and accountability in general, this guide focuses the evaluation
standards on these content areas.
Funders, such as the National
Science Foundation, U.S. Department of Education, and others may want to
require or prefer that grants and contracts awarded in the areas of student
assessment and evaluation in these content areas be contingent upon proper use
of The Student Evaluation Standards,
as has occurred in the past with the Joint Committee’s The Program
Evaluation Standards.
Content of This Guide
This publication is intended to
accompany The Student Evaluation
Standards, not to be used instead of that text. We have not rewritten the
book; the complete set of Student Evaluation Standards still stands in its original
form. Instead, we have shaped the guide in two ways for increased relevance and
user-friendliness.
The first is direct application to
the particular content areas and alignment with other standards that
educators need to employ. The second is by briefly citing cases: examples and
stories that match both content area and public K-12 settings. (The Student Evaluation Standards
are designed to have a much broader audience, content, and application.)
I. Prompts to the Standards
This section presents simplified,
question-based “prompts,” or windows into the Student Evaluation Standards.
These are not intended to replace the actual standards, only to assist the
process of reviewing and using them.
To review the actual standards,
please turn to page 36.
Propriety
Is the evaluation legal, ethical, and in your students’
best interests?
P1 Service to Students: Does
the evaluation promote effective work by students and serve their educational
needs?
P2 Appropriate Policies and Procedures: Is the evaluation fair? Is it consistent? Are there written
policies and procedures that are readily available?
P3 Access to Evaluation Information: Are you providing information about their evaluations to
your students and to others with permission to see it, while also protecting
confidentiality and privacy?
P4 Treatment of Students: Are your students treated with respect at all stages of the
process?
P5 Rights of Students: Does
the process protect your students’ legal and human rights?
P6 Balanced Evaluation: Does
the evaluation identify strengths to build on and weaknesses to address?
P7 Conflict of Interest: If any conflicts of interest arise, are they dealt with openly and
honestly?
Utility
Is the evaluation informative, timely, and influential? In
other words, is it useful?
U1 Constructive Orientation: Does each evaluation lead to good decisions in that
student’s best interests?
U2 Defined Users and Uses: Have you defined who can and will use the evaluation
results?
U3 Information Scope: Is
the information that you collect focused and comprehensive enough to create an
evaluation that fully addresses student needs?
U4 Evaluator Qualifications: Do the evaluators have the knowledge and skills they need
to produce useful results?
U5 Explicit Values: Does
the process make very clear the values that are used to judge student performance?
U6 Effective Reporting: Are
the evaluation reports clear, timely, accurate, and relevant?
U7 Follow-Up: Are the
evaluation reports easy to understand—and do they clearly say how students,
parents, and others can follow up?
Feasibility
Is the evaluation practical, diplomatic, and
well-supported? Does the process work as planned?
F1 Practical Orientation: Do the procedures produce the desired information
efficiently, without needless disruption?
F2 Political Viability: Are
you building cooperation by creating room in the procedure to receive and
answer questions from students, parents, and others?
F3 Evaluation Support: Are
you providing enough time and resources to fully plan, carry out, and follow up
on the evaluation?
Accuracy
Is the evaluation generating sound results?
A1 Validity Orientation: Does the process lead to clear and valid interpretations of your
students’ performance?
A2 Defined Expectations for Students: Have you clearly defined the student performance you
expect?
A3 Context Analysis: When
a student’s situation, or another variable, affects the evaluation, is that
considered in interpreting the results?
A4 Documented Procedures: Have you clearly documented the evaluation process?
A5 Defensible Information: Have you gathered enough information to make and defend
your decisions about student performance?
A6 Reliable Information: Have you gathered solid, reliable, consistent information?
A7 Bias Identification and Management: Is the evaluation free of bias?
A8 Handling Information and Quality Control: Are you systematically reviewing—and correcting where
necessary—how you collect, process, store, and report evaluation information?
A9 Analysis of Information: Are you systematically reviewing the evaluation data or information
itself to make sure your results are correct and that they tell you what you
want or need to know?
A10 Justified Conclusions: Are you able to explain your conclusions about student
performance well enough that your students and their parents believe them?
A11 Metaevaluation: Are
you periodically reexamining the whole evaluation process, to improve it over
time and make sure it meets these and other standards?
II. Tasks that Relate to Evaluation and the Standards
A Functional Table of Contents
is included in each of the three Standards volumes from the Joint
Committee, assisting users to know which standards might be useful for
particular tasks. Here we have taken the
larger sets of functions or activities for student evaluation and provided
greater detail for teachers of mathematics and science and other school leaders.
Teachers think about evaluation
when
Planning evaluations
Thinking
about purposes and uses
Planning for learning
Planning
for instruction targeted to learning goals
Designing assessments to drive instruction and to assess
learning
Selecting and
evaluating appropriate assessments from published or other designed
assessments, tests, or measures
Conducting evaluations
Assessing
students, formative or summative
Tests,
quizzes, observations, homework, etc.
Facilitating
and observing students on a daily basis
Collecting
data and information carefully
Analyzing data
Categorizing,
analyzing, or interpreting data and information correctly
Aggregating and disaggregating data
Creating rubrics and other frameworks for arraying data
Quantitative
Qualitative
Grading students
Scoring,
grading, ranking work
Using
multiple forms of assessment
Evaluating
students
Communicating and reporting student progress
Sharing
information with students on a daily or ongoing basis
Communicating
with parents/guardians at appropriate times
Reporting
data school- and districtwide
Making decisions
Using
multiple forms of assessment and test data
Deciding
how to use data
Deciding
grades
Deciding
eligibility for specific programs, placement
Managing and using data
Collecting,
organizing, and sorting data
Using
data that are collected
Throughout these processes,
teachers strive to
Ensure fairness and equity
Considering
diversity, equity, and fairness to all students
Ensure rigor and relevance
Considering
technical issues and methodology
II. For
Instructional Leaders
Principals, curriculum and
assessment coordinators, and superintendents think about evaluation when
Designing assessment plans
Supervising assessment implementation
Managing and using evaluation data
Overseeing methodology
Supporting professional competence
III. For
Professional Developers
Professional developers and
higher education faculty think about evaluation when
Preparing and developing professionals
Teaching
and evaluating
Planning
workshops
IV. For Policymakers
Local school board members and department
of education personnel think about evaluation when
Using evaluation data
Making decisions about instruction, curriculum, assessment,
and professional development
Developing policy
III. Case
Studies
Using the
Standards to Meet Assessment and Evaluation Challenges
The brief case studies that follow
describe realistic evaluation issues, opportunities, and challenges that math
and science teachers face, along with their school leaders and others.
These case studies show how the
Student Evaluation Standards can be useful in addressing each challenge and
developing the most effective, efficient solution. The standards can guide and
support evaluation efforts while at the same time helping to shape quality
instruction and good decision making about students.
At the end of each case study is a
list of the Student Evaluation Standards that apply, with a note on how and
why. For more on each standard, please refer to the question-based “Prompts to
the Standards” and to the Student Evaluation Standards themselves.
These case studies can be used as teachers plan or in professional
development exercises. In such situations, we suggest that participants be
provided with the case and the complete set of prompts and then asked to
develop their own standards-based analysis, as is prepared in the Facilitator’s
Guide.
Student Assessment and Evaluation Cases
Teachers and administrators in our “Case Example District”
encounter real questions, issues, and dilemmas in assessment instances and with
student evaluation scenarios or decision making. An assessment team is working
to design its Local Comprehensive Assessment Plan, and a new Director of
Assessment arrives with some familiarity of the Student Evaluation Standards! We present these case examples, analyze them
in light of all relevant standards, and resolve the situations with some
logical next steps and work to do. We
also offer resolution to some of the tension between the need for professional
development and the desire for the Standards to stand alone.
Through these windows of work, we visit with elementary,
middle school, and secondary school professionals. We learn that several of the issues that
arise present themselves at the middle level because of transitions between
schools and culminations of grade clusters used in most of the content
standards. We focus on math and science;
technology is embedded. Our cases look
at using assessments to inform instruction, to make decisions about
instructional units, to place students in specific learning groups, to align a
districtwide curriculum that supports student learning, and to look across the
district to see student knowledge strengths and weaknesses in order to address
teacher professional development needs.
We also explore representing and communicating assessment and evaluation
results.
In this Guide, we are committed to fostering increased use—including
individual use—of The Student
Evaluation Standards text and offer these common “real life” assessment and
evaluation cases. While we appreciate
that the best learning often comes through guided and embedded professional
development, we recognize that such professional learning opportunities are not
always available. Thus, we have woven
together these cases into one “Case District” to serve as an example of using The Student Evaluation Standards
within one context.
Sixth Grade Math Assessment Case
Two middle-level teachers wanted to design a better process for making placement decisions in mathematics for rising seventh graders, the many sixth grade students arriving from elementary schools throughout the district. Historically, sixth grade teachers recommended students for particular sections. Grades, test scores, and other data were certainly considered. While the various sections for mathematics purported to be heterogeneous, some tracking of students actually began at this level. Parents also would often request a certain teacher, which was the known route to advanced placement as the students progressed from middle to high school. These teachers were seeking a more sound method of assessing what students know and are able to do at the beginning of their secondary experience so that the process would better serve them.
The teachers worked with their colleagues at both middle schools to develop a Districtwide Sixth Grade Assessment, drawing upon the K-12 curriculum, which was grounded in NCTM standards, their state’s standards, and benchmarks of progress. They were learning about and applying the concept of “power standards” or those standards that represent the essential understandings at each grade cluster or level (Wiggins & McTighe, 2004.) They constructed a 35-item test of computation and problem solving in all of the relevant areas: number and operations, algebra, geometry, measurement, data analysis and probability, problem solving, reasoning and proof, communication, connections, and representation. The assessment was designed to be administered within 40 minutes. The teachers worked with one of their principals who supported the project and assumed the responsibility of working through the appropriate channels.
As the time approached to implement the assessment, the sixth grade teachers hit the roof! Totally caught by surprise, they were not supportive of the assessment and questioned its legitimacy. Elementary school principals decided to move forward due to the timing. Teachers ultimately agreed to administer the examination, but with the commitment to evaluate and rethink the procedures for the following year.
These teachers were doing precisely the right thing, but were doing it the wrong way. All of the knowledge in the world about standards does not replace knowledge of human nature and effective communication. Looking at The Student Evaluation Standards, particularly the Propriety Standards, was of great assistance to all in deconstructing this event. Because that particular domain of standards addresses student well-being, it was a great place to start.
The intent of the sixth grade assessment was to serve students (P1 Service to Students); the goal was to make better placement decisions. Teachers knew that, for better or worse, decisions made at this level correlated closely with student opportunities later in high school. That being the case, they wanted to use multiple, fair assessments and ones that actually tested the intended content.
What they did not do, although their administrator did not help them here, was to see this as something much broader than their grade cluster. Since this was a districtwide venture, there needed to be the appropriate policies and procedures (P2 Appropriate Policies and Procedures) to ensure that this was sanctioned fully, designed well, and administered fairly. There were policies regarding district testing, but no such procedure for a common sixth grade math test was in place. The process to approve and support such a practice would involve more than a few teachers and, in fact, more than one principal. The district’s new assessment team was responsible for researching and recommending such work but had not been included in the decision.
The sixth grade teachers were left out of the process of
deciding whether to implement this new practice and were not invited to
participate in the design. Thus, they
became defensive at the notion of this evaluation of “their students” and of
what they feared might become a personnel evaluation of them or their
teaching. They were concerned about who
would have access to the test data (P3 Access to Evaluation Information) and
how it might be used (U2 Defined Users and Uses).
The well-meaning teachers were naïve about (F2) Political Viability in terms of their locus of control, their colleagues’ wishes, and the timing of the introduction. (A11) Metaevaluation of the math assessment and of the procedures surrounding its introduction and use was more than welcomed.
The teachers, who were surprised initially at the strong reaction they received, learned a great deal from this deconstruction of events and agreed to move forward in the following manner. They agreed to work with members of the assessment team, who supported the concept and effort of these teachers. After careful review and consideration, they would move the procedure through the proper channels, including but not limited to the administrative council, district school board, and others who would be involved or affected, e.g., sixth grade teachers and principals and parents. The entire procedure would be evaluated. In addition, this prompted a review of available placement options; the existence of homogeneous, heterogeneous, and flexible grouping practices; and their influence on student outcomes in mathematics (U5 Explicit Values). While many more, if not all of the Standards, apply, teachers found it helpful to use only a sampling of those most pertinent.
Two third and fourth grade teachers were growing concerned that their colleagues were adhering unevenly to the district’s science curriculum, which had been developed several years ago. Since then, a number of things had occurred to justify their concern. New federal legislation was passed, which dramatically changed the district’s testing policies and procedures. While more testing was being done, the emphasis had moved to mathematics, reading, and writing. Revised state standards and benchmarks were emerging. There was a new assessment director, who was not the person who had been on watch during the development of the district curriculum. The current curriculum included a number of assessments insufficient to evaluate how well students were learning what was intended for them to know and be able to do. One primary concern was that teachers were still teaching units they had developed, in which they were invested and with which they were comfortable. These units aligned unevenly with the benchmarks, scope, and sequence. Thus, a student could reach the sixth grade and have studied butterflies three or four times, but had not been exposed to one unit about the human interaction with the environment or ecology in terms of Life Science.
With the support of their principals and district level administrators, the teachers invited a representative group to work with a science curriculum and assessment consultant to evaluate, revise, and update their science curriculum and assessments, to map the current work, and to align their intended work with updated benchmarks and standards. They also emphasized the development of assessments as a major component of the new work. In doing so, for starters they identified these:
· Apparent gaps in the instructional program
· Superficial coverage of content
· Continuity across grade level and clusters
· Common assessments in science districtwide
· Appropriate use of available resources
For each appropriate unit, they identified
· Science learning goals
· Grade expectation or benchmark cluster concepts (or what students will understand) in each area
· Expectations at the end of each unit (what students will do to demonstrate their understandings)
· Assessment task or tool (used to gather evidence of student understanding)
· Instructional strategies (leading to student understanding)
The teachers found that there was a lot of work to do, but they felt the satisfaction of knowing that they were well on their way to moving in the right direction. Ultimately, they would work to evaluate, revise, and update their science curriculum; the policies and procedures to support and enforce it; and the assessments to serve as the “proof of the pudding” as to whether students were learning appropriate science concepts and essential understandings. The district’s new student information system would provide templates and efficiency to managing this vast work.
The Utility Standards were the domain to focus this work. There were no glaring disagreements, but some teachers would be reluctant to change their science units. Ensuring that colleagues were on board to be serious about moving forward with a focus on science, in the best interests of students, was critical to this small group.
The teacher leaders knew that this work held (U1) Constructive Orientation (focus) in terms of identifying the purposes of the evaluation. The major goal was to align the curriculum with national and state standards and benchmarks and to ensure that all students were provided education in the appropriate domains and content standards. (U3) Information Scope was important in that assessments aligned with the curriculum goals and benchmarks. (U5) Explicit Values was considered useful because math and language arts had overshadowed science during this period, and these teachers wanted to ensure that the district leadership and board members valued science.
The district showed evidence of (F3) Evaluation Support by inviting the science assessment consultant to work with teachers. (A2) Defined Expectations for Students was important here, as well. In valuing science as a critical domain of the curriculum and in being clear about what students would know, understand, and be able to do, teachers would assess whether students were meeting the expectations.
After this work began and met with considerable support, there was a recommendation that work focus on coordinating grades 5-8, so that the 5-6 and 7-8 teachers would coordinate around students’ transitions. The 5-6 teachers would work with the K-4 teachers, while the 7-8 faculties would coordinate with 9-12 science teachers.
These elementary teachers continued their work, but made way for the 5–8 teachers to take the lead in terms of transitions from elementary to middle school.
A team of three grade seven
teachers was beginning a unit on electrical energy, focusing on its transfer
into heat, light, and sound. Before
moving into this new material, they wanted to do a preassessment to determine whether
or not their students understood some key concepts in electricity. This would inform their planning and
instruction. They were also learning to
use flexible grouping with students, a strategy to differentiate instruction,
where students are placed in different groups at different times according to
readiness, interest, or learning style (Tomlinson, 2000). They recognized that students would have
different points of entry into the work.
Teachers think about the many
purposes and uses of assessment when they design one. By designing a preassessment, they may hope
to gauge their students’ understanding of basic concepts before introducing
more sophisticated concepts within a topic.
In doing so, their instruction will likely result in increased
engagement, challenge, and learning on the part of students.
One key concept is that of a
circuit, the circular flow of electricity in a system. The teachers wanted to know if the students
could work with simple circuits.
According to the district’s spiraling curriculum, students are
introduced to electricity as fourth graders by using batteries, bulbs, and
wires to build circuits and by using experimental data to classify different
materials as conductors and insulators.
Because the curriculum was relatively new to the district, however,
there was a good chance that many of the seventh graders had never learned this
basic concept.
The teachers decided to give
students D-cell batteries, simple bulbs, pieces of wire, and ask them to find
out how many different arrangements they could come up with to make the bulb
light. Students would be asked to make
drawings of each trial, to keep track of which arrangements worked and which
didn’t, and then to explain why by showing the path of electricity in the
circuit.
The teachers found that many
students could make the bulb light, and many could draw the situations where
the bulb did or did not light, but few could successfully explain the
electricity’s path. Teachers and
students reviewed the assessment data together, using and revising classroom
rubrics so that students could participate, not only in their learning, but
also in their understanding of what was expected of them throughout the
process. They were also able to check in
on whether the “explanation” component tested science or language or both. This conversation became an integral component
of this and other units.
The teachers used the assessment
data to tailor and differentiate their instruction. With one group of students, they would create
a learning opportunity by teaching simple circuits, parallel circuits,
conductors, and insulators. Another
group of students would work to review some of these concepts before moving
into the new material, and the third group was ready to begin the new
unit. Ultimately, all students would
learn the new material, but they would begin at their readiness levels.
These teachers knew and understood
sound professional practice and were drawing upon an array of standards
principles. As a team, they were quite
familiar with the National Science Education Standards as well as their
district curriculum that was grounded in these national standards and state
standards, which also reflected the national work. Recently developed statewide benchmarks
helped clarify common understandings students should have at each grade level
or grade cluster. In their work, these
teachers were using Content Standards for Grades 5-8. Specifically, they were referencing the
following: Science as Inquiry,
Abilities Necessary to do Scientific Inquiry and Physical Science,
Content Standard B: Transfer of Energy. According to the state benchmarks, and the
local curriculum that aligned to the state standards and expectations by grade,
upon completing grades 7-8, students were expected to demonstrate their understanding
of electrical energy in two ways:
·
by building an
electric circuit and explaining the transfer of electrical energy into heat,
light, and sound, leaving the system but not destroyed
·
by describing the
effect of a change in voltage in the circuit system
These teachers were also using and
mindful of these Student Evaluation Standards, particularly those in the utility
domain. Assessment was used to inform
instruction, therefore resulting in good decisions, in service of students (U1 Constructive Orientation). The teachers were clear about the purposes and
uses of this assessment, as they were about who would use the data and how (U2 Defined Users and Uses). They would plan their instruction based on
having assessed what their students already knew (A2 Defined Expectations for Students). The teachers were careful to limit the scope
of the assessment to the principles of a complete circuit, although communication
skill was also needed (A6 Reliable
Information). The pretest and its
delivery were designed to provide reliable information for the team of teachers
to use in deciding what the students’ performance told them. Perhaps most important, (U7) Follow-Up
ensured that this work would be ongoing throughout the unit and beyond.
A team of math teacher
leaders from the elementary schools became increasingly concerned that
instructional and assessment practices within the district needed to be adapted
continuously to meet the ongoing and future needs of students. With national and global trends in education
increasingly focused on inquiry, problem solving, and communication skills,
these teachers convened to discuss how to strengthen the work. They were committed to supporting these
important concepts through portfolio assessment in grades K-12, particularly with
K-6, which was their responsibility.
Clearly, if the leaders did not create structures for student portfolios
to be properly assessed, then teachers were unlikely to take them seriously as
an instructional tool. The math teachers
brought their concerns to the assessment team.
Prior to doing so, they
engaged in some research and learned that using portfolios has many
merits. Portfolios
·
reveal a range of skills
and understanding that cannot be assessed on the state standardized tests
·
reflect change and growth
over a period of time, monitoring ongoing development of thinking and
decision-making processes
·
encourage divergent
thinking, allowing for more than one right answer
·
allow for group work,
editing, revising, and self-reflection
·
are linked directly to
instruction
·
carry the essence of
universal design, in that they are accessible to a wide range of students and
can be adapted easily for students with special needs
Over time, educators in
this district designed their portfolio system to be used both as a teaching
tool and as an assessment. In math,
portfolio work has been embedded into the curriculum and is to be used, by
design, to enhance the teaching of content and to assess students’
understanding of concepts as well as problem-solving skills. Portfolio tasks are linked directly to the
unit and occur throughout the year—according to the curriculum
expectations. Options were increasingly
available to use electronic portfolios, which provided easier management and
connections. Students loved them! These teachers doubted that much of this was
standard practice, however, and thought that implementation was uneven from
school to school, or classroom to classroom, depending on expectations of the
school leadership.
The teacher leader team
recommended to the assessment team that schools work toward the following
expectations in mathematics:
· Students in grades K-6 compile a math portfolio containing five problem-solving pieces reflecting each of five math content strands:
·
Number and Operations
·
Geometry
·
Measurement
·
Algebra
·
Data Analysis and
Probability
·
Teachers score each
portfolio based on the Mathematics Scoring Criteria or other agreed-upon rubric
and submit the scores to their building principals.
·
Principals assist teachers
by assigning scoring mentors, partners, or designated scorers based on
individual need and skills, using calibration scores.
The teacher leader team
also recommended that the district pilot a new local assessment endeavor that
would combine elements of both portfolio assessment and on-demand testing to
provide a consistent and efficient measure of student performance across the
district. Rather than collecting samples
of students’ portfolios at grades 4 and 5, as had been done in the past, they
recommended implementing a “performance event,” which would require students to
complete one common task for math within a given window of time.
To assess the math
problem-solving standard locally, all students in grades 1 through 6 will
complete one common math portfolio task in the third week of May. During the second week of May, the assessment
director will inform teachers what type of problem they will be receiving. This gives teachers the opportunity to review
the appropriate concepts and problem-solving procedures with their
students. One week later, all classes
will receive the same grade-specific problem, have one week to complete it, and
submit it for scoring. Teachers whose
calibration scores are 75 percent or better will score all portfolios.
The scores from the
performance event may be used in a variety of ways.
·
They will provide feedback
to teachers about how their students are performing.
·
The district will be
providing or supporting professional development opportunities using the
feedback from this event.
·
Schools may use the results
to inform their local action plans and for making program and instructional
decisions.
The scores from the
individual students’ portfolios may be used in these ways:
·
to provide feedback to the
students and their parents about their progress
·
to provide feedback to
teachers to inform curricular and instructional practices
·
to target specific areas
needed for professional development
·
to inform local action
plans and other school improvement initiatives
The assessment team brought the proposal to the administrative council and met with mixed reactions. They found The Student Evaluation Standards extremely helpful in debriefing their experience and in moving forward. By using that resource they were able to identify more accurately the biggest barrier to the initiative’s success and to design a route to resolution.
While significant concerns lingered within the accuracy domain about portfolio assessment, that was not the major concern. Research would inform some of those concerns; and since portfolio was not the only method of assessing progress and understandings in mathematics, validity and reliability would be addressed by other methods. The greatest concerns were actually about feasibility. So, while there were the usual questions about validity and newer concerns about bias—who would select the problems, who would score, who would see and use results—the major concerns were about whether or not this could be done well within reasonable time, effort and diplomacy.
The teacher leader team, which began this investigation, decided to create the opportunity for a professional learning community (DuFour, 2001) to focus on the following essential questions:
· How can portfolio assessment be administered well in a practical, efficient, and nondisruptive manner? (F1 Practical Orientation)
· What questions or concerns do the many audiences (students, parents, teachers, administrators, board members) have about using portfolios, and how can we address them? (F2 Political Viability)
· What appropriate resources and supports are needed for this instruction and assessment method to be effective? (F3 Evaluation Support)
Consuela is a mathematics teacher
leader for her school. In addition to
her classroom work, she helps her fellow teachers improve their teaching in
their selected mathematics program. Her
state has just developed a set of grade level or cluster expectations for
mathematics, based on state standards that were developed several years
ago. Both the standards and benchmarks
are grounded in the NCTM Principles and Standards for School Mathematics
(2000), which provides a thorough rationale and description of this standard:
Problem
Solving Standard
Instructional programs from
prekindergarten through grade 12 should enable all students to
Consuela is concerned that her school’s math
program does not address these standards and expectations for problem solving adequately. She is also concerned that the program does
not provide any tools for assessing student performance in problem solving.
Consuela calls together a group of
third grade teachers and shares what she thinks are the school’s needs around
the teaching and assessment of problem solving.
The teachers look at the grade expectations and then at their math curriculum
to see where connections can be made.
They come to the same conclusion as Consuela: that adequate
problem-solving assessment is missing in their school.
The group then spends several
sessions researching, identifying, and creating problem- solving tasks for each
unit in the math program. To assess
student performance on the tasks, they create a rubric that reflects state
standards for student performance in problem solving.
Before reading on, in your
opinion, which of the Student Evaluation Standards would guide these teachers’
work?
Suggestions include
P1 Service to Students. This effort serves students’ needs in that
instruction is enhanced for problem solving.
U3 Information Scope. This information expanded the scope of the
evaluation to include problem solving, thus making the assessments more
comprehensive.
U5 Explicit Values. The state expectations are explicit and portray
the values being examined.
F2 Political Viability. Teachers used the state framework, which was
both politically sensitive and important.
F3 Evaluation Support. The district provided a curriculum leader and
the time in which to support this work.
A2 Defined Expectations for Students. Problem solving is
clearly a priority for math standards.
A11 Metaevaluation.
The team is taking the time to examine their evaluation practices.
One school district has put
substantial time and money into improving the teaching and learning of
writing. Science, traditionally assessed
at grades four and eight, is increasing in priority, particularly at the
elementary level, where it has been addressed—both taught and
assessed—unevenly. There is a growing
interest in integrating the curriculum.
Now the staff wants to integrate
inquiry-based science with writing to measure student progress and to gather
the data that can best inform both science and writing instruction. Science provides even the very youngest
students with much to write about.
After a series of false starts,
the staff decides to use writing prompts, a technique that the state employs
for its yearly testing at grades three and seven. These prompts give students an essential
question or science topic to write on relating to a particular area or genre
within the state science and/or writing standards. The prompts need to be very rich and
connected to the specific instructional activities, so that they are meaningful
and fertile.
Teachers and administrators decide
that writing prompts should be given three times a year, from third through
eighth grade, because those are the years that are tested for adequate yearly
progress under federal guidelines. They
will focus on science for two of those six years. The teachers decide on a standard set of
conditions for prompt administration (test window, allowable accommodations,
time, etc.) and to collect samples from all their classrooms. They use the samples to create benchmarks,
illustrating the kind of work that would meet state standards in writing and
science.
The group then uses the standards
and benchmarks to create a rubric for scoring future student writing and understanding
of science concepts using the prompts.
Even though it is a complex task to align the many pieces of this work,
the teachers see this as a way to address feasibility issues with regard to
resources (human, fiscal, time) for instruction, assessment, and professional
development.
What are the Student
Evaluation Standards that the teachers can use to help them arrive at the most
effective solution?
P1 Service to Students. This authentic assessment is in service to
students in that it is aligned with instruction and follows natural procedures.
U3 Information Scope. The team is concerned about the scope of
content assessed, making sure that the samples are focused clearly with the
particular and meaningful science and writing concepts.
F1 Practical Orientation. Teachers find that this sampling assessment
is less disruptive than some of the other required tests they administer.
F3 Evaluation Support. Statewide networks provide professional
development support in scoring, calibration, and reliability measures.
A6 Reliable Information. Teachers take measures to encourage
consistency of scoring.
A8 Handling Information and Quality Control. The team is
concerned with quality control and management of such qualitative, narrative,
and content-rich data.
This school district moved from
traditional letter grades to standards-based report cards several years ago and
is preparing to revise them once again. With
the district’s adoption of a new student information system and linkages to the
statewide data warehouse for increased capacity to aggregate and disaggregate
data, the revisions are calling for increased clarity, consistency of
terminology, and fidelity of use. The
most recent report cards list each state standard for a particular grade level,
or cluster, and then students are rated on a scale of 1 to 4 according to how
far he or she has come toward meeting the standard. Most schools include significant narrative in
the evaluations as well. The
standards-based report cards have been controversial, so it is important to
proceed carefully.