Project Description and Goals


– developing an assessment instrument in support of the undergraduate preparation of elementary teachers to teach science as inquiry
(Funded under the NSF/ASA Program Announcement Track 1: New Development)

Under National Research Council and AAAS leadership, the United States is developing a national commitment to the teaching of science as inquiry across the K-12 grades (AAAS, 1990; NRC, 1996, 2000, 2001). Inquiry teaching of science refers to teaching (pedagogy) that reflects the investigative approach and techniques that scientists use to discover and construct knowledge. Science as inquiry involves both process and content; the doing of science is emphasized as much as science knowledge.
The science content knowledge of undergraduate students is regularly assessed throughout their education, as a matter of course. For prospective teachers, an equally important knowledge component is pedagogical knowledge of inquiry science teaching; that is, knowledge of teaching practices for specific topics that specifically reflect the investigative nature of science. However, by contrast this crucial ability is rarely assessed directly or regularly during students’ preparation. This is an anomaly which likely arises because understanding of inquiry pedagogy, relating to specific topics, is harder to assess, and because suitable items and instruments are simply not available. Surely a necessary component of our national effort to improve science education is the ability to effectively and efficiently assess teachers’ pedagogical knowledge of inquiry science teaching. Without such an assessment we can neither fine-tune our undergraduate science teacher education programs nor ensure the quality of graduates. Currently, such an assessment tool is not available. Iris Weiss, a leading authority on science assessment, concurs: “I don't know of any such set of items, and I agree with you that this work is sorely needed!”
Therefore, we propose to develop a field-validated, objective assessment tool for testing pre-service K-8 teachers’ pedagogical knowledge of inquiry science teaching. We dub this POSIT, the Pedagogy of Science Inquiry Teaching test. Our focus is on undergraduate students preparing to be teachers of science. Our particular interest is the inquiry pedagogical knowledge required for teaching science at the elementary grades (K-8), because this sets the foundation; it is critical that children leave these grades with a good grounding in both the content and processes of science as they go on to the more specialized study of the sciences at the secondary level.
Note that we are not trying to develop an observational instrument to evaluate inquiry teaching practice in the classroom; there are tools for this, such as the Reformed Teaching Observation Protocol (RTOP), and the Lesson Observation System used the Science and Math Program Improvement (SAMPI) group, of established validity and reliability. Rather, our instrument serves other functions and goals. It will be a readily administered objective instrument for use during undergraduate instruction of prospective teachers, to both assess and promote understanding of inquiry science pedagogy. Note also that our assessment will be problem-based; items will be based on realistic classroom teaching scenarios for specific science topics, rather than on broad statements about inquiry generally.
Once developed and validated, this assessment tool, addressing an important assessment area where there is currently a huge gap, will be of value to every undergraduate science teacher program in the country. For the first time we will be able to assess science inquiry pedagogy as readily and regularly as we assess science content knowledge.

Features and variations of classroom inquiry
Table1 below (National Research Council 2000, p 29) identifies features of classroom inquiry and its variations, depending on the degree of learner self-direction or teacher direction.

Table 1

Teacher instructs so that the:
1. Learner engages in scientifically oriented questions
Learner poses a question Learner selects among questions, poses new questions Learner sharpens or clarifies question provided by teacher, materials, or other source Learner engages in question provided by teacher, materials, or other source
2. Learner gives priority to evidence in responding to questions
Learner determines what constitutes evidence and collects it Learner directed to collect certain data Learner given data and asked to analyze Learner given data and told how to analyze
3. Learner formulate explanations from evidence
Learner formulates explanations after summarizing evidence Learner guided in process of formulating explanations from evidence Learner given possible ways to use evidence to formulate explanation Learner provided with evidence
4. Learner connects explanations to scientific knowledge
Learner independently examines other resources and forms the links to explanations Learner directed toward areas and sources of scientific knowledge Learner given possible connections  
5. Learner communi-cates and justifies explana-
Learner forms reasonable and logical argument to communicate explanations Learner coached in development of communication Learner provided broad guidelines to use sharpen communication Learner given steps and procedures for communication
More                Amount of Learner Self-Direction                    Less
Less            Amount of Direction from Teacher/Material           More

Planning and implementing successful inquiry-based learning in the science classroom is a task demanding both science content knowledge and inquiry pedagogy knowledge – and in combination, in the context of each science topic being taught. Our aim is to develop and make available a problem-based instrument to assess inquiry pedagogical content knowledge, across all content areas of the K-8 science curriculum.

Project Goals
The U.S. Department of Education (2002) has called for the use of scientifically based research and evidence as the foundation for education programs; and as The Condition of Education 2002 states: “Reliable data are critical in guiding efforts to improve education in America” (Livingston 2002, p. iii; also see Lesh & Lovitts 2000; Shavelson & Towne 2002; Towne & Hilton, 2004). Accordingly our project involves a multistage plan for test development followed by two rounds of piloting and revision, concluding with blinded field-testing against classroom practice, for studying the validity of the assessment tool. The development of a validated objective instrument for assessing pedagogical knowledge of inquiry science teaching will significantly aid our instructional and research efforts to improve undergraduate science education.
To that end, the proposed project will:
1) Develop Inquiry-Item Criteria regarding inquiry pedagogy, to guide development & evaluation of the instrument. Criteria will be consistent with the classroom inquiry features and variations in Table 1.
2) Compose problem-based items meeting these criteria and involving realistic classroom teaching scenarios across the range of K-8 elementary science topics. Our target is 90 item-situations, each in three alternative formats, for a total of 270 assessment items 3) Ensure that the item scenarios and questions reflect race and gender diversity, and various geographic and economic contexts. 4) Pilot the POSIT items on a relatively large scale with undergraduate pre-service student teachers. Piloting will involve over a thousand students, of racial and gender diversity, at several collaborating institutions across the country. There will be two successive pilots and corresponding item revisions.
5) Establish standardized scoring and administration directions for the instrument.
6) Establish initial estimates of reliability of the instrument.
7) Validate the POSIT instrument by studying predictive power with respect to observed teacher practice of inquiry teaching in actual classrooms. This will include both pre-and in-service teachers. 8) Make the assessment items, the instrument, project documents and reports broadly available. Dissemination will include the Internet via website. The project and results will be reported widely though conference presentations and journal articles.

Project Background

The inquiry teaching of science across the K-12 grade levels is a goal to which American education aspires, as documented by both the National Research Council’s (1996) National Science Education Standards and the American Association for the Advancement of Science’s (1990) Project 2061. According to the National Science Education Standards,

Learning science is something that students do, not something that is done to them. ‘Hands-on' activities, while essential, are not enough. Students must have ‘minds-on’ experiences as well. The Standards call for more than ‘science as process’ skills, in which students learn such skills as observing, inferring, and experimenting. Inquiry is central to science learning. When engaging in inquiry, students describe objects and events, ask questions, construct explanations, test those explanations against current scientific knowledge, and communicate their ideas to others. They identify their assumptions, use critical and logical thinking, and consider alternative explanations. In this way, students actively develop their understanding of science by combining scientific knowledge with reasoning and thinking skills. (p. 2)

AAAS espouses a similar view:

“Both want students working in teams, both want them raising questions and exploring ideas for themselves, both want students to learn to evaluate ideas using evidence. The pedagogy for science teaching, then, is one that actively engages students in reasoning about scientific phenomena” (Kennedy, 1997, p. 9).

The origins of the modern day concept of science teaching as inquiry lie with the 1960s NSF-funded curriculum projects (Anderson 2003; DeBoer 1991; Krajcik et al 2001; Rudolph 2002; Schwab 1962). The effectiveness of those inquiry-based curricula was document first by Shymansky et al in 1983 and again 1990, and by Secker and Lissitz (1999) and Secker (2002) using updated techniques. There are dissenters, however, the most prominent of whom is David Klahr (Chen & Klahr, 1999; Klahr, 2000; 2002, Klahr and Nigam 2004), who cautions that the “widespread belief that discovery learning is superior to direct instruction in early science education warrants careful empirical assessment” (2002). Based on his carefully crafted research, he concludes (p.1) that his

‘results challenge predictions derived from the presumed superiority of discovery approaches for deeper, longer lasting, and “more authentic” understanding of scientific reasoning processes, and suggest instead, a more nuanced examination of the most effective mixes and the most suitable matches between topic and pedagogy’.

Note that the ‘guided inquiry’ approach advocated by the NRC and AAAS is not synonymous with ‘discovery’. It is indeed a more nuanced approach than simple ‘discovery’ as tested by Klahr. Based on Robert Karplus’ (1977) ‘learning cycle’, inquiry instruction involves a judicious mix of methods from discovery to teacher-guided concept development. The focus is on the raising of questions, seeking data, and working with evidence to form concepts. As needed, the teacher intervenes to help the students toward the concept – which is what Klahr’s research in fact supports.

The move toward inquiry based teaching puts pressure on teachers not only to know not only the content of science, but also how to translate the content and methods of science into analogous instructional practices. Such knowledge is what Shulman called pedagogical content knowledge or PCK (Shulman 1986, 1987; also Barnett & Hodson 2001; Dean 1999). PCK is the knowledge of instructional practices pertinent to specific content areas. In science teaching, that emphatically includes understanding of inquiry as an approach to the subject. (Eick 2000; Lederman 1992, Lowery 2002). Most science methods courses today deal with the pedagogical content knowledge of inquiry and there are many documents to help pre-and in-service teachers, such as Classroom Assessment and the National Science Education Standards (NRC, 2001) and Inquiry and the National Science Education Standards: A Guide for Teaching and Learning (NRC, 2000), both from the National Research Council.

The importance of quality teacher education for the teaching of science is clear, and it follows that valid assessment of science teaching ability is critically important. Without such assessment we do not have suitable tools for informing the revision of science teacher education methods programs or for fully documenting the quality and abilities of our graduates.

As one would expect, undergraduate pre-service teachers do have their science teaching practice evaluated (Luft 1999), but this evaluation is by observation of practice (e.g. Jenness & Barley 1999). Although this type of evaluation is very important, detailed observations of individual practice can only be done in limited measure, and each is limited in scope – practically, only a few science lessons covering one or two topics can be evaluated in this labor-intensive way. This is usually done toward the end of a program, and evaluations of pre-service elementary teachers tend to done by teacher educators with little science background.

Thus, undergraduate science teacher education desperately needs an assessment tool specific to the teaching of science, of broad scope and ease of use, validated by research, and which can be readily used, formatively and summatively, during the undergraduate program. It is critical to ascertain to what degree students moving through their preparation programs as teachers of science are developing the knowledge of inquiry that will form the basis for their practice of inquiry teaching.

Broader Impacts of the Project
The “gold standard” for pedagogy assessment is what teachers actually do in the classroom. In our project, POSIT, a written assessment of understanding of inquiry pedagogy, based on specific case-based examples, will have been blind-tested against just such a standard. Once POSIT has been developed, refined and validated in this way, it will be made widely available, so that educators at any undergraduate institution will be able to assess the effectiveness of their programs in regard to pedagogical content knowledge of inquiry science teaching. They would then be in an informed position to make critical improvements to undergraduate science teacher education programs.

Such informed improvement includes practices that will help redress the under-representation of certain groups in science. Improving the preparation and quality of teacher graduates will help ameliorate teacher inequalities in schools that serve under-represented populations, by increasing the number of teachers well qualified to teach science. (Mayer et al. 2000).

POSIT as a tested and validated instrument will clearly be of value to researchers who investigate education in science and ways to improve the education and practice of science teachers.

Professors of regular undergraduate and graduate SMET courses may also find POSIT items useful. Previous research indicates that science professors can be motivated to improve their own teaching practice as a consequence of considering how science is best taught to young people. (Fedock et al, 1996). This could be of particular importance given the recent efforts to recruit doctoral level science majors into school science teaching, such as the NSF program for Graduate Teaching Fellows in K-12 Education (Vines, 2002). We also propose to have students in selected graduate-level science seminars discuss the items as a way to prompt thinking about what it means to teach science, at all levels.

Meeting NSF Goals
Earlier we stated our project goals. These are consistent with NSF goals, as noted below.
1) NSF would like projects to integrate research and education by advancing discovery and
understanding while at the same time promoting teaching, training, and learning. Our project clearly integrates research and education. The instrument that we propose to develop and validate serves both assessment and instructional purposes. Moreover, during the pilot testing of the tool, we have integrated instructional components for the courses where the piloting is to take place. The project uses research to develop evidence-based educational instruments, thus advancing knowledge through research while clearly promoting teaching and learning.
2)   NSF would like projects to broaden the participation of underrepresented groups. The participants in our project (both the subjects and the writing teams) will include under-represented groups. There will also be diversity in geographic location and economic situation. As already noted, the assessment tool developed will help ameliorate teacher inequalities, particularly in schools that serve underrepresented populations, by improving teacher preparation.
3) NSF would like projects to enhance the infrastructure for research and/or education. Our project is specifically designed to provide a tool that will significantly improve science teacher education at any institution. Moreover our project will provide a tool that will also have application in science education research. This assessment tool will be of great value to researchers in studying ways to improve science teacher education.
4) NSF would like project results to be disseminated broadly to enhance scientific and technological understanding. The assessment tool and project reports will be made freely available via the Internet. We will report our work at conferences and submit articles to appropriate scholarly and practice-oriented journals.
5) NSF would like projects to benefit society at large. The project will benefit society by helping to improve the education of teachers for teaching science. Good teachers of elementary school science make students enthusiastic, which leads on more interest and ability in science amongst secondary students, and this eventually leads to a more scientifically literate citizenry, better future teachers, more scientists and better public support for the work of science. A good education in science needs to start at the elementary level and continue from there.



3225 Wood Hall
Western Michigan University
Kalamazoo MI 49008-5444 USA
269-387-5398 | 269-387-4998 Fax