Local Implementation

Because much of the content in statistics, probability, and discrete mathematics is new for many teachers, and because some of the familiar material is developed more fully than in traditional mathematics, teachers need advice and support from other teachers and administrative support in order to implement the curriculum effectively. (Professional Development Opportunities)

Active involvement of students also requires a different type of planning by teachers. The Teacher Resource materials encourage teachers to be listening, observing, questioning, facilitating student work, and orchestrating class discussions in new ways. Professional development programs organized around reflecting on practice enable teachers to hone their skills in these areas.

At the very least, teachers should attend a professional development workshop led by an experienced CPMP teacher. In addition, schools should strongly consider providing the following supports:

We examined the classroom practices of 20 teachers during the field test of CPMP Course 1. Ten of these teachers comprised the top quartile of field-test teachers and the other 10 the bottom quartile with respect to their students' growth in mathematical achievement over the one-year course. Achievement was measured by a nationally standardized test called the Ability to Do Quantitative Thinking which is the mathematics subtest of the Iowa Tests of Educational Development. The primary data sources were: trained observer's holistic rating of the alignment of the instructional practice and classroom climate with CPMP's teaching for understanding model, self-perceptions of practice by the teachers, and expressed concerns of the teachers about the new curriculum.

The research results from this study, summarized below, are reported in a peer-reviewed article published in the Journal for Research in Mathematics Education:
Schoen, H. L., Finn, K. F., Cebulla, K. J., & Fi, C. (2003). Teacher variables that relate to student achievement when using a standards-based curriculum. Journal for Research in Mathematics Education, 34(3) 228-259.

The description of the "effective" (i.e., first-quartile) teacher that emerged from analyzing the data from these sources follows. This teacher may be of either gender, but we will use female pronouns for convenience.

A paper by Richard Hill and Thomas Parker, that has been circulating informally for many years on the Internet and among critics of Standards-based reform, appeared in print in the December, 2006 American Mathematical Monthly. Hill and Parker claim to provide "compelling" evidence that students who learn high school mathematics from the Core-Plus Mathematics program are poorly prepared for collegiate mathematics at Michigan State University (MSU). However:

• The Hill and Parker paper does not provide crucial facts that seriously compromise the design, analysis, and conclusions of the study.

The six high schools in the Hill and Parker study were participating in the Core-Plus field test from 1994 to 1999, so Core-Plus evaluators have direct knowledge of the extent and nature of implementation of the Core-Plus Mathematics field-test units in these schools during those years. An article in the Journal for Research in Mathematics Education includes a report of the implementation practices of Core-Plus teachers based on class observations and interviews and written surveys of teachers in 36 Core-Plus field-test schools, including the six in the Hill and Parker study. The abstract and downloadable article is at: my.nctm.org/eresources/article_summary.asp?from=B&uri=JRME2003-05-228a. In addition, more recent correspondence directly with the schools in the Hill and Parker study has confirmed the extent and nature of implementation of Core-Plus during the period of the Hill and Parker study. Based on this in-depth knowledge and analysis, it is our carefully considered conclusion that:

• Hill and Parker's study and their reporting of it are fundamentally flawed. The data and design do not support the stated conclusions and interpretations, and the report is strikingly misleading.

Hill and Parker analyzed trends in the MSU mathematics course-taking and achievement data from the 1996, 1997, 1998, and 1999 graduates of six high schools. By 1998 and 1999, some of the graduates had studied some of the Core-Plus Mathematics field-test units. In four of the six schools, classified as CP (Core-Plus), Hill and Parker found a downward trend in the data, while in the other two "…declines are not evident in the data." For unclear reasons, Hill and Parker decided that the latter two schools "supplemented Core-Plus," and so they did not include these schools in the CP group and they removed them from their main analysis.

In fact, virtually every high school teacher supplements any curriculum they use in various ways, and the Core-Plus curriculum is no exception. As for the two excluded schools, 1998 and 1999 graduates in the Hill and Parker study would have completed field-test versions of three or four Core-Plus courses. There was no alternative curriculum track in either school, and there is no evidence that more supplementing of Core-Plus occurred in the two excluded schools than in the four schools that Hill and Parker chose for their main analysis. These two schools clearly should have been included in Hill and Parker's CP group.

• With vague and unconvincing justification, Hill and Parker separate out the two Core-Plus schools in which there was no downward trend, and base their main conclusions on the declining trend in MSU mathematics course-taking and achievement data in the other four schools.

Hill and Parker analyzed trends in the MSU mathematics course-taking and achievement data from the 1996, 1997, 1998, and 1999 graduates of six high schools. Their negative conclusions about Core-Plus are based on four of the six schools, which had weaker data for the 1998 and 1999 graduates than for the 1996 and 1997 graduates. Since Hill and Parker conclude that their “results raise serious issues about the effectiveness of CPMP [Core-Plus] in preparing students to take college mathematics at Michigan State University,” most readers would naturally assume that all 1998 and 1999 graduates of the Core-Plus-identified schools in the analysis actually completed Core-Plus Mathematics courses in high school. Such readers would be wrong ­ as a very careful reading of Hill and Parker's definition of the CP (Core-Plus) group at the top of page 909 shows. The members of the CP group, by definition, were "implementing a system of offering only Core-Plus Mathematics" and possibly AP Calculus in 1998 and 1999. Further clarification comes from correspondence from Hill and Parker in which they state that, “our ‘Core-Plus Group’ consists of students from schools that were in the process of implementing Core Plus” [italics as in a written statement from Hill and Parker]. Furthermore, they openly state that they do not know whether or how much any individual student actually studied from the Core-Plus Mathematics curriculum.

• Hill and Parker do not know which of the 1998 and 1999 graduates of these four schools completed even one course of Core-Plus Mathematics in high school. Yet they assert that their student-level analysis provides compelling evidence of the ineffectiveness of the Core-Plus Mathematics program.

In fact, in at least two of the four high schools classified as CP, the graduating classes had been taught from a variety of curriculum materials. For example, in the 1998 graduating classes from these two schools, only 40 out of 830 graduates participated in the field-test of the Core-Plus Mathematics curriculum. Furthermore, there is no evidence that any of these 40 Core-Plus students enrolled at Michigan State University. To attribute the performance of graduates from these schools to their having completed the Core-Plus Mathematics curriculum, as implied and stated in the Hill and Parker report, is patently incorrect.

Thus, two schools in which all graduates completed three or more Core-Plus courses are excluded from the CP group, while two other schools in which very few students studied any Core-Plus Mathematics units are included in the CP group.

• Four of the six schools in the Hill and Parker study are misleadingly classified as Core-Plus schools or not. If these schools were more appropriately classified, then there might be the beginning of a valid study of Core-Plus students’ preparedness for collegiate mathematics at MSU. Unfortunately, this is not the case in the Hill and Parker study.

Hill and Parker's reporting of their study is misleading from beginning to end, starting with the title. Rather than being a “study of Core-Plus students at Michigan State University,” as the title states, it is actually, according to the authors’ own description, a study of “students from schools that were in the process of implementing Core Plus.” Their classification of schools as Core-Plus (CP) or not, does not reflect whether or not the students in those schools actually studied significantly from the Core-Plus Mathematics curriculum.

• To attribute the performance of graduates from these schools to whether or not they completed the Core-Plus Mathematics curriculum, as implied and stated in the Hill and Parker report, is patently incorrect.

A final and seriously misleading statement comes in the very last paragraph of the report where Hill and Parker imply that their study has to do with “the first edition of CPMP.” In fact, 1998 and 1999 graduates of the six schools in this study who actually completed Core-Plus courses in high school would have used some units of the unpublished field-test version of the curriculum. These draft Core-Plus Mathematics materials have not been used for many years. Based on our own field-test evaluations, this draft version was revised prior to publication in ways that maintain the well-documented strengths of the Core-Plus Mathematics curriculum (see the Evaluation page on the Core-Plus Web site at www.wmich.edu/cpmp/) while improving the curriculum in several ways, particularly as regards to preparation for collegiate mathematics. The changes made and some of their positive effects are discussed in Schoen & Hirsch (American Mathematical Monthly, February 2003). Moreover, a second edition of the Core-Plus Mathematics curriculum is now being completed, with Courses 1 and 2 published by Fall 2007.

• The Hill and Parker study is not about students who necessarily completed Core-Plus courses in high school. Furthermore, it does not address the published four-year Core-Plus Mathematics curriculum that has been used in schools since its completion in 2000 or the second edition of the curriculum that becomes available in Fall 2007.

We agree with Hill and Parker that there is a need for research assessing effects of different high school mathematics experiences on students’ progress in collegiate mathematics. However, the present Hill and Parker study does not contribute usefully to this research due to its substantial flaws and misleading interpretations. We urge concerned parties to look for valid research that addresses current published curricula. A brief report of our project evaluators’ start in this direction can be found at www.wmich.edu/cpmp/LongitudinalStudy1.html.

Another gauge of performance in college-level mathematics is provided by Advanced Placement courses. School districts using the published Core-Plus Mathematics program report increased enrollments and passing rates in Advanced Placement Calculus courses. Their students enter college with placement in courses above those described in the Hill and Parker study. See School Reports in the Evaluation section of the Core-Plus Web site. There are many valid publications and presentations about the Core-Plus Mathematics curriculum.

• For an annotated bibliography of research and other publications about the Core-Plus Mathematics curriculum, including journal articles, book chapters, papers presented at research conferences, and Ph.D. dissertations, see www.wmich.edu/cpmp/bibliography.html.

In 1997, an opinion survey of Core-Plus graduates and non-Core-Plus graduates in a Michigan school district was carried out by Greg Bachelis of Wayne State University. This survey was then analyzed by James Milgram of Stanford University and a report was widely disseminated on the Internet and to the media. This report attempts to conclude that Core-Plus students are not well-prepared for collegiate mathematics. However, the survey is invalid due to serious design flaws and the report draws incorrect conclusions. In spite of this, critics continue to draw attention to the study as a means to create fear of change.

Why is the Survey Invalid?

Self-reported data: The data are based on self-reported grades and test scores from students. This well-known error in survey research leads to unreliable data.

Self-selected sample: The survey is based on a self-selected sample, with no evidence of the makeup of that sample. This well-known error in survey research can lead to biased results.

Aggressively biased survey methods: The anti-Core-Plus group funding this survey aggressively campaigned among students. Such activity creates bias in the very group one is trying to survey.

Invalid generalizations: The school was using a 1997 pilot curriculum, which no longer exists. As part of the 4-year, data-driven curriculum development process, the curriculum has gone through several years of additional development since 1997. The final version of the Core-Plus curriculum maintains the well-documented strengths of Core-Plus (see frequently asked evaluation questions), while improving the curriculum in several ways.

Incorrect conclusions: This flawed opinion survey attempts to conclude that Core-Plus students are not well prepared for collegiate mathematics. On the contrary, data provided by the University of Michigan registrar indicates that in collegiate mathematics courses at the University of Michigan, graduates of the Core-Plus program perform as well as, or better than, graduates of a traditional mathematics curriculum.

Conclusion:

Due to fatal research flaws and incorrect conclusions, the Bachelis-Milgram study is not a valid study of the 1997 Core-Plus pilot program at Andover High School. Furthermore, it says nothing about the final Core-Plus curriculum in use today.

The invalid claims made on the basis of this single flawed study of one school are in marked contrast to a large and growing body of research that shows the positive effects of the Core-Plus curriculum in a wide range of schools nationally. Results of these rigorous research studies have appeared in refereed journals and presentations at professional conferences. They show the strong positive effects of Core-Plus Mathematics on students' conceptual understanding, problem solving ability, quantitative reasoning, attitudes toward mathematics, and success in advanced mathematical study. For more information about this research, see the Evaluation page, the annotated list of Research Publications, and School Reports from schools using the published version of the Core-Plus Mathematics program.

Schoen, H. L., & Hirsch, C. R. (2003). Responding to calls for change in high school mathematics: Implications for collegiate mathematics. American Mathematical Monthly, February, 109-123.

Schoen, H. L. & Hirsch, C. R. (2003). The Core-Plus Mathematics Project: Perspectives and student achievement. In S. Senk and D. Thompson (Eds.), Standards-Based School Mathematics Curricula: What Are They? What Do Students Learn? pp. 311-344. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Schoen, H. L., Finn, K. F., Cebulla, K. J., & Fi, C. (2003). Teacher variables that relate to student achievement when using a standards-based curriculum. Journal for Research in Mathematics Education, 34(3) 228-259.