Mythbusting Monday – Teacher influence

mythbusting

Mythbusting Monday is a series of blog posts where we invite different faculty members from the College of Education to discuss education myths and how or what we are doing to combat the myth, fix the myth, etc.  We want our faculty to have a voice in busting these myths and we hope by making these issues personal we can promote and provoke conversations.  We will reference the book 50 Myths and Lies That Threaten America’s Public Schools: The Real Crisis in Education by David Berliner, Gene V Glass and Associates, 2014.

Myth #9:  Teachers are the most important influence in a child’s education

Contributor: Dr. Kathryn Brooks

Good teachers matter.  Supportive schools matter. Every child deserves good teachers and schools. However, teachers and schools do not have a strong influence on student standardized test scores.  Teacher and school evaluation systems in most states are based on, at least in part, how well their students perform on standardized tests.  However, these systems are flawed because factors other than teacher and school level factors are the greatest influence over student performance on standardized tests.

The research unequivocally shows that there is no statistically significant relationship between student performance on standardized tests and school or teacher quality.  Basing high stakes decisions about schools and teachers on bad data will lead to bad decisions.  Thousands of research studies over the past 50 years have been consistent in showing that we cannot establish a link between student test scores and school/teacher quality.  This body of research is some of the most robust research that we have in education and psychometrics.  We could write thousands of pages about these studies, but the following statements capture the essence of this research:

  • 50+ years of extensive research on the impact of teachers and schools on student achievement indicate that typically only 7-10% of variability in student performance on standardized tests is attributable to teacher and school level factors (Coleman, 1966, Heubert & Hauser, 1999; Rivkin, Hanushek, & Kain, 1998; Schochet & Chiang, 2010).  Ninety percent or more of standardized test score performance is attributable to factors that are not related to schools and teachers. The National Research Council of the National Academy of Sciences considers value-added measures of teacher effectiveness “too unstable to be considered fair or reliable” (Heubert & Hauser, 1999).
  • Using the growth model, based on error rates for teacher-level analyses, Schochet and Chiang (2010) found an error rate of 26% over three years of comparison data.  This means that more than one in four teachers were erroneously identified as underperforming or overperforming when they were average performers.  Schochet and Chiang (2010) also found that looking at just one year of teacher data has an error rate of 35%, and they estimated that school level error rates fall between 16% and 21%.  This means that more than 1 in 6 schools is likely to be falsely identified as low or high growth schools when the school is actually of average growth.  These error rates will be much higher for schools that do not reflect the average demographics for the state.  While Indiana has not released the error rates for our growth model, our model was based on the growth models studied by Schochet and Chiang.  The American Statistical Association (2014) calls the statistical underpinning of growth model systems unstable, even under ideal conditions, due to its large error rates.

The difficulty of connecting school/teacher quality to student standardized test scores relates to the myriad of factors that influence student performance on standardized test scores.  Non-school variables such as:

  • low birth-weight and non-genetic prenatal influences on children;
  • inadequate medical, dental, and vision care, often a result of inadequate or no medical insurance;
  • food insecurity;
  • environmental pollutants;
  • family relations and family stress; and
  • neighborhood characteristics (Berliner, 2009, p. 1),

exert a much greater influence on student achievement than do school-related factors.  Even Betebenner (2008), the developer of Indiana’s school accountability model, raised concerns about attributing a direct link between student growth and school performance:

As with student achievement for a school, it would be a mistake to assert that the school is solely responsible for the growth of its students…  The median student growth percentile is descriptive and makes no assertion about the cause of student growth (p. 5).

Non-school related factors create too much statistical noise for even the most prominent statisticians in our country to determine how schools and teachers influence student standardized test scores.

References

American Statistical Association (2014). Executive Summary of the ASA Statement on Using Value-Added Models for Educational Assessment. Retrieved from https://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf

Baker, E. L., et al. (2010). Problems with the Use of Student Test Scores to Evaluate Teachers. Washington, DC: Economic Policy Institute. Retrieved from epi.3cdn.net/b9667271ee6c154195_t9m6iij8k.pdf

Betebenner, D. (2008).  A primer on student growth percentiles. Dover, NH: National Center of the Improvement of Educational Assessment.

Coleman, J. (1966). Equality of educational opportunity. Washington, D.C.: U.S. Government Printing Office.

Franco, M. S., & Seidel, K. (2012). Evidence for the need to more closely examine school effects in value-added modeling and related accountability policies. Education and Urban Society, 44(1).

Gong, B., Perie, M., and Dunn, J. (2006). Using student longitudinal growth measures for school accountability under No Child Left Behind: An update to inform design decisions. Center for Assessment. Available online at: http://www.nciea.org/publications/GrowthModelUpdate_BGMAPJD07.pdf

Heubert, J.P., & Hauser, R.M. (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.

Hout, M. and Elliott, S., eds. (2011). Incentives and Test-Based Accountability in Education. National Research Council of the National Academies of Science. http://books.nap.edu/openbook.php?record_id=12521

Jones, B., and R. Egley. 2004. Voices from the frontlines: Teachers’ perceptions of high-stakes testing. Education Policy Analysis Archives 12(39).

Jones, M. G., B. D. Jones, B. Hardin, L. Chapman, T. Yarbrough, and M. Davis. 1999. The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan 81(3): 199-203.

Rivkin. S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417-458.

Schochet, P. Z., & Chiang, H. S. (2010). Error rates in measuring teacher and school performance based on student test score gains (NCEE 2010-4004). Washington, DC: National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences, United States Department of Education.

Stevens, J., & Zvoch, K. (2006). Issues in the implementation of longitudinal growth models for student achievement. In R. W. Lissitz (Ed.) Longitudinal and value added models of student performance (pp. 170-209). Maple Grove, MN: JAM Press.

Mythbusting Monday – High school exit exams and “college readiness”


mythbusting

Mythbusting Monday is a series of blog posts where we invite different faculty members from the College of Education to discuss education myths and how or what we are doing to combat the myth, fix the myth, etc.  We want our faculty to have a voice in busting these myths and we hope by making these issues personal we can promote and provoke conversations.  We will reference the book 50 Myths and Lies That Threaten America’s Public Schools: The Real Crisis in Education by David Berliner, Gene V Glass and Associates, 2014.

Myth: #45  

High school exit exams guarantee that our graduates will be “college ready” and prepared to succeed as workers in a global economy.

Contributors: Dr. Susan Adams, Dr. Catherine Pangan, Dr. Katie Brooks, Dr. Arthur Hochman, Lindsay Williams, Cassandra Pixey

How do college admissions teams know which high school graduates are most ready for success in college? It might surprise you to know that cumulative high school GPA-what most people consider to be subjective, unreliable data-is the strongest predictor of college success. Standardized test scores are much less predictive of college readiness and success than teacher-made assessments. Why is this? Classroom teachers use multiple measures, whether homework, projects, participation, and a wide variety of summative and formative assessments, to measure student learning. In the College of Education at Butler University, our pre-service teachers develop critical thinking expertise, collaboration skills, and creativity that serve them well in the classroom as they design meaningful and engaging learning and assessment opportunities for K-12 students.

In our laboratory school partnerships at IPS Shortridge and IPS Butler-Lab School 60, we are seeing the power and impact K-12 students communicate through project-based assessment, multimedia documentation panels, and inquiry-based group. We assess our teacher candidates using similar measures: portfolios, credential files, and video analysis of our candidates teaching with local students. A standardized assessment could never tell the full story of what students know, are able to do, and are ready to do. Good assessment practices involve the extensive use of formative and summative assessments that students produce over time.

In our global economy, project-based and inquiry-focused assessments better measure the knowledge, dispositions, and skills required in college and in the workplace than do multiple choice and short answer standardized assessments.  The marketplace has told us that our approach to assessment and teacher education is what is necessary and needed, by hiring our teacher education, school counseling, and administrative graduates.  Our students learn a wide variety of assessments so that their toolbox is as full and comprehensive as possible.  We need to tell the full story of each child, classroom, school and district through authentic assessment that is as equally varied, deep and rich. Every student deserves the opportunity to learn from a teacher who is highly skilled and well prepared to teach these creative, collaborative and project-based skills as preparation for participation in this global economy.

Transforming Education – March 2015

dean newsletter bannerReinventing “Rigor” in Education

A Guest Column by Professor Cathy Hartman

I have been troubled by the word rigor, often used today to describe something lacking in contemporary education. Standardized testing of students can supposedly expose and even correct non-rigorous instruction, or perhaps even worse, there are perceptions that standardized tests in and of themselves define rigor in education.

The dictionary definition of rigor is “harsh, unsupportive, inflexible,” and, even as defined by a children’s dictionary, “strict precision.”

When I started rebuilding my understanding of the word rigor, I decided to first define what it means to me. To me, rigor is a task or goal that might be challenging, worthwhile, meaningful, worthy, and involves critical and abstract thinking. This is not how I would define standardized testing, hence my initial disconnect.

When I started rebuilding my understanding of the word rigor, I decided to first define what it means to me. To me, rigor is a task or goal that might be challenging, worthwhile, meaningful, worthy, and involves critical and abstract thinking. This is not how I would define standardized testing, hence my initial disconnect.

I decided this word must be reinvented, maybe even put to rest. My own reflective process, collaborative dialogue with colleagues, and intentional observations of learning helped me realize the focus and “ultimate goal” of instruction should not be on rigor. We should focus our attention on supporting students to feel empowered in their learning.

Abby Bucher ’05, a K–1 teacher at the Indianapolis Public Schools/Butler University Laboratory School, captured a moment with her students that documents metaphorical thinking. This example illustrates a learning process that moves beyond inflexible thinking or “strict precision,” as rigor is defined. It instead shows empowerment in thinking, learning, and understanding when students are given the opportunity to move beyond a single answer. Or, in my mind, it is the essence of reinventing rigor.

In January we all came back together after I had been gone on maternity leave since September. I asked the children, “How do we show how we have grown as a community using rocks?” I had some ideas, but I never expected this: one friend [the term Lab School students use to describe classmates] began saying that we started off as an igneous rock, and then, when we split up into the classes, that was the heat and pressure. …When we came back together, we changed into a metamorphic rock. Here we were taking our knowledge of rocks and turning it into a beautiful metaphor of how our community had evolved.

Clearly Mrs. Bucher’s students had a strong understanding of content from their classroom learning experiences and were able to merge these understandings with describing their classroom community.

Rigorous” standardized testing does not determine the depth that our children, our students, our grandchildren, and our neighbors are adding to the world. The most important component can and should be shown through the learning, thinking, and application process offered and facilitated by our classroom teachers. If we approach learning engagements with the lens that children are capable and creative individuals who have prior knowledge and experiences, their learning possibilities are endless.

An excellent example recently occurred in a Lab School second grade classroom taught by Brittany Shackleford ’11:

Wednesday morning, as a community of mathematicians, we had an exciting and powerful discussion about whether or not a dollar bill is a rectangle or a rectangular prism. We started the discussion during problem solving with our math partners [peer partnerships determined prior to a lesson] by sharing our thinking about the question, Is a dollar bill a rectangle or rectangular prism?

Each partnership came to share time ready to justify their thinking. At the beginning of our discussion, a majority of the class shared that they believed a dollar bill was a rectangle. They felt that, since the dollar bill was “flat,” it had to be two-dimensional. This is when the mathematical debate began. The students who believed the dollar bill was a rectangular prism passionately shared their thinking and justifications. This share led to a more in-depth discussion about the attributes of a 2D shape and 3D shapes. As a community of mathematicians, the students created a checklist of attributes for 3D shapes that included:

• A 3D shape had to be solid
• Could be held
• Had to have measurable thickness

As the students compared the dollar bill to their checklist, many of the students shared that they were beginning to change their thinking. Noah then had the idea that we should use a ruler to measure the thickness of the dollar bill, even though it appeared flat to most students. The dollar bill measured in at less than one millimeter in depth. This measurement was a defining moment in the discussion. The entire class was agreeing and sharing how their thinking had changed throughout the discussion. If the dollar bill met the criteria of their checklist, it must be 3D. At this point, Maddie then shared a final thought that, if the dollar bill was 3D, then everything in the world around is 3D—even a piece of paper. Xiaoyang then shared that even drawing a 2D shape on a piece of paper could make it 3D because of the thickness of the ink on the paper.

This discussion embodied the power of math workshop. As a class we have created a community of mathematicians where the students are engaged in critical thinking, work collaboratively to problem solve, and feel comfortable sharing their thinking.

Let’s work together to inform our community of the thoughtful, empowering work being completed in our schools today. Here are some questions to help you get started with your own reflection. Please share your thinking and stories on our College of Education Facebook page.

• How do you continue to share the important stories of learning like the dollar bill or thinking metaphorically like the K–1 students, instead of tales of rigor associated with a single test, which can derail us from the powerful teaching that should continue taking place?
• How will you commit to a teaching and learning environment that promotes joy, depth, intentionality of lesson design, intentionality of cultivating an environment of thinking and processing, and having an impact on the greater community and world?
• How might you engage in a conversation with your colleagues to help more accurately define the word rigor? Is this the word you want to keep using? What will you use to describe powerful learning?
We look forward to hearing your thoughts.

Dr. Ena Shelley
Dean, College of Education