Mythbusting Monday – Teacher influence

mythbusting

Mythbusting Monday is a series of blog posts where we invite different faculty members from the College of Education to discuss education myths and how or what we are doing to combat the myth, fix the myth, etc.  We want our faculty to have a voice in busting these myths and we hope by making these issues personal we can promote and provoke conversations.  We will reference the book 50 Myths and Lies That Threaten America’s Public Schools: The Real Crisis in Education by David Berliner, Gene V Glass and Associates, 2014.

Myth #9:  Teachers are the most important influence in a child’s education

Contributor: Dr. Kathryn Brooks

Good teachers matter.  Supportive schools matter. Every child deserves good teachers and schools. However, teachers and schools do not have a strong influence on student standardized test scores.  Teacher and school evaluation systems in most states are based on, at least in part, how well their students perform on standardized tests.  However, these systems are flawed because factors other than teacher and school level factors are the greatest influence over student performance on standardized tests.

The research unequivocally shows that there is no statistically significant relationship between student performance on standardized tests and school or teacher quality.  Basing high stakes decisions about schools and teachers on bad data will lead to bad decisions.  Thousands of research studies over the past 50 years have been consistent in showing that we cannot establish a link between student test scores and school/teacher quality.  This body of research is some of the most robust research that we have in education and psychometrics.  We could write thousands of pages about these studies, but the following statements capture the essence of this research:

  • 50+ years of extensive research on the impact of teachers and schools on student achievement indicate that typically only 7-10% of variability in student performance on standardized tests is attributable to teacher and school level factors (Coleman, 1966, Heubert & Hauser, 1999; Rivkin, Hanushek, & Kain, 1998; Schochet & Chiang, 2010).  Ninety percent or more of standardized test score performance is attributable to factors that are not related to schools and teachers. The National Research Council of the National Academy of Sciences considers value-added measures of teacher effectiveness “too unstable to be considered fair or reliable” (Heubert & Hauser, 1999).
  • Using the growth model, based on error rates for teacher-level analyses, Schochet and Chiang (2010) found an error rate of 26% over three years of comparison data.  This means that more than one in four teachers were erroneously identified as underperforming or overperforming when they were average performers.  Schochet and Chiang (2010) also found that looking at just one year of teacher data has an error rate of 35%, and they estimated that school level error rates fall between 16% and 21%.  This means that more than 1 in 6 schools is likely to be falsely identified as low or high growth schools when the school is actually of average growth.  These error rates will be much higher for schools that do not reflect the average demographics for the state.  While Indiana has not released the error rates for our growth model, our model was based on the growth models studied by Schochet and Chiang.  The American Statistical Association (2014) calls the statistical underpinning of growth model systems unstable, even under ideal conditions, due to its large error rates.

The difficulty of connecting school/teacher quality to student standardized test scores relates to the myriad of factors that influence student performance on standardized test scores.  Non-school variables such as:

  • low birth-weight and non-genetic prenatal influences on children;
  • inadequate medical, dental, and vision care, often a result of inadequate or no medical insurance;
  • food insecurity;
  • environmental pollutants;
  • family relations and family stress; and
  • neighborhood characteristics (Berliner, 2009, p. 1),

exert a much greater influence on student achievement than do school-related factors.  Even Betebenner (2008), the developer of Indiana’s school accountability model, raised concerns about attributing a direct link between student growth and school performance:

As with student achievement for a school, it would be a mistake to assert that the school is solely responsible for the growth of its students…  The median student growth percentile is descriptive and makes no assertion about the cause of student growth (p. 5).

Non-school related factors create too much statistical noise for even the most prominent statisticians in our country to determine how schools and teachers influence student standardized test scores.

References

American Statistical Association (2014). Executive Summary of the ASA Statement on Using Value-Added Models for Educational Assessment. Retrieved from https://www.amstat.org/policy/pdfs/ASA_VAM_Statement.pdf

Baker, E. L., et al. (2010). Problems with the Use of Student Test Scores to Evaluate Teachers. Washington, DC: Economic Policy Institute. Retrieved from epi.3cdn.net/b9667271ee6c154195_t9m6iij8k.pdf

Betebenner, D. (2008).  A primer on student growth percentiles. Dover, NH: National Center of the Improvement of Educational Assessment.

Coleman, J. (1966). Equality of educational opportunity. Washington, D.C.: U.S. Government Printing Office.

Franco, M. S., & Seidel, K. (2012). Evidence for the need to more closely examine school effects in value-added modeling and related accountability policies. Education and Urban Society, 44(1).

Gong, B., Perie, M., and Dunn, J. (2006). Using student longitudinal growth measures for school accountability under No Child Left Behind: An update to inform design decisions. Center for Assessment. Available online at: http://www.nciea.org/publications/GrowthModelUpdate_BGMAPJD07.pdf

Heubert, J.P., & Hauser, R.M. (1999). High stakes: Testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.

Hout, M. and Elliott, S., eds. (2011). Incentives and Test-Based Accountability in Education. National Research Council of the National Academies of Science. http://books.nap.edu/openbook.php?record_id=12521

Jones, B., and R. Egley. 2004. Voices from the frontlines: Teachers’ perceptions of high-stakes testing. Education Policy Analysis Archives 12(39).

Jones, M. G., B. D. Jones, B. Hardin, L. Chapman, T. Yarbrough, and M. Davis. 1999. The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan 81(3): 199-203.

Rivkin. S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417-458.

Schochet, P. Z., & Chiang, H. S. (2010). Error rates in measuring teacher and school performance based on student test score gains (NCEE 2010-4004). Washington, DC: National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences, United States Department of Education.

Stevens, J., & Zvoch, K. (2006). Issues in the implementation of longitudinal growth models for student achievement. In R. W. Lissitz (Ed.) Longitudinal and value added models of student performance (pp. 170-209). Maple Grove, MN: JAM Press.

Mythbusting Monday – High school exit exams and “college readiness”


mythbusting

Mythbusting Monday is a series of blog posts where we invite different faculty members from the College of Education to discuss education myths and how or what we are doing to combat the myth, fix the myth, etc.  We want our faculty to have a voice in busting these myths and we hope by making these issues personal we can promote and provoke conversations.  We will reference the book 50 Myths and Lies That Threaten America’s Public Schools: The Real Crisis in Education by David Berliner, Gene V Glass and Associates, 2014.

Myth: #45  

High school exit exams guarantee that our graduates will be “college ready” and prepared to succeed as workers in a global economy.

Contributors: Dr. Susan Adams, Dr. Catherine Pangan, Dr. Katie Brooks, Dr. Arthur Hochman, Lindsay Williams, Cassandra Pixey

How do college admissions teams know which high school graduates are most ready for success in college? It might surprise you to know that cumulative high school GPA-what most people consider to be subjective, unreliable data-is the strongest predictor of college success. Standardized test scores are much less predictive of college readiness and success than teacher-made assessments. Why is this? Classroom teachers use multiple measures, whether homework, projects, participation, and a wide variety of summative and formative assessments, to measure student learning. In the College of Education at Butler University, our pre-service teachers develop critical thinking expertise, collaboration skills, and creativity that serve them well in the classroom as they design meaningful and engaging learning and assessment opportunities for K-12 students.

In our laboratory school partnerships at IPS Shortridge and IPS Butler-Lab School 60, we are seeing the power and impact K-12 students communicate through project-based assessment, multimedia documentation panels, and inquiry-based group. We assess our teacher candidates using similar measures: portfolios, credential files, and video analysis of our candidates teaching with local students. A standardized assessment could never tell the full story of what students know, are able to do, and are ready to do. Good assessment practices involve the extensive use of formative and summative assessments that students produce over time.

In our global economy, project-based and inquiry-focused assessments better measure the knowledge, dispositions, and skills required in college and in the workplace than do multiple choice and short answer standardized assessments.  The marketplace has told us that our approach to assessment and teacher education is what is necessary and needed, by hiring our teacher education, school counseling, and administrative graduates.  Our students learn a wide variety of assessments so that their toolbox is as full and comprehensive as possible.  We need to tell the full story of each child, classroom, school and district through authentic assessment that is as equally varied, deep and rich. Every student deserves the opportunity to learn from a teacher who is highly skilled and well prepared to teach these creative, collaborative and project-based skills as preparation for participation in this global economy.

Mythbusting Monday – Class size

mythbusting

Mythbusting Monday is a series of blog posts where we invite different faculty members from the College of Education to discuss education myths and how or what we are doing to combat the myth, fix the myth, etc.  We want our faculty to have a voice in busting these myths and we hope by making these issues personal we can promote and provoke conversations.  We will reference the book 50 Myths and Lies That Threaten America’s Public Schools: The Real Crisis in Education by David Berliner, Gene V Glass and Associates, 2014.

Myth #17:  Class size does not matter; reducing class sizes will not result in more learning
Contributor:  Erin Garriott, M.S., Instructor

As I sit here and watch my 12 year-old daughter play with friends in a hotel swimming pool, I’m wondering who in their right mind thinks that class size doesn’t matter. I know my daughter is safe in that pool, she’s been a confident swimmer since she was five years old. I also know that I’d feel much more at ease if there weren’t about 300 other people (at least that’s what it seemed like at times) in that very same pool of water as her. It’s hard to keep track of her.

It’s the same in today’s classrooms. Everyday kids are swimming; some confident, some not so much. Some starting and finishing strong, while others are struggling to keep up. Some have parents (like me) who are sitting on the side of the pool keeping watch ready to help if needed, some kids’ parents are no where to be found or are splitting their attention with their other children. With so many kids, it’s hard to keep track of each one.

Here in the College of Ed, we model and celebrate the impact that relationships have on success in our classrooms. Those relationships are built during meaningful class discussions and activities. And those relationships grow stronger because our class sizes are small. We get the chance to know each other, to care about each other. Most importantly, though, we don’t lose track of each other.