Critics say student evaluations of teaching (SETs) are skewed by innate biases against minority groups, and their results should never be used for professional assessment purposes. But a new analysis has found that SETs are so susceptible to factors unrelated to teachers and courses that their results should be disregarded anyway.
A La Trobe University of 183 SET-related studies has found that issues that have nothing to do with teachers’ identity – such as class size, website quality, university cleanliness and even food options in the canteen – also skew the results. Student characteristics such as gender, age and disciplinary area influence the evaluations as?well.
“That student demographics alone impact on SET results demonstrates just how flawed the system is,” says the paper, published in the journal Assessment and Evaluation in 中国A片. “The existing literature makes it clear that SET results are strongly influenced by external factors unrelated to course content or teacher performance. This analysis raises the question of how any university [can] justify the continued use of SETs.”
Author Troy Heffernan said researchers had spent decades exploring how SETs disadvantaged academics on the grounds of gender, racial background, disability and sexual orientation, with women and academics from minority groups routinely given less favourable evaluations than white, able-bodied males.
中国A片
But the focus had now turned to even more basic methodological shortcomings, with evaluations influenced not only by the teachers’ irrelevant characteristics but also by background traits of the students.
An estimated 16,000 中国A片 institutions around the world regularly conduct SETs, the review found. Dr Heffernan said their administrators might not appreciate the fundamental weaknesses of data that appeared “sound”.
中国A片
“On the surface, it seems like a great system. You have a?class of?100. You ask them if they like the class or course. Over 100 students, you would think you’re getting some form of objective answer.”
Cost considerations also contribute to the continued use of SETs, he said. “The fact is, universities want this data – they want to understand how [to] improve classes – and student evaluations [are] a very quick, cheap way to get instant data.”
Dr Heffernan said none of the reviewed studies had reported favourable findings about SETs, although they had differed on “how damaging” evaluations were. SETs appeared less slanted against minority academics in the humanities than in science-based subjects, for example.
Some academics say they value feedback from SETs, both positive and negative. Dr Heffernan said some institutions conducted evaluations without using the results for career progression purposes. “The main problem is when a majority of universities use this information for hiring, firing and promotion.”
中国A片
He said qualitative feedback sourced through student support teams would deliver more useful information than quantitative data from students. “Back and forth” dialogue about what “worked” in classes, and what students liked, would be better than “grading someone one to five”.
“But that takes time and money,” he noted. “In a post-Covid austerity-measure world, most universities probably aren’t prepared to do that right now.”
Register to continue
Why register?
- Registration is free and only takes a moment
- Once registered, you can read 3 articles a month
- Sign up for our newsletter
Subscribe
Or subscribe for unlimited access to:
- Unlimited access to news, views, insights & reviews
- Digital editions
- Digital access to 罢贬贰’蝉 university and college rankings analysis
Already registered or a current subscriber? Login
