Instead of using the traditional Caracal format, teachers of the course Face-to-Face Communication piloted a playful alternative using colors, cards and conversations to gather student evaluations. This approach aimed to generate richer, more meaningful, and more varied input than the often-uninformative responses typically yielded by standard course evaluations.

Background information

Current practices of student evaluations of teaching have faced criticism. For instance, it is often claimed that student evaluations are unhelpful, since students may not be adequately equipped to assess teaching and quality of education is hard to measure on a five-point scale. At Utrecht University, the Caracal course evaluations have been criticized for low response rates and unconstructive student input.

Project description

Therefore, this project proposed an alternative form of evaluation to contribute to teacher professionalization and educational improvement. Drawing on literature highlighting the value of ecological and dialogic approaches, students were asked to draw their reflection on the lectures, tutorials, seminar teacher, and the course as a whole, or choose a fitting Dixit card per category. Afterwards, they discussed their drawings or cards in duo’s; those conversations were recorded and analyzed.

Aims

This project aimed to answer the following research questions:

• How does an alternative student evaluation (using colors, cards and conversations) work out?
• How do students and teachers value this alternative student evaluation?
• How does the input gained by these alternative student evaluations compare to input gained from Caracal evaluations?

Results & Conclusion

The experiment with drawings and Dixit cards yielded valuable conversations about the four subjects in course evaluations (lecture, tutorial, teacher, course as a whole). As a result of the open and associative nature of this method, students were less guided by predetermined questions and could share what was important to them. This provided teachers with richer information than Caracal would provide. For instance, the evaluations included more nuance, glimpses of students’ experiences, expressions of personal appreciation for the teacher and explanations for evaluations. The method can be very suitable for teacher professionalization, as it helped teachers gain insight into (students’ experiences with) their teaching methods, and gave them more targeted information about their teaching.

However, some students found it difficult to draw their experiences and feelings, and/or translate the drawings into words. The Dixit cards were a nice option for those who did not want to draw, but this is different than drawing. Moreover, this evaluation method is more privacy-sensitive than Caracal evaluations. Finally, the evaluation form elicited only limited reflection from students on their own role in the learning process.

Although this evaluation form offers valuable information, interpreting and processing that information is costly. More cost-effective adaptations for various purposes can be found via these links:

• Creative evaluations allow for richer, more nuanced feedback
• How students and teachers work together for better education

References

• De la Croix, A., & Veen, M. (2018). The reflective zombie: problematizing the conceptual framework of reflection in medical education. Perspectives on Medical Education, 7, 394-400.
• Esarey, J., & Valdes, N. (2020). Unbiased, reliable, and valid student evaluations can still be unfair. Assessment & Evaluation in Higher Education, 45(8), 1106-1120.
• Fawns, T., Aitken, G., & Jones, D. (2021). Ecological teaching evaluation vs the datafication of quality: Understanding education with, and around, data. Postdigital Science and Education, 3(1), 65-82.
• Scager, K., van Strien, J., & Lam, I. (2019). Laat de beoordeling van de docent maar (niet) over aan de student. TH&MA Hoger Onderwijs, 53-57.
• Van der Schoot, M. (2020, juni). Een onderwijsevaluatie past niet in een vragenlijst. Te lezen via https://www.scienceguide.nl/2020/06/een-onderwijsevaluatie-past-niet-in-een-vragenlijst/