Bias in Peer Assessment: Challenges, Solutions, and Best Practices for Fair Student Evaluation

Mohammad Ridwan; Ahmad Najib; Hasan Ruzakki; Rofiqi Rofiqi; Muslimin Muslimin

doi:10.35445/alishlah.v17i2.7086

Bias in Peer Assessment: Challenges, Solutions, and Best Practices for Fair Student Evaluation

Mohammad Ridwan, Ahmad Najib, Hasan Ruzakki, Rofiqi Rofiqi, Muslimin Muslimin

Abstract

Bias in student evaluations is a persistent challenge in peer assessment, often leading to inaccurate or unfair outcomes. This study investigates strategies to minimize bias and enhance the objectivity and reliability of peer assessment practices in educational settings. A Systematic Literature Review (SLR) was conducted, analyzing 53 peer-reviewed journal articles published between 2019 and 2023. Articles were sourced using the Publish or Perish application, which extracts data from Google Scholar, selected for its extensive coverage of open-access academic sources. Inclusion criteria focused on article quality, thematic relevance to bias in peer assessment, and novelty of insights. Key databases such as Scopus and Web of Science were used to validate and cross-check influential studies. The review identified several effective strategies for minimizing bias in peer assessment. These include maintaining student anonymity, using structured rubrics, providing evaluative training, and incorporating teacher supervision. Each approach demonstrated significant improvements in the fairness and accuracy of assessments across diverse educational contexts. Compared to previous literature, this study offers a more comprehensive synthesis of bias-reduction techniques and emphasizes their practical applicability. By adopting these strategies, educators can improve the validity of peer feedback, fostering more equitable assessment environments and promoting student growth through reflective learning.

Keywords

peer assessment; bias; fair student evaluation

Full Text:

PDF

References

Achadah, A. (2019). Evaluasi Dalam Pendidikan Sebagai Alat Ukur Hasil Belajar. An-Nuha : Jurnal Kajian Islam, Pendidikan, Budaya Dan Sosial, 6(1), 97–114. https://doi.org/10.36835/annuha.v6i1.296

Hosaini, H., Kandiri, K., Minhaji, M., & Alehirish, M. H. M. (2024). Human Values Based on Pancasila Viewed from Islamic Education. Al-Hayat: Journal of Islamic Education, 8(2), 539-549.

Adachi, C., Hong-Meng Tai, J., & Dawson, P. (2018). Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education. Assessment and Evaluation in Higher Education, 43(2), 294–306. https://doi.org/10.1080/02602938.2017.1339775

Arifin, S., Chotib, M., Rahayu, N. W. I., Hosaini, H., & Samsudi, W. (2024). Kiai's Transformative Leadership in Developing an Organizational Culture of Islamic Boarding Schools: Multicase Study. AL-ISHLAH: Jurnal Pendidikan, 16(2), 2608-2620.

Afsari, S., Safitri, I., Harahap, S. K., & Munthe, L. S. (2021). Systematic Literature Review: Efektivitas Pendekatan Pendidikan Matematika Realistik Pada Pembelajaran Matematika. Indonesian Journal of Intellectual Publication, 1(3), 189–197. https://doi.org/10.51577/ijipublication.v1i3.117

Qomar, M., & Fitri, A. Z. (2024). Innovative Learning Strategies for Islamic Religious Education Based on Merdeka Belajar Curriculum in Vocational High Schools. Al-Hayat: Journal of Islamic Education, 8(3), 966-981.

Amodio, D. M., & Devine, P. G. (2019). Stereotyping and evaluation in implicit race bias: Evidence for independent constructs and unique effects on behavior. Journal of Personality and Social Psychology, 91(4), 652–661. https://doi.org/10.1037/0022-3514.91.4.652

Minhaji, M., Hosaini, H., Prasetiyo, N. T., Maktumah, L., & Alehirish, M. H. M. (2024). Responsive Islamic Education in Exploring Social Values Through the War Takjil Phenomenon: Sociological Perspective in Indonesia. Jurnal Indo-Islamika, 14(1), 51-61.

Hosaini, H., Fitri, A. Z., Kojin, K., & Alehirish, M. H. M. (2024). The Dynamics of the Islamic Education System in Shaping Character. Edukasia: Jurnal Penelitian Pendidikan Islam, 19(1), 79-98.

Awacorach, J., Jensen, I., Lassen, I., Olanya, D. R., Zakaria, H. L., & Tabo, G. O. (2021). Exploring Transition in Higher Education: Engagement and Challenges in Moving from Teacher-Centered to Student-Centered Learning. Journal of Problem Based Learning in Higher Education, 9(2), 113–130. https://doi.org/10.5278/ojs.jpblhe.v9i2.5262

Cahyono, C., Judijanto, L., Hutahaean, E. S. H., Nisa, U. W., Mulyadi, M., & Hosaini, H. (2024). Pesantren Education as Indonesia's Indigenous Heritage: Nurturing Moral Education in the Digital Era. At-Ta'dib, 19(1), 177-193.

Supriadi, S., Hosaini, H., & Sain, Z. H. (2024). Transformation of Literacy-Based Islamic Education Learning Management Integration in Elementary Schools. International Journal of Social Learning (IJSL), 5(1), 294-304.

Hosaini, H., Qomar, M., Kojin, K., & Sibilana, A. R. (2024). Integration of School Curriculum and Islamic Boarding Schools in Preparing the Golden Generation with holistic intelligence. In SHS Web of Conferences (Vol. 205, p. 03001). EDP Sciences.

Ayuningtyas, D., Praherdhiono, H., & Wedi, A. (2018). Aplikasi Web Evaluasi Proses Pembelajaran Dengan Pendekatan Assessmen As Learning. Jktp, 1(1), 77–84.

Bauer, C. C., & Baltes, B. B. (2022). Reducing the effects of gender stereotypes on performance evaluations. Sex Roles, 47(9–10), 465–476. https://doi.org/10.1023/A:1021652527696

Carlsson Hauff, J., & Nilsson, J. (2022). Students’ experience of making and receiving peer assessment: the effect of self-assessed knowledge and trust. Assessment and Evaluation in Higher Education, 47(6), 959–971. https://doi.org/10.1080/02602938.2021.1970713

Del Pozo García, A. (2020). Raising awareness on assessment criteria through peer-assessment and self-reflection in the Spanish oral class. Five Years of ELEUK Conferences: A Selection of Short Papers from 2019, 2019(2020), 79–88. https://doi.org/10.14705/rpnet.2020.41.1077

Double, K. S., McGrane, J. A., & Hopfenbeck, T. N. (2020). The Impact of Peer Assessment on Academic Performance: A Meta-analysis of Control Group Studies. Educational Psychology Review, 32(2), 481–509. https://doi.org/10.1007/s10648-019-09510-3

Eaton, A. A., Saunders, J. F., Jacobson, R. K., & West, K. (2020). How Gender and Race Stereotypes Impact the Advancement of Scholars in STEM: Professors’ Biased Evaluations of Physics and Biology Post-Doctoral Candidates. Sex Roles, 82(3–4), 127–141. https://doi.org/10.1007/s11199-019-01052-w

El-Senousy, H. (2020). How peer assessment could be interactive and effective. South African Journal of Education, 40(2), 1–14. https://doi.org/10.15700/SAJE.V40N2A1651

Elmedina Nikoçeviq-Kurti. (2022). European Journal of Educational Research. European Journal of Educational Research, 11(3), 1245–1257. Retrieved from https://www.researchgate.net/profile/Suntonrapot-Damrongpanit/publication/356662582_Effects_of_Mindset_Democratic_Parenting_Teaching_and_School_Environment_on_Global_Citizenship_of_Ninth-grade_Students/links/61a6dda685c5ea51abc0f7b6/Effects-of-Mindset-Dem

Fathullah, H., & Sunarti. (2018). Kemampuan Guru Mendesain Dan Mengimplimentasikan Penilian Otentik Pada Pembelajaran IPS. Sosialita, 9(1), 27–40.

Golightly, A. (2021). Self- and peer assessment of preservice geography teachers’ contribution in problem-based learning activities in geography education. International Research in Geographical and Environmental Education, 30(1), 75–90. https://doi.org/10.1080/10382046.2020.1744242

Guo, X., & Lei, P. W. (2020). Effect of Quality Characteristics of Peer Raters on Rating Errors in Peer Assessment. International Journal of Testing, 20(3), 206–230. https://doi.org/10.1080/15305058.2020.1720216

Hidayat, A. (2018). Meta Analisis: Pentingnya Self Dan Peer Assesment Dalam Pembelajaran. Jurnal Basicedu, 2(1), 95–101. https://doi.org/10.31004/basicedu.v2i1.30

Hoang, L. P., Le, H. T., Van Tran, H., Phan, T. C., Vo, D. M., Le, P. A., … Pong-inwong, C. (2022). Does evaluating peer assessment accuracy and taking it into account in calculating assessor’s final score enhance online peer assessment quality? Education and Information Technologies, 27(3), 4007–4035. https://doi.org/10.1007/s10639-021-10763-1

Houser, K. W. (2022). Fairness and Bias in Peer Review: Anonymity, Open Science, and Preprints. LEUKOS - Journal of Illuminating Engineering Society of North America, 18(4), 415–416. https://doi.org/10.1080/15502724.2022.2103367

Hwang, G. J., & Chang, S. C. (2021). Facilitating knowledge construction in mobile learning contexts: A bi-directional peer-assessment approach. British Journal of Educational Technology, 52(1), 337–357. https://doi.org/10.1111/bjet.13001

Iezzoni, L. I. (2018). Explicit Disability Bias in Peer Review. Medical Care, 56(4), 277–278. https://doi.org/10.1097/MLR.0000000000000889

Jia, Q., Cui, J., Xiao, Y., Liu, C., Rashid, P., & Gehringer, E. F. (2021). ALL-IN-ONE: Multi-Task Learning BERT models for Evaluating Peer Assessments. (Edm), 525–532. Retrieved from http://arxiv.org/abs/2110.03895

Jundi, M., & Yasin, Z. (2020). Penilaian Sejawat dalam Pembelajaran Keterampilan Dasar Mengajar bagi Mahasiswa Pendidikan Bahasa Arab pada Mata Kuliah Pembelajaran Mikro. Maharat: Jurnal Pendidikan Bahasa Arab, 2(2), 51–70. https://doi.org/10.18196/mht.2217

Kaewsaiha, P., & Chanchalor, S. (2020). The development of an online peer assessment tool using google applications. ACM International Conference Proceeding Series, 5–9. https://doi.org/10.1145/3429630.3429638

Kartikarini, N., & Sugiarto. (2016). Pengaruh Gender, Keahlian dan Skeptisisme Profesional terhadap Kemampuan Auditor Mendeteksi Kecurangan (Studi pada Badan Pemeriksa Keuangan Republik Indonesia). Simposium Nasional Akuntansi, 19, 1–31.

Lerchenfeldt, S., & Taylor, T. A. (2020).

Best Practices in Peer Assessment: Training Tomorrow’s Physicians to Obtain and Provide Quality Feedback

. Advances in Medical Education and Practice, Volume 11, 571–578. https://doi.org/10.2147/amep.s250761

Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. Assessment and Evaluation in Higher Education, 45(2), 193–211. https://doi.org/10.1080/02602938.2019.1620679

Lorenzetti, N. L. (n.d.). It Starts in the Classroom : The Relationship Between Teacher Education Students ’ Implicit Racial Biases and Classroom Disciplinary Decisions. 31(1).

Magin, D. (2021). Reciprocity as a Source of Bias in Multiple Peer Assessment of Group Work. Studies in Higher Education, 26(1), 53–63. https://doi.org/10.1080/03075070020030715

Manzoor, E., & Shah, N. B. (2021). Uncovering Latent Biases in Text: Method and Application to Peer Review. 35th AAAI Conference on Artificial Intelligence, AAAI 2021, 6A, 4767–4775. https://doi.org/10.1609/aaai.v35i6.16608

May, G. L. (2018). The effect of rater training on reducing social style bias in peer evaluation. Business Communication Quarterly, 71(3), 297–313. https://doi.org/10.1177/1080569908321431

Meixiu Lu, M. M. C. (2022). Do Teamwork Guidelines Improve Peer Assessment Accuracy or Attitudes During Collaborative Learning? IEEE Transactions on Education, 65(4). Retrieved from 10.1109/TE.2021.3130242

Meletiadou, E. (2021). The Use of Peer Assessment as an Inclusive Learning Strategy in Higher Education Institutions. 1–26. https://doi.org/10.4018/978-1-7998-8579-5.ch001

Mohammed, S. S., & Baysen, E. (2022). Peer Assessment of Curriculum Content of Group Games in Physical Education: A Systematic Literature Review of the Last Seven Years. Sustainability (Switzerland), 14(11). https://doi.org/10.3390/su14116561

Mumpuni, K. E., Priyayi, D. F., & Widoretno, S. (2022). How do Students Perform a Peer Assessment? International Journal of Instruction, 15(3), 751–766. https://doi.org/10.29333/iji.2022.15341a

Murdianto. (2018). Stereotipe , Prasangka dan Resistensinya ( Studi Kasus pada Etnis Madura dan Tionghoa di Indonesia ). Qalamuna, 10(2), 137–160.

Nejad, A. M., & Mahfoodh, O. H. A. (2019). Assessment of oral presentations: Effectiveness of self-, peer-, and teacher assessments. International Journal of Instruction, 12(3), 615–632. https://doi.org/10.29333/iji.2019.12337a

Ng, W. S., & Yu, G. (2023). The Impacts of Dialogic Interaction to Engage Students in Peer Assessment. Asia-Pacific Education Researcher, 32(1), 53–64. https://doi.org/10.1007/s40299-021-00633-2

Panadero, E., & Alqassab, M. (2019). An empirical review of anonymity effects in peer assessment, peer feedback, peer review, peer evaluation and peer grading. Assessment and Evaluation in Higher Education, 44(8), 1253–1278. https://doi.org/10.1080/02602938.2019.1600186

Rezai, A., Namaziandost, E., Miri, M., & Kumar, T. (2022). Demographic biases and assessment fairness in classroom : insights from Iranian university teachers.

Shah, N. B. (2022). The Role of Author Identities in Peer Review. ArXiv Preprint ArXiv:2301.00221, 1–13.

Sridharan, B., Tai, J., & Boud, D. (2019). Does the use of summative peer assessment in collaborative group work inhibit good judgement? Higher Education, 77(5), 853–870. https://doi.org/10.1007/s10734-018-0305-7

Stelmakh, I., Shah, N. B., & Singh, A. (2019). On testing for biases in peer review. Advances in Neural Information Processing Systems, 32(i).

Suardipa, I. P., & Primayana, K. H. (2020). Peran Desain Evaluasi Pembelajaran Untuk Meningkatkan Kualitas Pembelajaran. Widyacarya, 4(2), 88–100. Retrieved from http://jurnal.stahnmpukuturan.ac.id/index.php/widyacarya/article/view/796

Tamblyn, R., Girard, N., Qian, C. J., & Hanley, J. (2018). Assessment of potential bias in research grant peer review in Canada. Cmaj, 190(16), E489–E499. https://doi.org/10.1503/cmaj.170901

Teplitskiy, M., Acuna, D., Elamrani-Raoult, A., Körding, K., & Evans, J. (2018). The sociology of scientific validity: How professional networks shape judgement in peer review. Research Policy, 47(9), 1825–1841. https://doi.org/10.1016/j.respol.2018.06.014

Triandini, E., Jayanatha, S., Indrawan, A., Werla Putra, G., & Iswara, B. (2019). Metode Systematic Literature Review untuk Identifikasi Platform dan Metode Pengembangan Sistem Informasi di Indonesia. Indonesian Journal of Information Systems, 1(2), 63. https://doi.org/10.24002/ijis.v1i2.1916

Tuzcu-Eken, D. (2021). Peer Evaluation in Writing: How to Implement Efficiently. International Online Journal of Education and Teaching, 8(2), 691–708. Retrieved from https://www.proquest.com/scholarly-journals/peer-evaluation-writing-how-implement-efficiently/docview/2540365459/se-2?accountid=15292%0Ahttps://bibcercador.uab.cat/openurl/34CSUC_UAB/34CSUC_UAB:VU1?genre=article&atitle=Peer+Evaluation+in+Writing%3A+How+to

Wijayanti, A. (2022). Efektivitas Self Assessment dan Peer Assessment Dalam Pembentukan Karakter Siswa. Realita : Jurnal Penelitian Dan Kebudayaan Islam, 15(2), 1895–1898. https://doi.org/10.30762/realita.v15i2.482

Wijayanti, E., & Mundilarto, M. (2019). Penembangan Istrumen Asesmen Diri Dan Teman Sejawat Kompetensi Bidang Studi Pada Mahasiswa. Jurnal Penelitian Dan Evaluasi Pendidikan, 19(2), 129–144. https://doi.org/10.21831/pep.v19i2.5572

Wride, M. (2017). Guide to peer-assessment. Academic Practice and ELearning (CAPSL) Resources, 7(4), 1–15. Retrieved from https://www.tcd.ie/CAPSL/assets/pdf/Academic Practice Resources/Guide to Student Peer Assessment.pdf

Yesil Cinar, S., & Sata, M. (2021). Examining Rater Biases of Peer Assessors in Different Assessment Environments. International Journal of Psychology and Educational Studies, 8(4), 136–151. https://doi.org/10.52380/ijpes.2021.8.4.622

Yuliasari, H. (2020). Pelatihan Konselor Sebaya Untuk Meningkatkan Self Awareness Terhadap Perilaku Beresiko Remaja. Jurnal Psikologi Insight, 4(1), 63–72. https://doi.org/10.17509/insight.v4i1.24638

Zhan, Y. (2021). What matters in design? Cultivating undergraduates’ critical thinking through online peer assessment in a Confucian heritage context. Assessment and Evaluation in Higher Education, 46(4), 615–630. https://doi.org/10.1080/02602938.2020.1804826

Zubaidah, S. (2020). Keterampilan Abad Ke-21: Keterampilan yang Diajarkan Melalui Pembelajaran. Online. (December 2016).

DOI: https://doi.org/10.35445/alishlah.v17i2.7086