The Effect of WhatsApp-Integrated Meta AI Chatbot on Vocational Students' Critical Thinking Skills in the Automotive Engineering Basic Subjects

Sumardi Sumardi; Yogi Yunefri; Zamzami Zamzami; Rizki Novendra; Yogo Turnandes; Safrianto Safrianto

doi:10.35445/alishlah.v18i1.9452

The Effect of WhatsApp-Integrated Meta AI Chatbot on Vocational Students' Critical Thinking Skills in the Automotive Engineering Basic Subjects

Sumardi Sumardi, Yogi Yunefri, Zamzami Zamzami, Rizki Novendra, Yogo Turnandes, Safrianto Safrianto

Abstract

The integration of artificial intelligence (AI) into vocational education offers new opportunities to enhance students’ higher-order thinking skills, particularly critical thinking. However, empirical evidence on the effectiveness of AI chatbots in supporting critical thinking within vocational automotive learning remains limited. This study investigates the effect of a WhatsApp-integrated Meta AI chatbot on vocational students’ critical thinking skills in the Automotive Engineering Basic Subjects. This research employed a quantitative quasi-experimental design with a non-equivalent control group. The participants were 40 tenth-grade vocational high school students divided into an experimental group (n = 20) and a control group (n = 20). The experimental group used a Meta AI chatbot integrated with WhatsApp as a learning companion during four instructional sessions, while the control group received conventional instruction. Data were collected using a validated critical thinking test measuring analysis, evaluation, and inference skills. Statistical analysis included descriptive statistics, assumption tests, independent samples t-tests, N-gain analysis, and simple linear regression. The results indicate a significant difference in critical thinking scores between the experimental and control groups (t(38) = 2.466, p = 0.018). The effect size was categorized as medium-to-large (Cohen’s d = 0.78). The experimental group also showed a moderate improvement (N-gain = 0.45). Regression analysis revealed that the use of the Meta AI chatbot explained 18% of the variance in students’ critical thinking performance (R² = 0.18, p = 0.029). These findings suggest that integrating AI chatbots through widely used communication platforms such as WhatsApp can support the development of critical thinking in vocational education, particularly in technical problem-solving contexts.

Keywords

artificial intelligence in education; educational chatbot; WhatsApp-assisted learning; critical thinking skills; vocational education

Full Text:

PDF

References

Afriani, F. (2024). Impact analysis of educational chatbot use in supporting students' online learning. Journal of Educational Technology Innovation, 11 (2), 145–158. https://doi.org/10.21831/jitp.v11i2

Almohammadi, K., Hagras, H., Alghazzawi, D., & Aldabbagh, G. (2023). A survey of artificial intelligence techniques employed for adaptive educational systems. Computers & Education: Artificial Intelligence, 4 , 100108. https://doi.org/10.1016/j.caeai.2023.100108

Ansari, M., Alas, Y., Yunus, N., Saturday, NP, & Hamid, MH (2022). Online learning: Trends, issues, and challenges in the era of COVID-19. Education and Information Technologies, 27 (1), 1–24. https://doi.org/10.1007/s10639-021-10709-0

Baskara, R. (2023). Investigating the impact of chatbots in different learning contexts on student engagement and critical thinking. Journal of English Teaching for Young and Adult Learners, 5 (2), 89–102. https://ejournal.stkippacitan.ac.id/ojs3/index.php/jeeyal/article/view/689

Crompton, H., Bernacki, M., & Greene, J. A. (2022). Psychological foundations of emerging technologies for teaching and learning. Educational Psychologist, 57 (3), 161–176. https://doi.org/10.1080/00461520.2022.2073813

Deng, L., & Yu, T. (2023). Roles and research trends of artificial intelligence in education: A bibliometric analysis. Interactive Learning Environments, 31 (4), 1–17. https://doi.org/10.1080/10494820.2021.195403 8

Fidan, O., & Gencel, İ. E. (2022). Chatbot-assisted discussions promote critical thinking and knowledge sharing. Education Sciences, 12 (10), 676. https://doi.org/10.3390/educsci12100676

Hake, R.R. (2022). Interactive-engagement methods in introductory physics: Update and meta-analysis. American Journal of Physics, 90 (2), 94–107. https://doi.org/10.1119/10.0009380

Hwang, G. J., & Tu, Y. F. (2023). Roles and research trends of artificial intelligence in education. Interactive Learning Environments, 31 (4), 1–17. https://doi.org/10.1080/10494820.2021.1954038

Kurniawan, D., & Suryadi, A. (2023). Developing critical thinking skills in vocational high school students through digital technology-based learning. Journal of Vocational Education, 13 (1), 65–78. https://doi.org/10.21831/jpv.v13i1

Luckin, R., Holmes, W., Griffiths, M., & Forcier, L. B. (2022). Intelligence unleashed: An argument for AI in education . Pearson Education.

Sari, RP, & Nugroho, A. (2024). Utilization of AI-based chatbots to improve students' higher-order thinking skills. Journal of Educational Technology, 26 (2), 201–214. https://doi.org/10.21009/jtp.v26i2

Schmid, R.F., Bernard, R.M., Borokhovski, E., Tamim, R.M., Abrami, P.C., Surkes, M.A., Wade, C.A., & Woods, J. (2022). Technology-supported personalization in education: A meta-analysis. Educational Research Review, 36 , 100456. https://doi.org/10.1016/j.edurev.2022.100456

Sugiyono. (2022). Quantitative, qualitative, and R&D research methods (Revised edition). Alfabeta.

UNESCO. (2023). Guidance on generative AI in education and research . UNESCO Publishing.

Yusuf, M., & Widodo, A. (2023). Integration of WhatsApp as an interactive learning medium in vocational high schools. Al-Ishlah: Journal of Education, 15 (2), 233–245. https://doi.org/10.35445/alishlah.v15i2

DOI: https://doi.org/10.35445/alishlah.v18i1.9452