Development and Validation of a Critical Thinking Test Instrument on Acid-Base for Senior High School Students

M. Raidil; Agustina Simorangkir

doi:10.24114/jq6ceh75

Authors

M. Raidil Cenderawasih University
Agustina Simorangkir Cenderawasih University

DOI:

https://doi.org/10.24114/jq6ceh75

Keywords:

critical thinking; evidence-based assessment; two-tier multiple-choice; Rasch model; chemistry assessment

Abstract

Critical thinking is an essential competency in chemistry learning, yet its assessment requires valid and reliable instruments capable of measuring students’ reasoning beyond answer selection. This study aimed to develop and validate an evidence-based critical thinking test instrument for senior high school students on acid-base chemistry. The study employed a Research and Development design using the 4D model consisting of Define, Design, Develop, and Disseminate stages. The instrument was developed as a 20-item open-ended two-tier multiple-choice test integrating five critical thinking indicators: interpretation, analysis, inference, evaluation, and self-regulation. Content validity was evaluated by three expert validators, while empirical validation involved 86 senior high school students in Jayapura. Item characteristics were analyzed using the Item Response Theory approach with QUEST software. Expert validation resulted in an overall validity score of 96.11%, categorized as very valid. Empirical analysis showed that 12 of 20 items met the psychometric criteria, with INFIT MNSQ values ranging from 0.74 to 1.26 and person reliability of 0.87. Eight items produced zero scores and were excluded from calibration. The findings indicate that the developed instrument demonstrates strong psychometric quality and can be used to assess students’ critical thinking in acid-base chemistry.

References

Adedoyin, O. O., & Mokobi, T. (2013a). Using IRT psychometric analysis in examining the qulity of junior certificate mathematics multiple choice examination test items. International Journal of Asian Social Science, 3(4), 992–1011.

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. doi.org

Anisah, D. C., Nasrudin, H., & Sukarmin. (2025). Profile of critical thinking skills of phase F learning in chemistry subjects acid-base material. Jurnal Penelitian Pendidikan IPA, 11(1), 17–27. https://doi.org/10.29303/jppipa.v11i1.9842.

Arrang, M. L., Pongkendek, J. J., & Sumanik, N. B. (2024). Pengembangan instrumen diagnostik two-tier multiple choice untuk mengidentifikasi miskonsepsi pada materi senyawa hidrokarbon. Arfak Chem: Chemistry Education Journal, 7(2), 623–632. https://doi.org/10.30862/accej.v7i2.762.

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (3rd ed.). Routledge.

Bortolotti, S., Di Cagno, D. T., Galliera, A., & Sternieri, M. (2013). Cognitive load and impatience: An experimental study. Alma Mater Studiorum Università di Bologna Working Paper.

Danczak, S., Thompson, C., & Overton, T. (2017). What does the term critical thinking mean to you? A qualitative analysis of chemistry undergraduate, teaching staff and employers’ views of critical thinking. Chemistry Education Research and Practice, 18(3), 420–434. doi.org

De Ayala, R. J. (2013). The theory and practice of item response theory. Guilford Publications.

Djafar, Y., Ishak, I., & Tangio, J. S. (2024). Analisis kemampuan berpikir kritis siswa dalam menyelesaikan instrumen tes two-tier multiple choice pada materi laju reaksi. Jurnal Inovasi Pendidikan Kimia, 18(2), 112–117. https://doi.org/10.15294/f677x032

Ennis, R. H. (2011). Critical thinking: A stream of reasoning. Pearson Education.

Facione, P. A. (2015). Critical thinking: What it is and why it counts. Insight Assessment.

Fadillah, E. N. (2017). Development of assessment instruments to measure the science process skills of high school students [Pengembangan instrumen penilaian untuk mengukur keterampilan proses sains siswa SMA]. Didaktika Biologi: Jurnal Penelitian Pendidikan Biologi, 1(2), 123–134.

Fulmer, G. W., Chu, H.-E., Treagust, D. F., & Neumann, K. (2015). Is it harder to know or to reason? Analyzing two-tier science assessment items using the Rasch measurement model. Asia-Pacific Science Education, 1(1), Article 5. https://doi.org/10.1186/s41029-015-0005-x

Haja, S., & Clarke, D. (2011). Middle school students’ responses to two-tier tasks. Mathematics Education Research Journal, 23(1), 67–76. https://doi.org/10.1007/s13394-011-0004-5

Ibrahim, A.-W. (2023). Reconsidering the validity of zero score's grading practices: Imperatives of paradigm shift in assessment strategies in Nigerian universities. Journal of Assessment and Evaluation, 42, 1–10.

Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of classical test theory and item response theory in individual change assessment. Applied Psychological Measurement, 40(8), 559–572. https://doi.org/10.1177/0146621616664046

Johnstone, A. H. (2000). Teaching of chemistry – Logical or psychological? Chemistry Education Research and Practice, 1(1), 9–15.

Purwanto, A., Rahmawati, Y., Rahmayanti, N., Mardiah, A., & Putri, R. A. (2022). Socio-critical and problem-oriented approach in environmental issues for students' critical thinking skills development in chemistry learning. Journal of Technology and Science Education, 12(1), 50–67. https://doi.org/10.3926/jotse.1341

Kohen, Z., Herscovitz, O., & Dori, Y. J. (2020). How to promote chemical literacy? Online question posing and communicating with scientists. Chemistry Education Research and Practice, 21(1), 250–266. https://doi.org/10.1039/c9rp00134d

Laksono, P. J. (2019). Pengembangan dan penggunaan instrumen two-tier multiple choice pada materi termokimia untuk mengukur kemampuan berpikir kritis. Orbital: Jurnal Pendidikan Kimia, 2(2), 80–92. https://doi.org/10.19109/ojpk.v2i2.2646

Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions, 16(2), 878.

Mater, N. R., Haj Hussein, M. J., Salha, S. H., Draidi, F. R., Shaqour, A. Z., Qatanani, N., & Affouneh, S. (2020). The effect of the integration of STEM on critical thinking and technology acceptance model. Educational Studies, 1–17. https://doi.org/10.1080/03055698.2020.1793736

Mokkink, L. B., Herbelet, S., Tuinman, P. R., & Terwee, C. B. (2025). Content validity: Judging the relevance, comprehensiveness, and comprehensibility of an outcome measurement instrument – A COSMIN perspective. Journal of Clinical Epidemiology, 185, Article 111879. https://doi.org/10.1016/j.jclinepi.2025.111879

Mondal, B. C., & Chakraborty, A. (2013). Misconceptions in chemistry. LAP LAMBERT Academic Publishing.

Mutiawani, V., Athaya, A., Saputra, K., & Subianto, M. (2022). Implementing Item Response Theory (IRT) Method in Quiz Assessment System. TEM Journal, 210–218. https://doi.org/10.18421/TEM111-26

Nurhasanah, Hidayatullah, Z., Yansa, H., Arif, M. B. S., & Asriadi, M. (2025). Integrating Technology and Psychometrics: Developing an App-Based Diagnostic Tool Using the Partial Credit Model to Uncover Student Misconceptions. Jurnal Penelitian Pendidikan IPA, 11(12), 861–872. https://doi.org/10.29303/jppipa.v11i12.12305

OECD. (2019). PISA 2018 results (Volume I): What students know and can do. OECD Publishing.

Oktaviani, S. Z., & Harjono. (2025). Development of two-tier multiple-choice diagnostic test instrument based on science literacy on chemical equilibrium material. Chemistry in Education, 14(1), 134–144. https://doi.org/10.15294/chemined.v14i1.21813

Partnership for 21st Century Skills. (2019). Framework for 21st century learning. P21.

Putri, P. N., Rachmadiarti, F., Purnomo, T., & Satriawan, M. (2025). Measuring scientific literacy of students through environmental issues based on PISA 2025 science framework. Jurnal Penelitian Pendidikan IPA, 11(3), 44–53. https://doi.org/10.29303/jppipa.v11i3.10413.

Quinn, S., Hogan, M., Dwyer, C., Finn, P., & Fogarty, E. (2020). Development and validation of the Student-Educator Negotiated Critical Thinking Dispositions Scale (SENCTDS). Thinking Skills and Creativity, 38, 100710. https://doi.org/10.1016/j.tsc.2020.100710.

Raidil, M., Damris, Syahri, W., & Triansyah, F. A. (2023). Pengaruh model team assisted individualization dan self-efficacy terhadap kemampuan berpikir kritis siswa materi asam basa. Jurnal Ilmiah Ilmu Pendidikan, 6(3), 1747–1753.

Rahmawati, A. S., Hamidah, I., Samsudin, A., & Rochintaniawati, D. (2025). Review of relevant literature about critical thinking skills in a science perspective. Jurnal Penelitian Pendidikan IPA, 11(5), 56–67. https://doi.org/10.29303/jppipa.v11i5.9687

Sadhu, S., & Laksono, E. W. (2018). Development and validation of an integrated assessment for measuring critical thinking and chemical literacy in chemical equilibrium. International Journal of Instruction.

Simorangkir, A., & Raidil, M. (2025). High school chemistry learning: The correlation between students’ critical thinking and self-efficacy. Jurnal Pendidikan Matematika dan Sains, 13(Special Issue), 143–153. https://doi.org/10.21831/jpms.v13iSpecial_issue.88280

Sunarti, T., Wasis, W., Setyarsih, W., Zainudin, A., & Jauhariyah, M. N. R. (2021). Analysis of the development of critical thinking instruments test in physics. Advances in Engineering Research, 209, 627–634. Atlantis Press.

Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional development for training teachers of exceptional children: A sourcebook.

Torres-Malca, J., Vera-Ponce, V., Zuzunaga-Montoya, F., Talavera, J., & De La Cruz-Vargas, J. (2022). Content validity by expert judgment of an instrument to measure knowledge, attitudes and practices about salt consumption in the peruvian population. Revista de la Facultad de Medicina Humana, 22(2), 273–279.

Varenina, L., Vecherinina, E., Shchedrina, E., Valiev, I., & Islamov, A. (2021). Developing critical thinking skills in a digital educational environment. Thinking Skills and Creativity, 41, Article 100906. https://doi.org/10.1016/j.tsc.2021.100906.

Walker, J. (2024). Understanding EBA (Evidence-Based Assessment) in education. Journal of Foreign Language Education and Technology, 9(4), 1–2.

Wardani, I. S., & Fiorintina, E. (2023). Building critical thinking skills of 21st century students through problem-based learning model. Jurnal Pendidikan Indonesia, 12(3), 461–470.

Watson, G., & Glaser, E. M. (2002). Watson-Glaser Critical Thinking Appraisal (WGCTA): UK Edition Manual. Psychological Corporation.

Wiyoko, T., Wulandari, T., & Irhas, M. (2024). Online-based literacy test instruments for evaluating prospective elementary school teachers’ science competencies. Jurnal Ilmiah Sekolah Dasar, 8(2), 282–292. https://doi.org/10.23887/jisd.v8i2.68342

Yolanda, F. (2019). The effect of problem-based learning on mathematical critical thinking skills of junior high school students. Journal of Physics: Conference Series.