AI-based assessment: Aiken index analysis of content validity in scientific literacy instruments on the basic laws of chemistry with ethnochemistry

Fila Delfia Zai, Sri Yamtinah, Ari Syahidul Shidiq, Hayuni Retno Widarti, Antuni Wiyarsi

Abstract


Scientific Literacy Assessment (SLA) is used to measure science literacy on basic chemical law material that has been qualified and tested. This study aims to prove the content validity of ethnochemistry SLA instruments integrated with artificial intelligence (AI) systems using Aiken's validity index so that they can measure students' scientific literacy in basic chemical law material. The research design uses the ADDIE development model (Analysis, design, development, implementation and evaluation which is simplified into analysis, design, and development. Content validity data was obtained from 10 experts, namely 7 chemistry education lecturers from UNS, UNY, and UM and 3 chemistry teachers in Surakarta through focus group discussions (FGD). The content validation test is based on aspects of context, language, and construct. A valid instrument is indicated by an Aiken index value of more than or equal to 0.73. The AI-integrated ethnochemistry SLA instrument on basic laws of chemistry material produced 15 valid items with an Aiken index greater than 0.73.

Keywords


Aiken index; Artificial intelligence; Content validity; Science literacy assessment

Full Text:

PDF

References


Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40(4), 955-959. https://doi.org/10.1177/001316448004000419

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131-142. https://doi.org/10.1177/0013164485451012

Asda, V. D. (2023). Development of Buffer Solution Students’ Worksheet Based on Problem Based Learning with Ethnochemistry to Improve Students Science Literacy Ability. Jurnal Penelitian Pendidikan IPA, 9(7), 5220-5227. https://doi.org/10.29303/jppipa.v9i7.4369

Asikin, N., & Yulita, I. (2019). Scientific literacy-based chemical teaching materials design of chemical solution materials on sea pollution context. Jurnal Penelitian Pendidikan IPA, 5(2), 204. https://doi.org/10.29303/jppipa.v5i2.249

Aulia, E. V., Poedjiastoeti, S., & Agustini, R. (2018). The effectiveness of guided inquiry-based learning material on students’ science literacy skills. Journal of Physics: Conference Series, 947, p. 012049. https://doi.org/10.1088/1742-6596/947/1/012049

Berland, L. K., Schwarz, C. V., Krist, C., Kenyon, L., Lo, A. S., & Reiser, B. J. (2016). Epistemologies in practice: Making scientific practices meaningful for students. Journal of Research in Science Teaching, 53(7), 1082-1112. https://doi.org/10.1002/tea.21257

Chasanah, N., Widodo, W., & Suprapto, N. (2022). Pengembangan instrumen asesmen literasi sains untuk mendeskripsikan profil peserta didik. PENDIPA Journal of Science Education, 6(2), 474-483. https://doi.org/10.33369/pendipa.6.2.474-483

Chiu, M. (2022). My Journey in Chemistry Education. Pure and Applied Chemistry, 94(8), 919-941.

Cooper, M. M., & Stowe, R. L. (2018). Chemistry education research—From personal empiricism to evidence, theory, and informed practice. Chemical Reviews, 118(12), 6053-6087. https://doi.org/10.1021/acs.chemrev.8b00020

Dang, H., Mecke, L., Lehmann, F., Goller, S., & Buschek, D. (2022). How to prompt? Opportunities and challenges of zero-and few-shot learning for human-AI interaction in creative applications of generative models. arXiv preprint arXiv:2209.01390. https://doi.org/10.48550/arXiv.2209.01390

DinanThompson, M., & Penney, D. (2015). Assessment literacy in primary physical education. European Physical Education Review, 21(4), 485-503. https://doi.org/10.1177/1356336X15584087

Dragoş, V., & Mih, V. (2015). Scientific literacy in school. Procedia-Social and Behavioral Sciences, 209, 167-172. https://doi.org/10.1016/j.sbspro.2015.11.273

Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T., ... & Williams, M. D. (2021). Artificial Intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management, 57, 101994. https://doi.org/10.1016/j.ijinfomgt.2019.08.002

Eilks, I., Witteck, T., & Pietzner, V. (2012). Eilks, I., Witteck, T., & Pietzner, V. (2012). The role and potential dangers of visualisation when learning about sub-microscopic explanations in chemistry education. Ceps Journal, 2(1), 125-145. https://doi.org/10.25656/01:5828

Goss, H. R., Shearer, C., Knowles, Z. R., Boddy, L. M., Durden-Myers, E. J., & Foweather, L. (2022). Stakeholder perceptions of physical literacy assessment in primary school children. Physical Education and Sport Pedagogy, 27(5), 515-530. https://doi.org/10.1080/17408989.2021.1911979

Jufri, A. W., Ramdani, A., Jamaluddin, J., & Azizah, A. (2019). Development of Scientific Literacy and Pedagogical Content Knowledge (PCK) of Prospective Science Teachers through Lesson Study-Based Courses. Jurnal Penelitian Pendidikan IPA, 5(2), 179-184. https://doi.org/10.29303/jppipa.v5i2.235

Kemendikbudristek. (2023, 5 Des). Peringkat Indonesia pada PISA 2022 Naik 5-6 Posisi Dibanding 2018. Diakses pada 14 Desember 2024, https://www.kemdikbud.go.id/main/blog/2023/12/peringkat-indonesia-pada-pisa-2022-naik-56-posisi-dibanding-2018

Khlaif, Z., Odeh, A., & Bsharat, T. R. K. (2024). Generative AI-Powered Adaptive Assessment. Power of Persuasive Educational Technologies in Enhancing Learning, 157–176. https://doi.org/10.4018/979-8-3693-6397-3.ch007

Laohapornchaiphan, J., & Chenprakhon, P. (2024). A Review of Research on Learning Activities Addressing the Submicroscopic Level in Chemistry. Journal of Chemical Education, 101(11), 4552-4565. https://doi.org/10.1021/acs.jchemed.4c00156

Morales‐Doyle, D., Childress Price, T., & Chappell, M. J. (2019). Chemicals are contaminants too: Teaching appreciation and critique of science in the era of Next Generation Science Standards (NGSS). Science Education, 103(6), 1347-1366. https://doi.org/10.1002/sce.21546

Muniroh, N., Rusilowati, A., & Isnaeni, W. (2022). Instrument Development of Science Literacy Assessment with Socio-Sciences Contains in Natural Science Learning for Elementary School. Journal of Research and Educational Research Evaluation, 11(1), 15-22. https://doi.org/10.15294/jere.v11i1.55421

OECD (2021), 21st-Century Readers: Developing Literacy Skills in a Digital World, PISA, OECD Publishing, Paris, https://doi.org/10.1787/a83d84cb-en

Pebrianti, P., & Suryani, O. (2024). Development of the Acid-Base Module Based on Problem Based Learning with Ethnochemisty to Improve Students Science Literacy Ability. Jurnal Penelitian Pendidikan IPA, 10(8), 4634-4640. https://doi.org/10.29303/jppipa.v10i8.8582

Permanasari, A. (2016, October). STEM education: Inovasi dalam pembelajaran sains. In Seminar Nasional Pendidikan Sains VI 2016. Sebelas Maret University.

Putri, V. L., & Dj, L. (2022). Description of students learning difficulties in the basic law of chemistry. Jurnal Pijar Mipa, 17(5), 597-603. https://doi.org/10.29303/jpm.v17i5.3855

Rahmawati, Y., Ridwan, A., Rahman, A., & Kurniadewi, F. (2019). Chemistry students’ identity empowerment through etnochemistry in culturally responsive transformative teaching (CRTT). Journal of Physics: Conference Series, 1156, 012032. https://doi.org/10.1088/1742-6596/1156/1/012032

Rosyidi, D. (2020). Teknik dan instrumen asesmen ranah kognitif. Tasyri': Jurnal Tarbiyah-Syari'ah Islamiyah, 27(1), 1-13. https://doi.org/10.52166/tasyri.v27i1.79

Rubini, B., Ardianto, D., Pursitasari, I. D., & Hidayat, A. (2018). Science Teachers’ Understanding on Science Literacy and Integrated Science Learning: Lesson from Teachers Training. Jurnal Pendidikan IPA Indonesia, 7(3), 259-265. https://doi.org/10.15294/jpii.v7i3.11443

Rusilowati, A., Nugroho, S. E., Susilowati, E. S. M., Mustika, T., Harfiyani, N., & Prabowo, H. T. (2018). The development of scientific literacy assessment to measure student’s scientific literacy skills in energy theme. Journal of Physics: Conference Series, 983, 012046. https://doi.org/10.1088/1742-6596/983/1/012046

Shwartz, Y., Ben-Zvi, R., & Hofstein, A. (2006). The use of scientific literacy taxonomy for assessing the development of chemical literacy among high-school students. Chemistry Education Research and Practice, 7(4), 203-225. https://doi.org/10.1039/B6RP90011A

Ulfah, A. A., Kartono, K., & Susilaningsih, E. (2020). Validity of content and reliability of inter-rater instruments assessing ability of problem solving. Journal of Research and Educational Research Evaluation, 9(1), 1-7. https://doi.org/10.15294/jere.v9i1.40423

UNESCO. (2021). Implementation of the UNESCO Recommendation on Open Science. https://www.unesco.org/en/open-science/implementation

Wardani, L. K., Mulyani, B., Ariani, S. R. D., Yamtinah, S., Masykuri, M., Ulfa, M., & Shidiq, A. S. (2023). The effect of an ethnochemistry-based culturally responsive teaching approach to improve cognitive learning outcomes on green chemistry material in high school. Jurnal Penelitian Pendidikan IPA, 9(12), 11029-11037. https://doi.org/10.29303/jppipa.v9i12.5532

Zou, Y., Xue, X., Jin, L., Huang, X., & Li, Y. (2024). Assessment of conceptual understanding in student learning of evaporation. Physical Review Physics Education Research, 20(2), 020107. https://doi.org/10.1103/PhysRevPhysEducRes.20.020107




DOI: https://doi.org/10.24114/jpkim.v16i3.67118

Article Metrics

Abstract view : 66 times
PDF - 11 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 fila delfia zai, sri yamtinah, sri yamtinah, Ari Syahidul Shidiq, Ari Syahidul Shidiq, Hayuni Retno Widarti, Hayuni Retno Widarti, Antuni Wiyarsi, Antuni Wiyarsi

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Jurnal Pendidikan Kimia
Contact: +62 853-1769-2813
Email: jpkim.pps@unimed.ac.id

Jl. Willem Iskandar, Pasar V, Medan Estate, Medan City, North Sumatra Province, Postal Code 20221. Phone/fax: (061) 661 3365 / +62 852-7802-1981.

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

slot gacor slot