UX test in the academic information system of vocational higher education

Jurnal Penelitian Guru Indonesia March 04, 2022 Volume 7, Number 1 4 authors

Abstract

The purpose of this study is to evaluate the user experience (UX) of academic information system (SIAKAD) using the User Experience Questionnaire (UEQ). Although SIAKAD has been running for two years, user experience testing has not yet been carried out to assess whether this application is sufficient to meet user needs or not. Questionnaires were distributed to all lecturers, administrative units, and cadets and 70 people filled out the survey. The results showed positive values on the scale of attractiveness, clarity, efficiency, accuracy, and stimulation with means > 0,8. The negative values are finding on the novelty scale where the mean is < 0,8 or 0.564. The reliability test showed that five of the six aspects were good with Cronbach-alpha > 0.7, while the UEQ weakness was on the novelty scale with an Alpha score of 0.16. A benchmark study conducts, and results shows a scale of accuracy, attractiveness, and stimulation to get an “Excellent” Product rating. Clarity and efficiency scale reach a “Good” category. Poor rating with “Below Average” obtained by novelty scale. Suggestions for further improvement is to re-evaluate the novelty aspect of SIAKAD with important points that must be considered related to innovation, invention, and creative design.

Authors

  1. Direstu Amalia

    Politeknik Penerbangan Palembang

  2. Dwi Cahyono

    Politeknik Penerbangan Palembang

  3. Virma Septiani

    Politeknik Penerbangan Palembang

  4. Muhammad Kristiawan

    Universitas Bengkulu

Keywords

No keywords available

References

  1. Amalia, D. (n.d.). Lihat artikel. Retrieved December 21, 2021, from JAET website: https://scholar.google.com/citations?view_op=view_citation&hl=id&user=aoaPeX4AAAAJ&citation_for_view=aoaPeX4AAAAJ:UeHWp8X0CEIC
  2. Björn, M. (2022). Development of an effective pedagogical EEG simulator: design-based research project among biomedical laboratory science students. Retrieved from https://dspace.uef.fi/bitstream/handle/123456789/27018/urn_isbn_978-952-61-4454-2.pdf?sequence=1&isAllowed=y
  3. Bonett, D., behavior, T. W.-J. of organizational, & 2015, undefined. (2014). Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Wiley Online Library, 36(1), 3–15. https://doi.org/10.1002/job.1960
  4. Callender, J., Measurement, H. O.-J. of E., & 1979, undefined. (n.d.). An empirical comparison of coefficient alpha, Guttman’s lambda-2, and MSPLIT maximized split-half reliability estimates. JSTOR. Retrieved from https://www.jstor.org/stable/1434452
  5. Folmer, E., software, J. B.-J. of systems and, & 2004, undefined. (n.d.). Architecting for usability: a survey. Elsevier. Retrieved from https://www.sciencedirect.com/science/article/pii/S0164121202001590
  6. Francesca, B., Francesca, B., Elisa, P., & Federica, R. (2022). A Psychometric Tool for Evaluating Executive Functions in Parkinson’s Disease. Retrieved from https://iris.unito.it/handle/2318/1842903
  7. Amalia, D. (n.d.). Lihat artikel. Retrieved December 21, 2021, from JAET website: https://scholar.google.com/citations?view_op=view_citation&hl=id&user=aoaPeX4AAAAJ&citation_for_view=aoaPeX4AAAAJ:UeHWp8X0CEIC
  8. Björn, M. (2022). Development of an effective pedagogical EEG simulator: design-based research project among biomedical laboratory science students. Retrieved from https://dspace.uef.fi/bitstream/handle/123456789/27018/urn_isbn_978-952-61-4454-2.pdf?sequence=1&isAllowed=y
  9. Bonett, D., behavior, T. W.-J. of organizational, & 2015, undefined. (2014). Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Wiley Online Library, 36(1), 3–15. https://doi.org/10.1002/job.1960
  10. Callender, J., Measurement, H. O.-J. of E., & 1979, undefined. (n.d.). An empirical comparison of coefficient alpha, Guttman’s lambda-2, and MSPLIT maximized split-half reliability estimates. JSTOR. Retrieved from https://www.jstor.org/stable/1434452
  11. Folmer, E., software, J. B.-J. of systems and, & 2004, undefined. (n.d.). Architecting for usability: a survey. Elsevier. Retrieved from https://www.sciencedirect.com/science/article/pii/S0164121202001590
  12. Francesca, B., Francesca, B., Elisa, P., & Federica, R. (2022). A Psychometric Tool for Evaluating Executive Functions in Parkinson’s Disease. Retrieved from https://iris.unito.it/handle/2318/1842903
  13. Galitz, W. (2007). The essential guide to user interface design: an introduction to GUI design principles and techniques. Retrieved from https://books.google.com/books?hl=id&lr=&id=Q3Xp_Awu49sC&oi=fnd&pg=PR5&dq=Galitz,+2007&ots=I063DTae-_&sig=Ltg1gSF2OJ10dv7G7YoKiapD_2Q
  14. Heo, M., Kim, N., medical, M. F.-B., & 2015, undefined. (n.d.). Statistical power as a function of Cronbach alpha of instrument questionnaire items. Bmcmedresmethodol.Biomedcentral …. Retrieved from https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-015-0070-6
  15. Lallemand, C., Gronier, G., Behavior, V. K.-C. in H., & 2015, undefined. (2015). User experience: A concept without consensus? Exploring practitioners’ perspectives through an international survey. Elsevier. https://doi.org/10.1016/j.chb.2014.10.048
  16. Laugwitz:Construction and evaluation of a user experience. (n.d.). Retrieved March 1, 2022, from https://scholar.google.de/scholar?start=0&hl=de&scisbd=1&as_sdt=0,5&sciodt=0,5&cites=16964264291568318743&scipsc=
  17. Law, E., Schaik, P. Van, Human, V. R.-I. J. of, & 2014, undefined. (n.d.). Attitudes towards user experience (UX) measurement. Elsevier. Retrieved from https://www.sciencedirect.com/science/article/pii/S1071581913001304
  18. Mandel, Т. (1997). The elements of user interface design. Retrieved from https://www.researchgate.net/profile/Theo-Mandel/publication/234796045_The_Elements_of_User_Interface_Design/links/56ba180908aed47e41f38c19/The-Elements-of-User-Interface-Design.pdf
  19. methods, H. O.-P., & 2000, undefined. (2000). Coefficient alpha and related internal consistency reliability coefficients. Psycnet.Apa.Org, 5(3), 343–335. Retrieved from https://psycnet.apa.org/journals/met/5/3/343/
  20. Newsletter, J. B.-J. T. & E. S., & 2002, undefined. (n.d.). The Cronbach alpha reliability estimate. Jalt.Org. Retrieved from http://www.jalt.org/test/bro_13.htm
  21. Santoso, H., Schrepp, M., Isal, R., … A. U.-J. of E., & 2016, undefined. (2016). Measuring user experience of the student-centered e-learning environment. ERIC, 13, 59. Retrieved from https://eric.ed.gov/?id=EJ1087680
  22. Schölkopf, L., Heyde, C. G. von der, … A. S.-M., & 2022, undefined. (2022). Reducing the Gap between Mental Models of Truck Drivers and Adaptive User Interfaces in Commercial Vehicles. Mdpi.Com. https://doi.org/10.3390/mti6020014
  23. Schrepp, M., Hinderks, A., & Thomaschewski, J. (2017). Construction of a benchmark for the user experience questionnaire (UEQ). International Journal of Interactive Multimedia and Artificial Intelligence, 4, 4–40. https://doi.org/10.9781/ijimai.2017.445
  24. Schrepp, M., & Thomaschewski, J. (n.d.). Handbook for the modular extension of the User Experience Questionnaire. Retrieved from www.ueq-online.org
  25. Septiani, V. (n.d.). Lihat artikel. Retrieved December 21, 2021, from International Journal of Recent Technology and Engineering website: https://scholar.google.com/citations?view_op=view_citation&hl=id&user=881Og84AAAAJ&citation_for_view=881Og84AAAAJ:u-x6o8ySG0sC
  26. Septiani, V., Amalia, D., & Cahyono, D. (2020). Implementation of Inspector Training System Certification Learning Based On E-Learning in Palembang Aviation Polytechnic. International Journal of Progressive Sciences and Technologies, 23(1), 342–346. https://doi.org/10.52155/ijpsat.v23.1.2271
  27. Shapira Galitz, Y., Halperin, D., Bavnik, Y., & Warman, M. (2016). Sino-Nasal Outcome Test-22: Translation, Cross-cultural Adaptation, and Validation in Hebrew-Speaking Patients. Otolaryngology - Head and Neck Surgery (United States), 154(5), 951–956. https://doi.org/10.1177/0194599816629378
  28. Stamm, O., Dahms, R., Reithinger, N., Ruß, A., & Müller-Werdan, U. (2022). Virtual reality exergame for supplementing multimodal pain therapy in older adults with chronic back pain: a randomized controlled pilot study. Virtual Reality. https://doi.org/10.1007/S10055-022-00629-3
  29. Tavakol, M., education, R. D.-I. journal of medical, & 2011, undefined. (n.d.). Making sense of Cronbach’s alpha. Ncbi.Nlm.Nih.Gov. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4205511/
  30. Widhiarso, W., … F. U.-P. U. Y. O. di: http, & 2017, undefined. (n.d.). Semantik Diferensial. Academia.Edu. Retrieved from https://www.academia.edu/download/34275951/3_-_semantik_diferensial.pdf