Explainable artificial intelligence in mental health assessment: a systematic review of interpretability techniques, trust, and clinical adoption

Jurnal Konseling dan Pendidikan December 30, 2025 Volume 13, Number 4 4 authors

Abstract

Machine learning is now woven into mental health assessment, and the predictive models it produces are impressively accurate. Yet the black-box nature of those models keeps raising the same worries: transparency, clinician trust, and whether they can be adopted safely in practice. Explainable AI, or XAI, is often proposed as the answer, though the evidence on where it helps and how much is still scattered. We ran a systematic review following PRISMA 2020 to pull that evidence together. A structured search of Scopus turned up 403 records. After 18 duplicates were removed, 385 went into screening; 75 full texts were assessed, and 15 studies, all published between 2021 and 2025, made it into the final pool. Study design, mental health condition, modality, XAI technique, interpretability, trust, and clinical utility were all captured and synthesised thematically. Three things stood out. First, SHAP and other post-hoc feature-attribution techniques dominated the field, appearing across work on depression, suicide risk, stress, and anxiety alike. Second, explanation quality did shape trust and acceptance, but remarkably few studies actually involved human participants. Third, heterogeneity, small samples, geographic bias, and a lack of standardised metrics all limited how far the results could be compared or translated into practice. Where should the field go next? Priorities include clinician-in-the-loop evaluation, standardised reporting of explanation quality, and equitable external validation.

Authors

  1. Nilma Zola

    Center for Educational Neuroscience, Trauma, and Human Behavior, Universitas Negeri Padang

  2. Ifdil Ifdil

    Center for Educational Neuroscience, Trauma, and Human Behavior, Universitas Negeri Padang

  3. M. Fahli Zatrahadi

    Universitas Islam Negeri Sultan Syarif Kasim

  4. Nur Adila Wafiqoh Zulvi

    Center for Educational Neuroscience, Trauma, and Human Behavior, Universitas Negeri Padang

Keywords

Explainable artificial intelligence (XAI)Mental health assessment interpretability (SHAP and LIME)Trust and clinical adoption machine learning

References

  1. Ahmed, A., Saleem, M., Alzeen, M., Birur, B., & Fargason, R. (2026). Explainable AI for mental health emergency returns: Integrating large language models with clinical records. JAMIA Open. https://doi.org/10.1093/jamiaopen/ooag065
  2. Ahmed, I., Brahmacharimayum, A., Ali, R. H., Khan, T. A., & Ahmad, M. O. (2025). Explainable AI for depression detection and severity classification from activity data: Development and evaluation study of an interpretable framework. JMIR Mental Health, 12, e72038. https://doi.org/10.2196/72038
  3. Ali, M., Ali, S., Abbas, Q., Abbas, Z., & Lee, S. W. (2025). Artificial intelligence for mental health: A narrative review of applications, challenges, and future directions. Digital Health, 11. https://doi.org/10.1177/20552076251395548
  4. Antoniou, G., Papadakis, E., & Baryannis, G. (2022). Mental health diagnosis: A case for explainable artificial intelligence. International Journal on Artificial Intelligence Tools, 31(3), 2241003. https://doi.org/10.1142/S0218213022410032
  5. Arias, M. I., Cadavid, L., & Velásquez, J. D. (2025). Advancing healthcare analytics: A thematic review of machine learning, health informatics, and real-world data applications. Journal of Biomedical Informatics, 171, 104934. https://doi.org/10.1016/j.jbi.2025.104934
  6. Bajpai, S., Mishra, G., Jain, R., Jain, D. K., & Kotecha, K. (2026). Large language model empowered explainable and interpretable mental health analysis. Scientific Reports. https://doi.org/10.1038/s41598-026-45992-2
  7. Balcombe, L., & De Leo, D. (2021). Digital mental health challenges and the horizon ahead for solutions. JMIR Mental Health, 8(3), e26811. https://doi.org/10.2196/26811
  8. Baydili, İ., Tasci, B., & Tasci, G. (2025). Artificial intelligence in psychiatry: A review of biological and behavioral data analyses. Diagnostics, 15(4), 434. https://doi.org/10.3390/diagnostics15040434
  9. Bhatt, P., Liu, J., Gong, Y., Wang, J., & Guo, Y. (2022). Emerging artificial intelligence-empowered mHealth: Scoping review. JMIR mHealth and uHealth, 10(6), e35053. https://doi.org/10.2196/35053
  10. Byeon, H. (2023). Advances in machine learning and explainable artificial intelligence for depression prediction. International Journal of Advanced Computer Science and Applications, 14(6), 520–526. https://doi.org/10.14569/IJACSA.2023.0140656
  11. Chadaga, K., Prabhu, S., Sampathila, N., Chadaga, R., Bhat, D., Sharma, A. K., & Swathi, K. S. (2024). SADXAI: Predicting social anxiety disorder using multiple interpretable artificial intelligence techniques. SLAS Technology, 29(2), 100129. https://doi.org/10.1016/j.slast.2024.100129
  12. Choi, M., Jo, J., & Jeong, J. (2026). An artificial intelligence-based detection of comorbid depression, anxiety, and stress. Diagnostics, 16(12), 1817. https://doi.org/10.3390/diagnostics16121817
  13. Dagnino, P., Salinas, M., & Salas, R. (2026). Patient complexity profiles in depression: A machine learning approach to personalization. Frontiers in Psychiatry, 17, 1741860. https://doi.org/10.3389/fpsyt.2026.1741860
  14. de Oliveira, A. C., Azevedo, J. P. C., Ruback, L., Moreira, R., Teixeira, S. S., & Teles, A. S. (2025). Effect of explainable artificial intelligence on trust of mental health professionals in an AI-based system for suicide prevention. IEEE Access, 13, 60987–61005. https://doi.org/10.1109/ACCESS.2025.3556245
  15. Enkhbayar, D., Ko, J., Oh, S., Ferdushi, R., Kim, J., Key, J., & Urtnasan, E. (2025). Explainable artificial intelligence models for predicting depression based on polysomnographic phenotypes. Bioengineering, 12(2), 186. https://doi.org/10.3390/bioengineering12020186
  16. Gholi Zadeh Kharrat, F., Gagné, C., Lesage, A., Gariépy, G., Pelletier, J. F., Brousseau-Paradis, C., Rochette, L., Pelletier, E., Lévesque, P., Mohammed, M., & Wang, J. (2024). Explainable artificial intelligence models for predicting risk of suicide using health administrative data. PLoS ONE, 19(4), e0301117. https://doi.org/10.1371/journal.pone.0301117
  17. Henninger, M., Debelak, R., Rothacher, Y., & Strobl, C. (2023). Interpretable machine learning for psychological research: Opportunities and pitfalls. Psychological Methods, 30(2), 271–305. https://doi.org/10.1037/met0000560
  18. Hoang, H., Ha, H., Nguyen, H., Watton, P., & Ngo, L. (2025). Advancing mental health diagnostics: A review on the role of smartphones, wearables, and artificial intelligence. Biomedical Engineering Letters. https://doi.org/10.1007/s13534-025-00505-4
  19. Jaber, D., Hajj, H., Maalouf, F., & El-Hajj, W. (2022). Medically-oriented design for explainable AI for stress prediction from physiological measurements. BMC Medical Informatics and Decision Making, 22, 38. https://doi.org/10.1186/s12911-022-01772-2
  20. Jha, I. P., Awasthi, R., Kumar, A., Kumar, V., & Sethi, T. (2021). Learning the mental health impact of COVID-19 in the United States with explainable artificial intelligence: Observational study. JMIR Mental Health, 8(4), e25097. https://doi.org/10.2196/25097
  21. Joyce, D. W., Kormilitzin, A., Smith, K. A., & Cipriani, A. (2023). Explainable artificial intelligence for mental health through transparency and interpretability for understandability. npj Digital Medicine, 6(1), 6. https://doi.org/10.1038/s41746-023-00751-9
  22. Kaur, G., Aggarwal, H., & Goel, N. (2025). Artificial intelligence driven neuropsychiatry: A systematic review of electroencephalography-based approaches. Neuroscience. https://doi.org/10.1016/j.neuroscience.2025.07.010
  23. Kerz, E., Zanwar, S., Qiao, Y., & Wiechmann, D. (2023). Toward explainable AI (XAI) for mental health detection based on language behavior. Frontiers in Psychiatry, 14, 1219479. https://doi.org/10.3389/fpsyt.2023.1219479
  24. Khan, A. E., Hasan, M. J., Anjum, H., Mohammed, N., & Momen, S. (2024). Predicting life satisfaction using machine learning and explainable AI. Heliyon, 10(10), e31158. https://doi.org/10.1016/j.heliyon.2024.e31158
  25. Kim, J. W., Lee, T., Lim, B., Park, S. H., Park, J. H., & Jeong, B. (2026). Actigraphy-based step analysis for the detection of depressed mood: An explainable approach. Journal of Affective Disorders. https://doi.org/10.1016/j.jad.2025.120104
  26. Kritsotaki, N., Diamantidis, D., Koutlaki, N., Machairiotis, N., & Tsikouras, P. (2026). Explainable artificial intelligence in assisted reproductive technology: Bridging prediction and clinical judgment. Biomedicines, 14(5), 1024. https://doi.org/10.3390/biomedicines14051024
  27. Lamba, K., Rani, S., & Shabaz, M. (2026). Explainable machine learning for mental health prediction from social media behaviour. Discover Mental Health. https://doi.org/10.1007/s44192-026-00373-z
  28. Lavelle-Hill, R., Smith, G., Deininger, H., & Murayama, K. (2025). An explainable artificial intelligence handbook for psychologists: Methods, opportunities, and challenges. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000772
  29. Li, J., Wu, S., & Gu, J. (2025). Explainable machine learning model for assessing health status in patients with comorbid coronary heart disease and depression: Development and validation study. International Journal of Medical Informatics, 196, 105808. https://doi.org/10.1016/j.ijmedinf.2025.105808
  30. Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions. PLoS Medicine, 6(7), e1000100. https://doi.org/10.1371/journal.pmed.1000100
  31. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765–4774.
  32. Malhotra, A., & Jindal, R. (2024). XAI transformer based approach for interpreting depressed and suicidal user behavior on online social networks. Cognitive Systems Research, 84, 101186. https://doi.org/10.1016/j.cogsys.2023.101186
  33. Mamun, A., Barua Soumma, S., & Ghasemzadeh, H. (2026). Trustworthy AI in digital health: A comprehensive review of robustness and explainability. Progress in Biomedical Engineering. https://doi.org/10.1088/2516-1091/ae4e74
  34. Naegelin, M., Weibel, R. P., Kerr, J. I., Schinazi, V. R., La Marca, R., von Wangenheim, F., Hoelscher, C., & Ferrario, A. (2023). An interpretable machine learning approach to multimodal stress detection in a simulated office environment. Journal of Biomedical Informatics, 139, 104299. https://doi.org/10.1016/j.jbi.2023.104299
  35. Ntakolia, C., Yotsidi, V., Rannou, I., & Gournellis, R. (2025). Interpretable machine learning approach for predicting clinically significant suicidal ideation. Psychiatry Research, 349, 116607. https://doi.org/10.1016/j.psychres.2025.116607
  36. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71
  37. Rajpura, P., Cecotti, H., & Kumar Meena, Y. (2024). Explainable artificial intelligence approaches for brain–computer interfaces: A review and design space. Journal of Neural Engineering, 21(4), 046055. https://doi.org/10.1088/1741-2552/ad6593
  38. Ramon, Y., Farrokhnia, R. A., Matz, S. C., & Martens, D. (2021). Explainable AI for psychological profiling from behavioral data: An application to Big Five personality predictions from financial transaction records. Information, 12(12), 518. https://doi.org/10.3390/info12120518
  39. Rehman, A. U., Iqbal, A., Sayyed, A., Aslam, Z., & Mohman, S. (2026). Explainable text-based depression and suicide risk prediction from social media. Healthcare, 14(11), 1440. https://doi.org/10.3390/healthcare14111440
  40. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?” Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. https://doi.org/10.1145/2939672.2939778
  41. Sa’ad, M. I., Irawan, A. W., & Putri, A. A. P. (2026). Beyond predictive accuracy: Explainability, clinical judgment, and the limits of artificial intelligence in mental health. Asian Journal of Psychiatry, 108, 105090. https://doi.org/10.1016/j.ajp.2026.105090
  42. Sarkar, S., Gaur, M., Chen, L. K., Garg, M., & Srivastava, B. (2023). A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement. Frontiers in Artificial Intelligence, 6, 1229805. https://doi.org/10.3389/frai.2023.1229805
  43. Sharma, S., Chakrabarty, M., Ray, S. B., & Shukla, J. (2025). Machine learning-driven analysis of temporal pupil dynamics for interpretable ADHD diagnosis. Computers in Biology and Medicine, 196, 110878. https://doi.org/10.1016/j.compbiomed.2025.110878
  44. Shen, A., Sun, J., Chen, X., & Gao, X. (2025). A data-centric and interpretable EEG framework for depression severity grading using SHAP-based feature analysis. Journal of NeuroEngineering and Rehabilitation, 22(1), 116. https://doi.org/10.1186/s12984-025-01645-5
  45. Shen, S., Qi, W., Zeng, J., Li, S., Liu, X., & Zhu, X. (2025). Passive sensing for mental health monitoring using machine learning with wearable and smartphone data. Journal of Medical Internet Research, 27, e77066. https://doi.org/10.2196/77066
  46. Tang, H., Miri Rekavandi, A., Rooprai, D., Dwivedi, G., Sanfilippo, F. M., Boussaid, F., & Bennamoun, M. (2024). Analysis and evaluation of explainable artificial intelligence on suicide risk assessment. Scientific Reports, 14, 6163. https://doi.org/10.1038/s41598-024-53426-0
  47. Taskynbayeva, M., & Gutoreva, A. (2025). Machine learning approaches to anxiety detection: Trends, model evaluation, and future directions. Frontiers in Artificial Intelligence, 8, 1630047. https://doi.org/10.3389/frai.2025.1630047
  48. Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, 45. https://doi.org/10.1186/1471-2288-8-45
  49. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. https://doi.org/10.1111/1467-8551.00375
  50. Turchioe, M. R., Hermann, A., & Benda, N. C. (2024). Recentering responsible and explainable artificial intelligence research on patients. Frontiers in Psychiatry, 14, 1321265. https://doi.org/10.3389/fpsyt.2023.1321265
  51. Wang, M., Luo, M., Li, L., Bao, S., Chen, F., & Zhao, Q. (2026). Primary-care-focused interpretable machine learning model for depression screening. Journal of Affective Disorders, 371, 120551. https://doi.org/10.1016/j.jad.2025.120551
  52. Zhang, R., Liu, Y., Zhang, Z., Luo, R., & Lv, B. (2025). Interpretable machine learning model for predicting postpartum depression: Retrospective study. JMIR Medical Informatics, 13, e58649. https://doi.org/10.2196/58649