Clinically Interpretable Depression Screening via Static Facial Images Using Deep Learning Feature Extraction and a Fine-Tuned Decision Tree

Document Type : Original Article

Authors

1 Department of Biomedical Engineering, Meybod University, Meybod, Iran

2 Department of Electrical Engineering, Vali-e-Asr University of Rafsanjan, Rafsanjan 7718897111, Iran

3 Department of Computer Engineering Technical and Vocational University (TVU), Tehran, Iran

10.30476/jhmi.2025.107906.1303

Abstract

Introduction: Early and accurate detection of depression remains a pressing clinical
challenge, especially in resource-limited environments. Facial expression analysis has
emerged as a promising, non-invasive screening method, yet many existing approaches are
either computationally intensive or lack clinical interpretability.
Methods: This study aims to develop a lightweight, explainable deep learning framework for
depression screening using static facial images, with a specific focus on clinical relevance and
diagnostic transparency.
Methods: We propose a hybrid architecture that leverages fine-tuned convolutional features
from ResNet-18, followed by classification with a decision tree optimized using Gini impurity.
Facial images were sourced from a publicly available dataset comprising over 20,000 labeled
samples, representing diverse adult populations. Images were preprocessed using contrast
enhancement and bilateral filtering to preserve subtle affective cues. The model was trained
and evaluated using stratified 5-fold cross-validation, with performance assessed via accuracy,
precision, recall, F1-score, and confusion matrix analysis.
Results: The proposed framework achieved an average classification accuracy of 91.4%,
outperforming several baseline visual-only models. Importantly, the use of a fine-tuned
decision tree classifier yielded clear, interpretable diagnostic rules that aligned with clinical
preferences. The model demonstrated robustness across folds and strong generalizability,
requiring minimal computational resources. Comparative analysis further highlighted
the method’s balance between performance and interpretability, making it well-suited for
integration into clinical decision support systems.
Conclusion: This study demonstrates potential in combining deep learning-based feature
extraction with interpretable classifiers for mental health screening. The method offers a
practical, explainable, deployable solution for early-stage depression detection using facial
imagery.

Highlights

Khosro Rezaee: Google Scholar

Keywords

Main Subjects

References

  1. Du N, Chong ES, Wei D, Liu Z, Mu Z, Deng S, et al. Prevalence, risk, and protective factors of self-stigma for people living with depression: a systematic review and meta-analysis. J Affect Disord. 2023;332:327-40. doi: 10.1016/j.jad.2023.04.013.

  2. Cuijpers P, Silverstein M, Gladstone T. Preventing depression: challenges and innovations. J Consult Clin Psychol. 2025;93(4):191. doi: 10.1037/ccp0000951.

  3. Squires M, Tao X, Elangovan S, Gururajan R, Zhou X, Acharya UR, et al. Deep learning and machine learning in psychiatry: a survey of current progress in depression detection, diagnosis and treatment. Brain Inform. 2023;10(1):10. doi: 10.1186/s40708-023-00188-6.

  4. Shangguan Z, Liu Z, Li G, Chen Q, Ding Z, Hu B. Dual-stream multiple instance learning for depression detection with facial expression videos. IEEE Trans Neural Syst Rehabil Eng. (سال و مشخصات کامل در فایل موجود نیست)

  5. Zhou X, Wei Z, Xu M, Qu S, Guo G. Facial depression recognition by deep joint label distribution and metric learning. IEEE Trans Affect Comput. 2020;13(3):1605-18. doi: 10.1109/TAFFC.2020.3022732.

  6. Muzammel M, Salam H, Othmani A. End-to-end multimodal clinical depression recognition using deep neural networks: a comparative analysis. Comput Methods Programs Biomed. 2021;211:106433. doi: 10.1016/j.cmpb.2021.106433.

  7. Khandelwal S, Sharma S, Agrawal S, Kalshetti G, Garg B, Jain R. Depression level analysis using face emotion recognition method. In: International Conference on Data Analytics & Management. Singapore: Springer Nature Singapore; 2023. p. 265-78. doi: 10.1007/978-981-99-6550-2_21.

  8. Sharmila M, Dharshinie RP, Keerthana A, Deepika K, Ananthi T. Depression level calculation for predicting child psychometric retardation using DepressNet approach through GPU accelerated Google cloud platform. Turk J Comput Math Educ. 2021;12(9):1879-87.

  9. Kumar G, Das T, Singh K. Early detection of depression through facial expression recognition and electroencephalogram-based artificial intelligence-assisted graphical user interface. Neural Comput Appl. 2024;36(12):6937-54. doi: 10.1007/s00521-024-09437-z.

  10. Sugiyanto S, Purnama IK, Yuniarno EM, Anggraeni W, Purnomo MH. Depression classification based on facial action unit intensity features using CNN-poolingless framework. Int J Intell Eng Syst. 2024;17(5):172-87. doi: 10.22266/ijies2024.1031.15.

  11. Krause FC, Linardatos E, Fresco DM, Moore MT. Facial emotion recognition in major depressive disorder: a meta-analytic review. J Affect Disord. 2021;293:320-8. doi: 10.1016/j.jad.2021.06.053.

  12. Porter-Vignola E, Booij L, Bosse-Chartier G, Garel P, Herba CM. Emotional facial expression recognition and depression in adolescent girls: associations with clinical features. Psychiatry Res. 2021;298:113777. doi: 10.1016/j.psychres.2021.113777.

  13. Rajawat AS, Bedi P, Goyal SB, Bhaladhare P, Aggarwal A, Singhal RS. Fusion fuzzy logic and deep learning for depression detection using facial expressions. Procedia Comput Sci. 2023;218:2795-805. doi: 10.1016/j.procs.2023.01.251.

  14. Zhou X, Jin K, Shang Y, Guo G. Visually interpretable representation learning for depression recognition from facial images. IEEE Trans Affect Comput. 2018;11(3):542-52. doi: 10.1109/TAFFC.2018.2828819.

  15. Liu D, Liu B, Lin T, Liu G, Yang G, Qi D, et al. Measuring depression severity based on facial expression and body movement using deep convolutional neural network. Front Psychiatry. 2022;13:1017064. doi: 10.3389/fpsyt.2022.1017064.

  16. De Melo WC, Granger E, Hadid A. A deep multiscale spatiotemporal network for assessing depression from facial dynamics. IEEE Trans Affect Comput. 2020;13(3):1581-92. doi: 10.1109/TAFFC.2020.3021755.

  17. Pan Y, Shang Y, Liu T, Shao Z, Guo G, Ding H, et al. Spatial-temporal attention network for depression recognition from facial videos. Expert Syst Appl. 2024;237:121410. doi: 10.1016/j.eswa.2023.121410.

  18. Wang Q, Yang H, Yu Y. Facial expression video analysis for depression detection in Chinese patients. J Vis Commun Image Represent. 2018;57:228-33. doi: 10.1016/j.jvcir.2018.11.003.

  19. Fu G, Yu Y, Ye J, Zheng Y, Li W, Cui N, et al. A method for diagnosing depression: facial expression mimicry is evaluated by facial expression recognition. J Affect Disord. 2023;323:809-18. doi: 10.1016/j.jad.2022.12.029.

  20. Li M, Wang Y, Yang C, Lu Z, Chen J. Automatic diagnosis of depression based on facial expression information and deep convolutional neural network. IEEE Trans Comput Soc Syst. 2024;11(5):5728-39. doi: 10.1109/TCSS.2024.3393247.

  21. Pan Y, Shang Y, Shao Z, Liu T, Guo G, Ding H. Integrating deep facial priors into landmarks for privacy-preserving multimodal depression recognition. IEEE Trans Affect Comput. 2023. doi: 10.1109/TAFFC.2023.3296318.

  22. Liu Z, Yuan X, Li Y, Shangguan Z, Zhou L, Hu B. PRA-Net: part-and-relation attention network for depression recognition from facial expression. Comput Biol Med. (سال و مشخصات کامل در فایل موجود نیست)

  23. Monferrer M, Garcia AS, Ricarte JJ, Montes MJ, Fernandez-Caballero A, Fernandez-Sotos P. Facial emotion recognition in patients with depression compared to healthy controls when using human avatars. Sci Rep. 2023;13(1):6007. doi: 10.1038/s41598-023-31277-5.

  24. Yang M, Wu Y, Tao Y, Hu X, Hu B. Trial selection tensor canonical correlation analysis (TSTCCA) for depression recognition with facial expression and pupil diameter. IEEE J Biomed Health Inform. 2023.

  25. Casado CA, Canellas ML, Lopez MB. Depression recognition using remote photoplethysmography from facial videos. IEEE Trans Affect Comput. 2023;14(4):3305-16. doi: 10.1109/TAFFC.2023.3238641.

  26. Chen X, Luo T. Catching elusive depression via facial micro-expression recognition. IEEE Commun Mag. 2023;61(10):30-6. doi: 10.1109/MCOM.001.2300003.

  27. Khan MT, Imran M, Kanwal M. MCNN: multi-channel neural network with channelwise attention for facial expression-based depression recognition. Multimed Tools Appl. 2025;1-24. doi: 10.1007/s11042-025-20962-4.

  28. Lu L, Jiang Y, Li X, Wang H, Zou Q, Wang Q. Depression and anxiety detection method based on serialized facial expression imitation. Eng Appl Artif Intell. 2025;149:110354. doi: 10.1016/j.engappai.2025.110354.

  29. He L, Zhao J, Zhang J, Jiang J, Qi S, Wang Z, et al. LMTformer: facial depression recognition with lightweight multi-scale transformer from videos. Appl Intell. 2025;55(3):195. doi: 10.1007/s10489-024-05908-x.

  30. Chen X, Liu L, Mei H, Jiang Z, Yan W, Shi L, et al. Efficacy evaluation and facial expressions biomarker of light therapy in youths with subthreshold depression: a randomized control trial study. J Affect Disord. 2025;380:357-65. doi: 10.1016/j.jad.2025.03.123.

  31. Wang Y, Lin Z, Yang C, Zhou Y, Yang Y. Automatic depression recognition with an ensemble of multimodal spatio-temporal routing features. IEEE Trans Affect Comput. 2025. doi: 10.1109/TAFFC.2025.3543226.

  32. Attar CH, Ridder N, Stein J, Klucznik D, Dittrich K, Jaite C, et al. Maladaptive mother-child interactions in mothers with remitted major depression are associated with blunted amygdala responses to child affective facial expressions. Psychol Med. 2025;55:e15. doi: 10.1017/S0033291724003404.

  33. Sharma S. Depression survey/dataset for analysis [Internet]. 2023 [cited 2025 May 31]. Available from: https://www.kaggle.com/datasets/sumansharmadataworld/depressionsurveydataset-for-analysis

  34. Mahayossanut Y, Nupairoj N, Hemrungrojn S, Vateekul P. Explainable depression detection based on facial expression using LSTM on attentional intermediate feature fusion with label smoothing. Sensors. 2023;23(23):9402. doi: 10.3390/s23239402.

  35. Jiang Z, Xu K, Gao X, Cao Y, Zhang Y, Dong G, et al. DNet: a depression recognition network combining residual network and vision transformer. BMC Psychiatry. 2025;25(1):1-20. doi: 10.1186/s12888-025-07322-0.

Health Management & Information Science
Volume 13, Issue 1
January 2026
Pages 36-52

Files

Share

How to cite

Statistics