Usability Evaluation of the Doctor Saina Online Consultation Application Using the Think-Aloud Method
Keywords:
Usability Evaluation, Mobile Health Applications, Doctor Saina, Telemedicine, Think-Aloud MethodAbstract
Introduction: The rapid advancement of technology and economic growth has created both opportunities and challenges in healthcare accessibility. The unequal distribution of medical resources, which disproportionately favors economically developed regions, has led to a decline in the quality and efficiency of healthcare services in rural and underdeveloped areas, highlighting the urgent need for innovative solutions. Telemedicine effectively overcoming geographical barriers and improving access to medical care. However, usability issues in some of these applications present significant challenges, potentially compromising service quality and user experience. This study aimed to evaluate the usability of the Doctor Saina application, identifying key factors that influence its effectiveness, user satisfaction, and overall success.
Material and Methods: In this study, digital health application Doctor Saina which facilitate online medical consultations, making healthcare services more accessible, was examined. A laboratory-based usability evaluation was conducted using a predefined scenario-driven approach and the think-aloud method with 15 participants. The identified usability issues were categorized using the Van den Haak classification framework, and their severity was assessed based on Nielsen’s heuristic evaluation model.
Results: The average duration of the usability evaluation per participant was 22.08 minutes. During the evaluation process, 23 issues were identified by the users, 5 of which had a severity greater than 2. The most frequent usability issues identified by users were in the Comprehensiveness category (43.5%). During the evaluation process, 9% of the issues were resolved by users without facilitator intervention.
Conclusion: Among the identified usability challenges, layout and comprehensiveness were reported as the most significant barriers affecting user experience. Addressing these issues is crucial for enhancing the overall usability, accessibility, and effectiveness of the Doctor Saina application.
References
1. Dorsey ER, Topol EJ. State of telehealth. N Engl J Med. 2016; 375(2): 154-61. PMID: 27410924 DOI: 10.1056/NEJMra1601705
2. Parikh D, Armstrong G, Liou V, Husain D. Advances in telemedicine in ophthalmology. Semin Ophthalmol. 2020; 35(4): 210-5. PMID: 32644878 DOI: 10.1080/08820538.2020.1789675
3. Hadziahmetovic M, Nicholas P, Jindal S, Mettu PS, Cousins SW. Evaluation of a remote diagnosis imaging model vs dilated eye examination in referable macular degeneration. JAMA Ophthalmol. 2019; 137(7): 802-8. PMID: 31095245 DOI: 10.1001/jamaophthalmol.2019.1203
4. Su Z, Li C, Fu H, Wang L, Wu M, Feng X. Development and prospect of telemedicine. Intelligent Medicine. 2024; 4(1): 1-9.
5. Naik N, Ibrahim S, Sircar S, Patil V, Hameed BM, Rai BP, et al. Attitudes and perceptions of outpatients towards adoption of telemedicine in healthcare during COVID-19 pandemic. Ir J Med Sci. 2022; 191(4): 1505-12. PMID: 34402031 DOI: 10.1007/s11845-021-02729-6
6. Ezeamii VC, Okobi OE, Wambai-Sani H, Perera GS, Zaynieva S, Okonkwo CC, et al. Revolutionizing healthcare: How telemedicine is improving patient outcomes and expanding access to care. Cureus. 2024; 16(7): e63881. PMID: 39099901 DOI: 10.7759/cureus.63881
7. Gholamzadeh M, Abtahi H, Safdari R. Telemedicine in lung transplant to improve patient-centered care: A systematic review. Int J Med Inform. 2022; 167: 104861. PMID: 36067628 DOI: 10.1016/j.ijmedinf.2022.104861
8. Aalaei S, Amini M, Mazaheri Habibi MR, Shahraki H, Eslami S. A telemonitoring system to support CPAP therapy in patients with obstructive sleep apnea: A participatory approach in analysis, design, and evaluation. BMC Med Inform Decis Mak. 2022; 22(1): 168. PMID: 35754055 DOI: 10.1186/s12911-022-01912-8
9. World Health Organization. Diabetes [Internet]. 2024 [cited: 25 Sep 2024]. Available from: https://www.who.int/en/news-room/fact-sheets/detail/diabetes
10. Agarwal E, Miller M, Yaxley A, Isenring E. Malnutrition in the elderly: A narrative review. Maturitas. 2013; 76(4): 296-302. PMID: 23958435 DOI: 10.1016/j.maturitas.2013.07.013
11. Agha Seyyed Esmaeil Amiri FS, Bohlouly F, Khoshkangin A, Razmi N, Ghaddaripouri K, Mazaheri Habibi MR. The effect of telemedicine and social media on cancer patients' self-care: A systematic review. Frontiers in Health Informatics. 2021; 10: 92.
12. Mubeen M, Iqbal MW, Junaid M, Sajjad MH, Naqvi MR, Khan BA, et al. Usability evaluation of pandemic health care mobile applications. IOP conference series: Earth and environmental science; 2021.
13. Al-Marsy A, Chaudhary P, Rodger JA. A model for examining challenges and opportunities in use of cloud computing for health information systems. Applied System Innovation. 2021; 4(1): 15.
14. Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: A scoping review. Int J Med Inform. 2019; 126: 95-104. PMID: 31029270 DOI: 10.1016/j.ijmedinf.2019.03.018
15. Watbled L, Marcilly R, Guerlinger S, Bastien JM, Beuscart-Zéphir MC, Beuscart R. Combining usability evaluations to highlight the chain that leads from usability flaws to usage problems and then negative outcomes. J Biomed Inform. 2018; 78: 12-23. PMID: 29305953 DOI: 10.1016/j.jbi.2017.12.014
16. Khajouei R, Hasman A, Jaspers MW. Determination of the effectiveness of two methods for usability evaluation using a CPOE medication ordering system. Int J Med Inform. 2011; 80(5): 341-50. PMID: 21435943 DOI: 10.1016/j.ijmedinf.2011.02.005
17. Campbell EM, Guappone KP, Sittig DF, Dykstra RH, Ash JS. Computerized provider order entry adoption: Implications for clinical workflow. J Gen Intern Med. 2009; 24(1): 21-6. PMID: 19020942 DOI: 10.1007/s11606-008-0857-9
18. Kim MS, Shapiro JS, Genes N, Aguilar MV, Mohrer D, Baumlin K, et al. A pilot study on usability analysis of emergency department information system by nurses. Appl Clin Inform. 2012; 3(1): 135-53. PMID: 23616905 DOI: 10.4338/ACI-2011-11-RA-0065
19. Khajouei R, Azizi AA, Atashi A. Usability evaluation of an emergency information system: A heuristic evaluation. Journal of Health Administration. 2013; 16(52): 61-72.
20. Nielsen J. Usability engineering. Academic Press; 1993.
21. Mazaheri Habibi MR, Khajouei R, Eslami S, Jangi M, Ghalibaf AK, Zangouei S. Usability testing of bed information management system: A think-aloud method. J Adv Pharm Technol Res. 2018; 9(4): 153-7. PMID: 30637234 DOI: 10.4103/japtr.JAPTR_320_18
22. Ghalibaf AK, Jangi M, Mazaheri Habibi MR, Zangouei S, Khajouei R. Usability evaluation of obstetrics and gynecology information system using cognitive walkthrough method. Electron Physician. 2018; 10(4): 6682-8. PMID: 29881531 DOI: 10.19082/6682
23. Jangi M, Khajouei R, Tara M, Mazaheri Habibi MR, Ghalibaf AK, Zangouei S, et al. User testing of an admission, discharge, transfer system: Usability evaluation. Frontiers in Health Informatics. 2021; 10: 77.
24. Narasimhan M, Kapila M. Implications of self-care for health service provision. Bull World Health Organ. 2019; 97(2): 76-A. PMID: 30728611 DOI: 10.2471/BLT.18.228890
25. Zhang X, Foo S, Majid S, Chang Y-K, Dumaual HTJ, Suri VR. Self-care and health-information-seeking behaviours of diabetic patients in Singapore. Health Commun. 2020; 35(8): 994-1003. PMID: 31303050 DOI: 10.1080/10410236.2019.1606134
26. Koirala B, Himmelfarb CRD, Budhathoki C, Davidson PM. Heart failure self-care, factors influencing self-care and the relationship with health-related quality of life: A cross-sectional observational study. Heliyon. 2020; 6(2): e03412. PMID: 32149197 DOI: 10.1016/j.heliyon.2020.e03412
27. Jin MX, Kim SY, Miller LJ, Behari G, Correa R. Telemedicine: Current impact on the future. Cureus. 2020; 12(8): e9891. PMID: 32968557 DOI: 10.7759/cureus.9891
28. Seppälä J, De Vita I, Jämsä T, Miettunen J, Isohanni M, Rubinstein K, et al. Mobile phone and wearable sensor-based mHealth approaches for psychiatric disorders and symptoms: Systematic review. JMIR Ment Health. 2019; 6(2): e9819. PMID: 30785404 DOI: 10.2196/mental.9819
29. Tas B, Lawn W, Traykova EV, Evans RA, Murvai B, Walker H, et al. A scoping review of mHealth technologies for opioid overdose prevention, detection and response. Drug Alcohol Rev. 2023; 42(4): 748-64. PMID: 36933892 DOI: 10.1111/dar.13645
30. Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: Evidence, theory and improvements. Evid Based Ment Health. 2018; 21(3): 116-9. PMID: 29871870 DOI: 10.1136/eb-2018-102891 [
31. Moghbeli F, Setoodefar M, Mazaheri Habibi MR, Abbaszadeh Z, Keikhay Moghadam H, Salari S, et al. Using mobile health in primiparous women: Effect on awareness, attitude and choice of delivery type, semi-experimental. Reprod Health. 2024; 21(1): 49. PMID: 38594731 DOI: 10.1186/s12978-024-01785-2
32. Khoshkangin A, Agha Seyyed Esmaeil Amiri FS, Ghaddaripouri K, Noroozi N, Mazaheri Habibi MR. Investigating the role of mobile health in epilepsy management: A systematic review. J Educ Health Promot. 2023; 12: 304. PMID: 38023071 DOI: 10.4103/jehp.jehp_1188_22
33. Agapito G, Cannataro M. An overview on the challenges and limitations using cloud computing in healthcare corporations. Big Data and Cognitive Computing. 2023; 7(2): 68.
34. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Hinder S, et al. Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017; 19(11): e367. PMID: 29092808 DOI: 10.2196/jmir.8775
35. Lawal FB, Omara M. Applicability of dental patient reported outcomes in low resource settings–a call to bridge the gap in clinical and community dentistry. J Evid Based Dent Pract. 2023; 23(1S): 101789. PMID: 36707169 DOI: 10.1016/j.jebdp.2022.101789
36. Oliveira Chaves L, Gomes Domingos AL, Louzada Fernandes D, Ribeiro Cerqueira F, Siqueira-Batista R, Bressan J. Applicability of machine learning techniques in food intake assessment: A systematic review. Crit Rev Food Sci Nutr. 2023; 63(7): 902-19. PMID: 34323627 DOI: 10.1080/10408398.2021.1956425
37. Damiani M, Sinkko T, Caldeira C, Tosches D, Robuchon M, Sala S. Critical review of methods and models for biodiversity impact assessment and their applicability in the LCA context. Environmental Impact Assessment Review. 2023; 101: 107134.
38. Thyvalikakath TP, Monaco V, Thambuganipalle H, Schleyer T. Comparative study of heuristic evaluation and usability testing methods. Stud Health Technol Inform. 2009; 143: 322-7. PMID: 19380955
39. Yen PY, Bakken S. A comparison of usability evaluation methods: Heuristic evaluation versus end-user think-aloud protocol – An example from a web-based communication tool for nurse scheduling. AMIA Annu Symp Proc. 2009; 2009: 714-8. PMID: 20351946
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Advances in Medical Informatics
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
