نشریه علمی پژوهشی طب انتظامی Journal of Police Medicine
Volume 11, Issue 1 (2022)
J Police Med 2022, 11(1) |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Mortezaei S, Mohammadnejad E. Usability Evaluation of a Military Medical Center’s Hospital Information System Based on ISO 9241. J Police Med 2022; 11 (1)
URL: http://jpmed.ir/article-1-1062-en.html
URL: http://jpmed.ir/article-1-1062-en.html
1- Research Center for Life & Health Sciences & Biotechnology of the Police, Directorate of Health, Rescue & Treatment, Police Headquarter, Tehran, Iran , kianm313@gmail.com
2- Faculty of Mechanics, Electrical Power & Computer, Science & Research Branch, Islamic Azad University, Tehran, Iran
2- Faculty of Mechanics, Electrical Power & Computer, Science & Research Branch, Islamic Azad University, Tehran, Iran
English Extended Abstract: (2697 Views)
INTRODUCTION
... [1, 2]. Due to the important role of information systems in improving the quality of health care services, their evaluation from various aspects is important [3]. Usability evaluation is one of the most famous evaluations in the field of information systems and is considered an important method of measuring the quality of information technology [2] and is directly related to productivity [4, 5]. ... [6]. The International Organization for Standardization (ISO) has introduced the interaction principles in ISO 9241 and provided guidelines for the ergonomic design of interactive software [7]. Information systems usability evaluation has been less studied in Iran [8, 9]. The methods used to evaluate usability are variable [2, 5, 10-12]. One of the common tools in assessing the usability of information systems is the isometric questionnaire, which has been developed according to the ISO 9241 standard [13]. ... [14-17].
AIM (s)
This study aimed to evaluate the usability of the hospital information system in one of the police medical centers.
RESEARCH TYPE
This is a descriptive cross-sectional study.
RESEARCH SOCIETY, PLACE, AND TIME
This study was performed in 2021 in one of the selected hospitals of Tehran, Iran police. All users of the hospital information system, including nurses, staff of the radiology unit, laboratory, emergency, surgery, pharmacy, and secretaries of the specialized unit were in the selected center of the statistical population of this study.
SAMPLING METHOD AND NUMBER
110 users were selected through convenience sampling.
USED DEVICES & MATERIALS
Users' views on system usability were collected through an isometric questionnaire [18] (Table 1). In this tool, to prevent unrealistic answers, an "No opinion" option was added, which was a score equivalent to the "average" answer [13]. The reliability of the tool was 0.7 and the face and content validity of the questionnaire were confirmed by three medical informatics specialists.
ETHICAL PERMISSIONS
The questionnaires were provided to the users with the necessary permissions and the needed explanations were provided regarding the confidentiality of the information received from the participants.
STATISTICAL ANALYSIS
In addition to descriptive and analytical statistical methods, to find the relationship of work experience and degree of education with usability, analysis of variance (ANOVA) was used in SPSS 26 software.
FINDING BY TEXT
110 people participated in the survey after distributing the questionnaires 23 of which were excluded due to the high number of unanswered questions or marking the option "No opinion" (N=87). The mean work experience of users was 10.62±14 years, with a minimum of less than one year and a maximum of 22 years. The distribution of users by occupation and degree was listed in Table 2. The mean score of all usability criteria in this study from the users' point of view was 3.23±0.36. Among the subjects, nurses had the highest satisfaction (3.44±0.36) and pharmacy users had the lowest satisfaction (2.41±0.11) from the information system (Table 3). Analysis of the variance test did not show any significant difference between the work experience of the subjects and the usability of the system. Similarly, no significant difference was observed between users' qualifications and usability. The degree of compliance of the studied information system with the ISO 9241-11 standard was shown in Figure 1 from the perspective of hospital wards users.
MAIN COMPARISON TO THE SIMILAR STUDIES
... [19, 20]. The compliance of the studied system with the ISO 9241 standard was equal to 65%, which is to some extent consistent with the results of the study of Ehteshami et al. (67%) [21]. The mean score of all usability criteria in this study from the perspective of the users was 3.23±0.36 while, the mean scores obtained in the study of Mousavi et al. were 2.61, in the study of Safdari et al. 2.94, in the study of Moghadasi et al. 2.90 and the study of Ahmadi et al. 2.95 [14-17]. It seems that the situation of usability in the present study community is more favorable than in the communities of the mentioned studies. The system controllability criterion had the lowest score in this study, which is consistent with the results of the study of Kumar et al. (Figure 2) [22]. ... [23-25]. In the present study, the users of the pharmacy unit assigned the lowest score to all the criteria of usability of the hospital information system, which complied with the standard of ISO 9241, 48%. Therefore, contrary to the findings of Bayati et al.'s study, in this study, the status of the pharmacy information system was unfavorable from the perspective of users [24]. The greatest dissatisfaction of the users of this unit was related to the fit for work and then the fit for learning. Therefore, it is recommended to pay more attention to the usability aspects, especially to the mentioned items in developing the pharmacy information system. ... [26]. In this study, the mean score of fitness for work was moderate, which is in line with the findings of previous studies [14-17]. The most dissatisfaction among users with this criterion is related to the large number of different steps to do things. The software produced should be designed in such a way that it does not need to perform different steps to perform a specific task [13, 27]. From the users' point of view, the system self-description had the highest average score and the users' view of this criterion was a relatively more favorable situation than previous studies [14-17]. ... [28]. The subjects in this study had less satisfaction with program control compared to other usability criteria. Regarding this criterion, the findings of this study were relatively similar to Moqaddasi et al. (0.3), Safdari et al. (0.9), and Ahmadi et al. (10.3) [14, 16, 17]. While users in the study of Mousavi et al., were less satisfied with the controllability of the system under study (63.2) [15]. The greatest dissatisfaction of users was related to the requirement of the system to do things in a fixed and specific sequence, which makes users not have enough flexibility and does not speed up the current tasks. Users' views on other controllable metrics, such as ease of switching between pages, ease of calling processes, and the ability to stop activities when needed, were relatively favorable. ... [29, 30]. In the criterion of meeting the user's expectations, the average score was 3.30±0.51, which compared to Mousavi et al., Moghadasi et al., Safdari et al., and Ahmadi et al., the users of this study had a relatively more favorable view [14-17]. In the studies of Mousavi et al., Moghadasi et al., Safdari et al., and Ahmadi et al., the mean error tolerance score was in line with the present study, and users' views on this criterion were relatively favorable [14-17]. In the present study, the fit criterion for personalization after control had the lowest mean score (3.15±0.87). In all previous studies, this criterion has the lowest average score from the perspective of users [14-17]. Therefore, despite the average score of users to this criterion, the findings of the present study were in line with previous studies. It can also be inferred from the findings of this study and previous studies that the fit criterion for personalization has been neglected by the developers of hospital information systems in Iran. Another criterion examined in the present study was fitness for learning. ... [31, 32]. The average fit for learning in the study of Safdari et al. in the information system was relatively low (98.2) [16]. However, in the study of Moghadasi et al. as well as Rashidi et al., this criterion has obtained the highest average among other applicability criteria (12.3 and 37.4, respectively) [17, 33]. This average was 97.2 in the study of Ahmadi et al. And 7.2 in the study of Mousavi et al. according to the mean of 3.22±0.59 obtained in the present study, it can be inferred that the country's information systems in some cases have performed weakly for learning and in some cases have performed well. In this study, the lowest score on the fitness criterion for learning was related to the amount of detail needed to remember the correct use of the software.
LIMITATIONS
One of the limitations of this study was the inability to estimate the number of hospital information system users. On the other hand, due to some limitations, the tool used in this study was a modified version of the isometric questionnaire, however the use of the original version of the questionnaire could provide a broader view of the information system under study.
SUGGESTIONS
Based on the findings of this study, a list of suggestions that can be used to improve the usability and performance of the information system of the studied hospital is given in Table 4. It is also recommended that such evaluations be performed periodically in other medical centers to ensure the efficiency, productivity, and proper functioning of these systems.
CONCLUSIONS
Examination of the hospital information system of one of the police force medical centers in terms of usability according to the ISO 9241-11 standard showed that despite the desirability of some features of this system, it can be improved in some cases, including the ability to control and fit for personalization.
ACKNOWLEDGMENTS
The authors would like to thank all users of the hospital information system understudy who contributed to the completion of the research questionnaires in this study.
CONFLICT OF INTEREST
The authors state that there is no conflict of interest in the present study.
FUNDING SOURCES
This research had no financial support.
TABLES & CHARTS
Table 2) Frequency distribution of users by work experience, degree and job
Table 3) Mean and standard deviation of the score of all users and segregation of hospital wards to the usability of the hospital information system
Table 4) Suggestions for improving the usability and improving the performance of the current hospital information system
Figure 1) Degree of compliance of the information system with the standard 9241-11, from the perspective of hospital ward users
Figure 2) Radar diagram of hospital information system usability from users' point of view
... [1, 2]. Due to the important role of information systems in improving the quality of health care services, their evaluation from various aspects is important [3]. Usability evaluation is one of the most famous evaluations in the field of information systems and is considered an important method of measuring the quality of information technology [2] and is directly related to productivity [4, 5]. ... [6]. The International Organization for Standardization (ISO) has introduced the interaction principles in ISO 9241 and provided guidelines for the ergonomic design of interactive software [7]. Information systems usability evaluation has been less studied in Iran [8, 9]. The methods used to evaluate usability are variable [2, 5, 10-12]. One of the common tools in assessing the usability of information systems is the isometric questionnaire, which has been developed according to the ISO 9241 standard [13]. ... [14-17].
AIM (s)
This study aimed to evaluate the usability of the hospital information system in one of the police medical centers.
RESEARCH TYPE
This is a descriptive cross-sectional study.
RESEARCH SOCIETY, PLACE, AND TIME
This study was performed in 2021 in one of the selected hospitals of Tehran, Iran police. All users of the hospital information system, including nurses, staff of the radiology unit, laboratory, emergency, surgery, pharmacy, and secretaries of the specialized unit were in the selected center of the statistical population of this study.
SAMPLING METHOD AND NUMBER
110 users were selected through convenience sampling.
USED DEVICES & MATERIALS
Users' views on system usability were collected through an isometric questionnaire [18] (Table 1). In this tool, to prevent unrealistic answers, an "No opinion" option was added, which was a score equivalent to the "average" answer [13]. The reliability of the tool was 0.7 and the face and content validity of the questionnaire were confirmed by three medical informatics specialists.
ETHICAL PERMISSIONS
The questionnaires were provided to the users with the necessary permissions and the needed explanations were provided regarding the confidentiality of the information received from the participants.
STATISTICAL ANALYSIS
In addition to descriptive and analytical statistical methods, to find the relationship of work experience and degree of education with usability, analysis of variance (ANOVA) was used in SPSS 26 software.
FINDING BY TEXT
110 people participated in the survey after distributing the questionnaires 23 of which were excluded due to the high number of unanswered questions or marking the option "No opinion" (N=87). The mean work experience of users was 10.62±14 years, with a minimum of less than one year and a maximum of 22 years. The distribution of users by occupation and degree was listed in Table 2. The mean score of all usability criteria in this study from the users' point of view was 3.23±0.36. Among the subjects, nurses had the highest satisfaction (3.44±0.36) and pharmacy users had the lowest satisfaction (2.41±0.11) from the information system (Table 3). Analysis of the variance test did not show any significant difference between the work experience of the subjects and the usability of the system. Similarly, no significant difference was observed between users' qualifications and usability. The degree of compliance of the studied information system with the ISO 9241-11 standard was shown in Figure 1 from the perspective of hospital wards users.
MAIN COMPARISON TO THE SIMILAR STUDIES
... [19, 20]. The compliance of the studied system with the ISO 9241 standard was equal to 65%, which is to some extent consistent with the results of the study of Ehteshami et al. (67%) [21]. The mean score of all usability criteria in this study from the perspective of the users was 3.23±0.36 while, the mean scores obtained in the study of Mousavi et al. were 2.61, in the study of Safdari et al. 2.94, in the study of Moghadasi et al. 2.90 and the study of Ahmadi et al. 2.95 [14-17]. It seems that the situation of usability in the present study community is more favorable than in the communities of the mentioned studies. The system controllability criterion had the lowest score in this study, which is consistent with the results of the study of Kumar et al. (Figure 2) [22]. ... [23-25]. In the present study, the users of the pharmacy unit assigned the lowest score to all the criteria of usability of the hospital information system, which complied with the standard of ISO 9241, 48%. Therefore, contrary to the findings of Bayati et al.'s study, in this study, the status of the pharmacy information system was unfavorable from the perspective of users [24]. The greatest dissatisfaction of the users of this unit was related to the fit for work and then the fit for learning. Therefore, it is recommended to pay more attention to the usability aspects, especially to the mentioned items in developing the pharmacy information system. ... [26]. In this study, the mean score of fitness for work was moderate, which is in line with the findings of previous studies [14-17]. The most dissatisfaction among users with this criterion is related to the large number of different steps to do things. The software produced should be designed in such a way that it does not need to perform different steps to perform a specific task [13, 27]. From the users' point of view, the system self-description had the highest average score and the users' view of this criterion was a relatively more favorable situation than previous studies [14-17]. ... [28]. The subjects in this study had less satisfaction with program control compared to other usability criteria. Regarding this criterion, the findings of this study were relatively similar to Moqaddasi et al. (0.3), Safdari et al. (0.9), and Ahmadi et al. (10.3) [14, 16, 17]. While users in the study of Mousavi et al., were less satisfied with the controllability of the system under study (63.2) [15]. The greatest dissatisfaction of users was related to the requirement of the system to do things in a fixed and specific sequence, which makes users not have enough flexibility and does not speed up the current tasks. Users' views on other controllable metrics, such as ease of switching between pages, ease of calling processes, and the ability to stop activities when needed, were relatively favorable. ... [29, 30]. In the criterion of meeting the user's expectations, the average score was 3.30±0.51, which compared to Mousavi et al., Moghadasi et al., Safdari et al., and Ahmadi et al., the users of this study had a relatively more favorable view [14-17]. In the studies of Mousavi et al., Moghadasi et al., Safdari et al., and Ahmadi et al., the mean error tolerance score was in line with the present study, and users' views on this criterion were relatively favorable [14-17]. In the present study, the fit criterion for personalization after control had the lowest mean score (3.15±0.87). In all previous studies, this criterion has the lowest average score from the perspective of users [14-17]. Therefore, despite the average score of users to this criterion, the findings of the present study were in line with previous studies. It can also be inferred from the findings of this study and previous studies that the fit criterion for personalization has been neglected by the developers of hospital information systems in Iran. Another criterion examined in the present study was fitness for learning. ... [31, 32]. The average fit for learning in the study of Safdari et al. in the information system was relatively low (98.2) [16]. However, in the study of Moghadasi et al. as well as Rashidi et al., this criterion has obtained the highest average among other applicability criteria (12.3 and 37.4, respectively) [17, 33]. This average was 97.2 in the study of Ahmadi et al. And 7.2 in the study of Mousavi et al. according to the mean of 3.22±0.59 obtained in the present study, it can be inferred that the country's information systems in some cases have performed weakly for learning and in some cases have performed well. In this study, the lowest score on the fitness criterion for learning was related to the amount of detail needed to remember the correct use of the software.
LIMITATIONS
One of the limitations of this study was the inability to estimate the number of hospital information system users. On the other hand, due to some limitations, the tool used in this study was a modified version of the isometric questionnaire, however the use of the original version of the questionnaire could provide a broader view of the information system under study.
SUGGESTIONS
Based on the findings of this study, a list of suggestions that can be used to improve the usability and performance of the information system of the studied hospital is given in Table 4. It is also recommended that such evaluations be performed periodically in other medical centers to ensure the efficiency, productivity, and proper functioning of these systems.
CONCLUSIONS
Examination of the hospital information system of one of the police force medical centers in terms of usability according to the ISO 9241-11 standard showed that despite the desirability of some features of this system, it can be improved in some cases, including the ability to control and fit for personalization.
ACKNOWLEDGMENTS
The authors would like to thank all users of the hospital information system understudy who contributed to the completion of the research questionnaires in this study.
CONFLICT OF INTEREST
The authors state that there is no conflict of interest in the present study.
FUNDING SOURCES
This research had no financial support.
TABLES & CHARTS
Table 1) usability evaluation criteria based on ISO 9241-11 (Gediga G, 1999)
| Criterion | Explanation |
| Fit for work | Support the user in order to achieve his goals in an effective and efficient way. Only the parts of the program that are needed to do the job are displayed. |
| Self-description | All stages of the work are intuitively understood and, if mistaken, lead to immediate feedback. At the request of the user, sufficient support is provided to the user. |
| Ability to control | The user can start the sequence of work and guide its speed and path to reach its goal. |
| In accordance with user expectations | The system is stable and robust and conforms to user specifications; For example, it considers the user's knowledge in terms of scientific education, work experience as well as general cases in a specific field of work. |
| Error tolerance | Despite incorrect input or misdirection, the desired results are achieved with minimal or no extra effort. |
| Fit for personalization | The personalization system is made possible according to the desired activity as well as the capacity and preferences of the user. |
| Fit for learning | The system accompanies the user in the various stages of learning and the effort to learn is as small as possible. |
Table 2) Frequency distribution of users by work experience, degree and job
| Variable | Number | Percentage | |
| work experience | Less than 1 year | 5 | 5.7 |
| 1 to 3 years | 9 | 10.3 | |
| 4 to 6 years | 7 | 8 | |
| 7 to 10 years | 30 | 34.5 | |
| Over 10 years | 33 | 37.9 | |
| Total* | 84 | 96.6 | |
| Degree | Diploma and high school diploma | 17 | 19.5 |
| Bachelor's Degree | 7 | 8 | |
| Master degree | 54 | 62.1 | |
| Graduate | 8 | 9.2 | |
| Total* | 86 | 98.8 | |
| The part | Laboratory | 9 | 10.3 |
| Radiology | 13 | 14.9 | |
| the reception | 26 | 29.9 | |
| Nursing | 18 | 20.7 | |
| Pharmacy and medicine warehouse | 6 | 6.9 | |
| Emergency | 15 | 17.2 | |
| Total | 87 | 100 | |
Table 3) Mean and standard deviation of the score of all users and segregation of hospital wards to the usability of the hospital information system
| Index | Laboratory | Radiology | reception | Nursing | Pharmacy | Emergency | Mean |
| Fit for work | 3.0±33.84 | 3.25±0.63 | 3.53±0.75 | 2.99±0.65 | 1.9±0.47 | 2.99±0.21 | 3.15±0.75 |
| Self-description | 3.67±0.46 | 3.34±0.81 | 3.51±0.63 | 3.5±0.78 | 3±0.49 | 3.39±0.27 | 3.44±0.63 |
| Ability to control | 3.16±0.71 | 3.15±0.43 | 2.95±0.61 | 3.44±0.56 | 2.53±0.21 | 3.24±0.2 | 3.13±0.55 |
| In accordance with user expectations | 3.13±0.78 | 3.18±0.43 | 3.35±0.46 | 3.56±0.54 | 2.9±0.24 | 3.29±0.423 | 3.3±0.51 |
| Error tolerance | 3.2±0.55 | 2.78±0.42 | 3.38±0.45 | 3.52±0.53 | 2.43±0.57 | 3.2±0.26 | 3.2±0.54 |
| Fit for personalization | 2.98±0.89 | 2.77±1 | 2.95±0.82 | 3.82±0.73 | 2.2±0.25 | 3.49±0.38 | 3.15±0.87 |
| Fit for learning | 3.27±0.46 | 3.43±0.54 | 3.53±0.5 | 3.22±0.32 | 1.93±0.48 | 2.99±0.32 | 3.22±0.59 |
| mean | 3.25±0.32 | 3.13±0.29 | 3.31±0.29 | 3.44±0.36 | 2.41±0.11 | 3.23±0.11 |
Table 4) Suggestions for improving the usability and improving the performance of the current hospital information system
| Criterion | Suggestion |
| Fit for work | Pay more attention to the user needs of each department, especially units with lower usability |
| Reduce the number of steps a system user needs to take to perform a specific activity to reach their goal | |
| Self-description | Show related descriptions if needed by the user, at any stage of working with the system, for example, placing a help button on each page to provide related explanations |
| control-ability | Increasing the flexibility of the system so that it does not need to do a task has always followed a fixed sequence For example, consider shortcut keys to move between different parts of the system |
| In accordance with user expectations | Facilitate activities by the user For example, in cases where it is possible to scan the barcode, it is not necessary for the user to enter information |
| Error tolerance | Prevent errors and serious consequences, in case of user error For example, requesting confirmation or warning when a malicious action is taken, such as deleting data and information, or the possibility of going back to the previous steps in case of a mistake by the user |
| Fit for personalization | Provide facilities that the user can adjust the software response speed according to their work speed. |
| For example, considering command codes and macros for various system activities for more experienced users | |
| Fit for learning | Reduce the details that the user needs to remember to use the software, for example, autocomplete or make suggestions when searching for a drug name |
Figure 1) Degree of compliance of the information system with the standard 9241-11, from the perspective of hospital ward users
Figure 2) Radar diagram of hospital information system usability from users' point of view
Article Type: Original Research |
Subject:
Police Medicine Related Technologies
Received: 2021/11/21 | Accepted: 2022/02/28 | Published: 2022/04/6
Received: 2021/11/21 | Accepted: 2022/02/28 | Published: 2022/04/6
References
1. Wager KA, Lee FW, Glaser JP. Health Care Information Systems. 4th ed. Jossey-Bass: United State; 2017. 608 p. https://www.amazon.com/Health-Care-Information-Systems-Management/dp/1119337186
2. Farzandipour M, Riazi H, Jabali MS. Proposing electronic health record usability requirements based on enriched ISO 9241 metric usability model. Acta Inform Med. 2018;26(1):29-34. [DOI:10.5455/aim.2018.26.29-34] [PMID] [PMCID]
3. Bürkle T, Ammenwerth E, Prokosch HU, Dudeck J. Evaluation of clinical information systems. What can be evaluated and what cannot? J Eval Clin Pract. 2001;7(4):373-85. doi:10.1046/j.1365-2753.2001.00291.x [DOI:10.1046/j.1365-2753.2001.00291.x] [PMID]
4. Viitanen J, Hyppönen H, Lääveri T, Vänskä J, Reponen J, Winblad I. National questionnaire study on clinical ICT systems proofs: Physicians suffer from poor usability. Int J Med Inform. 2011;80(10):708-25. doi:10.1016/j.ijmedinf.2011.06.010 [DOI:10.1016/j.ijmedinf.2011.06.010] [PMID]
5. Assila A, Ezzedine H. Standardized usability questionnaires: Features and quality focus. eJCIST. 2016;6(1). http://ejcsit.uniten.edu.my/index.php/ejcsit/article/view/96
6. Donahue GM. Usability and the bottom line. IEEE Softw. 2001;18(1):31-2. doi:10.1109/52.903161 [DOI:10.1109/52.903161]
7. Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts (ISO 9241-11:2018). [Internet]. International Organization for Standards; 2018 [cited 2020 Nov 17]. Available from: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
8. Atashi A, Khajouei R, Azizi A, Dadashi A. Erratum: user interface problems of a nationwide inpatient information system: A heuristic evaluation. Appl Clin Inform. 2016;07(1):e1. doi:10.1055/s-0040-1701485 [DOI:10.1055/s-0040-1701485]
9. Mohammadi R, Bostan H, Darabi H, Lazem M, Maleki M, Ghalaneh S et al. Assessment of laboratory information system (LIS) performance in two military hospitals: A cross sectional study. J Police Med. 2019;8(1):5. [Persian]. doi.org/10.30505/8.1.1
10. Seffah A, Donyaee M, Kline RB, Padda HK. Usability measurement and metrics: A consolidated model. Softw Qual J. 2006;14(2):159-78. doi:10.1007/s11219-006-7600-8 [DOI:10.1007/s11219-006-7600-8]
11. Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: A scoping review.Inter J Medi Informatics. 2019;126:95-104. doi:10.1016/j.ijmedinf.2019.03.018 [DOI:10.1016/j.ijmedinf.2019.03.018] [PMID]
12. Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Informatics Assoc. 2012;19(3):413-22. doi:10.1136/amiajnl-2010-000020 [DOI:10.1136/amiajnl-2010-000020] [PMID] [PMCID]
13. Gediga G, Hamborg KC, Düntsch I. IsoMetrics usability inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems. Behav Inf Technol. 1999;18(3):151-64. doi:10.1080/014492999119057 [DOI:10.1080/014492999119057]
14. Ahmadi M, Shahmoradi L, Barabadi M, Hoseini F. Usability evaluation of hospital information systems based on isometric 9241. Hakim Reseach J. 2011;13(4):226-33. [Persian]. https://www.sid.ir/fa/journal/ViewPaper.aspx?id=153015
15. Mousavi A, Hemati M, Kayyal S, Shamshiri A. Usability evaluation of hospital information systems based on isometric standard from the viewpoint of usersin education and treatment centers affiliated to Qom University of Medical Sciences, Iran. Qom Univ Med Sci J. 2014;8(4):58-63. [Persian]. https://www.sid.ir/fa/journal/ViewPaper.aspx?id=226075
16. Safdari R, Dargahi H, Shahmoradi L, Farzaneh Nejad A. Comparing four softwares based on ISO 9241 part 10. J Med Syst. 2021;36(5):2787-93. [Persian]. doi:10.1007/s10916-011-9755-5 [DOI:10.1007/s10916-011-9755-5] [PMID] [PMCID]
17. Moghaddasi H, Rabiei R, Asadi F, Ostvan N. Evaluation of nursing information systems : Application of usability aspects in the development of systems. Health Inform Res. 2017;23(2):101-8. [Persian]. doi:10.4258/hir.2017.23.2.101 [DOI:10.4258/hir.2017.23.2.101] [PMID] [PMCID]
18. Kasunic M. Designing an effective survey. Softw Eng Inst. 2005. doi:10.1184/R1/6573062.v1
19. Rogers ML, Patterson E, Chapman R, Render M. Usability testing and the relation of clinical information systems to patient safety. Nat Library Med. 2005;2. https://pubmed.ncbi.nlm.nih.gov/21249828/
20. Azizi A, Maniati M, Ghanbari-Adivi H, Aghajari Z, Hashemi S, Hajipoor B,. Usability evaluation of hospital information system according to heuristic evaluation. front health informatics. 2021;10(1).69 doi:10.30699/fhi.v10i1.271 [DOI:10.30699/fhi.v10i1.271]
21. Ehteshami A, Sadoughi F, Saeedbakhsh S, Isfahani MK. Assessment of medical records module of health information system according to ISO 9241-10. Acta Inform Med. 2013;21(1):36-41. doi:10.5455/aim.2012.21.36-41 [DOI:10.5455/aim.2012.21.36-41] [PMID] [PMCID]
22. Kumar KUV, Subramoniam S. Usability analysis of an Indian e-governance software. Electron Gov. 2013;10(2):211-21. doi:10.1504/EG.2013.052602 [DOI:10.1504/EG.2013.052602]
23. Mahoney CD, Berard-Collins CM, Coleman R, Amaral JF, Cotter CM. Effects of an integrated clinical information system on medication safety in a multi-hospital setting. Am J Health Pharm. 2007;64(18):1969-77. doi:10.2146/ajhp060617 [DOI:10.2146/ajhp060617] [PMID]
24. Bayati S, Bastani P, Sagheb ZM, Jamalabadi S, Samadbeik M. The performance implications of pharmacy information system at the university teaching hospitals of Shiraz, Iran: Cluster approach. J Adv Pharm Technol Res. 2017;8(4):125-30. doi:10.4103/japtr.JAPTR_13_17 [DOI:10.4103/japtr.JAPTR_13_17] [PMID] [PMCID]
25. Asadi F, Moghaddasi H, Hosseini A, Maserrat E. A survey on pharmacy information system at hospitals affiliated to Shahid Beheshti University of medical sciences 2009. J Health Adm. 2010;13(41):31-40. Available from: http://jha.iums.ac.ir/article-1-685-en.html [Persian]
26. Hamborg K-C, Vehse B, Bludau H-B. Questionnaire based usability evaluation of hospit al information systems. Electron J Inf Syst Eval. 2004;7(1):21-30. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.67.7129&rep=rep1&type=pdf
27. Lin HX, Choong YY, Salvendy G. A proposed index of usability: A method for comparing the relative usability of different software systems. Behav Inf Technol. 1997;16(4-5):267-77. doi:10.1080/014492997119833 [DOI:10.1080/014492997119833]
28. 10 Usability Heuristics for User Interface Design [Internet]. Nielsen J. 1995. [cited 2019 Nov]. Available from: https://www.nngroup.com/articles/ten-usability-heuristics/
29. Pribeanu C. A revised set of usability heuristics for the evaluation of interactive systems. Inform Econ. 2017;21(3):31-8. doi:10.12948/issn14531305/21.3.2017.03 [DOI:10.12948/issn14531305/21.3.2017.03]
30. Conrath DW, Mignen OP. What is being done to measure user satisfaction with EDP/MIS. Inf Manag. 1990;19(1):7-19. doi:10.1016/0378-7206(90)90010-F [DOI:10.1016/0378-7206(90)90010-F]
31. Paymans TF, Lindenberg J, Neerincx M. [proceeding]. Usability trade-offs for adaptive user interfaces: Ease of use and learnability.2004. New York, NY, United States.2004. 396p. doi:10.1145/964442.964512 [DOI:10.1145/964442.964512]
32. Jeng J. Usability assessment of academic digital libraries: Effectiveness, efficiency, satisfaction, and learnability. Libri. 2005;55(2-3). doi:10.1515/LIBR.2005.96 [DOI:10.1515/LIBR.2005.96]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |