Evaluation Of Information Systems Research Articles

SOFTWARE QUALITY EVALUATION USING ISO/IEC 9126 ON THE ITB STIKOM BALI ONLINE ACADEMIC INFORMATION SYSTEM (SION)

The success of the software that is built is seen based on whether or not the software works according to the ongoing business processes. One of the private universities that has used software as a tool in providing the information needed by students and other community members is ITB STIKOM Bali. The problem that occurs at ITB STIKOM Bali is that the Online Information System (SION) as application software has been implemented for a long time but its reliability has not been tested. The method used in this research is observation with a literature study flow, initial data collection, identification of evaluation needs, evaluation specifications, evaluation design, evaluation implementation and conclusions. This research uses the ISO 9126 framework and the characteristics taken are Usability and Functionality. The results obtained by the researchers were a successful evaluation of the ITB STIKOM Bali Online Academic Information System (SION) using a valid and consistent questionnaire and produced Usability results of 88.12% and Functionality results of 99.55%. These results show that the ITB STIKOM Bali Online Academic Information System (SION) can be used and functions well.

CSIMQ Anniversary Editor-in-Chief Thoughts and Editorial Introduction to Issue 38: Model-based and Decision Support Methods for Next-generation Information Systems

CSIMQ enters its second decade of existence, therefore the preface for the current issue starts with a message from the Editor-in-Chief, highlighting the journal’s mission of dealing with complexity management and systems modeling as meta-fields that have emerged from Information Science but remain relevant across diverse domains. Afterwards, the 38th issue is introduced by summarizing the four articles selected by guest issue editors on topics related to novel decision support methods or model-based frameworks for evolving or evaluating information systems. Design-oriented research is the dominant approach in these works, balancing technical design decisions insights with empirical evaluation cases. Three of the selected articles contribute with decision-support methods or frameworks – for ESG (Environment-Social-Governance) accounting, for democratized decision services and for information security management. Another article revisits UML-based model-driven software development from a new perspective.

Evaluation and Development of Building Material Sales Information System to Improve Inventory Management and Customer Service

This research investigates the Evaluation and Development of Building Material Sales Information Systems (BMIS) to Enhance Inventory Management and Customer Service within the construction industry. Through a qualitative analysis approach, the study explores the challenges faced by building material suppliers, identifies the requirements for effective BMIS, and evaluates the impacts of BMIS implementation on inventory management practices and customer service. Data were collected through interviews, focus group discussions, and case studies with stakeholders representing diverse perspectives within the construction supply chain. The findings highlight the critical role of BMIS in addressing inventory management challenges, improving order accuracy, and enhancing communication with customers. The study contributes to the development of practical recommendations for building material suppliers seeking to optimize their operations and deliver superior service experiences to their clientele.

Modelling of physicians’ clinical information-seeking behaviour in Iran: a grounded theory study

ObjectivesExploring clinical information-seeking behaviour (CISB) and its associated factors contributes to its theoretical advancement and offers a valuable framework for addressing physicians’ information needs. This study delved into the dimensions,...

Development and Evaluation of Drug Management Information System in Mijen Community Health Center Demak

Background: Drug management information systems can serve as tools for decision-making. The effectiveness of such a system can be assessed through evaluation. One of the evaluation methods used is the PIECES method.Objectives: This research aims to evaluate and develop a drug management information system using the PIECES method.Methods: The research conducted was a qualitative descriptive study with inductive analysis. Sampling for this research was conducted purposive sampling. System development in this study utilized the prototype method, with interviews conducted for Management Information System (MIS) evaluation using the PIECES Framework. The design of the MIS system was created using Data Flow Diagrams and Entity Relationship Diagrams. The designed MIS system development was evaluated using the same method. Data processing in qualitative research consists of valid, clear, and accountable word descriptions, sentences, or narratives provided by the researchers, followed by data validation using triangulation techniques and Nvivo 12 Plus software.Results: Evaluation results using the PIECES Framework indicate that the implemented MIS system benefits and aids staff performance in daily drug dispensing. However, there are still manual tasks and lacking features such as distribution, drug expiration warnings, and minimum stock on the dashboard. The proposed design for MIS system development includes several features such as stock in, stock out, and distribution, complemented with drug expiration warnings, minimum stock alerts on the dashboard, and reports for each feature.Conclusion: Implementing the proposed MIS system design can assist and streamline work processes by providing comprehensive information, thus serving as a decision-making tool.

Development of a parametric-based Analytical Hierarchy Process (AHP) utilizing Geographic Information Systems (GIS) for wheat land suitability evaluation

Wheat is considered one of the most essential crops for Egypt. Nevertheless, it is also one of its largest imports. Therefore, it is important to develop an accurate wheat suitability model to define the most suitable areas for its production. This study aimed to develop a parametric-based Analytical Hierarchy Process (AHP) using Geographic Information Systems (GIS) for land suitability evaluation of wheat in a selected area in El-Beheira governorate, Egypt. The climatic and land parameters influencing wheat production in the studied area were selected and rated according to the parametric method. These parameters included slope, texture, calcium carbonate, sum of basic cations, pH, organic matter, salinity, exchangeable sodium percentage and mean temp. of the growing cycle. The rated parameters were processed according to the AHP. The results were compared with Storie and the Square root methods and field observations. When validated using field observations, the developed method had a higher accuracy suitability evaluation for wheat cultivation in the studied area than the other two methods. According to the developed method, almost all of the studied areas could be classified as very suitable (S1) for wheat cultivation. On the other hand, the wheat suitability evaluation according to the other two methods indicated that most of the studied area could be classified as moderately suitable (S2) and marginally suitable (S3), which contradicted the field survey.

OPTIMALISASI PENERAPAN SISTEM APLIKASI E-PUSKESMAS: TINJAUAN TERHADAP PROSES DAN KEEFEKTIFAN DI PUSKESMAS TAJINAN KABUPATEN MALANG

The development of digital technology in this modern era has resulted in a digital transformation, including in the realm of health services. This effort is strengthened by the Republic of Indonesia Ministry of Health Regulation Number 24 of 2022, which regulates legal certainty in the management andadministration of Electronic Medical Records (RME). This regulation must be implemented by all health service facilities no later than the end of 2023. This research aims to analyze the implementation of the EPuskesmas Application System at the Tajinan Health Center, Malang Regency. The method used in this research is descriptive qualitative. Data collection was carried out through observations and interviews with health workers using the HOT-Fit information system evaluation model, which includes Human, Organizational, Technology and Benefit aspects. The research results show that the E-Puskesmas application has a variety of menus and features that are considered very profitable and useful, able to help lighten the workload of health workers. Community health center management also supports the implementation of this application by providing adequate hardware. However, there are stillshortcomings, namely the absence of official Standard Operating Procedures (SOP) regarding the implementation of E-Puskesmas, as well as a lack of special training for officers to operate the application. The implications of these findings indicate the importance of preparing official SOPs and appropriatetraining to optimize the implementation of the E-Puskesmas Application System at the Tajinan Health Center, Malang Regency.

Analisis Kepuasan Pengguna Terhadap Penerapan Sistem Informasi Manajemen Puskesmas Di Puskesmas Sibabangun Menggunakan Metode EUCS Tahun 2023

Puskesmas services are progressing day by day, making them increasingly complex, therefore a puskesmas management information system is needed that is able to guarantee the availability of data quickly, accurately, up to date, continuously and responsibly. The community health center management information system (SIMPUS) is a community health center management application whose main function is as a tool for recording, processing and presenting information on all patient data from initial registration to reporting. Every Community Health Center is obliged to implement (SIMPUS) both electronic and non-electronic which is part of the health information system. One method used to evaluate the success of a system as seen from user satisfaction is End User Computing Statistics (EUCS). The EUCS definition of an information system itself is an overall evaluation of information system users with more emphasis on end user satisfaction (statisfaction) with 5 satisfaction assessment variables including content, format, accuracy, timeliness, ease of use. This research was conducted at the Sibabangun Health Center, Sibabangun sub-district, Central Tapanuli district in 2023, involving 30 respondents actively using SIMPUS with the EUCS method by sending questionnaires using Google Form to the respondents with the research results obtained being that on average respondents were satisfied with the implementation of SIMPUS but not closes the possibility that there are still many problems experienced by users, especially in the dimensions of accuracy and ease of use, with problems with system errors and ease of use which needs to be considered by providing training to all SIMPUS users.

The Application of COBIT Framework to Evaluate Information System Governance in National Business Technology Transformation Companies

The purpose of this study is to determine how to assess information technology governance by utilizing the COBIT 5 framework and the domains of APO01 (Manage the IT Management Framework), EDM04 (Ensure Resource Optimization), and APO04 (Manage Innovation). The COBIT 5 architecture is employed in this study. Both qualitative and quantitative research methodologies were used in this study. Through surveys, interviews, and observations, we gathered primary data. The secondary data came from various existing sources, including websites, literature reviews, and the findings of other investigations. This study's information technology governance data analysis approach complies with COBIT 5's Assessment Process Activities. According to the analysis, the current state's capability value in the EDM04 domain is 1.8. In the meantime, the capability value is 2.9 for the anticipated circumstance. The APO01 domain's recent state results in a capability value of 1.7. In the meantime, the capability value is 3.1 for the expected circumstance. In the APO04 domain, the current state has a capability value 1.6. Under ideal circumstances, the business receives a capacity value of 3. This implies that the business needs to fulfill the prerequisites for level 2 process capacity indicators, which are now unfulfilled. For instance, producing papers outlining the management of the innovation process and documents evaluating rejected innovative ideas.

Evaluasi Sistem Informasi Layanan Antrian Paspor Online Menggunakan Framework Information Technology Infrastructure Library V3

Palembang Class 1 Immigration Office is a service provider agency in the immigration sector that has the authority to arrange passports or Travel Letters of the Republic of Indonesia (SPRI). In meeting the needs of the community in passport processing, Palembang Class 1 Immigration Office has implemented an Online Passport Queuing Service System. Until now, the system still experiences problems such as errors in the system, exhausted quotas, inappropriate applicant NIK, slow access to the system, and system handling that always involves the Immigration Head Office. Therefore, an evaluation of the online passport queue service information system is needed which can be used as a benchmark for the sustainability of the system. The framework used is the Information Technology Infrastructure Library (ITIL) V3 framework with the domain of Service Operation and Continual Service Improvement. This research was conducted by distributing questionnaires, interviews and observations to obtain maturity level scores. The results of the maturity level calculation in the Service Operation (SO) domain show that the average value of maturity level in the SO domain is 3.49 or is at level 3 (Define Process) and for the Continual Service Improvement (CSI) domain is 3.47 or is at level 3 (Define Process). From the evaluation results of the Online Passport Queuing Service System based on the maturity level calculation, several recommendations for the Service Operation (SO) and Continual Service Improvement (CSI) domains can be taken into consideration in improving service operations and improving the Online Passport Queuing Service System on an ongoing basis.

Evaluation of logistics management information system and availability of non-program tracer drugs in public health facilities in Bahir Dar City, Northwest Ethiopia.

The logistics management information system (LMIS) plays a crucial role in effective record-keeping and reporting, ensuring efficient management of stock status and consumption data. A proficient LMIS improves accountability and supports informed logistic decisions in healthcare. Conversely, a subpar LMIS negatively affects essential medicine availability, compromising overall healthcare service efficiency. This study aimed to evaluate the status of the logistics management information system and the availability of non-program tracer drugs (NPTDs) in public health facilities within Bahir Dar City. This study employed an institutional-based cross-sectional study. Data were collected from February 20 to April 30, 2022 in 12 public health facilities located in Bahir Dar City. Structured questionnaire and data abstraction formats were used to gather pertinent data. After checking for completeness, the data were analyzed using Microsoft Excel and SPSS version 23. The primary analytical outcomes involved descriptive statistics, encompassing frequencies, averages, and percentages, which were subsequently presented in tables and figures. Bin card and Internal Facility Report and Resupply Form were the only blank LMIS tools available in all health facilities, while stock-record card was the least available, present in only 2(16.7%) facilities. Nine health facilities (75%) used self-prepared forms to request NPTDs from the Ethiopian Pharmaceutical Supply Agency (EPSA) at the end of the review period, whereas 7(58.3%) used official letters for emergency orders. Additionally, seven health facilities (58.3%) used the Health Commodities Management Information System On the day of the visit, 78.68% of NPTDs were available. Tetracycline eye ointment 1% had the longest stock-out duration, lasting for a mean 69.64 days. Rather than using the RRF, most of facilities opted for their own forms to request NPTDs from EPSA. While it is advisable for all health facilities to maintain continuous availability of tracer drugs, this study revealed that the current state of non-program tracer drug availability falls short of meeting this expectation.

Student Satisfaction Assessment Study of E-Learning Users: User Satisfaction with IT (USIT)

E-Learning has changed the traditional learning paradigm that is limited to a certain physical space and time. Through eLearning, students can access various educational contents, such as learning modules, learning videos, interactive exercises, and discussion forums, which can be accessed anytime and anywhere by using electronic devices such as computers, laptops, or smartphones.User Satisfaction with IT (USIT) was developed by DeLone and McLean in the context of Information Systems Evaluation in 1992. DeLone and McLean proposed a framework for measuring the success of information systems based on several dimensions including user satisfaction. This USIT framework has undergone development and modification from time to time by researchers and practitioners in the field of Information Systems Evaluation. This study aims to determine the level of satisfaction on the online simulation using the USIT model which is serious in user satisfaction which consists of variables in USIT.This study shows the level of satisfaction of students as e-learning users. It can be concluded that the majority of students are satisfied with their experience using the e-learning platform. This can be seen from the level of positive responses to questions relating to user satisfaction.

Analisis Sistem Informasi Driver Maxim Indonesia Menggunakan Pendekatan Evaluasi Usability Heuristik dan Human Resource Information System

Driver information system analysis includes evaluation and in-depth understanding of various aspects of the system used by drivers in a service or platform. Driver information system analysis helps ensure that the technology infrastructure that supports driver operations in application-based transportation services is efficient, safe, and in line with business and driver needs. This research aims to analyze the usability of the driver information system with a case study of the Maxim driver information system. The method used to analyze Maxim drivers uses the Heuristic Evaluation (HE) and Human Resource Information System (HRIS) approaches. The research results found that the Maxim driver information system is easy to use and the information contained in it is quite clear.

Analysis of The Level of Satisfaction of Researchers and Devotioners with The Performance

The development of information systems and technology has certainly changed people's behavior in carrying out their activities which were initially carried out manually and can now be carried out digitally so that they are more effective and efficient. The Service Research Information System (SIPP) is a transformation carried out to digitize research and service services within the Semarang State University (UNNES). As a service provider, to find out whether the UNNES Service Research Information System (SIPP) is running as it should, an evaluation process is needed on the performance of the information system. In carrying out information system analysis and evaluation activities, the PIECES Framework analysis model is used. PIECES framework is a framework that has six categories for classification and problem solving, namely Performance, Information and Data, Economics, Control and Security, Efficiency, and Service. The use of the PIECES framework method is very suitable for analyzing each system variable studied because it can provide real results on the effectiveness of the system used by UNNES. The aim of this research is to measure the level of satisfaction, find out strengths and weaknesses, analyze what components need to be improved in the quality of service at SIPP.

A qualitative analysis of the key contextual factors for the scaling-up of the ADLIFE toolbox in the Basque Health Service

Introduction: The translation of new digital solutions in different settings requires a preliminary analysis of contextual factors. The results might contribute to the development of strategies addressing barriers, for a successful implementation.
 ADLIFE is an EU funded project to improve the quality of life of older people with advanced chronic diseases by providing integrated personalised care via a toolbox including innovative digital solutions. The contextual factors of the participating organizations were evaluated prior to implementing this toolbox to identify the most relevant ones for the translation of this innovation action into routine practice.
 To structure this evaluation process, a bespoke framework for implementation assessment was developed, based on the Health Information System evaluation framework HOT-fit and the Consolidated Framework for Implementation Research (CFIR). These frameworks provide a structured and systematic way to identify constructs influencing the implementation of ADLIFE.
 This work presents the analysis of the contextual factors that would influence the uptake and effectiveness of ADLIFE tools in the Basque Health Service.
 Methods: Semi-structured interviews were conducted with different stakeholders before the implementation of the ADLIFE toolbox. A standardized set of questions was utilized, based on the main dimensions of the developed framework (human, organization, technology).
 These interviews were conducted online. They were recorded, transcribed, pseudonymized and subsequently analyzed using qualitative content analysis. These analyses were conducted using the software MAXQDA.
 Results: A total of 9 interviews were conducted: 2 primary care physicians, 2 specialized care physicians, 2 nurses, 2 IT staff and a manager.
 Through the implementation of the ADLIFE toolbox, interviewees expect improvements in communication between professionals and with patients, as well as facilitation of patient empowerment.
 Training about the new systems and time to get used to them were considered essential, which would help to overcome the digital gap that currently influences the lack of adherence to this type of intervention.
 Finally, direct linkage between the existing and the new systems were considered key to motivate professionals to participate and to facilitate the integration of patient care.
 Discussion and conclusions: The scaling up of the ADLIFE toolbox in the Basque Health Service requires it to be integrated into the digital systems currently used for the management of the Electronic Health Record. ADLIFE can serve to promote an improvement, not only in quality of life, but also in the quality of care, provided that professionals who care for these patients are involved.
 This knowledge will help to establish the most effective strategies to facilitate the intervention in the Basque Health Service.
 Acknowledgements
 We are grateful for the participation of the professionals who were interviewed.
 ADLIFE (GA 875209) received funding from the EU Horizon 2020. The authors thank all partners within ADLIFE.

Evaluasi Sistem Informasi Akuntansi Persediaan Barang Dalam Meningkatkan Pengendalian Internal Pada Percetakan Isma Kreasindo

This research was conducted at Isma Kreasindo, a printing services company in Surabaya. In this research, the evaluation of the raw material inventory accounting information system uses SAS 94 as the basis for the research. From the evaluation results, it can be concluded that the internal control system and accounting information system at Isma Kreasindo have not been implemented properly. It can be seen from the dual duties between the admin section and the warehouse section. There is also a duplication of the purchasing department with the inventory control department. So there is no guarantee regarding monitoring of raw material stock in the warehouse.

Identification and classification of indicators for evaluating health information systems

BackgroundSeveral evaluation methods are used to determine the advantages and disadvantages of healthcare information systems and their contribution to attaining organizational goals. Despite the existence of many evaluation frameworks, there is no comprehensive set of indicators that evaluate different dimensions of information systems. This study aimed to develop a set of indicators for evaluating health information systems. MethodsThis research was conducted in three phases. First, based on a literature review of PubMed, Web of Science, Scopus, and Embase databases, studies using the health information system evaluation methods were extracted. Second, consecutive focus group meetings were held with scientific and executive experts to discuss the list of evaluation indicators extracted from the studies. In these meetings, the experts agreed on including, removing, adding, combining, and grouping the indicators. Third, the indicators were weighted using the Analytical Network Process (ANP) method, and the set of evaluation indicators was finalized. ResultsThe review of 177 relevant articles resulted in the extraction of 360 indicators. During the focus group meetings, 174 overlapping and duplicate indicators were eliminated and 61 indicators were added to the model based on experts’ suggestions. The remaining 247 indicators were classified into a four-level hierarchy. The final set consisted of 4 dimensions, 16 criteria, 47 markers, and 180 indicators. ConclusionWe developed a comprehensive general set of indicators that helps researchers, designers, and developers of health information systems to evaluate different dimensions of these systems. This set can also be used to improve the design of relevant systems.

Usability evaluation of emergency information systems in educational hospitals in Kerman, Iran

BackgroundSmart and practical health information systems and applications with fewer errors are crucial for healthcare facilities. One method that ensures the proper design of health information systems (HIS) and applications is usability evaluation.ObjectiveThis study aimed to evaluate the usability of the emergency information systems used at the emergency departments of four educational hospitals in Kerman, Iran.MethodThis study was conducted in two phases. In the first phase, the information systems' errors and shortages were identified using a semi-structured questionnaire by users (nurses and the IT staff). In the second phase, based on the results of the first phase, two questionnaires were designed for each group of users to their opinions about the usability of the emergency information systems.ResultsThe average score of “reducing and facilitating user’s daily activities” was significantly different among hospitals (p = 0.03). Shahid Beheshti Hospital obtained the lowest usability score (17.5), and Afzalipour Hospital received the highest usability score (21.75). Moreover, the average score in “use of the HIS” for nurses and IT staff was 2.93 and 3.54 on a scale of 5, respectively.ConclusionUsability evaluation of health information systems is essential to ensure that these systems provide sufficient and accurate information and requirements for users and health care providers. Also, modifying health information systems based on the user views and expectations improves the quality of the system and user-system Interaction.

Evaluation of the Service Quality of the Jaticempaka Village Website, Bekasi City, Using the PIECES Method

Government agencies have widely used advances in information technology in the modern era to improve public services. Websites have become one of the popular media for accessing information. Jati Cempaka Village, Bekasi City, is one of the institutions that has implemented websites to provide, convey, and publish public information in the context of good governance. This research aims to understand, analyze, and improve the quality of local government website services to support transparency, community participation, and more efficient and effective services. Provide recommendations for appropriate improvements or enhancements to improve the service quality of the Jati Cempaka Village website so that it can provide better benefits to the community and users. The research method used is the PIECES method, which can be used to review and evaluate information systems in an organization, company, or institution. This method analyzes several aspects: performance, information, economy, security, efficiency, and service. In addition, the PIECES method helps analyze the quality of website services with a structured approach and can provide concrete guidance for necessary improvements. User satisfaction based on the analysis of website quality using the PIECES method, it can be concluded that the quality of website-based information services provided by Jati Cempaka Village, Bekasi City, as a community media is considered good. The results of this research can also guide the authorities in Jati Cempaka Village to improve the quality of their website services to provide better benefits to the community and users.

Development of the National Policy for Quality in Healthcare for Malaysia

BackgroundQuality in healthcare is a fundamental pillar of health systems performance, leading to improved health outcomes and reduced waste. The World Health Organization recommends that countries establish a national quality policy and strategy (NQPS) to steer the provision of safe and high-performing healthcare services and foster a quality culture. This paper describes the development process and key content of Malaysia’s new 5-year National Policy for Quality in Healthcare.MethodsThe development process was managed by a technical working group led by the Institute for Health Systems Research in the Ministry of Health. Situational analysis was conducted through a multi-pronged approach, underpinned by a review of the past and present healthcare sectoral and quality plans and guided by the WHO NQPS framework. This approach involved: (i) review of quality-related policy documents, (ii) online surveys of healthcare providers and the public, (iii) key-informant facilitated discussions and (iv) mapping of existing quality improvement initiatives (QIIs). Data gathered from these approaches informed the content of the new policy. Following thematic analysis, the findings were grouped into specific domains, which were then organized into a strengths, weaknesses, opportunities, and threats (SWOT) framework. ResultsTen key areas of concern identified were (i) a people-centred holistic approach, (ii) governance for quality, (iii) resources, (iv) quality culture, (v) stakeholder engagement, (vi) health management information system, (vii) workforce competency, (viii) knowledge exchange, (ix) quality indicators and (x) monitoring and evaluation of quality activities. These led to the formulation of seven strategic priorities for the planning of improvements aimed at addressing the key areas of concern. The national definition of quality was affirmed. A total of 40 QIIs were mapped and grouped into three broad categories, namely (i) regulatory, (ii) domain-specific QIIs and (iii) Quality Improvement (QI) method.ConclusionsThe National Policy for Quality in Healthcare for Malaysia was developed through a comprehensive situational analysis using a multi-method approach that identified priorities across national, state, institutional and community levels. This evidence-informed approach led to meaningful contextual adaptation of the NQPS framework to shape the strategic direction to advance quality and achieve effective and safe outcomes for all Malaysians.