Software Usability Evaluation Research Articles

Augmented Reality For Learning Children With Asd Software Usability Evaluation With Heuristic Rules

The objective of this research is to develop an augmented reality mobile application for learning mathematics and reading for children between 5 and 7 years of age who suffer from autism spectrum disorders. An own methodology for the development of the mobile application is presented and as results the finished mobile application is shown, which was evaluated by the tutors of the autistic children. 90% of the tutors affirmed that augmented reality facilitated educational understanding, 80% assured that learning through augmented reality applications is more attractive for children and finally 90% affirmed that they would recommend the use of the mobile application to tutors with similar situations.

Development of Open Backend Structures for Health Care Professionals to Improve Participation in App Developments: Pilot Usability Study of a Medical App.

Efficient digitization in medicine still is in its infancy but undeniably has great potential for current and future challenges in health care. Thus far, the rollout of medical apps has not resulted in widespread use of smartphones in the German health care sector-the reasons for this have not been clarified so far. Nevertheless, the lack of user involvement in the development process and content creation might contribute to low acceptance of these products. This study aims to outline an approach to involve medical expertise without any coding knowledge for developing medical app content and functions. An end user-operable backend was built. Its usability was evaluated using a usability evaluation test protocol. The results of the usability tests were evaluated by the app development team, and the usability test was repeated for optimizing backend usability. In total, 40 criteria to measure the ease of app usage were defined a priori. The usability test comprised 20 tasks that had to be fulfilled. Usability tasks were analyzed for completion, dropout, and test duration. Due to the COVID-19 pandemic, digital videoconferencing platforms (Zoom and QuickTime Player) were used to complete usability questionnaires. Finally, several backend-based apps for several specialties (infectiology, plastic and reconstructive surgery, and orthopedics) were developed by health care professionals as prototypes. Initial usability testing was conducted with 5 participants (4 men and 1 woman; mean age 39.2, SD 5.97 years). All of them could complete the assigned backend tasks with only a few workflow interruptions and some minor errors. After usability optimization, the workflow completion time decreased from 5.03 minutes to 3.50 minutes, indicating a time saving. The basic backend structure was clear to all test users and the handling was intuitive to learn. Some minor errors in the backend occurred during the test rounds. The apps developed using the aforementioned approach are in clinical use as a proof of concept. Backends offering operability for medical professionals might have great potential for app development in the mobile health sector. Sophisticated and time-saving usability are pivotal for the acceptance of medical software, as illustrated by the backend-based apps presented herein, which are in clinical use as a proof of concept. Basic interventions are essential and sufficient for adequate usability optimization. Practicable, well-structured software usability evaluation is possible based on the usability evaluation test protocol.

Usability inspection: Novice crowd inspectors versus expert

Objective:This research study aims to investigate the use of novice crowd inspectors for usability inspection with respect to time spent and the cost incurred. This study compares the results of the novice crowd usability inspection guided by a single expert’s heuristic usability inspection (novice crowd usability inspection henceforth) with the expert heuristic usability inspection. Background:Traditional usability evaluation methods are time-consuming and expensive. Crowdsourcing has emerged as a cost-effective and quick means of software usability evaluation. Method:In this regard, we designed an experiment to evaluate the usability of two websites and a web dashboard. Results:The results of the experiment show that novice crowd usability inspection guided by a single expert’s heuristic usability inspection:a) Finds the same usability issues (w.r.t. content & quantity) as expert heuristic usability inspection.b) Is cost-effective than expert heuristic usability inspection employing less time duration. Conclusion:Based on the findings of this research study, we can conclude that the novice crowd usability inspection guided by a single expert’s heuristic usability inspection and expert heuristic usability inspection, on average, gives the same results in terms of issues identified.

Feature selection algorithm for usability engineering: a nature inspired approach

AbstractSoftware usability is usually used in reference to the hierarchical software usability model by researchers and is an important aspect of user experience and software quality. Thus, evaluation of software usability is an essential parameter for managing and regulating a software. However, it has been difficult to establish a precise evaluation method for this problem. A large number of usability factors have been suggested by many researchers, each covering a set of different factors to increase the degree of user friendliness of a software. Therefore, the selection of the correct determining features is of paramount importance. This paper proposes an innovative metaheuristic algorithm for the selection of most important features in a hierarchical software model. A hierarchy-based usability model is an exhaustive interpretation of the factors, attributes, and its characteristics in a software at different levels. This paper proposes a modified version of grey wolf optimisation algorithm (GWO) termed as modified grey wolf optimization (MGWO) algorithm. The mechanism of this algorithm is based on the hunting mechanism of wolves in nature. The algorithm chooses a number of features which are then applied to software development life cycle models for finding out the best among them. The outcome of this application is also compared with the conventional grey wolf optimization algorithm (GWO), modified binary bat algorithm (MBBAT), modified whale optimization algorithm (MWOA), and modified moth flame optimization (MMFO). The results show that MGWO surpasses all the other relevant optimizers in terms of accuracy and produces a lesser number of attributes equal to 8 as compared to 9 in MMFO and 12 in MBBAT and 19 in MWOA.

Evaluating Usability for e-Learning andldquo;Madrasatiandrdquo; Platform in Saudi Arabia

Software usability evaluation is an important concept in the discipline of Human Computer Interaction (HCI) due to its benefits in the process of software development. Little research has been done to evaluate the accessibility and usability of the e-learning websites in Saudi Arabia due to limited availability of e-learning solutions offered in Saudi in general, especially during the pre-tertiary education which lacked any sort of e-learning capabilities before the COVID-19 pandemic. The current study was conducted to evaluate the usability and the content accessibility for the Madrasti platform in Saudi Arabia. Different web diagnostic tools and measures have been used in the evaluation process. The result shows that Madrasti platform has several issues in terms of usability and content accessibility.

Towards a unified criteria model for usability evaluation in the context of open source software based on a fuzzy Delphi method

A plethora of models are available for open-source software (OSS) usability evaluation. However, these models lack consensus between scholars as well as standard bodies on a specific set of usability evaluation criteria. Retaining irrelevant criteria and omitting essential ones will mislead the direction of the usability evaluation. This study introduces a three-step method to develop a usability evaluation model in the context of OSS. The fuzzy Delphi method has been employed to unify the usability evaluation criteria in the context of OSS. The first step in the method is the usability criteria analysis, which involves redefining and restructuring all collected usability criteria reported in the literature. The second step is fuzzy Delphi analysis, which includes the design and validates the fuzzy Delphi instrument and the utilisation of the fuzzy Delphi method to analyse the fuzziness consensus of experts' opinions on the usability evaluation criteria. The third step is the proposal of the OSS usability evaluation model. A total of 124 usability criteria were identified, redefined, and restructured by creating groups of related meaning criteria. The result of the groupings generated 11 main criteria; the findings of the fuzzy Delphi narrowed down the criteria to only seven. The final set of criteria was sent back to the panellists for reconsideration of their responses. The panellists verified that these criteria are suitable in the evaluation of the usability of OSS. The empirical analysis confirmed that the proposed evaluation model is acceptable in assessing the usability of OSS. Therefore, this model can be used as a reference metric for OSS usability evaluation which will have a practical benefit for the community in public and private organisations in helping the decision-maker to select the best OSS software package amongst the alternatives.

Design and validation of a questionnaire in Spanish language for software usability evaluation.

Usability is an essential characteristic of software development, which needs to be evaluated mainly through questionnaires. However, the questionnaires available in Spanish are limited and not available for most software's developers and users. This work aimed to design and validate a new questionnaire to evaluate software usability, considering the characteristics of efficiency, effectiveness, and satisfaction as defined by the International Standards Organization. The Software Usability Assessment Questionnaire or SUAQ in Spanish language was designed and validated using Kendall's coefficient, Cronbach's Alpha, Factorial Analysis and Scree test. The questionnaire's validation was carried out by five experts through an evaluation of the clarity, coherence, and relevance of the proposed items. Once the questionnaire was administered to 95 participants, an overall reliability value of Cronbach's alpha of 0.839 was obtained. The factorial analysis is viable since a value of 0.757 was found in the Kaiser-Meyer-Olkin (KMO) test, and a significant value was obtained in Bartlett's sphericity test. The proposed questionnaire proved to be valid and reliable, and therefore, suitable to evaluate software usability in the Spanish language.

Heuristic Evaluation on Usability of Educational Games: A Systematic Review

This work presents a systematic review whose objective was to identify heuristics applicable to the evaluation of the usability of educational games. Heuristics are usability engineering methods that aim to detect problems in the use of a system during its development and / or when its interface is in interaction with the user. Therefore, applying heuristics is an essential part of developing digital educational games. Search sources were articles available in all the databases present in the Capes / MEC / Brazil periodicals portal, in the available languages. The descriptors adopted were "educational games", "heuristic" and "usability" in Boolean search in titles, abstracts and keywords, with AND operator, for publications starting in 2014. The inclusion criteria were: (a) articles with a clear description of the methodology used in the usability analysis; (b) studies presenting primary data and (c) articles whose focus corresponds to the investigated question. Two examiners conducted the searches in the databases and a third the evaluation and general review of the data. Initially, 93 articles were identified, of which 19 were repeated, 5 were literature reviews. Of the 69 that remained, 57 were elected as not eligible with only 12 selected for full studies, of which 6 entered the final review. With this review we can deduce that the field of heuristics and usability for educational games is still little explored, with few specific evaluations validated or in the process of validation, requiring greater investment in the area. Through this review, we found at least one heuristic that meets the usability evaluation of educational software: Game User Experience Satisfaction Scale (GUESS).

Usability Evaluation of Software Tools for Engineering Design

AbstractMuch of the work in design research focusses on the development of methods and tools to support engineering designers. Many of these tools are nowadays implemented in software. Due to the strongly growing use of computers and smart devices in the last two decades, the expectations of users increased dramatically. In particular users expect good usability, for example little effort for learning to apply the software. Therefore, the usability evaluation of design software tools is crucial. A software tool with bad usability will not be used in industrial practice. Recommendations for usability evaluation of software often stem from the field of Human Computer Interaction. The aim of this paper is to tailor these general approaches to the specific needs of engineering design. In addition, we propose a method to analyse the results of the evaluation and to derive suggestions for improving the design software tool. We apply the usability evaluation method on a use case - the KoMBi software tool for bio-inspired design. The case study provides additional insights with regards to problem, causes and improvement categories.

Incorporating user feedback in the design of a genetics analysis tool: A two-part approach.

While general usability assessment models for websites have been developed for a wide variety of contexts, research literature on incorporating user feedback in the design of online scientific tools is lacking. In this article, we present an approach that we developed and illustrate how it was used to elicit user feedback of the AnalyzeMyVariant tool, which enables geneticists to use family pedigree data to calculate pathogenicity likelihood ratios for variants of unknown significance. We reviewed existing usability literature and developed a survey instrument emphasizing concepts of importance to online, data-driven, scientific tools. The items on the survey instrument were grouped in four categories: usability, quality, privacy and security, and satisfaction. We performed a two-part evaluation using the survey and a semi-structured interview protocol. The survey instrument was used to collect data about the use experience of AnalyzeMyVariant from 57 genetic experts and trainees who were recruited via email invitations. We also conducted semi-structured interviews with six genetics experts to explore work contexts in which users might use the tool and further delve into issues faced in tool use. Interviews were inductively coded and major themes identified using the constant comparative method. We found that the needs of genetics professionals vary for research- and clinically-focused work. These differences can inform the design of tools to serve their needs. The major contribution of this work is the description of a two-part method to elicit user feedback to inform the design of online, data-driven, scientific tools, which focuses on constructs of particular relevance to these tools such as usability, quality, privacy, security, and satisfaction. The survey instrument that we developed, coupled with contextual interviews, may serve as an example that can be used by others conducting usability studies of similar tools. In addition, our results emphasize the importance of considering contextual factors such as background knowledge, situational factors, and the intended application of results, in the usability evaluation of scientific software. It is our hope that this two-part approach might be adapted to assess the usability of other online scientific tools and facilitate the design of tools to meet the needs of their target audiences.

Usability Evaluation of a Web Academic Software Developed to Measure

.

Usability feature selection via MBBAT: A novel approach

In this paper, a metaheuristic algorithm has been introduced for software usability feature selection and evaluation. Usability is becoming one of the most significant aspects of quality of software. The term ‘usability’ has already been defined by the authors in their previous work in a reference to the hierarchical software usability model. This model combines various usability factors and features in a hierarchical manner. Here, we introduced MBBAT (Modified Binary Bat Algorithm) for usability feature selection to get an optimal solution for the search of useful usability features out of a given set of usability features. MBBAT is an extension of Binary Bat Algorithm(BBA) which is based on the bat's behavior and to the best of our knowledge, this algorithm is introduced for the first time in software engineering practices. The selected number of features and accuracy of proposed MBBAT algorithm is compared with the original BBA and the proposed metaheuristic algorithm outperforms the original BBA as it generates a fewer number of selected features and having low accuracy.

Usability Evaluation Approach of Educational Resources Software Using Mixed Intelligent Optimization

Aiming at the problems of strong subjectivity and uncertain fuzziness of attribute weights in the software usability evaluation approach, an evaluation approach based on mixed intelligent optimization was proposed, which combines subjective and objective methods to measure software usability for educational resources software. Firstly, the usability evaluation index system of educational resources software was established, and the basic probability assignment was generated by the interval method from the historical sample data. Then the weight optimization problem was adapted to the smooth optimization problem by the maximum entropy function method, and the hybrid social cognitive optimization (HSCO) algorithm was introduced to solve the optimal weights of evidence. Finally, the software usability level was fused by DS evidence theory. The experimental results show that the educational resources software usability evaluation approach can objectively and truly reflect the usability of the software. It provides an efficient way to evaluate the usability of the software.

Service Desk Software Usability Evaluation: The Case of Brazilian National Cancer Institute

The use of control tools to manage services in Information Technology (IT) is a reality in contemporary organizations, this can be stated on the understanding that the controls can generate financial results and contribute to reach organizational goals. Brazilian laws state the technology services should be executing by private companies, justifying the need to control and supervise the implementation of these services. Brazilian National Cancer Institute acquired a new Service Desk Manager Software in order to simplify support operations through narrow integration between service request management process, incident, problem, changes, knowledge, assets and configuration. This paper aim to measure how the usability of a service desk manager software can influence the efficiency of the service operation. This survey found that the change of application, particularly those that directly influence the performance of an activity in the organization, requires an effort on your deployment project. Factors such as usability can affect the success of the adoption of a system for a particular team.

Software Usability Evaluation Using Opinion Mining

Usability is critical for any system, but in software it is one of the most important features. In fact, one of the main reasons for software failure is the system lacking to achieve users specified goals and satisfaction. For this reason, usability evaluation is becoming an important part of software development. Software usability evaluation can be costly in terms of time and human. Therefore, automation is promising way to augment existing approaches especially if the evaluation is subjective where the usability concentrated about user's opinion. This paper proposes to use opinion mining as an automatic technique to evaluate subjective usability. Opinion mining is a research subtopic of data mining aiming to automatically obtain useful opinioned knowledge in subjective texts. We propose a novel model to extract knowledge from opinions to improve subjective software usability. This is the first time opinion mining used in software usability. To evaluate our proposed model, a set of experiments was designed and conducted and we got an average accuracy of 85.41%. Also, we propose to use graphics to visualize user's opinion in software and to compare the usability of two software.

A usability evaluation of medical software at an expert conference setting

IntroductionA usability test was employed to evaluate two medical software applications at an expert conference setting. One software application is a medical diagnostic tool (electrocardiogram [ECG] viewer) and the other is a medical research tool (electrode misplacement simulator [EMS]). These novel applications have yet to be adopted by the healthcare domain, thus, (1) we wanted to determine the potential user acceptance of these applications and (2) we wanted to determine the feasibility of evaluating medical diagnostic and medical research software at a conference setting as opposed to the conventional laboratory setting. MethodsThe medical diagnostic tool (ECG viewer) was evaluated using seven delegates and the medical research tool (EMS) was evaluated using 17 delegates that were recruited at the 2010 International Conference on Computing in Cardiology. Each delegate/participant was required to use the software and undertake a set of predefined tasks during the session breaks at the conference. User interactions with the software were recorded using screen-recording software. The ‘think-aloud’ protocol was also used to elicit verbal feedback from the participants whilst they attempted the pre-defined tasks. Before and after each session, participants completed a pre-test and a post-test questionnaire respectively. ResultsThe average duration of a usability session at the conference was 34.69min (SD=10.28). However, taking into account that 10min was dedicated to the pre-test and post-test questionnaires, the average time dedication to user interaction of the medical software was 24.69min (SD=10.28). Given we have shown that usability data can be collected at conferences, this paper details the advantages of conference-based usability studies over the laboratory-based approach. For example, given delegates gather at one geographical location, a conference-based usability evaluation facilitates recruitment of a convenient sample of international subject experts. This would otherwise be very expensive to arrange. A conference-based approach also allows for data to be collected over a few days as opposed to months by avoiding administration duties normally involved in laboratory based approach, e.g. mailing invitation letters as part of a recruitment campaign.Following analysis of the user video recordings, 41 (previously unknown) use errors were identified in the advanced ECG viewer and 29 were identified in the EMS application. All use errors were given a consensus severity rating from two independent usability experts. Out of a rating scale of 4 (where 1=cosmetic and 4=critical), the average severity rating for the ECG viewer was 2.24 (SD=1.09) and the average severity rating for the EMS application was 2.34 (SD=0.97). We were also able to extract task completion rates and times from the video recordings to determine the effectiveness of the software applications. For example, six out of seven tasks were completed by all participants when using both applications. This statistic alone suggests both applications already have a high degree of usability. As well as extracting data from the video recordings, we were also able to extract data from the questionnaires. Using a semantic differential scale (where 1=poor and 5=excellent), delegates highly rated the ‘responsiveness’, ‘usefulness’, ‘learnability’ and the ‘look and feel’ of both applications. ConclusionThis study has shown the potential user acceptance and user-friendliness of the novel EMS and the ECG viewer applications within the healthcare domain. It has also shown that both medical diagnostic software and medical research software can be evaluated for their usability at an expert conference setting. The primary advantage of a conference-based usability evaluation over a laboratory-based evaluation is the high concentration of experts at one location, which is convenient, less time consuming and less expensive.

Applying fuzzy logic to software usability evaluation

Applying fuzzy logic to software usability evaluation

Combining experimental and inquiry methods in software usability evaluation

PurposeThe purpose of this paper is to present a methodology combining experimental and inquiry methods used for software usability evaluation. The software product of LeViS project funded by the European Commission (Socrates/Lingua II) is used as an evaluation paradigm. The aim of the paper is twofold: to present the results of the usability evaluation using this software as an example and to suggest a number of improvements for the next version of the software tool; and to portray the advantages of combining methods from different evaluation approaches and the experiences from their application.Design/methodology/approachThe evaluation for this experiment combined different usability methods, both experimental and inquiry ones. More specifically, the methods employed were the Thinking Aloud Protocol and the User Logging, which were performed in a usability evaluation laboratory, as well as the inquiry methods of Interview and Focus Group.FindingsIn this study, usability problems regarding the Learning via Subtitling (LvS) educational software were revealed as well as issues regarding the use of Thinking Aloud Protocol method and involving users with a specific profile. The research findings presented in this paper constitute an innovative and effective methodology for software usability evaluation and are useful for laboratories aiming to conduct similar evaluations.Research limitations/implicationsAlthough this methodology has been successfully applied for over 20 software products, due to practical purposes related to this paper's extent, only one software is used as an example.Originality/valueThrough the evaluation process, apart from discovering certain usability problems related to the software, there are a number of important conclusions drawn, regarding the methods used and the methodology followed in software usability evaluation.

Evaluation of Software Usability Using Electroencephalogram-Comparison of Frequency Component between Different Software Versions

Evaluation of Software Usability Using Electroencephalogram-Comparison of Frequency Component between Different Software Versions

How relative absolute can be: SUMI and the impact of the nature of the task in measuring perceived software usability

This paper addresses the possibility of measuring perceived usability in an absolute way. It studies the impact of the nature of the tasks performed in perceived software usability evaluation, using for this purpose the subjective evaluation of an application’s performance via the Software Usability Measurement Inventory (SUMI). The paper reports on the post-hoc analysis of data from a productivity study for testing the effect of changes in the graphical user interface (GUI) of a market leading drafting application. Even though one would expect similar evaluations of an application’s usability for same releases, the analysis reveals that the output of this subjective appreciation is context sensitive and therefore mediated by the research design. Our study unmasked a significant interaction between the nature of the tasks used for the usability evaluation and how users evaluate the performance of this application. This interaction challenges the concept of absolute benchmarking in subjective usability evaluation, as some software evaluation methods aspire to provide, since subjective measurement of software quality will be affected most likely by the nature of the testing materials used for the evaluation.