Software Product Evaluation Research Articles

Evaluating software quality using software metrics for a software product

As software becomes more and more pervasive, there has been a growing concern in the information technology industry about software quality. There are various software quality models to standardise the software quality. However, there is no consensual relation among software quality models and the process of assessing the software quality remains an open issue. The main concern of this paper is to measure the characteristics of software quality defined by the International Standard for Software Product Evaluation ISO/IEC 9126 using a set of software metrics proposed in this research study. The results are then validated using statistical techniques. Our results provide guidance to the software engineering team to concentrate on the specific software quality attributes which leads to the poor quality of the software. The significance of our research study is emphasised and future research directions are indicated.

생체인식 소프트웨어의 품질 평가모듈에 관한 연구

최근 생체인식 분야는 IT 분야의 보안기술과 함께 빠르게 진전되어 왔다. 현재 생체인식의 중요성이 인식되면서 국내외 생체인식 소프트웨어 시장이 급격히 증가하고 있는 추세이다 이에 따라 생체인식 소프트웨어의 고신뢰성과 고품질 소프트웨어의 요구가 증대되고 있다. 생체인식 소프트웨어의 품질인증을 위해서는 평가항목 및 평가기준이 마련되어 있어야 한다 본 논문에서는 생체인식 소프트웨어의 품질요구와 시험에 관한 표준인 ISO/IEC 12119, 소프트웨어 제품평가를 위한 표준인 91뽀, 평가모듈의 구성을 위한 국제 표준인 ISO/IEC 14598-6을 기반으로 하여 생체인식 소프트웨어 시험을 위한 평가모듈을 개발하였다. 본 논문에서 제시하는 품질 평가모듈은 생체인식 소프트웨어 제품의 구성요소(제품설명서, 사용자문서, 프로그램과 데이터)를 대상으로 하고 있으므로 ISO/IEC 9126-3과 같은 소프트웨어 개발과정에서 적용 가능한 표준과 병행함으로써 소프트웨어의 품질 향상을 기대할 수 있다. The latest biometric field have marched fast with security technology of IT. As importance of present biometrics is realized, internal and external biometrics software market is trend that is soaring. Accordingly, high reliability of biometric software and request of high quality software are enlarged. Evaluation items and criteria must be established for biometric software quality assurance. In this paper, we development the evaluation module for biometric software test based on ISO/IEC 12119 that is the standard about software quality requirement and test, and ISO/IEC 9126 that is standard about evaluation of software product, and ISO/IEC 14598-6 that is the standard about construction of the evaluation module. Constituents of biometric software products(product descriptor, user document program and data) is subject to the quality evaluation module that we developed in this paper, we can expect improvement in the quality of software by using with a standard such as ISO/IEC 9126-3 that can be used in software development process.

Evaluation and comparison of search engines using the LSP method

We present a comprehensive model for quantitative evaluation and comparison of search engines. The model is based on the LSP method for system evaluation. The basic contribution of our approach is the aggregation of all relevant attributes that reflect functionality, usability, and performance of search engines. In this respect our model is fully consistent with the ISO 9126 standard for software product evaluation. Performance analysis of competitive search engines is based on our search engine benchmarking tool (SEben) that is also described in the paper.

The W-Process for Software Product Evaluation: A Method for Goal-Oriented Implementation of the ISO 14598 Standard

The importance of software product evaluations will grow with the awareness of the need for better software quality. The process to conduct such evaluations is crucial to get evaluation results that can be applied and meet customers' expectations. This paper reviews a well-known evaluation process: the ISO 14598 standard. The review focuses on the difficulties in selecting and evaluating the appropriate evaluation techniques. The review shows that the standard has problems in applying evaluation processes in practice due to insufficient attention to goal definition and to relationships between activities being implicit. Also, the standard ignores the trade-off between goals and resources and pays insufficient attention to feedback. To address these deficiencies, the W-process is proposed. It extends the standard through an improved process structure and additional guidelines.

Integrating scenario-based and measurement-based software product assessment

The software industry needs means to evaluate software products and compare development and implementation technologies in the context of actual projects. Solutions need to be cost-effective but also technically sound. This paper presents a methodology to combine two software product evaluation techniques: measurement of structural design properties, and evaluation of change scenarios. The goal is to use these two approaches together so that they can address each other's limitations. In a case study in the context of the European aerospace industry, this combined methodology was used to assess the impact of choice of programming language and distribution technology on the maintainability of resulting systems. It encompasses the comparison of C++ and Java, as well as distribution/communication technologies such as IPC via sockets, and CORBA implementation. Lessons learned in terms of benefits and limitations are presented. The study shows the usefulness of the approach presented but it is also clear that it needs to be used in combination with other means of evaluation and with a critical mind, as for any engineering solution.

A formal process for evaluating COTS software products

A software product evaluation process grounded in mathematics and decision theory can effectively determine product quality and suitability with less risk and at lower cost than conventional methods.

A software quality evaluation system: JT-SQE

JT-SQE system is a software quality and measurement system. Its design was based on the Chinese national standards of software product evaluation and quality characteristics. The JT-SQE system consists of two parts. One is the model for software quality measurement, which is of hierarchical structure. The other is the process of requirements definition, measurement and rating. The system is a feasible model for software quality evaluation and measurement, and it has the advantage of a friendly user interface, simple operation, ease of revision and maintenance, and expansible measurements.

XML

In the identification, evaluation and acquisition of suitable computer related software products and skills, as well as Internet products and skills, organisations have continuously sought to streamline their business processes. Historically the route taken has been dictated by the outcome of previous decisions — organisations have been governed by their chosen hardware and software environments or had to justify significant re-investment in a change of direction. This could, however, all be about to change radically in the immediate future.

Evaluation of Workflow-type software products: a case study

The main objective of this paper is to propose a set of indicators for the evaluation of Workflow software-type products within the context of Information Systems. This paper is mainly based on a comprehensive bibliographical review of all topics referring to the Workflow Technology and Information Systems. Next, sets of indicators are presented for the selection of a Workflow software based on the realities of the business world, including a method of examination so as to obtain an integral evaluation on the Workflow software. Finally, the evaluation method for two types of Workflow software is applied: Lotus Domino/ Notes ® and Microsoft Exchange ®, for the billing subsystems of a company called MANAPRO Consultants, Inc. ®.

Applying independent verification and validation to the automatic test equipment life cycle

Automatic Test Equipment (ATE) is becoming increasingly more sophisticated and complex, with an even greater dependency on software. With the onset of Versa Modular Eurocard (VME) Extensions for Instrumentation (VXI) technology into ATE, which is supposed to bring about the promise of interchangeable instrument components. ATE customers are forced to contend with system integration and software issues. One way the ATE customer can combat these problems is to have a separate Independent Verification and Validation (IV&V) organization employ rigorous methodologies to evaluate the correctness and quality of the ATE product throughout its life cycle. IV&V is a systems engineering process where verification determines if the ATE meets its specifications, and validation determines if the ATE performs to the customer's expectations. IV&V has the highest potential payoff of assuring a safe and reliable system if it is initiated at the beginning of the acquisition life cycle and continued throughout the acceptance of the system. It is illustrated that IV&V effects are more pronounced when IV&V activities begin early in the software development life cycle, but later application of IV&V is still deemed to have a significant impact. IV&V is an effective technique to reducing costs, schedule, and performance risks on the development of complex ATE, and a "tool" to efficiently and effectively manage ATE development risks. The IV&V organization has the ability to perform a focused and systematic technical evaluation of hardware and software processes and products. When performed in parallel with the ATE development life cycle, IV&V provides for early detection.

IusWare: a methodology for the evaluation and selection of software products

IusWare (IUStitia SoftWARis) is a methodology designed to evaluate software products in a formal and rigorous way. The methodology is based on the multicriteria decision aid approach and encompasses activities such as comparison, assessment and selection of software artefacts. The methodology defines an evaluation process which consists of two main phases, designing an evaluation model and applying it. The design phase is made up of the following activities: first, identifying the actors relevant to the evaluation, their role, the purpose of the evaluation, the resources available and the object(s) of the evaluation; secondly, identifying the type of evaluation required: either a formal description of products or the ranking of products from the most preferred to the least preferred or a partitioning into two sets of the best and the remaining products; thirdly, defining a nonredundant hierarchy of evaluation attributes, often corresponding with the quality characteristics of quality models; fourthly, associating a measure, a criterion scale and a function to transform the measure scale into the criterion scale to each basic attribute; and finally, choosing an aggregation technique so as to aggregate values on criteria to form a recommendation for the selection. In the application phase attributes of products are measured, measures are transformed into values on criteria and aggregated to form a recommendation.

An empirical study of skills assessment for software practitioners

Software professionals face the difficult challenge of keeping up with today's fast-paced technological environment. There has been much discussion about technical obsolescence in a field where the half-life of an undergraduate education is only a few years. Moreover, assessments provide measurable proof of behavioral changes, legitimizing the human resource department's role in improving productivity by rendering it quantifiable. In this paper, we describe an empirical study of the skills assessment of software practitioners. This study is based on a survey performed collaboratively by the Software Engineering Research Center (SERC), Purdue University, and IBM Training and Education, with direct participation from the IEEE. The goal of this research was the determination of the critical skills necessary for software professionals. This paper describes the survey, the structure of the questionnaire, and the skills assessment process. Skills assessment stages such as data collection, data analysis, data representation, and follow-up reassessment are also described. Detailed results of the survey and selected critical skills relating to both object-oriented and client-server technologies are presented in this paper. These assessments provide a systematic approach through which human resources departments can improve productivity during downturns by increasing the working effectiveness of software developers. We believe that university software engineering students must understand the differences between academic programming and industry software development and engineering. They must also be able to perform the activities involved with plan development, project management, and software product evaluation. We also conclude that these assessments will foster genuine commitment and motivate software practitioners to grow in a field of technology that changes daily.

A model for software product quality

A model for software product quality is defined, it has been formulated by associating a set of quality-carrying properties with each of the structural forms that are used to define the statements and statement components of a programming language. These quality-carrying properties are in turn linked to the high-level quality attributes of the International Standard for Software Product Evaluation ISO-9126. The model supports building quality into software, definition of language-specific coding standards, systematically classifying quality defects, and the development of automated code auditors for detecting defects in software. >

Appraisal and evaluation of software products

Software product evaluation can be problematical as products from different backgrounds are likely to exhibit different strengths and weaknesses. A weakness in one area will not necessarily eliminate a product from consideration as project requirements are seldom absolute What therefore is needed is a systematic approach which helps to identity the product or products which best meet the needs of the project or organisation. A method will be described which has been designed to assess the suitability of software products against the total requirements of a project. The method consists of a weighted scoring technique based on a customisable criteria hierarchy Different scenarios in which the method can be used will be covered, together with a description of a hierarchy for the evaluation of text-based information management systems

Users, a software usability model and product evaluation

The paper advances an amalgamated model of software usability in conjunction with a method for improving the usability of commercial software products. The model organises the substantial amount of prior research on software usability into seven basic principles, their underlying attributes, and associated relative weights. This model of software usability for human-computer interaction has two primary goals: first, to allow software designers to make quantitative decisions about which usability attributes should be included in a design; and second, to provide a usability metric by which software designs can be consistently rated and compared. The paper focuses on the second goal. Since it is ultimately the users of a software system who decide its usability, the method proposed suggests that the users be made an integral part of the software design and development process. It recommends that the users of current and future software products be asked, through questionnaires, how well a product meets the principles and underlying attributes of usability as defined by the model. Only if the ultimate users of a product are pleased is a product likely to succeed. The paper reviews the model and its evaluation. It then illustrates the method by describing its use in evaluting a particular wordprocessing product, WordPerfect. In that evaluation, 988 questionnaires were sent to users of the product.

The Case for Independent Software Usability Testing: Lessons Learned from a Successful Intervention

This report presents the lessons learned from a software usability test for an external customer. An initial evaluation with naive users revealed problems in the user interface and that the customer's objectives were not being met. After initial resistance to making changes in the software, the customer decided to delay release of its product to implement some of the recommendations and changed the focus of initial release to experienced users. The results of a second evaluation conducted on the revised product with experienced users were positive. Several lessons can be learned from the above evaluation: (1) Usability evaluation should be incorporated earlier in the software development cycle to minimize resistance to changes in a hardened user interface; (2) Organizations should have an independent usability evaluation of software products to avoid the temptation to overlook problems to release the product; (3) Multiple categories of dependent measures should be employed in usability testing because subjective measurement is not always consonant with user performance; and (4) Even though usability testing at the later stages of development may not impact software changes, it is useful to point out areas where training is needed to overcome deficiencies in the software.

Evaluation of Computer Software Products

Evaluation of Computer Software Products

Software Product Evaluation Metrics: AMethodological Approach

This paper reports essentials of a software product evaluation methodology, called CDSEM (Checklist Driven Software Evaluation Methodology), designed by Software Quality Laboratory of Tecnopolis CSATA Novus Ortus. Our intention is to focalize the use of software evaluation metrics in the framework of our methodology. We consider software product as composed by different parts: software system, product documentation, user documentation, support services and distribution media. Each component needs a specific set of metrics and tools for the evaluation process. After each component has been evaluated, the methodology provides an unified assessment process. The methodology proposes the evaluation models in accordance with the standard ISO 9126 (Information technology - Software product evaluation Quality characteristics and guidelines for their use) taking also into account the emerging new parts of the standard. The six characteristics defined in ISO 9126 (functionality, reliability, usability, maintainability, portability, efficiency), are exploded, for every component, into sublayers of abstractions till to the identification of the measurable items (metrics). Moreover, the methodology identifies, for each metric, tools and procedures for the evaluation (code measures, inspection etc.). A tool has been developed on PC platform (CDSET, Checklist Driven Software Evaluation Tool) to manage the methodology information base, results and reports.

A Model For Assessment Module Selection InProduct Certification

A Model For Assessment Module Selection InProduct Certification