Accuracy Performance Measures Research Articles

Psychometric evaluation of a working memory assessment measure in young children with Down syndrome

BackgroundWorking memory involves the temporary storage and manipulation of information and is frequently an area of challenge for individuals with Down syndrome (DS). Despite the potential benefits of intervention, laboratory assessments of working memory that could capture intervention effects have not undergone rigorous evaluation for use with young children with DS. It is critical to evaluate assessments of working memory in young children with DS to ensure the reliable and accurate measurement of performance. AimThis study evaluated an adapted laboratory measure of working memory for young children with DS 2–8 years old. MethodA self-ordered pointing task, the Garage Game, was administered to 78 children with DS (mean = 5.17 years; SD = 1.49). Adaptations were made to the task to minimize potential DS phenotype-related language and motor confounds. ResultsResults indicate that the measure is feasible, scalable, and developmentally sensitive, with minimal floor and practice effects for this population within this chronological and developmental age range. ConclusionThese findings demonstrate that the Garage Game is promising for use in studies of early working memory and treatment trials that aim to support the development of this critical dimension of executive functioning for children with DS.

DHHoE: Deep hybrid homogenous ensemble for digital histological breast cancer classification

AbstractThe progress of deep learning architectures, machine learning models and pathology slide digitization is an encouraging step toward meeting the growing demand for more precise classification and prediction diagnosis for the breast tumours. The BreakHis dataset with four magnification factors (40X, 100X, 200X and 400X), as well as seven deep learning architectures used for feature extraction (DenseNet 201, Inception ResNet V2, Inception V3, ResNet 50, MobileNet V2,VGG16 and VGG19), four machine learning models for classification (MLP, SVM, DT, and KNN), and two combination rules (hard and weighted voting) were investigated in this paper to design and evaluate a new proposed approach consisting of building deep hybrid homogenous ensemble. Additionally, the best proposed models were compared to deep stacked, deep bagging, deep boosting, and deep hybrid heterogenous ensemble to choose the best strategy in building deep ensemble learning techniques. The four performance measures accuracy, precision, recall, and F1‐score were used in the empirical evaluations, as well as 5‐fold cross‐validation, the Scott Knott statistical test, and the Borda Count voting method. The results demonstrated the new approach's potential since it outscored both singles and other deep ensemble learning strategies, achieving accuracy values of 98.3% and 97.7% for the MFs 40X, 100X and 200X, 400X, respectively. The empirical results demonstrated that the proposed ensembles are impactful for histopathological breast cancer images classification, and they provided a promising tool to assist pathologists in the diagnosis of breast cancer.

Under My Skin: Reducing Bias in STEM through New Approaches to Assessment of Spatial Abilities Considering the Role of Emotional Regulation

Reducing gender bias in STEM is key to generating more equality and contributing to a more balanced workforce in this field. Spatial ability and its components are cognitive processes crucial to success in STEM education and careers. Significant gender differences have consistently been found in mental rotation (MR), the ability to mentally transform two- and three-dimensional objects. The aim of this pilot study is to examine factors in psychological assessment which may contribute to gender differences in MR performance. Moreover, findings will inform the development of the new approaches to assessment using computer adaptive testing (CAT). (1) Background: The study examines the impact of emotional regulation on MR performance in primary school children whose mean age was 9.28 years old. (2) Methods: Skin conductance was measured to assess the impact of emotional reactivity (ER) on performance during an MR task. (3) Results: Patterns of ER influence response time (RT) on specific items in the task. (4) Conclusions: Identifying the effects of emotional arousal and issues of test construction such as stereotyped stimuli and item difficulty in tests of spatial ability warrants ongoing investigation. It is vital to ensure that these factors do not compromise the accurate measurement of performance and inadvertently contribute to the gender gap in STEM.

Does It MultiMatch? What Scanpath Comparison Tells us About Task Performance in Teams

Teamwork and collaboration form the cornerstones of organizational performance and success. It is important to understand how the attention allocation of team members is linked to performance. One approach to studying attention allocation in a team context is to compare the scanpath similarity of two people working in teams and to explore the link between scanpath similarity and team performance. In this study, participants were recruited to work in pairs on an unmanned aerial vehicle (UAV) task that included low and high workload conditions. An eye tracker was used to collect the eye movements of both participants in each team. The scanpaths of two teammates were compared in low and high workload conditions using MultiMatch, an established scanpath comparison algorithm. The obtained scanpath similarity values were correlated with performance measures of response time and accuracy. Several MultiMatch measures showed significant strong correlations across multiple dimensions, providing insight into team behavior and attention allocation. The results suggested that the more similar each team member’s scanpath is, the better their performance. Additional research and consideration of experimental variables will be necessary to further understand how best to use MultiMatch for scanpath similarity assessment in complex domains.

Evaluation of cyclic repetition frequency based algorithm for fetal heart rate extraction from fetal phonocardiography

The passive and non-invasive approach of fetal heart rate (FHR) estimation using fetal phonocardiography (fPCG) provides an alternative to the commonly used Doppler ultrasound-based technique. The acoustic signal recorded on the mother’s abdomen is corrupted by noise, making FHR estimation difficult. Thus, an accurate and reliable FHR measurement algorithm from the fPCG signal is needed. This study evaluates a cyclic repetition frequency-based algorithm with additional FHR validation steps on the clinical recordings from three datasets with varying signal quality. The FHR from the algorithms is compared against the reference FHR measurements, and the accuracy and reliability performance measure is determined. The cyclic repetition algorithm significantly improves (p < 0.05) against the Hilbert-envelope autocorrelation algorithm for poor-quality signals. The algorithm's average reliability across three datasets is 93%, and accuracy is 4.5 beats per minute. The algorithm performs well for the gestation week ranging from 26 −40 and for the labor and antenatal ward data.

Comparison of Arrivals on Green Estimations from Vehicle Detection and Connected Vehicle Data

Many agencies use Automated Traffic Signal Performance Measures (ATSPMs), which require vehicle detection and communication equipment, to evaluate traffic signal efficiency. ATSPMs usually rely on projections based on spatially limited vehicle trajectory samples to estimate performance. Recently, connected vehicle (CV) data has become available that provides entire vehicle trajectories that can be used to generate accurate signal performance measures without the need for projections or infrastructure upgrades. This paper analyzes over 50 intersections in Utah to evaluate how traditional detector-based arrivals on green (AOG) calculations compare with their CV-based calculations. The effects that saturation and queue- lengths have on estimations are analyzed. In general, there is close correlation between computations when queues are short and undersaturated conditions exist. However, if queues extend past the advance detector, detector-based calculations tend to overestimate AOG since vehicles are detected during green but may stop before. This impact is particularly large when the approach is oversaturated, which led to overestimations of around 40% in this study. In addition, detector-based estimations can underestimate AOG in undersaturated scenarios with short queues if vehicles reach the advance detector on red but reduce their speed afterwards, allowing them to not stop. In some cases, the detector-based technique underestimated AOG by over 50%. The findings can help practitioners understand how detector-based estimations vary by traffic conditions. This is particularly important as the industry moves toward a hybrid blend of detector- and CV-based signal performance measures. It is recommended that CV trajectories be used to measure AOG during periods with long queues, oversaturated conditions, or both.

Multimodal face shape detection based on human temperament with hybrid feature fusion and Inception V3 extraction model

ABSTRACT In this research, the multi model (image and video) based face shape detection with human temperament is developed. Here, the video is captured by webcam via live recording session and the proposed model undergoes three major stages namely, pre-processing, extracted feature fusion and temperament detection. Extraction of facial landmark and facial boundary takes place in the pre-processing stage. In the feature extraction stage, the handcrafted features from image are extracted (i.e. face, forehead, eyes, cheeks, nose and mouth). From the video frames, the intrinsic features related to face region are extracted using pretrained Inception V3 model. Then robust principal component analysis (RPCA) is introduced to reduce the dimension of extracted features. Further, the feature fusion process is performed using discriminant correlation analysis (DCA) and canonical correlation analysis (CCA) at hybrid phase. Finally, the gated recurrent unit (GRU) classifier model is applied to identify the human temperaments based on face shapes. In the experimental scenario, the performance measures of accuracy (98.51%, 98.86%), precision (96.14%, 97.89%), recall (96.34%, 97.95%), F-measures (96.24%, 97.94%), etc are evaluated and compared with state-of-the-art methods under two datasets. In addition to this, the statistical test is also conducted to validate the efficacy of the proposed model.

Classifying histopathological images of oral squamous cell carcinoma using deep transfer learning

Oral cancer is a prevalent malignancy that affects the oral cavity in the region of head and neck. The study of oral malignant lesions is an essential step for the clinicians to provide a better treatment plan at an early stage for oral cancer. Deep learning based computer-aided diagnostic system has achieved success in many applications and can provide an accurate and timely diagnosis of oral malignant lesions. In biomedical image classification, getting large training dataset is a challenge, which can be efficiently handled by transfer learning as it retrieves the general features from a dataset of natural images and adapted directly to new image dataset. In this work, to achieve an effective deep learning based computer-aided system, the classifications of Oral Squamous Cell Carcinoma (OSCC) histopathology images are performed using two proposed approaches. In the first approach, to identify the best appropriate model to differentiate between benign and malignant cancers, transfer learning assisted deep convolutional neural networks (DCNNs), are considered. To handle the challenge of small dataset and further increase the training efficiency of the proposed model, the pretrained VGG16, VGG19, ResNet50, InceptionV3, and MobileNet, are fine-tuned by training half of the layers and leaving others frozen. In the second approach, a baseline DCNN architecture, trained from scratch with 10 convolution layers is proposed. In addition, a comparative analysis of these models is carried out in terms of classification accuracy and other performance measures. The experimental results demonstrate that ResNet50 obtains substantially superior performance than selected fine-tuned DCNN models as well as the proposed baseline model with an accuracy of 96.6%, precision and recall values are 97% and 96%, respectively.

Design and experimentation of remote driving system for robotic speed sprayer operating in orchard environment

AbstractThe automation of agricultural machines is an irreversible trend considering the demand for improved productivity and lack of labor in handling agricultural tasks. Unstructured working environments and weather often inhibit a seemingly simple task from being fully autonomously performed. In this context, we propose a remote driving system (RDS) to aid agricultural machines designed to operate autonomously. Particularly, we modify a commercial speed sprayer for orchard environments into a robotic speed sprayer to evaluate the proposed RDS's usability and test three sensor configurations in terms of human performance. Furthermore, we propose a confidence error ellipse‐based task performance measure to evaluate human performance. In addition, we present field experimental results describing how the sensor configurations affect human performance. We find that a combination of a semiautonomous line tracking device and a wide‐angle camera is the most effective for spraying. Finally, we discuss how to improve the proposed RDS in terms of usability and obtain a more accurate measure of human performance.

A Novel Multi-Task Learning Network Based on Melanoma Segmentation and Classification with Skin Lesion Images.

Melanoma is known worldwide as a malignant tumor and the fastest-growing skin cancer type. It is a very life-threatening disease with a high mortality rate. Automatic melanoma detection improves the early detection of the disease and the survival rate. In accordance with this purpose, we presented a multi-task learning approach based on melanoma recognition with dermoscopy images. Firstly, an effective pre-processing approach based on max pooling, contrast, and shape filters is used to eliminate hair details and to perform image enhancement operations. Next, the lesion region was segmented with a VGGNet model-based FCN Layer architecture using enhanced images. Later, a cropping process was performed for the detected lesions. Then, the cropped images were converted to the input size of the classifier model using the very deep super-resolution neural network approach, and the decrease in image resolution was minimized. Finally, a deep learning network approach based on pre-trained convolutional neural networks was developed for melanoma classification. We used the International Skin Imaging Collaboration, a publicly available dermoscopic skin lesion dataset in experimental studies. While the performance measures of accuracy, specificity, precision, and sensitivity, obtained for segmentation of the lesion region, were produced at rates of 96.99%, 92.53%, 97.65%, and 98.41%, respectively, the performance measures achieved rates for classification of 97.73%, 99.83%, 99.83%, and 95.67%, respectively.

Violence Detection with Machine Learning: A Sociodemographic Approach

This study suggests that by implementing machine learning methods on a sociodemographic data set can be helpful in preventing domestic violence. This approach is important in predicting high-risk factors that an offender may cause and it offers treatment, and financial or mental health aids in order to prevent domestic violence. In this sense, this proposal is critical at a personal and social level in creating a secure and healthy environment as well as empowering an equal society. In our study, we use k-nearest neighbor (k-nn), support vector machine (SVM), decision tree (DT), and Gaussian Naive Bayes (GNB) machine learning algorithms for the prediction analysis. We provide the comparison of the classifiers with precision, recall, F1 score, and accuracy performance measures. According to our analysis, the decision tree (DT) performs the best performance in terms of accuracy.

Quantitative Upper Limb Assessment With Natural User Interface in Children With Hemiparesis

Quantitative, efficient, and accurate measurement of upper limb motor performance is relevant for monitoring upper limb progress in daily activities of people with cerebral palsy, which is helpful to define an appropriate rehabilitation procedure. Currently, motion capture sensors using a Natural User Interface (NUI) are the most feasible solutions due to the low-cost implementation, portability, and effectiveness for upper limb motion analysis. A quantitatively assessment study of the upper limb motor performance for children with spastic hemiparesis using the NUI Kinect v2 sensor is described in this paper. The study participants were eighteen children with an average age of 8.28 ± 2.32 years having a Manual Ability Classification System level I and II. The assessment was done for each participant before and after the application of a Modified Constrained-Induced Movement Therapy (mCIMT) applied five days a week along twelve weeks. According to obtained results significant differences (p<0.05) in the Fugl-Meyer Assessment (FMA) score for motor performance in abduction movement of upper limb between the first and last sessions were quantitatively detected. The described NUI assessment method helped to quantitatively show that the applied therapy sessions were effective to improve upper extremity motor performance. Additionally, the upper limb movements motorically limited after the rehabilitation therapy were also identified. The main contribution of the described NUI assessment method is its potential use as a quantitative measurement tool, which might be used by specialists to objectively diagnose and consequently define an appropriate rehabilitation therapy for patients with hemiparesis.

Forecasting stochastic consumer portability visitation pattern in fair price shops of India

In India, the Public Distribution System (PDS) is a critical tool for accomplishing the aim of “Zero Hunger”. Despite the enormous resources used, PDS has several inefficiencies that are caused by the monopoly of agents engaged in last-mile grain supply. Various state governments in India have been employing portability as an innovative solution to address this problem. In this article, we examined a huge-scale data on the deployment of portable beneficiaries arriving in a particular FPS of Kerala state in India over three years. A comparison is made between Auto-Regressive Integrated Moving Average (ARIMA) method which makes forecasts in univariate data and ARIMA with exogenous variables called ARIMAX. We followed Mean Absolute Percentage Error (MAPE) and Mean Absolute Deviation (MAD) as the accuracy performance measure of the models and observed that the ARIMAX model outperforms the ARIMA model with the least forecasting errors.

Separation of arteries and veins in retinal fundus images with a new CNN architecture

ABSTRACT Retinal blood vessels are directly or indirectly associated with many diseases. The retinal blood vessels consist of artery and vein vessels. With the automatic correct identification of these vessels, many diseases can be prevented. In this paper, a method is proposed to separate between arteries and veins on retinal blood vessel images. In the proposed method, firstly, the image preprocessing step is applied. Then, image patches are obtained from pre-processed retinal fundus images. These patches are prepared as input to the deep learning network architecture. The proposed deep learning network architecture is presented as a new CNN architecture. This architecture decides whether the blood vessel pixels in the fundus image are arteries or veins. The proposed method was evaluated on publicly available given DRIVE, INSPIRE datasets, and the recently created LES-AV dataset. The performance of the proposed method was evaluated by using the most commonly used sensitivity, specificity, and accuracy performance measures. The accuracy measure for all vessel pixels is 0.9110 for DRIVE, 0.9654 for INSPIRE, and 0.9531 for LES-AV dataset. The proposed method is compared with other state-of-the-art artery/vein separation methods. The experimental results of the proposed method are promising. This method is suitable for automatic artery/vein separation.

Performance measurement of safety-critical systems based on ordinary differential equations and Petri nets: A case study of nuclear power plant

This article proposes a novel approach to measure the performance of Safety-Critical Systems (SCS). Such systems contain multiple processing nodes that communicate with each other is modeled by a Petri nets (PN). The paper uses the PN for the performance evaluation of SCS. A set of ordinary differential equations (ODEs) is derived from the Petri net model that represent the state of the system, and the solutions can be used to measure the system's performance. The proposed method can avoid the state space explosion problem and also introduces new metrics of performance, along with their measurement: deadlock, liveness, stability, boundedness, and steady state. The proposed technique is applied to Shutdown System (SDS) of Nuclear Power Plant (NPP). We obtained 99.887% accuracy of performance measurement, which proves the effectiveness of our approach.

How Should Researchers Measure Perovskite‐Based Monolithic Multijunction Solar Cells’ Performance? A Calibration Lab's Perspective

Perovskite multijunctions (PVSK MJs) have made remarkable progress with monolithic PVSK/PVSK tandems surpassing the efficiency of single‐junction (1 J) PVSK cells and PVSK/Si cells reported to exceed the 30% efficiency mark. These efficiencies are reported at standard test conditions (STC), established by the photovoltaic (PV) community to facilitate comparison between devices and technologies. Herein, it is discussed why an accurate STC performance measurement for a MJ is more complicated than for a 1 J cell and the special aspects to be considered when measuring the current–voltage characteristics and the performance of PVSK‐containing MJ cells are emphasized. It is discussed why a spectrally adjustable solar simulator is needed and the sequence of accurate performance measurement, namely measurement of the spectral response of each junction, adjustment of the spectrum, and appropriate protocols for measuring the power output of the device at STC, is presented. For all these, common errors and pitfalls that sometime lead to misleading interpretation of the results are presented, and the methods to evaluate the accuracy of the data when a spectrally adjustable solar simulator is not available are recommended. Finally, first step is taken toward recommending performance measurement approaches when high throughput is required as will eventually be in a production line.

THE IMPORTANCE OF ORGANIZATIONS PERFORMANCE ON ORGANIZATIONAL GROWTH

Accurate and appropriate measurement of performance is a critical element in entrepreneurship research (Aktan, Bulut, 2008). According to Achemfuor,(2012), a well-organised system of performance measurement may be the single most powerful mechanism at management’s disposal to enhance the probability of successful strategy implementation. In many of the studies in the field of strategic entrepreneurship, firm performance is defined as a dependent variable and the entrepreneurship activity of the firm is considered an independent variable. There is strong agreement among the researchers that the results of successful entrepreneurial activities positively correlate with the improvement of a company’s performance (Fairoz, Hirobumi&amp; Tanaka, 2010).&#x0D; Measuring absolute firm performance is very difficult, because the concept is complex and multi-dimensional. Researchers suggest that multiple performance indicators should be used to measure this complex construct (Farrington, Matchaba, 2011). The predictable approach to firm’s performance has been to consider financial performance (Frank, Kessler &amp; Fink, 2010). Another approach to organisational performance is to use measurement against purpose, using perceptual measures of company performance (Fatoki, 2012).

Product-level circularity metrics based on the “Closing–Slowing Future–Past” quadrant model

Critical literature review coupled with conceptual analysis provides a holistic framework for measuring a product's circularity performance based on its integrated contribution to the operationalization of various configurations of circular economy (CE) strategies. The following main results were obtained. First, we outline the “Сlosing–Slowing and Future–Past” (“CSFP”) quadrant model (“CSFP” quadrant model) of CE-related product attributes and the circular product categories derived as findings of our previous study in the form of modifications to develop a product-level circularity metric. Second, we propose fifteen product circularity profiles with singular and configurative CE strategy-related contributions building on the “СSFP” model and circular product categories. Third, we quantify the 6Rs (reuse, repair, refurbishment, remanufacturing, repurposing, recycling) for the strategy-related contributions derived from the “CSFP” quadrant model to propose a product circularity data profile coupled with a metrical scale for measuring product circularity performance and visualizing quantified circular contributions. To trial, showcase and validate the relevance of the developed model and associated visualization tool, we conduct two case studies comparing the circularity performances of smartphones and packaging bags, respectively. Companies can use the proposed framework as a maturity scale to enhance the circularity performance of a product. This work contributes to a more accurate measurement of product circularity performance and a sharper understanding of its integrated contribution to circular strategies.

Practical Applications of Option-Enhanced Tax-Smart Portfolio Value

Since the origination of the investment discipline, tax implications have always been secondary to investment decisions. Investment education resources and those for training investment professionals do not emphasize after-tax performance. Another idea favored by the investment community is that when investing, a more accurate measure of performance is not how much one makes but how much one keeps. These two ideas conflict since potential tax obligations from selling securities can significantly impact the actual value of investments. In <b><i>Option-Enhanced Tax-Smart Portfolio Value</i></b>, from the April 2022 issue of <b><i>The Journal of Portfolio Management</i></b> and other articles about the impact of taxes on investments, author <b>Andrew Kalotay</b> of <b>Kalotay Advisors</b> contends that there must be a change in how taxable securities are valued and how the performance of taxable investment portfolios is derived. “To discuss how you can improve after-tax performance intelligently, you must develop a way to measure it,” says Kalotay. This philosophical change in investment valuation would facilitate tax-loss harvesting decisions, enhance investment performance measurement, and provide a better way to compare the performance of fund managers and advisers. Kalotay believes that “there should be much more attention by academics and practitioners on the critical topic of after-tax performance.”

DESIGN OF HOTEL PERFORMANCE LEVEL MEASUREMENT SYSTEM BASED ON HALAL VALUE WITH BSC METHOD

In running the hotel business, it is always required to be able to compete and maintain the loyalty of the stakeholders in order to continue to exist. One of the efforts to be able to compete and maintain the loyalty of stakeholders is to find out how far the performance of the hospitality business is and what efforts are being made to improve its performance. In measuring company performance, it does not only focus on general aspects, but also considers other aspects, namely measuring company performance based on its halal value. In this study, the Balanced Scorecard method was used based on halal values ​​to be able to measure the company's performance. The results obtained after measuring using the Balanced Scorecard, namely the final score of the company's overall performance was obtained at 82.08% which was included in the very good category. From a financial perspective, the results of achievement, level of achievement, and scores from 2013 to 2015 were 32.95%, 71.04%, and 71.04%, respectively. Then from the customer's perspective, the results of achievement, level of achievement, and scores from 2013 to 2015 were 83.98%, 92.48%, and 92.48%, respectively. From the perspective of internal business processes, the results of achievement, level of achievement, and scores from 2013 to 2015 were 83.34%, 83.34%, and 83.34%, respectively. From the growth and learning perspective, the results of achievement, level of achievement, and scores from 2013 to 2015 were 60.78%, 86.26%, and 86.26%, respectively. However, the measurement using the Balanced Scorecard only focuses on the company's financial results and the unfavorable correlation between non-financial measures and the results. This situation will be a weakness of the Balanced Scorecard method because it cannot continue to take more specific steps forward so it cannot perform more accurate performance measurements.