Automatic Assessment Method Research Articles

Predicting Engagement of Older People’s Virtual Teams from Video Call Analysis

This study examines seniors’ creative engagement in group activities using synchronous communication tools and explores automatic assessment methods through behavioral and psychophysiological measurements. Working with a small senior group on collaborative creative tasks, we implemented a comprehensive data collection approach using audio-visual and physiological measurements. Machine learning models were used to evaluate group creative engagement levels using various data subsets. Results show that engagement assessment can be effective with different feature combinations, allowing flexibility across contexts and constraints. The multimodal approach, combining facial, audio, and body analysis, achieved optimal performance and is recommended when conditions permit. Our research provides insights into seniors’ online creative participation and presents an automated system for detecting creative engagement in virtual teams, supporting active participation strategies.

FEI-YOLO: A Lightweight Soybean Pod-Type Detection Model

Identifying and statistically analyzing soybean pod types are crucial for seed evaluation and yield estimation. Traditional visual assessment by breeding personnel is time-consuming, labor-intensive, and prone to subjective bias, especially with large datasets. Automatic assessment methods usually struggle with the highly confusing pod types with two and three seeds, affecting the model’s identification accuracy. To address these issues, we propose to improve the standard YOLOv5s object detection model to enhance the differentiation between pod types and to boost the model’s efficiency in prediction. To reduce the number of parameters and the computational load, we propose to introduce the FasterNet Block module in the FasterNet model into the original C3 module, leading to improvements in both detection accuracy and speed. To strengthen the feature extraction and representation for specific targets, the Efficient Multi-Scale Attention (EMA) module is incorporated into the C3 module of the backbone network, improving the identification of similar pod types. The Inner-IoU is combined with the CIoU as the loss function to further enhance detection accuracy and generalization. Experiments comparing FEI-YOLO with the baseline YOLOv5s show that FEI-YOLO achieves an mAP@0.5 of 98.6% and an mAP@0.5:0.95 of 81.1%, with improvements of 1.5% and 1.4%, respectively. Meanwhile, the number of parameters is reduced by 13.2%, and FLOPs decreased by 10.8%, demonstrating the model’s effectiveness and efficiency, enabling rapid and accurate identification of soybean pod types from images.

Application of an artificial intelligence for quantitative analysis of endothelial capillary beds in vitro.

The use of endothelial cell cultures has become fundamental to study angiogenesis. Recent advances in artificial intelligences (AI) offer opportunities to develop automated assessment methods in medical research, analyzing larger datasets. The aim of this study was to compare the application of AI with a manual method to morphometrically quantify in vitro angiogenesis. Co-cultures of human microvascular endothelial cells and fibroblasts were incubated mimicking endothelial capillary-beds. An AI-software was trained for segmentation of endothelial capillaries on anti-CD31-labeled light microscope crops. Number of capillaries and branches and average capillary diameter were measured by the AI and manually on 115 crops. The crops were analyzed faster by the AI than manually (3 minutes vs 1 hour per crop). Using the AI, systematically more capillaries (mean 48/mm2 vs 27/mm2) and branches (mean 23/mm2 vs 11/mm2) were counted than manually. Both methods had a strong linear relationship in counting capillaries and branches (r-capillaries = 0.88, r-branches = 0.89). No correlation was found for measurements of the diameter (r-diameter = 0.15). The present AI reduces the time required for quantitative analysis of angiogenesis on large datasets, and correlates well with manual analysis.

Investigation of users’ opinions, perceptions and attitudes towards the use of artificial intelligence in online learning environments

Abstract The impact and importance of AI are being increasingly recognised, although there are still uncertainties regarding its innovative potential. These same uncertainties and effects are also evident in educational contexts. In this study, teacher, student and parents’ opinions, perceptions and attitudes towards the use of AI in online education were examined through a systematic review of the literature on AI applications in online learning environments. Quantitative data on AI studies in education, the role of AI in education and teacher, student and parents’ opinions, perceptions and attitudes towards AI were compiled. The study outcomes present an overview of AI’s role in education and offer recommendations for researchers in this field. In line with the findings, the role of AI in education is divided into six categories, namely ‘chatbot, content, learning assistant, learning incentive, measurement and evaluation, and early warning system’. The study’s results indicate that students hold favourable opinions towards AI applications, particularly in relation to chatbots, adaptive learning techniques and automatic assessment methods. They appreciate the support, friendship and easy use of chatbots. Students perceive AI as a tool that enhances learning experiences, increases motivation and engagement, and improves learning outcomes.

Detection of diffusely abnormal white matter in multiple sclerosis on multiparametric brain MRI using semi-supervised deep learning

In addition to focal lesions, diffusely abnormal white matter (DAWM) is seen on brain MRI of multiple sclerosis (MS) patients and may represent early or distinct disease processes. The role of MRI-observed DAWM is understudied due to a lack of automated assessment methods. Supervised deep learning (DL) methods are highly capable in this domain, but require large sets of labeled data. To overcome this challenge, a DL-based network (DAWM-Net) was trained using semi-supervised learning on a limited set of labeled data for segmentation of DAWM, focal lesions, and normal-appearing brain tissues on multiparametric MRI. DAWM-Net segmentation performance was compared to a previous intensity thresholding-based method on an independent test set from expert consensus (N = 25). Segmentation overlap by Dice Similarity Coefficient (DSC) and Spearman correlation of DAWM volumes were assessed. DAWM-Net showed DSC > 0.93 for normal-appearing brain tissues and DSC > 0.81 for focal lesions. For DAWM-Net, the DAWM DSC was 0.49 ± 0.12 with a moderate volume correlation (ρ = 0.52, p < 0.01). The previous method showed lower DAWM DSC of 0.26 ± 0.08 and lacked a significant volume correlation (ρ = 0.23, p = 0.27). These results demonstrate the feasibility of DL-based DAWM auto-segmentation with semi-supervised learning. This tool may facilitate future investigation of the role of DAWM in MS.

Deep learning-assisted automatic quality assessment of concrete surfaces with cracks and bugholes

Cracks and bugholes are common factors affecting the quality of concrete surface, thus automatic detection of apparent defects is essential for concrete surface quality assessment. This study proposes an automatic quality assessment method based on deep learning for concrete surfaces with different scales of cracks and bugholes. In particular, a modified cascade mask region-based convolutional neural network (MC Mask R-CNN) with ResNet and feature pyramid network (FPN) is presented. Furthermore, a novel quantitative analysis algorithm for instance segmentation and size quantification of cracks and bugholes is proposed. Multi-scale features extracted by ResNet can be integrated using FPN. Closed operations and fixed elliptical kernels are adopted to reduce inaccuracies associated with crack discontinuities. Datasets collected at construction sites are used for training and testing of the network after data augmentation. Comparison work was carried out to evaluate the proposed model, and the box mean average precision (APbbox) and the mask mean average precision (APmask) index can reach up to 0.588 and 0.553. The overall quantization errors of cracks and bugholes were within acceptable limits. Validation experiments were conducted to illustrate the validity of the quality assessment method.

Enhancing Retinal Fundus Image Quality Assessment With Swin-Transformer-Based Learning Across Multiple Color-Spaces.

The assessment of retinal image (RI) quality holds significant importance in both clinical trials and large datasets, because suboptimal images can potentially conceal early signs of diseases, thereby resulting in inaccurate medical diagnoses. This study aims to develop an automatic method for Retinal Image Quality Assessment (RIQA) that incorporates visual explanations, aiming to comprehensively evaluate the quality of retinal fundus images (RIs). We developed an automatic RIQA system, named Swin-MCSFNet, utilizing 28,792 RIs from the EyeQ dataset, as well as 2000 images from the EyePACS dataset and an additional 1,000 images from the OIA-ODIR dataset. After preprocessing, including cropping black regions, data augmentation, and normalization, a Swin-MCSFNet classifier based on the Swin-Transformer for multiple color-space fusion was proposed to grade the quality of RIs. The generalizability of Swin-MCSFNet was validated across multiple data centers. Additionally, for enhanced interpretability, a Score-CAM-generated heatmap was applied to provide visual explanations. Experimental results reveal that the proposed Swin-MCSFNet achieves promising performance, yielding a micro-receiver operating characteristic (ROC) of 0.93 and ROC scores of 0.96, 0.81, and 0.96 for the "Good," "Usable," and "Reject" categories, respectively. These scores underscore the accuracy of RIQA based on Swin-MCSF in distinguishing among the three categories. Furthermore, heatmaps generated across different RIQA classification scores and various color spaces suggest that regions in the retinal images from multiple color spaces contribute significantly to the decision-making process of the Swin-MCSFNet classifier. Our study demonstrates that the proposed Swin-MCSFNet outperforms other methods in experiments conducted on multiple datasets, as evidenced by the superior performance metrics and insightful Score-CAM heatmaps. This study constructs a new retinal image quality evaluation system, which will contribute to the subsequent research of retinal images.

Automatic rehabilitation assessment method of upper limb motor function based on posture and distribution force.

The clinical rehabilitation assessment methods for hemiplegic upper limb motor function are often subjective, time-consuming, and non-uniform. This study proposes an automatic rehabilitation assessment method for upper limb motor function based on posture and distributed force measurements. Azure Kinect combined with MediaPipe was used to detect upper limb and hand movements, and the array distributed flexible thin film pressure sensor was employed to measure the distributed force of hand. This allowed for the automated measurement of 30 items within the Fugl-Meyer scale. Feature information was extracted separately from the affected and healthy sides, the feature ratios or deviation were then fed into a single/multiple fuzzy logic assessment model to determine the assessment score of each item. Finally, the total score of the hemiplegic upper limb motor function assessment was derived. Experiments were performed to evaluate the motor function of the subjects' upper extremities. Bland-Altman plots of physician and system scores showed good agreement. The results of the automated assessment system were highly correlated with the clinical Fugl-Meyer total score (r = 0.99, p < 0.001). The experimental results state that this system can automatically assess the motor function of the affected upper limb by measuring the posture and force distribution.

Design and Use of Deep Confidence Network Based on Crayfish Optimization Algorithm in Automatic Assessment Method of Hearing Effectiveness

INTRODCTION: Listening strategy analysis and assessment not only need objective and fair sound listening strategy analysis, but also need high-precision and high real-time assessment model, and even more need in-depth analysis and feature extraction of the influencing factors of listening assessment.OBJECTIVES: To address the problems of current automatic assessment methods, such as non-specific application, poor generalization, low assessment accuracy, and poor real-time performance.METHODS: This paper proposes an automatic assessment method based on a deep confidence network based on crawfish optimization algorithm. First, the multi-dimensional listening strategy evaluation system is constructed by analyzing the listening improvement strategy; then, the depth confidence network is improved by the crayfish optimization algorithm to construct the automatic evaluation model; finally, through the analysis of simulation experiments.RESLUTS: The proposed method improves the evaluation accuracy, robustness, and real-time performance. The absolute value of the relative error of the automatic evaluation value of the proposed method is controlled in the range of 0.011, and the evaluation time is less than 0.005 s. The method is based on a deep confidence network based on the crayfish optimization algorithm.CONCLUSION: The problems of non-specific application of automated assessment methods, poor generalization, low assessment accuracy, and poor real-time performance are addressed.

Automatic Scoring of Verbal Divergent Thinking Tests: From Lexical Databases to Large Language Models

The article explores the evolution of methods for automatically assessing verbal divergent thinking tests. Resear - chers have increasingly focused on the ability to evaluate the originality of respondents' answers by calculating their semantic distance from the stimulus task. From 2009 to 2019, latent semantic analysis became the primary method for assessing semantic distances. Overall, in terms of internal consistency and correlation with expert ratings, its application yielded satisfactory results, maintaining an acceptable balance of quality and effort expended. However, issues emerged (dependence on the corpus used, result instability, systematic distortions related to the length of analyzed responses), prompting researchers to transition to more advanced models of distributional semantics (GloVe, Word2Vec etc.), large language models, and supervised learning. Large language models, especially those fine-tuned on creativity test materials, demonstrated higher effectiveness compared to models assessing semantic distances and approached expert evaluations. In addition to evaluating originality, the article considers works proposing methods for automatic assessment of elaboration, flexibility, associative flow, and divergent semantic integration. References to online platforms that allow for automatic assessments of originality in responses to divergent tests are provided. The issue of interpreting results obtained through large language models is discussed. A drawback of using these models is the lack of understanding of the basis on which judgments of the originality of creative products are made. The perspectives of applying explainable artificial intelligence for evaluating results of verbal and non-verbal tests of creative thinking are being discussed.

Automatic Assessment Method and Device for Depression Symptom Severity Based on Emotional Facial Expression and Pupil-wave

Automatic Assessment Method and Device for Depression Symptom Severity Based on Emotional Facial Expression and Pupil-wave

MSMTSeg: Multi-Stained Multi-Tissue Segmentation of Kidney Histology Images via Generative Self-Supervised Meta-Learning Framework.

Accurately diagnosing chronic kidney disease requires pathologists to assess the structure of multiple tissues under different stains, a process that is timeconsuming and labor-intensive. Current AI-based methods for automatic structure assessment, like segmentation, often demand extensive manual annotation and focus on single stain domain. To address these challenges, we introduce MSMTSeg, a generative self-supervised meta-learning framework for multi-stained multi-tissue segmentation in renal biopsy whole slide images (WSIs). MSMTSeg incorporates multiple stain transform models for style translation of inter-stain domains, a self-supervision module for obtaining pre-trained models with the domain-specific feature representation, and a meta-learning strategy that leverages generated virtual data and pre-trained models to learn the domain-invariant feature representation across multiple stains, thereby enhancing segmentation performance. Experimental results demonstrate that MSMTSeg achieves superior and robust performance, with mDSC of 0.836 and mIoU of 0.718 for multiple tissues under different stains, using only one annotated training sample for each stain. Our ablation study confirms the effectiveness of each component, positioning MSMTSeg ahead of classic advanced segmentation networks, recent few-shot segmentation methods, and unsupervised domain adaptation methods. In conclusion, our proposed few-shot cross-domain technology offers a feasible and cost-effective solution for multi-stained renal histology segmentation, providing convenient assistance to pathologists in clinical practice. The source code and conditionally accessible data are available at https://github.com/SnowRain510/MSMTSeg.

An X-Ray-Based Multiexpert Inspection Method for Automatic Welding Defect Assessment in Intelligent Pipeline Systems

An X-Ray-Based Multiexpert Inspection Method for Automatic Welding Defect Assessment in Intelligent Pipeline Systems

Quality assessment of Low-cost retinal Videos for Glaucoma screening

Low-cost smartphone-compatible portable ophthalmoscopes can capture visuals of the patient’s retina to screen several ophthalmological diseases like glaucoma. The images captured have lower quality and resolution than standard retinography devices but enough for glaucoma screening. Small videos are captured to improve the chance of inspecting the eye properly; however, those videos may not always have enough quality for screening glaucoma, and the patient needs to repeat the inspection later. In this paper, a method for automatic assessment of the quality of videos captured using the D-Eye lens is proposed and evaluated with a personal dataset with 539 videos. Based on two methods developed for retina localization on the images/frames, the Circle Hough Transform method with a precision of 78,12% and the YOLOv7 method with a precision of 99,78%, the quality assessment method automatically decides on the quality of the video by measuring the number of frames of good-quality in each video, according to the chosen threshold.

PSAA-nnUNet: An Efficient Method for CT Carotid Artery Image Segmentation.

Carotid artery (CA) stenosis (CAS) constitutes a significant factor to ischaemic cerebrovascular events which exhibiting no overt symptoms in the early stages. Early detection of CAS can prevent ischaemic stroke and improve patient prognosis. In this study, we developed a non-invasive CAS automatic assessment method based on deep learning, intended for the early detection of CAS with CT imaging. The method proposed in this paper consists of three main components. First, we utilised thresholding and the Hessian-based Frangi filter to eliminate irrelevant tissue and enhance vascular structures. Second, we introduced a novel neural network named parameter shared axial attention (PSAA)-nnUNet for the automatic segmentation of CA. Finally, we assessed the degree of CAS with the North American Symptomatic Carotid Endarterectomy Trial (NASCET) formula. The PSAA-nnUNet algorithm proposed in this study achieved a segmentation accuracy of 0.82. The non-invasive CAS automatic assessment method based on PSAA-nnUNet exhibits excellent accuracy and great application potential.

Magnetocardiography-based coronary artery disease severity assessment and localization using spatiotemporal features

Objective. This study aimed to develop an automatic and accurate method for severity assessment and localization of coronary artery disease (CAD) based on an optically pumped magnetometer magnetocardiography (MCG) system. Approach. We proposed spatiotemporal features based on the MCG one-dimensional signals, including amplitude, correlation, local binary pattern, and shape features. To estimate the severity of CAD, we classified the stenosis as absence or mild, moderate, or severe cases and extracted a subset of features suitable for assessment. To localize CAD, we classified CAD groups according to the location of the stenosis, including the left anterior descending artery (LAD), left circumflex artery (LCX), and right coronary artery (RCA), and separately extracted a subset of features suitable for determining the three CAD locations. Main results. For CAD severity assessment, a support vector machine (SVM) achieved the best result, with an accuracy of 75.1%, precision of 73.9%, sensitivity of 67.0%, specificity of 88.8%, F1-score of 69.8%, and area under the curve of 0.876. The highest accuracy and corresponding model for determining locations LAD, LCX, and RCA were 94.3% for the SVM, 84.4% for a discriminant analysis model, and 84.9% for the discriminant analysis model. Significance. The developed method enables the implementation of an automated system for severity assessment and localization of CAD. The amplitude and correlation features were key factors for severity assessment and localization. The proposed machine learning method can provide clinicians with an automatic and accurate diagnostic tool for interpreting MCG data related to CAD, possibly promoting clinical acceptance.

Applying artificial intelligence to determination of legal age of majority from radiographic data

Forensic odontologists use biological patterns to estimate chronological age for the judicial system. The age of majority is a legally significant period with a limited set of reliable oral landmarks. Currently, experts rely on the questionable development of third molars to assess whether litigants can be prosecuted as legal adults. Identification of new and novel patterns may illuminate features more dependably indicative of chronological age, which have, until now, remained unseen. Unfortunately, biased perceptions and limited cognitive capacity compromise the ability of researchers to notice new patterns. The present study demonstrates how artificial intelligence can break through identification barriers and generate new estimation modalities. A convolutional neural network was trained with 4003 panoramic-radiographs to sort subjects into 'under-18' and 'over-18' age categories. The resultant architecture identified legal adults with a high predictive accuracy equally balanced between precision, specificity and recall. Moving forward, AI-based methods could improve courtroom efficiency, stand as automated assessment methods and contribute to our understanding of biological ageing.

Pose estimation methods for automatic general movements assessment in infants: influence of video acquisition settings and processing

Pose estimation methods for automatic general movements assessment in infants: influence of video acquisition settings and processing

Memory-based unsupervised video clinical quality assessment with multi-modality data in fetal ultrasound

In obstetric sonography, the quality of acquisition of ultrasound scan video is crucial for accurate (manual or automated) biometric measurement and fetal health assessment. However, the nature of fetal ultrasound involves free-hand probe manipulation and this can make it challenging to capture high-quality videos for fetal biometry, especially for the less-experienced sonographer. Manually checking the quality of acquired videos would be time-consuming, subjective and requires a comprehensive understanding of fetal anatomy. Thus, it would be advantageous to develop an automatic quality assessment method to support video standardization and improve diagnostic accuracy of video-based analysis. In this paper, we propose a general and purely data-driven video-based quality assessment framework which directly learns a distinguishable feature representation from high-quality ultrasound videos alone, without anatomical annotations. Our solution effectively utilizes both spatial and temporal information of ultrasound videos. The spatio-temporal representation is learned by a bi-directional reconstruction between the video space and the feature space, enhanced by a key-query memory module proposed in the feature space. To further improve performance, two additional modalities are introduced in training which are the sonographer gaze and optical flow derived from the video. Two different clinical quality assessment tasks in fetal ultrasound are considered in our experiments, i.e., measurement of the fetal head circumference and cerebellar diameter; in both of these, low-quality videos are detected by the large reconstruction error in the feature space. Extensive experimental evaluation demonstrates the merits of our approach.

Evaluating the kit-build concept mapping process using sub-map scoring

Concept mapping allows learners to visually represent their knowledge by connecting nodes (concepts) and links (relations between concepts). The kit-build (KB) concept map framework enhances this process by enabling learners to recompose a concept map from provided nodes and links, leading to improved learning outcomes. Additionally, KB employs an automatic assessment method called “Full Map Scoring (FMS)”, which evaluates the learner’s understanding based on the recomposed concept map. However, FMS only evaluates the final product of the recomposition activity, neglecting the process itself. This is a potential limitation because different processes leading to the same result could reflect different levels of understanding among learners. Therefore, it is crucial to incorporate process analysis into learner assessment. To address this issue, our research proposes a new assessment procedure termed “Sub-Map Scoring (SMS)”. A concept map is generally composed of several sub-maps with each sub-map representing a set of meanings. We hypothesize that if a learner comprehends the meaning of a sub-map, the learner will recompose the sub-map as a continuous activity. Therefore, SMS evaluates the recomposition process of each sub-map from the viewpoint of continuity, and the overall SMS score is derived from these evaluations. To verify the effectiveness of SMS, we compared SMS and FMS scores using data from a practical use of the KB framework. A multiple linear regression analysis confirmed that the SMS score was a more precise predictor of learning gain than the FMS score.