Evaluation Metrics Research Articles

3D human avatar reconstruction with neural fields: A recent survey

3D human avatar reconstruction aims to reconstruct the 3D geometric shape and appearance of the human body from various data inputs, such as images, videos, and depth information, acting as a key component in human-oriented 3D vision in the metaverse. With the progress in neural fields for 3D reconstruction in recent years, significant advancements have been made in this research area for shape accuracy and appearance quality. Meanwhile, substantial efforts on dynamic avatars with the representation of neural fields have exhibited their effect. Although significant improvements have been achieved, challenges still exist in in-the-wild and complex environments, detailed shape recovery, and interactivity in real-world applications. In this survey, we present a comprehensive overview of 3D human avatar reconstruction methods using advanced neural fields. We start by introducing the background of 3D human avatar reconstruction and the mainstream paradigms with neural fields. Subsequently, representative research studies are classified based on their representation and avatar partswith detailed discussion. Moreover, we summarize the commonly used available datasets, evaluation metrics, and results in the research area. In the end, we discuss the open problems and highlight the promising future directions, hoping to inspire novel ideas and promote further research in this area.

Predicting Electricity Consumption using an Innovative Deep Energy Predictor Model for Enhanced Accuracy and Efficiency

Accurate forecasting of electricity consumption is crucial for efficient energy management and planning. Traditional models often struggle to capture the complex, nonlinear relationships inherent in energy consumption data, leading to less reliable predictions. In the quest to improve the accuracy and efficiency of electricity consumption predictions, we introduce the Deep Energy Predictor Model (DEPM). This innovative hybrid model combines the strengths of XGBoost for classification and Deep Neural Networks (DNN) for deep learning (DL) capabilities. The model leverages advanced feature extraction techniques using Cascaded ResNet, ensuring the capture of intricate patterns within the data. Implemented in Python, the DEPM highlights an impressive accuracy of 98%, with a precision of 0.97, recall of 0.99, and an F1-score of 0.98, setting a new standard for predictive models in the energy sector. This study elaborates on the model architecture, implementation details, and the evaluation metrics that underscore the model's superior performance. Our results demonstrate the potential of DEPM to significantly enhance the reliability of electricity consumption forecasts, providing a robust tool for energy management and planning.

Preference learning based on adaptive graph neural network for multi-criteria decision support

Intelligent decision-making assists decision-makers (DMs) in making choices through data analysis, model prediction, and automated processes. Central to this field are two key concepts: multi-criteria decision making (MCDM) and preference learning (PL). While MCDM and PL both aim to develop decision models that rank alternatives based on observed or revealed preferences, they diverge in focus. MCDM concentrates on the DMs' perspectives, whereas PL emphasizes model-driven approaches. This divergence presents significant challenges in integrating these methodologies, particularly in ensuring the integrated method remains scalable and interpretable amidst the complexity of decision scenarios. To bridge this gap, our study introduces the use of graph structures to frame decision problems and proposes a novel PL method employing graph neural network (GNN) for multi-criteria decision support. This method is anchored in the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) technique and combines an adaptive GNN model with a weight determination model. The GNN model updates embeddings from the alternative's criteria and category features, utilizing an attention mechanism to adaptively assess their importance. Concurrently, the weight determination model contains a weight neural network module to set objective criterion weights and a game theory-based module for calculating combined criterion weights. The method not only inherits the interpretability and intuitive appeal of decision models but also leverages the computational efficiency and high accuracy of machine learning. In experiments conducted on benchmark datasets, our method exhibits significant performance improvements, especially in ranking-related evaluation metrics, outperforming the best baseline by 5.78%.

WEKA-based machine learning for traffic congestion prediction in Amman City

<p>Traffic congestion leads to wasted time, pollution, and increased fuel consumption. Traffic congestion prediction has become a developing research topic in recent years, particularly in the field of machine learning (ML). The evaluation of various traffic parameters is used to predict traffic congestion by relying on historical data. In this study, we will predict traffic congestion in Amman City, specifically at the 8th circle, using different ML classifiers. The 8th circle links four main streets: Westbound, Northbound, Eastbound, and Southbound. Datasets were collected from the greater Amman municipality hourly. The logistic regression (LR), k-nearest neighbor (KNN), decision tree (DT), random forest (RF), support vector machine (SVM), and multi-layer perceptron (MLP) classifiers have been chosen to predict traffic congestion at each street linked with the 8th circle. The waikato environment for knowledge analysis (WEKA) data mining tool is used to evaluate chosen classifiers by determining accuracy, F-measure, sensitivity, and precision evaluation metrics. The results obtained from all experiments have demonstrated that SVM is the best classifier to predict traffic congestion. The accuracy of SVM to predict traffic congestion at Westbound Street, Northbound Street, Eastbound Street, and Southbound Street was 99.4%, 99.7%, 99.6%, and 99.1%, respectively.</p>

Multifeature extraction based MobileViTv3 model for fish feeding behavior recognition from video streaming

The recognition of fish feeding behavior based on machine vision is essential for optimizing fish feeding strategies and enhancing the efficiency of aquaculture. Building an efficient, multi-feature extraction model for fish feeding recognition, especially on mobile and edge devices, remains a significant challenge. In the paper, we propose a novel multi-feature extraction (MFE)-MobileViTv3 model, which improve MobileViTv3 with the MFE blocks. It can extract spatio-temporal features while obtaining the lightweight characteristic. The MFE block is designed by improving ActionNet, with the frequency channel attention (FCA) and multi-head self-attention (MHSA) mechanisms. It can fully extract spatio-temporal, motion, and channel features from video streaming, thereby further improving the feature extraction capabilities, which subsequently enhances the model's recognition accuracy. The experiments were carried out on an industrial aquaculture farm. We built a dataset of Micropterus salmoides, and then conducted the compare experiments. Compared with C3D, R3D, ResNet50, SlowFast, AlexNet, and MobileNetV3, our model can achieve a classification accuracy of 96.7 % for feeding intensity, with fewer parameters (0.96 M) and FLOPs (8.44G). The results show that the proposed model can effectively recognize fish feeding behavior with fewer parameters. Additionally, we introduce two evaluation metrics for the feeding process: Average Feeding Intensity and Strong Feeding Ratio. The metrics are conducive to the quantitative evaluation of the fish's vigor and health status.

DeepCervix: enhancing cervical cancer detection through transfer learning with VGG-16 architecture

Cervical cancer remains a significant global health concern, emphasizing the urgent need for improved detection methods to ensure timely treatment. This research introduces a sophisticated methodology leveraging recent advances in medical imaging and deep learning algorithms to enhance the accuracy and efficiency of cervical cancer detection. The proposed approach comprises meticulous data preprocessing to ensure the integrity of input images, followed by the training of deep learning models including ResNet-50, AlexNet, and VGG-16, renowned for their performance in computer vision tasks. Evaluation metrics such as accuracy, precision, recall, and F1-score demonstrate the efficacy of the methodology, with an outstanding accuracy rate of 98% achieved. The model’s proficiency in accurately distinguishing healthy cervical tissue from cancerous tissue is underscored by precision, recall, and F1-score values. The primary strength of this deep learning-based approach lies in its potential for early detection, promising significant impact on cervical cancer diagnosis and treatment outcomes. This methodology contributes to advancements in medical imaging techniques, facilitating improved outcomes in cervical cancer detection and treatment.

Early Detection of Parkinson’s Disease Using Novel Deep Learning Frameworks Optimized with Advanced Techniques and Machine Learning Models Evaluated through Performance Metrics

Parkinson’s disease (PD) is a progressive neurodegenerative condition that frequently remains undiagnosed in its early stages due to subtle and overlapping symptoms. This study presents an innovative approach that integrates advanced deep learning architectures, optimization techniques, and machine learning models to enhance the accuracy of early PD detection. By utilizing performance metrics for model evaluation and comparison, the research identifies the most effective methods for achieving precise and reliable diagnoses. The proposed framework exhibits exceptional performance in differentiating early-stage Parkinson’s cases from healthy individuals, contributing to improved clinical decision-making and enabling timely interventions. Experimental results provide to prove the proposed frame work techniques using ML and DL with better optimizations and its performance metrics.

Classification of Skin Diseases using Decision Tree Algorithm on an Imbalanced Dataset

Skin infections caused by pathogens such as bacteria and fungi are common and can lead to serious health complications if not properly managed. Accurate classification of these infections is crucial for effective treatment and management. This study focuses on classifying two skin diseases, Chickenpox and Shingles, using a Decision Tree algorithm applied to an imbalanced dataset sourced from Kaggle. The dataset, which is imbalanced by nature, was split into training (80%) and testing (20%) subsets. Pre-processing involved segmentation using Thresholding to isolate regions of interest and feature extraction using Hu Moments to capture shape characteristics of the lesions. The dataset was scaled to ensure that all features had a mean of 0 and variance of 1. The classifier's performance was evaluated using 5-fold cross-validation, yielding a mean accuracy of 66.06%, with precision, recall, and F1-scores indicating moderate performance. The study highlights the challenges posed by imbalanced datasets and the limitations of the Decision Tree algorithm in this context. The results underscore the importance of proper pre-processing and feature extraction but also suggest the need for more advanced classification techniques and data balancing methods. This research contributes to the field by providing a detailed methodology and comprehensive evaluation metrics, offering insights into the application of machine learning for medical image classification. Future work should focus on improving classifier performance through data augmentation, advanced feature extraction, and exploring other machine learning models better suited for imbalanced datasets.

Breaking Down the Metrics: A Comparative Analysis of LLM Benchmarks

Due to the fast advancement in large language models (LLMs), the natural language processing (NLP) technology domain has witnessed a massive change. It has brought ground breaking developments in how machines understand and generate human language. However, even with these advancements, it remains hard to objectively compare and evaluate LLMs' performance because various benchmarks and metrics are used in the assessment. Such a method requires this study's writers to conduct a comparative analysis of various benchmark LLMs to explain the viability of different evaluation metrics. Analyzing the recognized standards, including GLUE, SuperGLUE, and SQuAD, reveals weaknesses and potential in the present evaluating systems. It embraces quantitative assessments of model performances and benchmarking and critical evaluations of benchmark designs and their scope. The paper discusses methodological issues in benchmarking research and explains that current measures are insufficient and lack comprehensive evaluation patterns. The conclusions of this research underline the significance of creating fully integrated and efficient reference sets that match the rate of innovation in LLM ability and inform other studies and the overall subsequent advancement of more sophisticated and flexible language models.

Batik Image Representation using Multi Texton Co-occurrence Histogram

This paper introduces a novel approach to batik image representation using the texton-based and statistical Multi Texton Co-occurrence Histogram (MTCH). The MTCH framework is leveraged as a robust batik image descriptor, capable of encapsulating a comprehensive range of visual features, including the intricate interplay of color, texture, shape, and statistical attributes. The research extensively evaluates the effectiveness of MTCH through its application on two well-established public batik datasets, namely Batik 300 and Batik Nitik 960. These datasets serve as benchmarks for assessing the performance of MTCH in both classification and image retrieval tasks. In the classification domain, four distinct scenarios were explored, employing various classifiers: the K-Nearest Neighbors (K-NN), Support Vector Machine (SVM), Decision Tree (DT), and Naïve Bayes (NB). Each classifier was rigorously tested to determine its efficacy in correctly identifying batik patterns based on the MTCH descriptors. On the other hand, the image retrieval tasks were conducted using several distance metrics, including the Euclidean distance, City Block, Bray Curtis, and Canberra, to gauge the retrieval accuracy and the robustness of the MTCH framework in matching similar batik images. The empirical results derived from this study underscore the superior performance of the MTCH descriptor across all tested scenarios. The evaluation metrics, including accuracy, precision, and recall, indicate that MTCH not only achieves high classification performance but also excels in retrieving images with high similarity to the query. These findings suggest that MTCH is a highly effective tool for batik image analysis, offering significant potential for applications in cultural heritage preservation, textile pattern recognition, and automated batik classification systems.

URDO&RG: A Tool for User Role Discovery and Optimization & Requirement Generation Based on User Check-in Data in LBSN

The regularities shared by a set of users in a long-term in Location-Based Social Network (LBSN) can be abstracted as user roles. Although methods are proposed to identify user roles, the quantitative analysis of the effectiveness of the method is insufficient, and as far as we know, tools to generate user roles automatically from user data are missing. Therefore, a tool named URDO&RG (User Role Discovery and Optimization & Requirements Generation) is realized. The tool includes four main functions: (1) visualization of the data from multiple dimensions in diverse display diagram, (2) discovery of user roles using various algorithms, (3) optimization of the method to achieve better user role discovery outcome. (4) generation of user requirements from user data. Data set is used to validate the effectiveness of the tool. Result shows that, the tool can discover user roles and generate user requirements effectively, and the optimization method outperforms the original methods in many evaluation metrics.

Adaptive demand planning models: Utilizing reinforcement learning to address contingencies and dynamic market conditions in supply chains

There are remarkable opportunities in supply chain management associated with using the reinforcement learning (RL) approach in demand planning. As opposed to numerous other techniques, such as modeling and forecasting and applying them over a certain period, RL allows for the making and adapting these decisions in real-time, depending on the current demand and other conditions in the market. Supplementary to this, RL-based models enable supply chains to constantly adapt to shifts since they directly update knowledge from incoming data, making them less susceptible to economic shocks or other supply uncertainties. This flexibility is particularly important for contemporary supply chains in uncertain global environments where conventional, deterministic demand planning techniques cannot address changing needs. In this research, we discover how RL-based models could reduce demand volatility and react to contingencies. As such, it is best suited for industries that are required to respond flexibly and faster. This paper enlightens RL on the benefits of demand planning. It provides live examples that give insight into how it may be used to improve inventory, reduce cost, and improve decision-making. Regarding the specific research questions, this study helps investigate the methodologies, evaluation metrics, and case outcomes that shape RL's potential for changing demand planning and improving supply chain adaptability for future research on adaptive supply chain management.

Cascaded frameworks in underwater optical image restoration

Optical imaging and vision technology have become crucial research topics in the field of underwater and ocean scenes. These technologies play a vital role in advancing underwater exploration, scientific research, and smart aquaculture. Despite their importance, optical systems face substantial challenges in complex water environments, especially in turbid conditions where light absorption and scattering by water and suspended particles drastically reduce the quality of captured images. This degradation highlights the need for sophisticated image restoration algorithms that can effectively recover the quality of images impaired by turbid water conditions. Cascaded frameworks, which incorporate multiple modules, including that of color correction, detail enhancement, and information fusion, have emerged as particularly efficacious, offering significant improvements over traditional direct or end-to-end restoration methods. This review elaborates on the comparative benefits of various cascading structures — serial, parallel, and hybrid — and examines different restoration algorithms that utilize intrinsic image features, physical models, prior knowledge, and deep learning technologies. Furthermore, it provides a comprehensive overview of publicly available datasets and evaluation metrics, both referential and non-referential, which are crucial for developing, validating, and optimizing underwater image restoration algorithms. Additionally, by conducting experimental comparative studies of several open restoration algorithms, the effectiveness and applicability of different methods are validated. Finally, the review proposes potential research directions, aiming to deliver valuable insights to scholars in underwater application and computer vision, thereby inspiring innovative research initiatives that could further enhance the capabilities of underwater optical imaging.

Key Node Identification Method Based on Multilayer Neighbor Node Gravity and Information Entropy

In the field of complex network analysis, accurately identifying key nodes is crucial for understanding and controlling information propagation. Although several local centrality methods have been proposed, their accuracy may be compromised if interactions between nodes and their neighbors are not fully considered. To address this issue, this paper proposes a key node identification method based on multilayer neighbor node gravity and information entropy (MNNGE). The method works as follows: First, the relative gravity of the nodes is calculated based on their weights. Second, the direct gravity of the nodes is calculated by considering the attributes of neighboring nodes, thus capturing interactions within local triangular structures. Finally, the centrality of the nodes is obtained by aggregating the relative and direct gravity of multilayer neighbor nodes using information entropy. To validate the effectiveness of the MNNGE method, we conducted experiments on various real-world network datasets, using evaluation metrics such as the susceptible-infected-recovered (SIR) model, Kendall τ correlation coefficient, Jaccard similarity coefficient, monotonicity, and complementary cumulative distribution function. Our results demonstrate that MNNGE can identify key nodes more accurately than other methods, without requiring parameter settings, and is suitable for large-scale complex networks.

Adaptive CLIP for open-domain 3D model retrieval

In order to effectively enhance the practicality of 3D model retrieval, we adopt a single real image as the query sample for retrieving 3D models. However, the significant differences between 2D images and 3D models in terms of lighting conditions, textures and backgrounds, posing a great challenge for accurate retrieval. Existing work on 3D model retrieval mainly focuses on closed-domain research, while the open-domain condition where the category relationship between the query image and the 3D model is unknown is more in line with the needs of real scenarios. CLIP shows significant promise in comprehending open-world visual concepts, facilitating effective zero-shot image recognition. Based on this multimodal pre-training large language model, we introduce Adaptive Open-domain Semantic Nearest-neighbor Contrast (AOSNC), a method for learning and aligning multi-modal text, image, and 3D model. In order to solve the issue of inconsistent cross-domain categories and difficult sample correlation in open-domain, we construct a cross-modal bridge using CLIP. This model utilizes textual features to bridge the gap between 2D images and 3D model views. Additionally, we design an adaptive network layer to address the limitations of the pre-training model for 3D model views and enhance cross-modal alignment. We propose a mutual nearest-neighbor semantic alignment loss to address the challenge of aligning features from disparate modalities (text, images, and 3D models). This loss function enhances cross-modal learning by effectively associating and distinguishing features, improving retrieval accuracy. We conducted comprehensive experiments using the image-based 3D model retrieval dataset MI3DOR and the cross-domain 3D model retrieval dataset NTU-PSB to validate the superiority of the proposed method. Our results show significant improvements in several evaluation metrics, underscoring the efficacy of our method in augmenting cross-modal feature alignment and retrieval performance.

An enhanced interval-valued PM2.5 concentration forecasting model with attention-based feature extraction and self-adaptive combination technology

Accurate predictions of air quality enable governments and relevant authorities to take promptly measures for protecting public health. With the increasing time-varying nature of air pollutants, predicting daily average concentrations alone is no longer sufficient for environmental management and risk warning. Hence, this paper proposes a multi-resolution interval-valued PM2.5 concentration combination prediction model, which based on interval decomposition and attention mechanism reconstruction. Firstly, the interval-valued time series (ITS) was decomposed and adaptively reconstructed using the binary empirical mode decomposition (BEMD) algorithm and attention-based reconstruction. Subsequently, multi-resolution linear projection layers were applied to extract temporal features from the time series. Finally, a hybrid prediction module was implemented that combines CNN and LSTM to predict each subsequence and integrate them to derive the final interval prediction values for PM2.5. In the proposed framework, the reconstruction technique effectively resolved the issue of inconsistent numbers of different feature decomposition subsequences, while the linear projection layer fully captured the multi-resolution characteristics of the time series. Empirical studies conducted in three districts of Beijing showed that, compared to state-of-the-art baseline models, the framework reduced the average values of five interval evaluation metrics by 11.2%, 17.4%, 11.7%, 10.5%, and 14.8%, respectively. This interval-valued prediction framework can effectively assist urban air quality management and warning.

Combining MRI radiomics and clinical features for early identification of drug-resistant epilepsy in people with newly diagnosed epilepsy

ObjectiveTo identify newly diagnosed patients with drug-resistant epilepsy (DRE) based on radiomics and clinical features. MethodsA radiomics approach was used to combine clinical features with magnetic resonance imaging (MRI) features extracted by the ResNet-18 deep learning model to predict DRE. Three machine learning classifiers were built, and k-fold cross-validation was used to assess the classifier outcomes, and other evaluation metrics of accuracy, sensitivity, specificity, F1 score, and area under the curve (AUC) were used to evaluate the performance of these models. ResultsOne hundred and thirty-four newly diagnosed epilepsy patients with 13 available clinical features and 1394 MRI features extracted by the ResNet-18 model were included in our study. Then three machine learning classifiers were built based on5 clinical features and 8 MRI features, including Support Vector Machine (SVM), Gradient-Boosted Decision Tree (GBDT) and Random Forest. After internally validation, the GBDT model performed the best, with an average accuracy of 0.85 [95% confidence interval (CI) 0.77–0.91], sensitivity of 0.97 [95% CI 0.85–1.00], specificity of 0.96 [95% CI 0.83–1.00], F1 score of 0.81 [95% CI 0.77–0.89], AUC of 0.95 [95% CI 0.82–0.99], and ten-fold cross validation avg score of 0.96 [95% CI 0.89–0.99] in test set. SignificanceThis study offers a novel approach for early diagnosis of DRE. Radiomics can provide potential diagnostic and predictive information to support personalized treatment decisions.

Automated Food Weight and Content Estimation Using Computer Vision and AI Algorithms

The work aims to leverage computer vision and artificial intelligence technologies to quantify key components in food distribution services. Specifically, it focuses on dish counting, content identification, and portion size estimation in a dining hall setting. An RGB camera is employed to capture the tray delivery process in a self-service restaurant, providing test images for plate counting and content identification algorithm comparison, using standard evaluation metrics. The approach utilized the YOLO architecture, a widely recognized deep learning model for object detection and computer vision. The model is trained on labeled image data, and its performance is assessed using a precision–recall curve at a confidence threshold of 0.5, achieving a mean average precision (mAP) of 0.873, indicating robust overall performance. The weight estimation procedure combines computer vision techniques to measure food volume using both RGB and depth cameras. Subsequently, density models specific to each food type are applied to estimate the detected food weight. The estimation model’s parameters are calibrated through experiments that generate volume-to-weight conversion tables for different food items. Validation of the system was conducted using rice and chicken, yielding error margins of 5.07% and 3.75%, respectively, demonstrating the feasibility and accuracy of the proposed method.

Classifying Dry Eye Disease Patients from Healthy Controls Using Machine Learning and Metabolomics Data

Background: Dry eye disease is a common disorder of the ocular surface, leading patients to seek eye care. Clinical signs and symptoms are currently used to diagnose dry eye disease. Metabolomics, a method for analyzing biological systems, has been found helpful in identifying distinct metabolites in patients and in detecting metabolic profiles that may indicate dry eye disease at early stages. In this study, we explored the use of machine learning and metabolomics data to identify cataract patients who suffer from dry eye disease, a topic that, to our knowledge, has not been previously explored. As there is no one-size-fits-all machine learning model for metabolomics data, choosing the most suitable model can significantly affect the quality of predictions and subsequent metabolomics analyses. Methods: To address this challenge, we conducted a comparative analysis of eight machine learning models on two metabolomics data sets from cataract patients with and without dry eye disease. The models were evaluated and optimized using nested k-fold cross-validation. To assess the performance of these models, we selected a set of suitable evaluation metrics tailored to the data set’s challenges. Results: The logistic regression model overall performed the best, achieving the highest area under the curve score of 0.8378, balanced accuracy of 0.735, Matthew’s correlation coefficient of 0.5147, an F1-score of 0.8513, and a specificity of 0.5667. Additionally, following the logistic regression, the XGBoost and Random Forest models also demonstrated good performance. Conclusions: The results show that the logistic regression model with L2 regularization can outperform more complex models on an imbalanced data set with a small sample size and a high number of features, while also avoiding overfitting and delivering consistent performance across cross-validation folds. Additionally, the results demonstrate that it is possible to identify dry eye in cataract patients from tear film metabolomics data using machine learning models.

Semi-PKD: Semi-supervised Pseudoknowledge Distillation for saliency prediction

In saliency prediction, Knowledge Distillation (KD) is leveraged to improve the predictive performance of compact Student Networks. However, the challenge is searching for an optimal teacher–student pair while handling the unavailability of large-scale annotations in the Pseudoknowledge Distillation (PKD). To overcome this challenge, a semi-supervised method is proposed; Semi-PKD. This method involves pseudo-label generation on unlabeled data by a Teacher Network trained using the exponential moving average KD (EMA-KD) method. The EMA-KD method utilizes only the Student Network by acquiring self-knowledge, solving the problem of optimal teacher–student pair selection. Semi-PKD outperforms other state-of-the-art saliency prediction models across various evaluation metrics. The code is available at https://github.com/chakkritte/Semi-PKD.