Model For Task Research Articles

Analysis on dendritic deep learning model for AMR task

This study introduces a novel hybrid deep learning model featuring a dendritic layer for enhancing the performance of automatic modulation recognition (AMR). By replacing the fully connected layer, the proposed model demonstrates superior classification accuracy in AMR tasks. Comparative experiments with nine state-of-the-art deep learning models on the RadioML2016.10a dataset reveal its consistent superiority. Statistical analyses, including the Friedman test and Wilcoxon signed-rank test, confirm the significant advantage of the HDM-D model.

Network Security Intrusion Detection Methods Combining Optimization Algorithms and Neural Networks

Cyber-attacks are becoming increasingly common and sophisticated, and the security of net-works is a pressing concern for companies everywhere. To spot and stop attempts to access network resources without authorization, intrusion. There is, however, a problem that many traditional IDS methods can generate a lot of false alarms. This means that it misidentifies safe activity as a threat and is not readily accepted by security teams for which real issues. Scalability and resistance to evolving threats a hybrid approach of a neural network combined with an adaptive algorithm was used in this paper focuses on improvement the performance and accuracy of the intrusion detection system proposed by us. Combining genetic algorithms and particle swarm optimization with neural networks using a framework optimized for comprehensive performance is present in this tutorial. More to the point, parameters and training procedures for Deep learning models in intrusion detection tasks. The objective of this research paper model is to make a false alarm the rate should be decreased, search accuracy increased and in real-time network environments, the system should adapt easily. Experimental results show that the combination of optimization algorithms and neural networks outperforms traditional IDS methods.

Unveiling anomalies: transformative insights from transformer-based autoencoder models

Nowadays, Intrusion Detection Systems (IDS) play a critical role in safeguarding networks by identifying and mitigating unauthorized access and malicious activities, yet face challenges in accurately detecting sophisticated threats while minimizing false positives. In this paper, we present a comprehensive analysis of a Transformer-based Autoencoder model for network intrusion detection using the NSL-KDD dataset. We evaluate the model's anomaly detection performance based on key metrics such as Mean Squared Error (MSE), Precision-Recall Area Under Curve (AUC), and Root Mean Squared Error (RMSE) on the test set. Our findings reveal promising results, with the model achieving low MSE, high AUC, and relatively low RMSE values, indicating its effectiveness in detecting anomalies. Additionally, we provide an in-depth examination of the model architecture, highlighting the role of encoder and decoder layers, dropout regularization, and optimization techniques such as Adam optimizer. Our analysis sheds light on the efficacy of Transformer-based Autoencoder models for network intrusion detection tasks, offering insights into their architectural design and performance evaluation. The novelty of this study lies in the implementation of a Transformer-based Autoencoder architecture for network intrusion detection, showcasing its superior performance in anomaly detection compared to traditional methods.

A Rule-Based Agent for Unmanned Systems with TDGG and VGD for Online Air Target Intention Recognition

Air target intention recognition (ATIR) is critical for unmanned systems in modern air defense operations. Through the analysis of typical air defense combat scenarios, first, the paper defines the intention space and intention parameters of air units based on military experience and domain knowledge. Then, a rule-based agent for unmanned systems for online intention recognition is proposed, with no training, no tagging, and no big data support, which is not only for intention recognition and parameter prediction, but also for formation identification of air targets. The most critical point of the agent is the introduction and application of a thermal distribution grid graph (TDGG) and virtual grid dictionary (VGD), where the former is used to identify the formation information of air targets, and the latter is used to optimize the storage space and simplify the access process for the large-scale and real-time combat information. Finally, to have a performance evaluation and application analysis for the algorithm, we carried out a data instance analysis of ATIR for unmanned systems and an air defense warfare simulation experiment based on a Wargame platform; the comparative experiments with the classical k-means, FCNIRM, and the sector-based forward search method verified the effectiveness and feasibility of the proposed agent, which characterizes it as a promising tool or baseline model for the battlefield situational awareness tasks of unmanned systems.

Domain adaptation spatial feature perception neural network for cross-subject EEG emotion recognition

Emotion recognition is a critical research topic within affective computing, with potential applications across various domains. Currently, EEG-based emotion recognition, utilizing deep learning frameworks, has been effectively applied and achieved commendable performance. However, existing deep learning-based models face challenges in capturing both the spatial activity features and spatial topology features of EEG signals simultaneously. To address this challenge, a domain-adaptation spatial-feature perception-network has been proposed for cross-subject EEG emotion recognition tasks, named DSP-EmotionNet. Firstly, a spatial activity topological feature extractor module has been designed to capture spatial activity features and spatial topology features of EEG signals, named SATFEM. Then, using SATFEM as the feature extractor, DSP-EmotionNet has been designed, significantly improving the accuracy of the model in cross-subject EEG emotion recognition tasks. The proposed model surpasses state-of-the-art methods in cross-subject EEG emotion recognition tasks, achieving an average recognition accuracy of 82.5% on the SEED dataset and 65.9% on the SEED-IV dataset.

Evaluating YOLO Models for Efficient Crack Detection in Concrete Structures Using Transfer Learning

The You Only Look Once (YOLO) network is considered highly suitable for real-time object detection tasks due to its characteristics, such as high speed, single-shot detection, global context awareness, scalability, and adaptability to real-world conditions. This work introduces a comprehensive analysis of various YOLO models for detecting cracks in concrete structures, aiming to assist in the selection of an optimal model for future detection and segmentation tasks. The YOLO models are initially trained on a dataset containing both images with and without cracks, producing a generalized model capable of extracting abstract features beneficial for crack detection. Subsequently, transfer learning is employed using a dataset that reflects real-world conditions, such as occlusions, varying crack sizes, and rotations, to further refine the model. Crack detection in concrete remains challenging due to the wide variation in crack sizes, aspect ratios, and complex backgrounds. To achieve optimal performance, we test different versions of YOLO, a state-of-the-art single-shot detector, and aim to balance inference speed and mean average precision (mAP). Our results indicate that YOLOv10 demonstrates superior performance, achieving a mean average precision (mAP) of 74.52% with an inference time of 19.5 milliseconds per image, making it the most effective among the models tested.

Quantum Convolutional Neural Network: A Hybrid Quantum-Classical Approach for Iris Dataset Classification

This paper presents a hybrid quantum-classical machine learning model for classification tasks, integrating a 4-qubit quantum circuit with a classical neural network. The quantum circuit is designed to encode the features of the Iris dataset using angle embedding and entangling gates, thereby capturing complex feature relationships that are difficult for classical models alone. The model, which we term a Quantum Convolutional Neural Network (QCNN), was trained over 20 epochs, achievinga perfect 100% accuracy on the Iris dataset test set on 16 epoch. Our results demonstrate the potential of quantum-enhanced models in supervised learning tasks, particularly in efficiently encoding and processing data using quantum resources. We detail the quantum circuit design, parameterizedgate selection, and the integration of the quantum layer with classical neural network components.This work contributes to the growing body of research on hybrid quantum-classical models and theirapplicability to real-world datasets.

A foundation model for enhancing magnetic resonance images and downstream segmentation, registration and diagnostic tasks.

In structural magnetic resonance (MR) imaging, motion artefacts, low resolution, imaging noise and variability in acquisition protocols frequently degrade image quality and confound downstream analyses. Here we report a foundation model for the motion correction, resolution enhancement, denoising and harmonization of MR images. Specifically, we trained a tissue-classification neural network to predict tissue labels, which are then leveraged by a 'tissue-aware' enhancement network to generate high-quality MR images. We validated the model's effectiveness on a large and diverse dataset comprising 2,448 deliberately corrupted images and 10,963 images spanning a wide age range (from foetuses to elderly individuals) acquired using a variety of clinical scanners across 19 public datasets. The model consistently outperformed state-of-the-art algorithms in improving the quality of MR images, handling pathological brains with multiple sclerosis or gliomas, generating 7-T-like images from 3 T scans and harmonizing images acquired from different scanners. The high-quality, high-resolution and harmonized images generated by the model can be used to enhance the performance of models for tissue segmentation, registration, diagnosis and other downstream tasks.

Models optimized for real-world tasks reveal the task-dependent necessity of precise temporal coding in hearing

Neurons encode information in the timing of their spikes in addition to their firing rates. Spike timing is particularly precise in the auditory nerve, where action potentials phase lock to sound with sub-millisecond precision, but its behavioral relevance remains uncertain. We optimized machine learning models to perform real-world hearing tasks with simulated cochlear input, assessing the precision of auditory nerve spike timing needed to reproduce human behavior. Models with high-fidelity phase locking exhibited more human-like sound localization and speech perception than models without, consistent with an essential role in human hearing. However, the temporal precision needed to reproduce human-like behavior varied across tasks, as did the precision that benefited real-world task performance. These effects suggest that perceptual domains incorporate phase locking to different extents depending on the demands of real-world hearing. The results illustrate how optimizing models for realistic tasks can clarify the role of candidate neural codes in perception.

A Hybrid RNN-CNN based Motor Imagery Tasks Classification Approach Using MEG Brain Signals for BCI Applications

<p dir="ltr"><span>Magnetoencephalography (MEG) has become a pioneering technology in Brain-Computer Interfaces (BCIs) for neurorehabilitation, which significantly improves communication and motor rehabilitation for people with neurological conditions, especially stroke survivors. MEG-based BCIs allow users to regain control over their motor and cognitive functions by providing precise temporal and spatial resolution for detecting neural activity. The MEG has several benefits including reduced distortion from the skull and scalp, its non-invasive nature, and the ability to capture deep brain activity, all of which enhance the effectiveness of BCI systems. This study focuses on classifying motor and cognitive tasks using various models, including Recurrent Neural Networks (RNNs), one-dimensional Convolutional Neural Networks (1DCNNs), and a hybrid approach. The effectiveness of the models in classification tasks was evaluated using various metrics such as accuracy, recall, precision, and F1-score. Significantly, the hybrid model exhibited enhanced performance relative to the other models, achieving marked improvements in both classification accuracy and robustness. These results underscore the promise of MEG BCI technology for neurorehabilitation and stress the need for the development of sophisticated classification models to support the recovery of motor and cognitive abilities in people with neurological conditions. This study adds to the expanding research focused on enhancing the quality of life for affected individuals by utilizing innovative BCI solutions, leveraging the unique capabilities of MEG technology to enable more effective neurorehabilitation interventions.</span></p>

Improved Osteoporosis Prediction in Breast Cancer Patients Using a Novel Semi-Foundational Model.

Small cohorts of certain disease states are common especially in medical imaging. Despite the growing culture of data sharing, information safety often precludes open sharing of these datasets for creating generalizable machine learning models. To overcome this barrier and maintain proper health information protection, foundational models are rapidly evolving to provide deep learning solutions that have been pretrained on the native feature spaces of the data. Although this has been optimized in Large Language Models (LLMs), there is still a sparsity of foundational models for computer vision tasks. It is in this space that we provide an investigation into pretraining Visual Geometry Group (VGG)-16, Residual Network (ResNet)-50, and Dense Network (DenseNet)-121 on an unrelated dataset of 8500 chest CTs which was subsequently fine-tuned to classify bone mineral density (BMD) in 199 breast cancer patients using the L1 vertebra on CT. These semi-foundational models showed significant improved ternary classification into mild, moderate, and severe demineralization in comparison to ground truth Hounsfield Unit (HU) measurements in trabecular bone with the semi-foundational ResNet50 architecture demonstrating the best relative performance. Specifically, the holdout testing AUC was 0.99 (p-value < 0.05, ANOVA versus no pretraining versus ImageNet transfer learning) and F1-score 0.99 (p-value < 0.05) for the holdout testing set. In this study, the use of a semi-foundational model trained on the native feature space of CT provided improved classification in a completely disparate disease state with different window levels. Future implementation with these models may provide better generalization despite smaller numbers of a disease state to be classified.

МОЖЛИВОСТІ ТА ОБМЕЖЕННЯ ВЕЛИКИХ МОВНИХ МОДЕЛЕЙ

The work is dedicated to the study of large language models (LLMs) and approaches to improving their efficiency in a new service. The rapid development of LLMs based on transformer architecture has opened up new possibilities in natural language processing and the automation of various tasks. However, fully utilizing the potential of these models requires a thorough approach and consideration of numerous factors. A review of the evolution of large language models was conducted, highlighting leading companies engaged in the research and development of efficient systems. The structure of these models and ways of representing internal knowledge were examined. Key approaches to training were described, including data collection, preprocessing, and selecting appropriate neural network architectures used in large language models. It was noted that the greatest breakthrough was achieved with the Transformer neural network, which is based on the attention mechanism. A comparison of popular transformer-based chatbots was presented, namely: ChatGPT, Claude AI, and Gemini AI. Their metrics, capabilities, and limitations were identified. The relevance of the topic lies in the rapid development of natural language processing technologies and the growing demand for large language models across various industries. The effective use of these models has tremendous potential to improve productivity and the quality of work with textual data. However, due to the complexity of the architecture and the large amounts of data required for training, selecting and configuring the optimal model for a specific task is a challenging process. As a result of the study, recommendations for developers were provided on the use of popular open-source models in the new service or integration with third-party programs. The characteristics of the models, their strengths, limitations, and certain caveats regarding trust in the generated results were indicated. Keywords: large language models, transformer architecture, neural networks, chatbot, content generation.

A Novel Image Formation Model for Descattering.

In the field of image descattering, the image formation models employed for restoration approaches are often simplified. In these models, scattering distribution is uniform in homogeneous media when transmission is fixed. Through specifically designed experiments, we discover that scattering exhibits non-uniform characteristics even in homogeneous media. Neglecting non-uniform scattering in these models limits their accuracy in representing scattering distribution, resulting in existing image descattering approaches inadequate. To tackle these issues, this paper proposes a novel image formation model for image descattering, considering more physical parameters, such as zenith angle, azimuth angle, scattering phase function, and camera focal length. Our model describes the light transfer process in scattering media more accurately. For image descattering, we introduce corresponding algorithms for parameter estimation in our model and simultaneous restoration from degraded images. Experimental evaluations demonstrate the effectiveness of our proposed model in various tasks, including physical parameter estimation, pure-scattering removal, image dehazing, and underwater image restoration. In terms of calculating parameters, our results are close to the real values; in terms of underwater image restoration, our work outperforms the state-of-art methods; in terms of image dehazing, our work promotes the performance of existing methods by replacing previous models with our model.

MVP-HOT: A moderate visual prompt for hyperspectral object tracking

The growing attention to hyperspectral object tracking (HOT) can be attributed to the extended spectral information available in hyperspectral images (HSIs), especially in complex scenarios. This potential makes it a promising alternative to traditional RGB-based tracking methods. However, the scarcity of large hyperspectral datasets poses a challenge for training robust hyperspectral trackers using deep learning methods. Prompt learning, a new paradigm emerging in large language models, involves adapting or fine-tuning a pre-trained model for a specific downstream task by providing task-specific inputs. Inspired by the recent success of prompt learning in language and visual tasks, we propose a novel and efficient prompt learning method for HOT tasks, termed Moderate Visual Prompt for HOT (MVP-HOT). Specifically, MVP-HOT freezes the parameters of the pre-trained model and employs HSIs as visual prompts to leverage the knowledge of the underlying RGB model. Additionally, we develop a moderate and effective strategy to incrementally adapt the HSI prompt information. Our proposed method uses only a few (1.7M) learnable parameters and demonstrates its effectiveness through extensive experiments, MVP-HOT can achieve state-of-the-art performance on three hyperspectral datasets.

Comparative Analysis of Machine Learning Algorithms: KNN, SVM, Decision Tree and Logistic Regression for Efficiency and Performance

This paper presents a comparative study of machine learning models namely K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Decision Tree, and Logistic Regression on two datasets: 20Newsgroups and Wine datasets. Based on basic accuracy, and precision, and recall of the models, F1 measures were assessed. When compared with other classifiers such as KNN and Decisions Trees, SVM and Logistic regression gave better results especially in the case of the 20Newsgroups dataset dominated by textual high dimensional data. KNN had poor recall results and Decision Tree was moderate. In the Wine dataset, since the structure of data is comparatively less complex in our context then all the models yielded almost similar results with accuracy and precision factors very close to 1.0 which of course manifested that the choice of the model does not affect much on simple data. These results stress the importance of the choice of the appropriate model for tasks with a certain level of data complexity; detailed models show the greatest efficiency at the accomplishment of complicated tasks, though at the same time, they are not required for simple, structured data.

The deep continual learning framework for prediction of blast-induced overbreak in tunnel construction

Blast-induced overbreak, characterized by the excessive removal of rock mass beyond the planned tunnel profile, poses significant safety risks, increases costs, and causes project delays during tunneling. Traditional machine/deep learning models have been developed to predict overbreak. However, these models are often inadequate because they are static and lack the flexibility to adapt to new, real-world data continuously. This study addresses this limitation by introducing a novel data-driven approach based on deep continual learning. The primary objective is to develop an adaptable predictive model with the ability of continual learning, which is particularly advantageous in dynamic environments like tunnel blasting. To achieve this, a self-attention multi-layer perceptron (MLP) model for overbreak prediction, integrated with two continual learning strategies (elastic weight consolidation (EWC) and memory replay (MR)), is developed. This step enables the overbreak prediction model to possess the ability to continuously learn real-world scenarios and adapt to the dynamic environment of tunnel blasting. The findings show that the continuous MLP model, empowered by EWC and MR, demonstrates superior adaptability and accuracy in predicting overbreak. Compared with the standard MLP model, which achieves a predictive accuracy of 0.831, the continuous MLP model achieves a predictive accuracy of 0.845 on unseen data. The integration of EWC and MR strategies proves to be a pivotal factor in developing deep learning models for the dynamic task of predicting overbreak. The continual learning strategies ensure that the models remain adaptable and accurate over time, which is essential for practical applications in dynamic environments of tunnel blasting operations.

LLMOverTab: Tabular data augmentation with language model-driven oversampling

In recent years, Large Language Model (LLM) have seen significant advancements, attracting attention for their applications in various fields. These models have shown promising results in handling tabular data, especially in cases with limited datasets, by leveraging pre-trained knowledge. However, their effectiveness in addressing imbalanced data in tabular formats is less explored. To bridge this gap, our study introduces LLMOverTab, a novel approach using LLMs for oversampling in imbalanced tabular data. We conducted comprehensive experiments on diverse tabular datasets to assess the effectiveness of LLMOverTab, demonstrating its potential in improving the handling of imbalanced data. The study also explores application of LLMOverTab in zero-shot and few-shot learning contexts, providing insights into its adaptability. Additionally, we analyze the oversampled data, offering reflections on the quality of generated samples. Our research not only showcases the utility of LLMOverTab in managing imbalanced tabular data, but also opens new avenues for the application of language models in various tasks of tabular data. This study adds to the increasing interest in applying LLMs to various task domains. It provides new perspectives for the innovative use of LLMs in structured tabular data fields, highlighting their potential in a range of applications.

EMGE: Entities and Mentions Gradual Enhancement with semantics and connection modelling for document-level relation extraction

Relation extraction is the process of identifying connections between entities in unstructured text and is a critical component of entity-centred information extraction to uncover latent knowledge structures in complex documents. Although graph-based methods have pushed the state-of-the-art forward in relation extraction, current approaches still exhibit limitations. These include incomplete capture of graph structural features, inadequate modelling of long-distance dependencies and imprecise representation of complex entity interactions. A novel Entities and Mentions Gradual Enhancement framework called EMGE is proposed. It integrates both contextual and structural information to robustly enhance entity representations for document-level relation extraction. It comprises three primary components: 1) a dynamic relation aware enhancement mechanism to comprehensively encode graph structural features; 2) a multi-scale feature enhancement module to effectively capture long-distance dependencies; and 3) an entity-mention pair enhancement mechanism to yield precise representations of classification targets. Extensive empirical evaluation on five widely-adopted datasets demonstrates that EMGE achieves promising performance. Particularly noteworthy are the substantial gains obtained on the challenging CDR dataset, where EMGE achieved relative improvements of 1.5%, 8.8%, and 3.5% over the strongest baseline in terms of the Intra-F1, Inter-F1 and Overall-F1 metrics, respectively. Further experimental results demonstrate that the proposed model outperforms the popular large language model in relation extraction tasks. Our code is available on github. 11EMGE: https://github.com/GZU-SAMLab/EMGE.

A Stakeholder-Related Procedure Model for Social Sustainability Assessment

The importance of assessing the sustainability of companies and their activities is increasing. Especially since the implementation of the new Corporate Social Reporting Directive, even more companies are committed to reporting on the impacts of their businesses on sustainability. This is a challenge especially concerning the social dimension of sustainability. Some frameworks present various relevant social criteria that can be used as a basis for assessment. However, these do not provide detailed suggestions for handling the numerous methodological challenges of such a multi-criteria assessment: identification and selection of the relevant stakeholders, categories and indicators to measure the impacts on social sustainability, weighting and aggregation of these criteria, etc. Therefore, this paper contributes to the methodology of social sustainability assessment by presenting a procedure model for this specific assessment task. The novelty of the model results from its foundation by a review of methods for selection, normalisation, weighting and aggregation of social criteria, existing decision theory-based procedure models, as well as a stakeholder-oriented catalogue of criteria. The procedure model is structured hierarchically by subdividing the overall social sustainability-assessment task into different levels: stakeholders, categories, and indicators. Furthermore, appropriate methods are suggested for the single steps of the procedure model.

DR-Z2AN: dual-recurrent neural network with a tri-channel attention mechanism for financial management prediction

Financial management prediction, often known as financial forecasting, is the act of estimating future financial outcomes using past data and present trends. It is an essential component of financial analysis and planning that aids businesses in making well-informed decisions and preparing for potential future events. In the healthcare domain, financial management prediction is a crucial task that helps patients track and predict the expenses required for their medical services. The established methods for financial management prediction have some flaws, such as the requirement of labeled data, data quality, time complexity, under fitting problems, and longer execution times. Therefore, in order to resolve these limitations; a deep learning-based model is developed in this study for efficient financial management prediction. Specifically, this research proposes a dual-recurrent neural network with a tri-channel attention mechanism (DR-Z2AN) for accurate prediction. The proposed DR-Z2AN model combines the tri-channel attention mechanism with dual-RNN and multi-head attention, which enhances the robustness and interpretability of the systems. The multi-head attention learns the complex relationships between the data, which develops the generalization capability of the model in prediction tasks. The combined model efficiently processes the sequence data, and the tri-channel attention improves the model's capacity to extract meaningful characteristics from the input. The integration of the incentive learning approach helps the model improve the learning parameters to get better results with the minimum error. The experimental results demonstrate that the DR-Z2AN model attains minimal error in terms of MAE, MAPE, MSE, and RMSE of 1.46, 3.83, 4.32, and 2.08, respectively; thus, the proposed approach gives better results than the other traditional methods. Overall, the DR-Z2AN model offers accurate predictions with reduced computational time and improved interpretability.