Neural Network Model Research Articles

Baseline model for deep neural networks in resource-constrained environments: an empirical investigation

Baseline model for deep neural networks in resource-constrained environments: an empirical investigation

HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs.

The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20 K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ( ∼5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448×448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224×224 inputs.

Early macular edema recognition after cataract surgery based on U-Net assisted optical coherence tomography

ObjectiveThe optical coherence tomography (OCT) model of macular edema in postoperative cataract patients is constructed by using convolutional neural network (CNN) U-Net, and its value is evaluated by relevant indicators. MethodsFrom January 2020 to December 2023, 200 binocular OCT images of 140 patients with macular edema after cataract surgery and 280 images of normal subjects in our hospital were included as data sets. They were randomly divided into a training set (70%) and valiadation set (30%). Res-SE U-Net was used to construct an automatic recognition model of macular edema in OCT images of patients after cataract surgery. The model was evaluated by the change of loss function and accuracy, and by drawing the receiver operating characteristic curve (ROC). ResultsThe segmentation efficacy between the U-Net model and its enhanced counterpart, Res-SE U-Net, was assessed in this study. Notably, the Dice coefficient for macular edema segmentation exhibited substantial improvement with the Res-SE U-Net model, reaching 0.96 compared to 0.90 for the conventional U-Net model. Additionally, evaluation based on the intersection over union (IoU) coefficient revealed a superiority of the Res-SE U-Net model with a coefficient of 0.79, in contrast to 0.76 for the U-Net model. Moreover, the area under the curve (AUC) analysis further supported the enhanced performance of the Res-SE U-Net model. The AUC values were determined as 0.837 for the Res-SE U-Net model and 0.811for the U-Net model. ConclusionThe Res-SE U-Net model of convolutional neural network based on OCT images can effectively identify and segment macular edema in patients after phacoemulsification.

Sqrt-Loglogish CNN and Markov model for 5G spectrum sensing application

The research presents innovative methods for spectrum sensing in 5G networks, using the Sqrt-Loglogish convolutional neural network (SL-CNN) and hidden orthogonal intuitionistic fuzzy Markov model (HOIFMM). The proposed methods aim to tackle issues related to detecting principal user signals accurately, mitigating interference, and efficiently utilizing the spectrum in wideband spectrum environments due to their diverse and ever changing characteristics. The Sqrt-Loglogish CNN improves spectrum sensing by addressing static threshold dependency and potential overfitting. The HOIFMM offers a complex framework for predicting sparsity levels and primary user patterns. The results highlight the effectiveness of the suggested techniques in differentiating primary user signals from noise and interference, resulting in enhanced interference management tactics and overall network performance. MATLAB simulation is performed and compared the proposed methods performance with existing state-of-the-art methods such as CNN, deep neural network (DNN), long short-term memory (LSTM) and artificial neural networks (ANN). The proposed method has outperformed existing methods in terms of sensitivity, accuracy, and precision. Future endeavors include improving these methods, investigating sophisticated machine learning algorithms, and doing real world validations to guarantee scalability and resilience in various 5G deployment situations. This research advances the spectrum sensing capabilities in 5G networks, potentially improving efficiency, reliability, and quality of service.

Application of Lightweight Deep Learning Model-Based Shear Wave Elastic Imaging with Abdominal Probe to Diagnose Prostate Cancer: A Biomedical Approach

We aimed to investigate the diagnostic value of lightweight convolutional neural network (CNN) model abdominal probe shear wave elastic imaging (SWE) in the perineal diagnosis and guided puncture biopsy of prostate cancer (PCa), and to provide reference for the clinical diagnosis of PCa. 100 PCa patients were assigned to group I (malignant) and group II (benign), with 50 cases in each. Ultrasonic elastic imaging based on lightweight convolutional neural network denoising model was adopted for detection. In both systolic and diastolic blood pressure (SBP/DBP), there was not a significant intergroup difference (P > 0.05). The levels of prostate specific antigen (PSA) and its free variant (fPSA) in group II were markedly lower (P < 0.05). Patients in group II had obviously more cystic components and fewer solid components. Patients with hyperechogenicity was more in group II. Patients had clearly fewer irregular margins and outward margin spread in group II. Patients without focal hyperechogenicity and punctate hyperechogenicity was more in group II, and the number of calcifications in group II was less. Patients with type 0 and type I was more and patients with type IIa and type IIb was less in group II. The Emean level of patients in group II was clearly higher, and the Emax level and Esd level of patients in group II were clearly lower. The SI level of patients was clearly lower in group II TTP was higher in group II (P < 0.05). Multivariate logistic regression analysis of abdominal probe SWE for transperineal diagnosis of PCa and guided puncture biopsy showed that internal echoes had the greatest OR and were associated with the occurrence of PCa. Ultrasonic elastic imaging index based on the lightweight convolutional neural network denoising model can be used for the benign and malignant diagnosis of PCa patients.

Enhancing emotion detection with synergistic combination of word embeddings and convolutional neural networks

Recognizing emotions in textual data is crucial in a wide range of natural language processing (NLP) applications, from consumer sentiment research to mental health evaluation. The word embedding techniques play a pivotal role in text processing. In this paper, the performance of several well-known word embedding methods is evaluated in the context of emotion recognition. The classification of emotions is further enhanced using a convolutional neural network (CNN) model because of its propensity to capture local patterns and its recent triumphs in text-related tasks. The integration of CNN with word embedding techniques introduced an additional layer to the landscape of emotion detection from text. The synergy between word embedding techniques and CNN harnesses the strengths of both approaches. CNNs extract local patterns and features from sequential data, making them well-suited for capturing relevant information within the embeddings. The results obtained with various embeddings highlight the significance of choosing synergistic combinations for optimum performance. The combination of CNNs and word embeddings proved a versatile and effective approach.

Optimizing Lung Cancer Diagnosis with Machine Learning and Feature Selection Methods

Lung cancer is a prevalent disease, with nearly 238,000 new cases diagnosed in 2023. This study utilizes clinical predictors from a Kaggle dataset containing 309 observations across 15 variables to aid in lung cancer diagnosis. The variables include swallowing difficulty, peer pressure, gender, allergy, yellow fingers, anxiety, wheezing, alcohol consumption, chronic disease, chest pain, coughing, fatigue, smoking, age, and shortness of breath. The research aims to develop and compare various supervised machine learning models for classifying and predicting lung cancer, while also identifying key clinical tests and parameters using unsupervised statistical models. The dataset was divided into training and test sets, balanced, and preprocessed for unbiased training. Feature selection and machine learning models were applied to identify crucial predictors. The study explored tree models, logistic regression, Naïve Bayes, support vector machine (SVM), ensemble, neural network, and kernel models. Among these, the linear SVM achieved the highest accuracy of 93.75% with 5-fold cross-validation. However, it showed overfitting, with a lower test accuracy of 82.55%. The Gaussian Naïve Bayes model emerged as the optimal choice, providing consistent performance between validation and test cases. It achieved the highest cross-validation classification accuracy of 82.81% using only 9 variables: swallowing difficulty, peer pressure, gender, allergy, yellow fingers, anxiety, wheezing, alcohol consumption, and chronic disease. This model allows for effective training with fewer predictors without compromising classification

On-device training of artificial intelligence models on microcontrollers

<p><span>Numerous studies are currently training artificial intelligence (AI) models on tiny devices constrained by computing power and memory limitations by implementing model optimization algorithms. The question arises whether implementing traditional AI models directly on small devices like micro-controller units (MCUs) is feasible. In this study, a library has been developed to train and predict the artificial neural network (ANN) model on common MCUs. The evaluation results on the regression problem indicate that, despite the extensive training time, when combined with multitasking programming on multi-core MCUs, the training does not adversely affect the system's execution. This research contributes an additional solution that enables the direct construction of ANN models on MCU systems with limited resources.</span></p>

Quantitative strategies of different loss functions aggregation for knowledge distillation

<p>Deep learning models have been successfully applied to many visual tasks. However, they tend to be increasingly cumbersome due to their high computational complexity and large storage requirements. How to compress convolutional neural network (CNN) models while still maintain their efficiency has received increasing attention from the community, and knowledge distillation (KD) is efficient way to do this. Existing KD methods have focused on the selection of good teachers from multiple teachers, or KD layers, which is cumbersome, expensive computationally, and requires large neural networks for individual models. Most of teacher and student modules are CNN-based networks. In addition, recent proposed KD methods have utilized cross entropy (CE) loss function at student network and KD network. This research focuses on the quantifiable evaluation of teacher-student model, in which knowledge is not only distilled from training models that have the same CNN architecture but also from different architectures. Furthermore, we propose combination of CE, balance cross entropy (BCE), and focal loss functions to not only soften the value of loss function in transferring knowledge from large teacher model to small student model but also increase classification performance. The proposed solution is evaluated on four benchmark static image datasets, and the experimental results show that our proposed solution outperforms the state-of-the-art (SOTA) methods from 2.67% to 9.84% at top 1 accuracy.</p>

New insight in cervical cancer diagnosis using convolution neural network architecture

<p>The Pap smear is a screening method for early cervical cancer diagnosis. The selection of the right optimizer in the convolutional neural network (CNN) model is key to the success of the CNN in image classification, including the classification of cervical cancer Pap smear images. In this study, stochastic gradient descent (SGD), root mean square propagation (RMSprop), Adam, AdaGrad, AdaDelta, Adamax, and Nadam optimizers were used to classify cervical cancer Pap smear images from the SipakMed dataset. Resnet-18, Resnet-34, and VGG-16 are the CNN architectures used in this study, and each architecture uses a transfer-learning model. Based on the test results, we conclude that the transfer learning model performs better on all CNNs and optimization techniques and that in the transfer learning model, the optimization has little influence on the training of the model. Adamax, with accuracy values of 72.8% and 66.8%, had the best accuracy for the VGG-16 and Resnet-18 architectures, respectively. Resnet-34 had 54.0%. This is 0.034% lower than Nadam. Overall, Adamax is a suitable optimizer for CNN in cervical cancer classification on Resnet-18, Resnet-34, and VGG-16 architectures. This study provides new insights into the configuration of CNN models for Pap smear image analysis.</p>

Modeling of neural network monitoring agent to predict traffic spikes and agent training

Modeling of neural network monitoring agent to predict traffic spikes and agent training

Graph Alignment Neural Network Model With Graph to Sequence Learning

Graph Alignment Neural Network Model With Graph to Sequence Learning

Spectral characteristics of dissolved organic matter in Plateau Lakes: Identifying eutrophication indicators in Southwest China

Dissolved organic matter (DOM) acts as a chemical intermediary between terrestrial and lacustrine ecosystems and significantly affects the structure and function of lakes. The trophic state of lakes, driven by terrestrial input and phytoplankton biomass, alters the optical properties of DOM. From November 2018 to July 2019, we collected 119 water samples from the Erhai watershed and analyzed them using UV–Vis and EEM-PARAFAC to study the optical properties of DOM in relation to the trophic conditions. Our result indicated that the tyrosine-like protein (C1), tryptophan-like protein (C2), and humic-like compounds (C3) were among the mostly autochthonous components of the DOM. The percentage of the C3 was higher in eutrophic lakes than in mesotrophic and light-eutrophic lakes. The ultraviolet absorption coefficients at 254 nm (aCDOM(254)) and fluorescence intensity at 355 nm (Fn(355)) increased significantly (p < 0.01) with an increased trophic state. Our findings indicate that the influence of nutrients and environmental factors (such as pH and water temperature) on DOM varies with the trophic state. The development of novel predictive models for trophic state assessment was largely based on the significant correlations between TSI and aCDOM(254) (R2 = 0.762, p < 0.01) and Fn(355) (R2 = 0.705, p < 0.01). This neural network model facilitates the creation of a novel fast assessment tool by highlighting the connection between DOM features and the trophic state index. By enabling swift experimental measurements, this model offers a high-resolution monitoring solution for tracking the eutrophication of plateau lakes and rivers.

Design of novel convolution neural network model for lung cancer detection by using sensitivity maps

&lt;p&gt;Despite the existence of numerous models for detecting lung cancer, there is still room for achieving higher levels of accuracy. In this paper, a maximum sensitivity neural network (MSNN) has been proposed. As the name suggests, the model aims to achieve high sensitivity and offers a viable remedy to minimize the number of false positive in oder to improve the overall accuracy for lung cancer detection. The MSNN model is a promising model since it can efficiently interpret grayscale lung computed tomography (CT) scan images as inputs and can be trained using just a few images also. This model has surpassed previous deep learning models by obtaining a remarkable sensitivity of 94.6% and an accuracy of 96.9%. A sensitivity map is created, offering important insights into the critical regions for finding malignant nodules. This innovative method has shown outstanding performance in identifying lung cancer with a low false positive rate which can increase the accuracy of medical diagnoses.&lt;/p&gt;

Optical character recognition for Telugu handwritten text using SqueezeNet convolutional neural networks model

Optical character recognition for Telugu handwritten text using SqueezeNet convolutional neural networks model

Detection and recognition of the invasive species, Hylurgus ligniperda, in traps, based on a cascaded convolution neural network.

Hylurgus ligniperda, an invasive species originating from Eurasia, is now a major forestry quarantine pest worldwide. In recent years, it has caused significant damage in China. While traps have been effective in monitoring and controlling pests, manual inspections are labor-intensive and require expertise in insect classification. To address this, we applied a two-stage cascade convolutional neural network, YOLOX-MobileNetV2 (YOLOX-Mnet), for identifying H. ligniperda and other pests captured in traps. This method streamlines target and non-target insect detection from trap images, offering a more efficient alternative to manual inspections. Two cascade convolutional neural network models were employed in two stages to detect both target and non-target insects from images captured in the same forest. Initially, You Only Look Once X (YOLOX) served as the target detection model, identifying insects and non-insects from the collected images, with non-insect targets subsequently filtered out. In the second stage, MobileNetV2, a classification network, classified the captured insects. This approach effectively reduced false positives from non-insect objects, enabled the inclusion of additional classification terms for multi-class insect classification models, and utilized sample control strategies to enhance classification performance. Application of the cascade convolutional neural network model accurately identified H. ligniperda, and Mean F1-score of all kinds of insects in the trap was 0.98. Compared to traditional insect classification, this method offers great improvement in the identification and early warning of forest pests, as well as provide technical support for the early prevention and control of forest pests. © 2024 Society of Chemical Industry.

VQ-NeRF: Neural Reflectance Decomposition and Editing With Vector Quantization.

We propose VQ-NeRF, a two-branch neural network model that incorporates Vector Quantization (VQ) to decompose and edit reflectance fields in 3D scenes. Conventional neural reflectance fields use only continuous representations to model 3D scenes, despite the fact that objects are typically composed of discrete materials in reality. This lack of discretization can result in noisy material decomposition and complicated material editing. To address these limitations, our model consists of a continuous branch and a discrete branch. The continuous branch follows the conventional pipeline to predict decomposed materials, while the discrete branch uses the VQ mechanism to quantize continuous materials into individual ones. By discretizing the materials, our model can reduce noise in the decomposition process and generate a segmentation map of discrete materials. Specific materials can be easily selected for further editing by clicking on the corresponding area of the segmentation outcomes. Additionally, we propose a dropout-based VQ codeword ranking strategy to predict the number of materials in a scene, which reduces redundancy in the material segmentation process. To improve usability, we also develop an interactive interface to further assist material editing. We evaluate our model on both computer-generated and real-world scenes, demonstrating its superior performance. To the best of our knowledge, our model is the first to enable discrete material editing in 3D scenes.

Stand-off concealed firearm detection using motion tracking and convolutional neural networks

&lt;p&gt;The standoff detection of concealed firearms is crutial in managing public security in public spaces. Currently employed standoff concealed weapon detection techniques employ electromagnetic wave imaging which has been found to be extremely slow and may require expensive hardware and may not be applicable in public open spaces. Inorder to maintain safety in open spaces, artificial intelligence enabled video surveillance systems have been widely adopted. This poses an opportunity to explore video surveillance cameras as concealed weapon detectors. A review of existing video surveillance based automated weapon detection approaches discovered that the focus was on the detection of unconcealed firearms leaving a gap in the detection of concealed firearms. This study addresses the aforementioned gap by providing a standoff concealed firearm detection approach on video based on skeletal-based human motion tracking and convolutional neural networks. The motion of armed and unarmed persons was tracked using a depth camera and further classified using convolutional neural networks model. The developed model reported 100% accuracy, precision and recall scores. These results outperformed results obtained from traditional machine learning models therefore highlighting the superior capability of the proposed approach for concealed firearm detection on video to complement the efforts of human video surveillance operators.&lt;/p&gt;

“Darling, time horizon matters”: applying the CNN-LSTM method for predicting US equity ETFs

ABSTRACT The paper uses a hybrid model of convolutional neural network and long short-term memory (CNN-LSTM) to examine the impact of the prediction (or input) window length on the prediction accuracy of trading decisions for US equity ETFs. Results demonstrate that the prediction window length plays a vital role in the prediction accuracy of the trading decision. A hump-shaped relationship is observed between prediction accuracy and prediction window length over monthly trading days.

Classification of Brain Infarction using Deep Learning techniques on Magnetic Resonance Imaging (MRI)

Background: This research concentrated on employing Convolutional Neural Networks (CNNs) to classify brain infarctions. A stroke occurs due to a blockage or hemorrhage in the blood vessels, which disrupts or diminishes the blood supply to the brain, leading to the death of brain cells. With the progress in machine learning techniques in medical imaging, early detection of stroke has become increasingly feasible, playing a crucial role in the diagnosis and treatment of this potentially fatal condition. Objective: Building an intelligent CNN model to classify types of brain infarction using up algorithms Median Filter preserving edges and efficient minimization of noise with durability against flowing noise, Fuzzy C – Mean to Image Segmentation and Recognition of Region of Interest for infarction location and size of the infarction for brain infarction Identification, GLCM, and FOSF. Methodology: The research was carried out in a private hospital; in Baghdad, Iraq, on 60 patients with brain infarction. Included were 9 hyperacute (&gt;6 hours post brain infarction), 19 acute (7-72 hours), 12 subacute (4-7 days), and 20 chronic (&lt; 15 days). We have introduced a Convolutional Neural Network (CNN) model as a solution to predict the likelihood of a patient experiencing a stroke at an early stage, aiming for maximize effectiveness and precision. Results: The ROC curve demonstrates that the average apparent diffusion coefficient (aveg (ADC) mm²/s) and relative apparent diffusion coefficient (rADC%) are two reliable and effective measures for monitoring brain infarction developments. High sensitivity indicates a strong ability to detect true positives (cases of infarction), while the shape and position of the curve suggest that average (ADC) mm²/s also maintains a reasonable specificity, minimizing false positives. This balance results in high overall diagnostic accuracy. Discussion: Both rACD (%) and aveg (ADC) mm²/s are robust indicators for the imaging monitoring of brain infarction developments. The inclusion of AI in the assessment process may result marginal improvements in specificity and accuracy.