Single-layer Neural Network Research Articles

Acoustic Event Detection in Vehicles: A Multi-Label Classification Approach

Autonomous driving technologies for environmental perception are mostly based on visual cues obtained from sensors like cameras, RADAR, or LiDAR. They capture the environment as if seen through “human eyes”. If this visual information is complemented with auditory information, thereby also providing “ears”, driverless cars can become more reliable and safer. In this paper, an Acoustic Event Detection model is presented that can detect various acoustic events in an automotive context along with their time of occurrence to create an audio scene description. The proposed detection methodology uses the pre-trained network Bidirectional Encoder representation from Audio Transformers (BEATs) and a single-layer neural network trained on the database of real audio recordings collected from different cars. The performance of the model is evaluated for different parameters and datasets. The segment-based results for a duration of 1 s show that the model performs well for 11 sound classes with a mean accuracy of 0.93 and F1-Score of 0.39 for a confidence threshold of 0.5. The threshold-independent metric mAP has a value of 0.77. The model also performs well for sound mixtures containing two overlapping events with mean accuracy, F1-Score, and mAP equal to 0.89, 0.42, and 0.658, respectively.

Mixed photonic/electronic neural network based on microLED arrays

Abstract We present a novel approach for the implementation of a mixed photonic/electronic neural network using gallium nitride based microLEDs as an efficient key element. The system is fully analog, with the synaptic fan-out and weighting implemented in free-space optics, while the fan-in and nonlinear activation functions are realized by optoelectronic components. Activations are represented by patterns of incoherent light generated by microLED arrays. Spatial light modulators represent the weighting matrices in the optical path, and the summation of the weighted activations is realized by photodetectors. We practically demonstrate our approach with a prototype, implementing a single-layer artificial neural network for the classification of the MNIST dataset. A test accuracy of 86.8% is achieved. A key feature of this photonic/electronic implementation is that an efficient conversion between the electrical and optical domains is realized by microLEDs, combining the advantages of the efficiency of synaptic connections through optics and nonlinear activation functions in electronics, which have the potential for miniaturization and integration. We estimate, that an upscaled version of our prototype can achieve an energy efficiency of around 14 fJ per multiply-accumulate operation using state-of-theart microLEDs. Finally we present a scalability analysis, showing that for a fully connected layer, for which a total of N multiply-accumulate operations have to be performed, in the average case the energy requirement scales with O(√N), while the energy requirement of conventional computers, as well as digital neuromorphic computing approaches, scales with O(N). Therefore, we suggest to follow this approach in order to fully exploit the potential of mixed optical/electrical neuromorphic systems and possibly demonstrate superiority over conventional GPUs.

Single-layer feedforward neural networks with dynamic width for domain adaptation

Single-layer feedforward neural networks with dynamic width for domain adaptation

An evolutionary extreme learning machine algorithm for multi-cube unit single-layer neural networks

An evolutionary extreme learning machine algorithm for multi-cube unit single-layer neural networks

Approximation of BV functions by neural networks: A regularity theory approach

In this paper, we are concerned with the approximation of functions by single hidden layer neural networks with ReLU activation functions on the unit circle. In particular, we are interested in the case when the number of data-points exceeds the number of nodes. We first study the convergence to equilibrium of the stochastic gradient flow associated with the cost function with a quadratic penalization. Specifically, we prove a Poincaré inequality for a penalized version of the cost function with explicit constants that are independent of the data and of the number of nodes. As our penalization biases the weights to be bounded, this leads us to study how well a network with bounded weights can approximate a given function of bounded variation (BV). Our main contribution concerning approximation of BV functions, is a result which we call the localization theorem. Specifically, it states that the expected error of the constrained problem, where the length of the weights are less than [Formula: see text], is of order [Formula: see text] with respect to the unconstrained problem (the global optimum). The proof is novel in this topic and is inspired by techniques from regularity theory of elliptic partial differential equations. Finally, we quantify the expected value of the global optimum by proving a quantitative version of the universal approximation theorem.

Estimating Single Layer Bi-Channel Neural Networks Architecture for Speed Control of Variable Reluctance Motors

This study explores the speed adjustment characteristics of the adaptive control of estimated single-layer Bi-channel neural networks (NNs) for Variable Reluctance Motor (VRM) drives. The innovative algorithm incorporates an estimation method to manage the nonlinear behavior of a VRM through a collection of Bi-channel NNs nodes. Each node acts as a single-layer Bi-channel NNs controller at a local linear operating point extending across the nonlinear surface of the system. Although this algorithm demonstrates strong tracking performance, a significant challenge arises from the motor speed being integrated into the machine model, which means that the controller is directly influenced by speed variations. This results in a sluggish speed adjustment response due to the learning process involved. To address this challenge, a grid of NNs must update the NNs-matrices whenever rotational speed changes. Finally, a simulation of the proposed novel speed control has been conducted to illustrate the speed adjustment behavior of the VRM under various operating conditions and to validate the effectiveness of the control approach.

Extreme Learning Techniques for Enhanced Sentiment Analysis

Extreme learning approaches are used to perform sentiment analysis on restaurant evaluations. Sluggish training and overfitting are two problems that traditional supervised learning techniques frequently face. In order to prevent overfitting and speed up training, extreme learning makes use of single hidden layer neural networks with randomly assigned weights. The goal is to create a quick and accurate model for identifying sentiment in meal reviews and to assess the differences between supervised learning techniques and models based on extreme learning. In order to map the scores to attitudes, the study uses a dataset of food reviews, where scores larger than three are interpreted as favourable and scores below that as negative. Training and testing sets are created from the dataset following preparation, which includes handling missing values and choosing pertinent columns. For modeling purposes, text data is transformed into Term Frequency-Inverse Document Frequency (TF-IDF) characteristics. The network is given randomly initialized weights and biases in both single layer and multi layer perceptron implemented. During model training, the loss function is computed, weights and biases are adjusted by backpropagation, and predictions are computed using forward propagation technique. By using a threshold of 0.5, the model's accuracy is assessed. Accuracy scores are used as a statistic for reporting training and testing accuracy. The model further validates the effectiveness for sentiment analysis in the context of food reviews by showing quicker training times and less sensitivity to overfitting. The work presents extreme learning approaches as a competitive substitute for supervised learning, which advances sentiment analysis tools. Comparing the model based on extreme learning to traditional supervised learning methods, experimental results show that the latter achieves competitive accuracy in sentiment analysis. Faster training times and less sensitivity to overfitting are further features. As a strong substitute for supervised learning techniques, this study highlights the effectiveness of extreme learning approaches for sentiment analysis in meal evaluations. Extreme learning improves the efficacy and precision of sentiment analysis models, especially in areas such as restaurant evaluations, furthering the practical uses of sentiment analysis tools.

Of Lyme disease and machine learning in a One Health world.

Lyme disease is a vector-borne emerging zoonosis in Ontario driven by human population growth and climate change. Lyme disease is also a prime example of the One Health concept. While little can be done to immediately reverse climate change and population growth, public health must resort to health communication as its best option for disease control until an effective vaccine becomes available. Disease surveillance enabling precision public health has an important role in this respect: one of the goals of disease surveillance is to forecast the future burden of disease to inform those who need to know. The goal of this study was to forecast the burden of Lyme disease using automated machine learning and statistical learning approaches. Lyme disease reports were retrieved from Ontario's integrated Public Health Information System surveillance system from January 2005 to December 2023. The reports from January 2005 to December 2021 were used as training data, and reports from January 2022 to December 2023 served as validation data. Forecasts from a seasonal autoregressive integrated moving-average model were used as a benchmark for forecasts from a feed-forward single-layer neural network machine learning algorithm. The Lyme disease burden in Ontario is predicted to increase dramatically. Neither the neural network nor the seasonal autoregressive integrated moving-average model proved to be generally more accurate. The increasing burden of human Lyme disease is concerning to public health, further indicating ecosystem changes and challenges for canine health. Human Lyme disease surveillance provides useful information to veterinarians.

Development, validation, and clinical evaluation of a machine-learning based model for diagnosing early infection after cardiovascular surgery (DEICS): a multi-center cohort study.

This study addresses the critical need for timely and accurate diagnosis of early postoperative infection (EPI) following cardiac surgery. EPI significantly impacts patient outcomes and healthcare costs, making its early detection vital. To develop, validate, and clinically implement a machine-learning-based model for diagnosing EPI post-cardiac surgery, enhancing postoperative care. In this multi-center cohort study spanning 2020 to 2022, data from four medical centers involved 2001 participants. Of these, 1400 were used for trainingand 601 for validation. Several machines-learning algorithms, including XGBoost, random forest, support vector machines, least absolute shrinkage and selection operator, and single-layer neural networks, were applied to develop predictive models. These were compared against a traditional logistic regression model. The model with the highest area under the receiver operating characteristic curve (AUROC) was deemed optimal. Implemented across four centers since 1 January 2023, a retrospective real-world study assessed its clinical applicability. Among 400 patients with an estimated EPI risk above 10%, identified by the optimal model, 55 followed its antibiotic upgrade recommendations (DEICS group). The remaining 345 patients upgraded antibiotics empirically, with 55 in the control group, matched 1:1 with the DEICS group. Clinical utility was evaluated through antibiotic use density (AUD), hospital costs, and ICU stay duration. The XGBoost model achieved the highest performance with an AUROC of 0.96 (95% CI: 0.93-0.98). The calibration curve exhibited strong agreement with Brier scores of 0.02. According to the XGBoost model, the DEICS group significantly demonstrated reduced AUD ( P < 0.01) in the matched cohort, along with decreased ICU stay time (median: 5 vs. 6days, P =0.01) and hospital costs (median: ¥150000 vs. median: ¥200000, P =0.01) in the EPI cohort. The successful implementation of the XGBoost model facilitates accurate EPI diagnosis, improves postoperative recovery, and lowers hospital costs.

Assessment of matting agent effect on polyurethane coatings' perceived blackness through geometric properties modeling: An AI approach

Assessment of matting agent effect on polyurethane coatings' perceived blackness through geometric properties modeling: An AI approach

Development of a Data‐Driven Lightning Scheme for Implementation in Global Climate Models

AbstractThis study proposes a new lightning scheme applicable at the global scale, predicting lightning rates from climatic variables. Using satellite lightning records spanning a period of 29 years, we apply machine learning methods to derive a functional relationship between lightning and climate reanalysis data. In particular, we design a tree‐based regression scheme, representing different lightning regimes with separate single hidden layer neural networks of low dimensionality. We apply multiple complexity constraints in the development stages, which makes our lightning scheme straightforward to implement within global climate models (GCMs). We demonstrate that, for years not used for training, our lightning scheme captures of the daily global spatio‐temporal lightning variability, which corresponds to a relative improvement compared to well‐established lightning schemes. Similarly, the scheme correlates well with lightning observations for the monthly climatology , inter‐annual variability , and latitudinal and longitudinal distributions . Most notably, the lightning scheme brings a critical improvement in representing lightning magnitude and variability in the three tropical lightning chimney regions: central Africa, the Amazon, and the Maritime Continent. We implement the lightning scheme in the Community Earth System Model to verify its stability and performance as a GCM component, and we provide detailed implementation guidelines. As an intermediate approach between high‐dimensional machine learning models and first‐order lightning parameterizations, our lightning scheme offers GCMs a straightforward and efficient tool to improve lightning simulation, which is critical for representing atmospheric chemistry and naturally ignited wildfires.

Interface-engineered non-volatile visible-blind photodetector for in-sensor computing

Ultraviolet (UV) detection is extensively used in a variety of applications. However, the storage and processing of information after detection require multiple components, resulting in increased energy consumption and data transmission latency. In this paper, a reconfigurable UV photodetector based on CeO2/SrTiO3 heterostructures is demonstrated with in-sensor computing capabilities achieved through interface engineering. We show that the non-volatile storage capability of the device could be significantly improved by the introduction of an oxygen reservoir. A photodetector array operated as a single-layer neural network was constructed, in which edge detection and pattern recognition were realized without the need for external memory and computing units. The location and classification of corona discharges in real-world environments were also simulated and achieved an accuracy of 100%. The approach proposed here offers promising avenues and material options for creating non-volatile smart photodetectors.

МОДЕЛИРОВАНИЕ УСПЕШНОСТИ ЗАВЕРШЕНИЯ СДЕЛОК СЛИЯНИЙ И ПОГЛОЩЕНИЙ В ЭЛЕКТРОЭНЕРГЕТИКЕ С ИСПОЛЬЗОВАНИЕМ ЛОГИСТИЧЕСКИХ РЕГРЕССИЙ

The paper considers mergers and acquisitions to be often the only possible option for implement-ing long-term development strategies due to the limited possibilities for creating new production capacity in electric power companies. The study aims to identify the factors that influence the success of mergers and acquisitions in the electric power industry. The success of a deal is crucial for decision making due to the long period of its implementation. The main hypothesis of the study is that not only the deal parameters themselves, the strategy implemented by the participants, but also the economic, social and political fac-tors of the environment influence the successful completion of deals. The authors use logistic regressions in the form of single-layer neural networks with a sigmoidal activation function as an economic and mathematical toolkit. The constructed models show that larger deals are less likely to be successful. The authors also conclude that political factors have a significant impact on the success of a deal. This reflects the modern realities of long-term decision making. The models are tested on the example of large mer-gers and acquisitions in the Russian energy industry; the results obtained allow us to conclude that the hypothesis is verified and the models are applicable to strategic planning. The models can be used in the planning and execution of M&amp;A deals both at the stage of preliminary selection of counterparties and for the purpose of choosing the best time for the deal execution.

Study of the theory of neural networks in the educational process of students of the Belarusian State University of Transport

The article presents the experience of studying the situational-heuristic method of planning and standardization in the educational process of the university. The main stages of calculations performed using heuristic methods are shown. The problems that students encounter when studying the above methods are named. It is proposed to use the mathematical apparatus of artificial neural networks to determine the duration of technological operations at railway stations. Descriptions of fragments of the software implementation of a single-layer artificial neural network are given. A similar problem can be solved when determining the duration of other technological operations: loading, unloading, processing of a train by teams of production and technical departments, etc.Recommendations are given on the use of neural network theory in the educational process of BelSUT students when performing laboratory work both by constructing a neural network and by training it based on training samples (depending on the student’s specialty); course projects, within the framework of which the tasks of determining the duration of technological operations for a given operational situation may be additionally set, as well as within the framework of diploma design. The tasks may be set of both determining the optimal sample size that guarantees a given level of accuracy in establishing the values of the duration of technological operations, and determining the accuracy for a given sample size.

Comparative Analysis of Backpropagation and Genetic Algorithms in Neural Network Training

The exploration of artificial neural networks (ANNs) has seen significant advancements, yet the optimal approach for training these networks remains a topic of debate. This study investigates the efficacy and computational efficiency of two prominent optimization techniques, backpropagation and genetic algorithms (GAs), in training traditional neural networks. The research conducted three experiments: the first involved training a single-layer neural network using a simple mathematical function; the second utilized the diabetes dataset for regression analysis; and the third applied the iris dataset for multi-class classification. The networks were trained using Google Colab, leveraging generative AI tools to expedite the experimentation process. Results indicate that backpropagation consistently achieved lower mean square error (MSE) in shorter training times compared to GAs, especially in high-dimensional data. GAs demonstrated robustness in escaping local minima, making them suitable for complex, noisy datasets where backpropagation might converge prematurely. The study concludes that while backpropagation is preferable for tasks requiring precision and speed, genetic algorithms offer valuable advantages in explorative scenarios, highlighting the importance of task-specific algorithm selection in neural network training.

PySpice-Simulated In Situ Learning with Memristor Emulation for Single-Layer Spiking Neural Networks

This paper presents a novel approach to in situ memristive learning by training spiking neural networks (SNNs) entirely within the circuit using memristor emulators in SPICE. The circuit models neurons using Lapicque neurons and employs pulse-based spike encoding to simulate spike-timing-dependent plasticity (STDP), a key learning mechanism in SNNs. The Lapicque neuron model operates according to the Leaky Integrate-and-Fire (LIF) model, which is used in this study to model spiking behavior in memristor-based SNNs. More exactly, the first memristor emulator in PySpice, a Python library for circuit simulation, was developed and integrated into a memristive circuit capable of in situ learning, named the “In Situ Memristive Learning Method for Pattern Classification.” This novel technique enables time-based computation, where neurons accumulate incoming spikes and fire once a threshold is reached, mimicking biological neuron behavior. The proposed method was rigorously tested on three diverse datasets: XPUE, a custom non-dominating 3 × 3 image dataset; a 3 × 5 digit dataset ranging from 0 to 5; and a resized 10 × 10 version of the Modified National Institute of Standards and Technology (MNIST) dataset. The neuromorphic circuit achieved successful pattern learning across all three datasets, outperforming comparable results from other in situ training simulations on SPICE. The learning process harnesses the cumulative effect of memristors, enabling the network to learn a representative pattern for each label efficiently. This advancement opens new avenues for neuromorphic computing and paves the way for developing autonomous, adaptable pattern classification neuromorphic circuits.

Diversifying Multi-Head Attention in the Transformer Model

Recent studies have shown that, due to redundancy, some heads of the Transformer model can be pruned without diminishing the efficiency of the model. In this paper, we propose a constrained optimization algorithm based on Hebbian learning, which trains specific layers in the Transformer architecture in order to enforce diversification between the different heads in the multi-head attention module. The diversification of the heads is achieved through a single-layer feed-forward neural network that is added to the Transformer architecture and is trained with the proposed algorithm. We utilize the algorithm in three different architectural variations of the baseline Transformer model. In addition to the diversification of the heads, the proposed methodology can be used to prune the heads that capture redundant information. Experiments on diverse NLP tasks, including machine translation, text summarization, question answering and large language modeling, show that our proposed approach consistently improves the performance of baseline Transformer models.

Backpropagation-based inference for spatial interpolation to estimate the blastability index in an open pit mine

Backpropagation-based inference for spatial interpolation to estimate the blastability index in an open pit mine

Investigating the Limits of Familiarity-Based Navigation.

Insect-inspired navigation strategies have the potential to unlock robotic navigation in power-constrained scenarios, as they can function effectively with limited computational resources. One such strategy, familiarity-based navigation, has successfully navigated a robot along routes of up to 60 m using a single-layer neural network trained with an Infomax learning rule. Given the small size of the network that effectively encodes the route, here we investigate the limits of this method, challenging it to navigate longer routes, investigating the relationship between performance, view acquisition rate and dimension, network size, and robustness to noise. Our goal is both to determine the parameters at which this method operates effectively and to explore the profile with which it fails, both to inform theories of insect navigation and to improve robotic deployments. We show that effective memorization of familiar views is possible for longer routes than previously attempted, but that this length decreases for reduced input view dimensions. We also show that the ideal view acquisition rate must be increased with route length for consistent performance. We further demonstrate that computational and memory savings may be made with equivalent performance by reducing the network size-an important consideration for applicability to small, lower-power robots-and investigate the profile of memory failure, demonstrating increased confusion across the route as it extends in length. In this extension to previous work, we also investigate the form taken by the network weights as training extends and the areas of the image on which visual familiarity-based navigation most relies. Additionally, we investigate the robustness of familiarity-based navigation to view variation caused by noise.

Predicting Bacterial Antibiotic Resistance using MALDI-TOF Mass Spectrometry Databases with ELM Applications.

Early detection of antibiotic resistance is a crucial task, especially for vulnerable patients under prolonged treatments with a single antibiotic. To solve this, machine learning approaches have been reported in the state of art. Researchers have used MALDI-TOF MS in order to predict antibiotic resistance and/or susceptibility in bacterial samples. Weis, et al. implemented LR, LightGBM and ANN to study the antibiotic resistance on bacterial strains of Escherichia Coli, Staphylococcus Aureus, and Klebsiella Pneumoniae. Despite promising results, the models have not achieved perfect accuracy, specifically when the classes are unbalanced. On the other hand, Extreme Learning Machine (ELM) is a training algorithm for forward propagation of single hidden layer neural networks, which converges much faster than traditional methods and offers promising performance along with less programmer intervention. In this way, this study introduced improved ELMs, including two weighted ELMs proposed by Zong, and the SMOTE technique in order to create new synthetic samples of the minority class. After heuristic optimization of ELM hiper-parameters, results demonstrated 85% in accuracy and 85% in geometric mean for the classification problem in the case of weighted ELM 1 subject to the SMOTE technique of oversampling.