Input To Network Research Articles

Is Football Unpredictable? Predicting Matches Using Neural Networks

The growing sports betting market works on the premise that sports are unpredictable, making it more likely to be wrong than right, as the user has to choose between win, draw, or lose. So could football, the world’s most popular sport, be predictable? This article studies this question using deep neural networks to predict the outcome of football matches using publicly available data. Data from 24,760 matches from 13 leagues over 2 to 10 years were used as input for the neural network and to generate a state-of-the-art validated feature, the pi-rating, and the parameters proposed in this work, such as relative attack, defence, and mid power. The data were pre-processed to improve the network’s interpretation and deal with missing or inconsistent data. With the validated pi-rating, data organisation methods were evaluated to find the most fitting option for this prediction system. The final network has four layers with 100, 80, 5, and 3 neurons, respectively, applying the dropout technique to reduce overfitting errors. The results showed that the most influential features are the proposed relative defending, playmaking, and midfield power, and the home team goal expectancy features, surpassing the pi-rating. Finally, the proposed model obtained an accuracy of 52.8% in 2589 matches, reaching 80.3% in specific situations. These results prove that football can be predictable and that some leagues are more predictable than others.

State-of-charge estimation based on improved back-propagation neural network for lithium-ion batteries in energy storage power stations

This paper develops a novel approach for the state of charge (SOC) estimation of Lithium-ion batteries in energy storage power stations, leveraging an improved back-propagation (BP) neural network optimized by an immune genetic algorithm (IGA). Addressing the paramount importance of accurate SOC estimation for enhancing battery management systems, this work proposes a methodological enhancement aimed at refining estimation precision and operational efficiency. First, the mechanisms of temperature, current, and voltage impacts on SOC are revealed, which serve as the inputs of the neural network. Second, the improved BP neural network’s structure and optimization through an IGA are designed, emphasizing the mitigation of traditional BP neural networks’ limitations including slow convergence speed and complex parameterization. Through an extensive experimental setup, the proposed model is validated against standard BP neural networks across various discharge experiments at different temperatures and discharge currents. Results prove that the estimation accuracy of the proposed method reaches as high as 98.15% and faster converges compared to the traditional BP network, thereby being valuable practically.

Comparison of radiated noise classification methods for underwater targets based on different enhanced images and convolutional neural networks

With the continuous development of economy and society, factors such as the variety of underwater targets and the high level of environmental noise have a great impact on the classification accuracy of underwater target radiation noise, and the traditional classification method based on signal features can no longer meet the requirements of underwater target identification. In this paper, we propose an underwater target radiation noise classification method based on enhanced image and convolutional neural network. First, the underwater target radiation noise signal is converted into enhanced image by various methods, then the converted image data set is used as the input of convolutional neural network for model training, and finally the great advantage of convolutional neural network in image classification is used to accurately classify underwater target radiation noise. In order to propose an optimal augmented image transformation method, this paper uses several augmented image transformation methods and compares the classification results. The experimental results show that the augmented image and convolutional neural network methods based on lagomorphs and corner fields have the highest classification accuracy and the best classification efficiency.

Network algebraization and port relationship for power‐electronic‐dominated power systems

AbstractIn the classical differential‐algebraic equations (DAEs) framework for the traditional power system stability analysis, synchronous generators are depicted by differential equations and network by algebraic equations under the quasi‐steady‐state assumption. Differently, in the power‐electronic‐dominated power system (PEDPS), the dynamics of transmission lines of network for fully differential equations should be considered, due to the rapid response of converters' controls, for example, the alternating current controls. This poses a great challenge for the cognition, modeling, and analysis of the PEDPS. In this article, a nonlinear DAE model framework for the PEDPS is established with differential equations for the source nodes and algebraic equations for the dynamical electrical network, by generalizing the application scenarios of Kron reduction. The internal and terminal voltages of source nodes of converters are chosen as ports of nodes and network. Namely, the internal and terminal voltages of source nodes work as their output and input, respectively, whereas they work as the input and output of the algebraic network, respectively. The impact of dynamical network becomes clear, namely, it serves as a (linear) voltage divider and generates the terminal voltage based on the internal voltage of the sources simultaneously. By keeping only useful independent state variables, all differential equations for the transmission lines can be transferred to algebraic equations. With this simple model, the roles of both nodes and network become apparent, and it enhances the understanding of the PEDPS dynamics. On the other hand, broad simulations are conducted and compared to verify the proposed DAE framework for the PEDPS. As all independent variables have been kept in the model, it is found that they show the same computational accuracy, but better efficiency in computational time, compared to the electromagnetic‐transient simulation results.

A Fast Adaptive AUV Control Policy Based on Progressive Networks with Context Information

Deep reinforcement learning models have the advantage of being able to control nonlinear systems in an end-to-end manner. However, reinforcement learning controllers trained in simulation environments often perform poorly with real robots and are unable to cope with situations where the dynamics of the controlled object change. In this paper, we propose a DRL control algorithm that combines progressive networks and context as a depth tracking controller for AUVs. Firstly, an embedding network that maps interaction history sequence data onto latent variables is connected to the input of the policy network, and the context generated by the network gives the DRL agent the ability to adapt to the environment online. Then, the model can be rapidly adapted to a new dynamic environment, which was represented by the presence of generalized force disturbances and changes in the mass of the AUV, through a two-stage training mechanism based on progressive neural networks. The results showed that the proposed algorithm was able to improve the robustness of the controller to environmental disturbances and achieve fast adaptation when there were differences in the dynamics.

Класифікація електрокардіограм як динамічного процесу на основі спайкінгової нейронної мережі

The article analyzes the ECG classifying methods. A method of dynamic ECG classification using spiking neural networks is proposed. Dynamic parameters are selected, which are a representation of the ECG signal in a time series. These parameters are fed into the input of a spiking neural network, which outputs both a single heartbeat and a full ECG study. The developed spiking neural network has fast learning and uses large amounts of data for training.

Minimally interactive segmentation of soft-tissue tumors on CT and MRI using deep learning.

Segmentations are crucial in medical imaging for morphological, volumetric, and radiomics biomarkers. Manual segmentation is accurate but not feasible in clinical workflow, while automatic segmentation generally performs sub-par. To develop a minimally interactive deep learning-based segmentation method for soft-tissue tumors (STTs) on CT and MRI. The interactive method requires the user to click six points near the tumor's extreme boundaries in the image. These six points are transformed into a distance map and serve, with the image, as input for a convolutional neural network. A multi-center public dataset with 514 patients and nine STT phenotypes in seven anatomical locations, with CT or T1-weighted MRI, was used for training and internal validation. For external validation, another public dataset was employed, which included five unseen STT phenotypes in extremities on CT, T1-weighted MRI, and T2-weighted fat-saturated (FS) MRI. Internal validation resulted in a dice similarity coefficient (DSC) of 0.85 ± 0.11 (mean ± standard deviation) for CT and 0.84 ± 0.12 for T1-weighted MRI. External validation resulted in DSCs of 0.81 ± 0.08 for CT, 0.84 ± 0.09 for T1-weighted MRI, and 0.88 ± 0.08 for T2-weighted FS MRI. Volumetric measurements showed consistent replication with low error internally (volume: 1 ± 28 mm3, r = 0.99; diameter: - 6 ± 14 mm, r = 0.90) and externally (volume: - 7 ± 23 mm3, r = 0.96; diameter: - 3 ± 6 mm, r = 0.99). Interactive segmentation time was considerably shorter (CT: 364 s, T1-weighted MRI: 258s) than manual segmentation (CT: 1639s, T1-weighted MRI: 1895s). The minimally interactive segmentation method effectively segments STT phenotypes on CT and MRI, with robust generalization to unseen phenotypes and imaging modalities. Question Can this deep learning-based method segment soft-tissue tumors faster than can be done manually and more accurately than other automatic methods? Findings The minimally interactive segmentation method achieved accurate segmentation results in internal and external validation, and generalized well across soft-tissue tumor phenotypes and imaging modalities. Clinical relevance This minimally interactive deep learning-based segmentation method could reduce the burden of manual segmentation, facilitate the integration of imaging-based biomarkers (e.g., radiomics) into clinical practice, and provide a fast, semi-automatic solution for volume and diameter measurements (e.g., RECIST).

The Combination of Topological Data Analysis and Mathematical Modeling Improves Sleep Stage Prediction From Consumer-Grade Wearables.

Wearable devices have become commonplace tools for tracking behavioral and physiological parameters in real-world settings. Nonetheless, the practical utility of these data for clinical and research applications, such as sleep analysis, is hindered by their noisy, large-scale, and multidimensional characteristics. Here, we develop a neural network algorithm that predicts sleep stages by tracking topological features (TFs) of wearable data and model-driven clock proxies (CPs) reflecting the circadian propensity for sleep. To evaluate its accuracy, we apply it to motion and heart rate data from the Apple Watch worn by young subjects undergoing polysomnography (PSG) and compare the predicted sleep stages with the corresponding ground truth PSG records. The neural network that includes TFs and CPs along with raw wearable data as inputs shows improved performance in classifying Wake/REM/NREM sleep. For example, it shows significant improvements in identifying REM and NREM sleep (AUROC/AUPRC improvements >13% and REM/NREM accuracy improvement of 12%) compared with the neural network using only raw data inputs. We find that this improvement is mainly attributed to the heart rate TFs. To further validate our algorithm on a different population, we test it on elderly subjects from the Multi-ethnic Study of Atherosclerosis cohort. This confirms that TFs and CPs contribute to the improvements in Wake/REM/NREM classification. We next compare the performance of our algorithm with previous state-of-the-art wearable-based sleep scoring algorithms and find that our algorithm outperforms them within and across different populations. This study demonstrates the benefits of combining topological data analysis and mathematical modeling to extract hidden inputs of neural networks from puzzling wearable data.

Deep Learning Assisted Diagnosis of Chronic Obstructive Pulmonary Disease Based on a Local-to-Global Framework

To aid the diagnosis of chronic obstructive pulmonary disease (COPD), a local-to-global deep framework with group attentions and slice-aware loss is designed in this paper, which utilizes the chest CT sequences of the patients as the network input. To fully mine the medical hints submerged in the CT slices, two types of group attentions are designed to extract local–global features of the grouped slices. Specifically, in each group, a group local attention block (GLAB) and a group global attention block (GGAB) are designed to extract local features in the CT slices and long-range dependencies among the grouped slices. To alleviate the influence of different numbers of CT slices in the chest CT sequences for different patients, a slice-aware loss is proposed by incorporating a normalized coefficient into the cross-entropy loss. Experimental results indicate that the designed deep model performs a good COPD identification on a real COPD dataset with 96.08% accuracy, 94.12% sensitivity, 97.06% specificity, and 95.32% AUC, which is superior to some existing deep learning methods.

From Envelope Spectra to Bearing Remaining Useful Life: An Intelligent Vibration-Based Prediction Model with Quantified Uncertainty.

Bearings are pivotal components of rotating machines where any defects could propagate and trigger systematic failures. Once faults are detected, accurately predicting remaining useful life (RUL) is essential for optimizing predictive maintenance. Although data-driven methods demonstrate promising performance in direct RUL prediction, their robustness and practicability need further improvement regarding physical interpretation and uncertainty quantification. This work leverages variational neural networks to model bearing degradation behind envelope spectra. A convolutional variational autoencoder for regression (CVAER) is developed to probabilistically predict RUL distributions with confidence measures. Enhanced average envelope spectra (AES) are used as network input for its physical robustness in bearing condition assessment and fault detection. The use of the envelope spectrum ensures that it contains only bearing-related information by removing other rotor-related frequencies, hence it improves the RUL prediction. Unlike traditional variational autoencoders, the probabilistic regressor and latent generator are formulated to quantify uncertainty in RUL estimates and learn meaningful latent representations conditioned on specific RUL. Experimental validations are conducted on vibration data collected using multiple accelerometers whose natural frequencies cover bearing resonance ranges to ensure fault detection reliability. Beyond conventional bearing diagnosis, envelope spectra are extended for statistical RUL prediction integrating physical knowledge of actual defect conditions. Comparative and ablation studies are conducted against benchmark models to demonstrate their effectiveness.

Method of Motion Path Planning Based on a Deep Neural Network with Vector Input

The article deals with the problem of path planning in a two-dimensional environment based on deep learning neural networks. Deep neural networks require large amounts of data and place high computational requirements on computing tools. The lack of sufficient data leads to a decrease in the accuracy of the neural network, and high computational requirements at the learning stage limit the use of this technology in engineering practice. In this paper, the forms of representation of the environment for the input of a neural network are studied. Vector form allows to reduce the amount of information supplied to the input of a neural network, but it leads to the need to use more complex neural networks. In this article, a combined form of representation is proposed, including a vector global and local map layout. The vector part of the map includes the position of the robot, the position of the target point and a description of obstacles. The local raster map describes the area closest to the robot. Using numerical research, the effectiveness of this form of data representation for a precise neural network is shown, compared with the raster representation of the map. In this article, two structures of neural networks are studied, one of which uses 8 possible directions of movement, and the other uses 3 possible directions of movement. It is shown that when using 3 possible directions, the cycling of trajectories planned by the neural network is eliminated, which leads to an increase in accuracy.

Enhanced Short-Term Temperature Prediction of Seasonally Frozen Soil Subgrades Using the NARX Neural Network

Accurate prediction of subgrade temperatures in seasonally frozen regions is crucial for understanding thermal states, frost heave phenomena, stability, and other critical characteristics. This study employs a nonlinear autoregressive with exogenous input (NARX) network to predict short-term subgrade temperatures in the Golmud-Nagqu section of China’s National Highway 109. The methodology involves preprocessing subgrade monitoring data, including temperature, water content, and frost heave, followed by developing a temperature prediction model. This tailored NARX neural network, compared to the traditional BP neural network, integrates feedback and delay mechanisms for monitoring data, offering superior memory and dynamic response capabilities. The precision of the NARX model is assessed with the backpropagation (BP) network, indicating that the NARX neural network significantly outperforms the BP model in both precision and stability for temperature prediction in seasonally frozen subgrades. These findings suggest that the NARX model is a valuable tool for accurately predicting subgrade temperatures in seasonally frozen regions, offering significant insights for practical engineering applications.

Ambisonics neural speech extraction with directional feature and rotary steering

In scenes with noise and overlapping speakers, directionally extracting audio tracks corresponding to individual speakers is crucial for immersive and interactive spatial audio systems. Although neural networks have been successful in this task, existing steering approaches for adjusting the direction of neural speech extraction mainly target spatial audio directly collected by microphone arrays, while directional speech extraction with Ambisonics spatial audio is less well studied. Therefore, to encode the target directional information as input for the neural network, this paper proposes two Ambisonics directional features based on the spatial feature difference and beamforming principle: the relative harmonic difference and the directional signal enhancement ratio. Using the special property of Ambisonics' rotation transform, a rotary steering pre-processing is also proposed to align the target speaker's direction with a fixed reference by inversely rotating the sound field, thereby simplifying multi-directional extraction to fixed-directional extraction. Finally, we integrate these proposed approaches with the existing temporal-spectral-spatial filtering neural networks to establish a generalized framework for steerable speech extraction and conduct experiments on a simulated Ambisonics dataset containing multiple speakers and noise sources. The experiments show that the proposed approaches outperform existing conditional steering and can be applied to various existing neural network architectures.

Quasi-visualizable detection of deep sub-wavelength defects in patterned wafers by breaking the optical form birefringence

Abstract In integrated circuit (IC) manufacturing, fast, nondestructive, and precise detection of defects in patterned wafers, realized by bright-field microscopy, is one of the critical factors for ensuring the final performance and yields of chips. With the critical dimensions of IC nanostructures continuing to shrink, directly imaging or classifying deep-subwavelength defects by bright-field microscopy is challenging due to the well-known diffraction barrier, the weak scattering effect, and the faint correlation between the scattering cross-section and the defect morphology. Herein, we propose an optical far-field inspection method based on the form-birefringence scattering imaging of the defective nanostructure, which can identify and classify various defects without requiring optical super-resolution. The technique is built upon the principle of breaking the optical form birefringence of the original periodic nanostructures by the defect perturbation under the anisotropic illumination modes, such as the orthogonally polarized plane waves, then combined with the high-order difference of far-field images. We validated the feasibility and effectiveness of the proposed method in detecting deep subwavelength defects through rigid vector imaging modeling and optical detection experiments of various defective nanostructures based on polarization microscopy. On this basis, an intelligent classification algorithm for typical patterned defects based on a dual-channel AlexNet neural network has been proposed, stabilizing the classification accuracy of λ/16-sized defects with highly similar features at more than 90%. The strong classification capability of the two-channel network on typical patterned defects can be attributed to the high-order difference image and its transverse gradient being used as the network’s input, which highlights the polarization modulation difference between different patterned defects more significantly than conventional bright-field microscopy results. This work will provide a new but easy-to-operate method for detecting and classifying deep-subwavelength defects in patterned wafers or photomasks, which thus endows current online inspection equipment with more missions in advanced IC manufacturing.

Motor Bearing Failure Identification Using Multiple Long Short-Term Memory Training Strategies

In the context of condition-based maintenance of rotating machines in manufacturing systems, the early diagnosis of possible faults related to rolling elements of the bearing is mainly based on techniques from artificial intelligence, namely, Machine Learning (ML) and Deep Learning (DL). Approaches based on using Deep Learning methods have been the most coveted in recent years. Among a variety of models, the type of architecture known as Long-Short-Term Memory (LSTM) of Recurrent Neural Network (RNN) has both the ability to capture long-term dependencies and to adapt to sequential data modeling. It is therefore able to work on data without any preprocessing. This paper studies using four types of LSTM networks to diagnose bearing faults in a classification approach. It aims to intervene on both the input parameters and the network architecture, to achieve high performance. The proposed method is carried out in two different ways. In the first case, the data inputs are raw frames of vibration signals. However, in the second case, the network inputs are pre-computed time-frequency features. The results clearly showed that LSTMs are more accurate with the latter.

Cuff-less wearable biosensor in continuous noninvasive human radial artery pulse waveform and blood pressure measurement using self-mixing interferometry

In this paper, we propose a compact, wearable biosensor for the noninvasive measurement of human radial artery pulse waveform curve (PWC) and blood pressure (BP). In this system, self-mixing interferometry (SMI) technology is employed to measure the weak arterial vascular deformation, enabling accurate PWC retrieval. Based on the reconstructed PWC features, BP values are precisely estimated by means of deep learning method. Here continuous wavelet transform (CWT), enabling visualization of the relationship between the SMI signal temporal frequency components and the PWC characteristics, is highlighted for PWC flipping points seeking and convolutional neural network (CNN) input parameter acquisition. For the first time, a novel deep learning network preprocessing method is proposed that allows direct feature extraction from the CWT scalogram of SMI signal without the complicated PWC reconstruction algorithm. The robustness and accuracy of our device are validated by a series of clinical measurements, mean absolute error (MAE) and standard deviation (STD) values are calculated and compared with the existing models. We approach minimal BP estimation results (MAE ± STD) of 1.41 ± 1.89 mmHg for systolic blood pressure (SBP) and 1.78 ± 2.01 mmHg for diastolic blood pressure (DBP), respectively. The luxuriant novelties and remarkable performance clearly demonstrate our wearable sensor’s great potential in BP monitoring, and other clinical applications.

Multi-view graph contrastive representation learning for bundle recommendation

Bundle recommendation can recommend a collection of associated items that can be consumed together to a user rather than recommending these items separately, making it extremely suitable for some scenarios such as product bundle recommendation and game bundle recommendation. Recent bundle recommendation approaches consider auxiliary data to mitigate sparse user-bundle interactions. However, these approaches obtain the node embeddings directly from the established user-bundle graph and do not explicitly exploit the relationships between users (bundles) when constructing recommendation models. Moreover, bundle recommendation approaches based on graph contrastive learning usually construct contrastive views by randomly discarding nodes (edges) in the graph, while discarding some essential nodes or edges will destroy the structure of the original graph, thereby deteriorating the quality of the learned node embeddings. Aiming at these limitations, we propose a bundle recommendation approach based on multi-view graph contrastive representation learning. First, we present a multi-view modeling method to model the relations between entities as several views from different perspectives. These views serve as inputs of graph neural networks for graph representation learning and provide contrastive views for the contrastive learning tasks. Second, we propose a novel framework for bundle recommendation. This framework obtains the user (bundle) embeddings from different views by performing multi-view graph representation learning and enhances the learned user and bundle embeddings through a two-level contrastive learning strategy. On this basis, the enhanced user (bundle) embeddings are fused for prediction. Finally, we design a joint optimization objective to optimize the model parameters, combining the prediction loss that supports multiple negative samples and the contrastive losses. Experiments on the Netease and Youshu datasets reveal that our approach outperforms the state-of-the-art (SOTA) baselines. Furthermore, the average improvements of Recall@K and NDCG@K of our approach over the SOTA baselines are approximately 3.38% and 2.80% on Netease and 3.94% and 4.84% on Youshu.

Spectral-Spatial Transformer for Hyperspectral Image Sharpening.

Convolutional neural networks (CNNs) have recently achieved outstanding performance for hyperspectral (HS) and multispectral (MS) image fusion. However, CNNs cannot explore the long-range dependence for HS and MS image fusion because of their local receptive fields. To overcome this limitation, a transformer is proposed to leverage the long-range dependence from the network inputs. Because of the ability of long-range modeling, the transformer overcomes the sole CNN on many tasks, whereas its use for HS and MS image fusion is still unexplored. In this article, we propose a spectral-spatial transformer (SST) to show the potentiality of transformers for HS and MS image fusion. We devise first two branches to extract spectral and spatial features in the HS and MS images by SST blocks, which can explore the spectral and spatial long-range dependence, respectively. Afterward, spectral and spatial features are fused feeding the result back to spectral and spatial branches for information interaction. Finally, the high-resolution (HR) HS image is reconstructed by dense links from all the fused features to make full use of them. The experimental analysis demonstrates the high performance of the proposed approach compared with some state-of-the-art (SOTA) methods.

An active two-stage class-J power amplifier design for smart grid’s 5G wireless networks

<span>The wireless communication networks in the smart grid’s advanced metering infrastructure (AMI) applications need 5G technology to support large data transmission efficiently. As the 5G wireless communication network’s overall bandwidth (BW) and efficiency depend on its power amplifier (PA), in this work, a two-stage class-J power amplifier’s design methodology that operates at 3.5 GHz centre frequency by utilizing the CGH40010F model gallium nitride (GaN) transistor is presented. The proposed design methodology involves proper designing of input, output, and interstage matching networks to achieve class-J operation with improved power gain over desired BW using the advanced design system (ADS) electronic design automation (EDA) tool and estimating its integration feasibility through active element-based design approach using the Mentor Graphics EDA tool. The proposed PA provides 54% drain efficiency (D.E), 53% power added efficiency (PAE) with a small signal gain of 27 dB at 3.5 GHz and 41 dBm power output with 21 dB of improved power gain across a BW of around 400 MHz using 28 V power supply into 50 Ω load. By replacing the two-stage PA's passive elements with active elements, its layout size is estimated to be (15.5×29.2) μm2 . The results of the proposed PA exhibit its integration feasibility and suitability for the smart grid’s 5G wireless networks.</span>

Automated Hematoma Detection and Outcome Prediction in Patients With Traumatic Brain Injury.

To develop a tool for automated subtype classification and segmentation of intracranial hemorrhages (ICH) on CT scans of patients with traumatic brain injury (TBI). Furthermore, outcome prediction for patients can effectively facilitate patient management. This study presents a cascade framework for two-stage segmentation and multi-label classification. The hematoma region of interest (ROI) is localized, and then the ROI is cropped and resized to the original pixel size before being input into the model again to obtain the segmentation results. In multilabel classification, the mask obtained from automatic segmentation is superimposed onto the corresponding ROI and CT slices, respectively, to constitute the input image. Subsequently, the ROI image is employed as the local network input to obtain local features. Third, the CT image is utilized to construct a feature extraction network to obtain global features. Ultimately, the local and global features are fused dimensions in the pooling layer,and calculated to generate the final retrieval results. For the prediction of 14-day in-hospital mortality, automatically extracted hematoma subtype and volume features were integrated to enhance the widely used CRASH model. The proposed segmentation method achieves the best estimates on the Dice similarity coefficient and Jaccard Similarity Index. The proposed multilabel classification method achieved an average accuracy of 95.91%. For mortality prediction, the best model achieved an average area under the receiver operating characteristic curve (AUC) of 0.91 by 5-fold cross-validation. The proposed method enhances the precision of hematoma segmentation and subtype classification. In clinical settings, the method can streamline the evaluation of ICH for radiologists, and the automatically extracted features are anticipated to facilitate prognosis assessment.