Multiple-layer Neural Network Research Articles

FORKLENS: Accurate weak-lensing shear measurement with deep learning

Context. Weak gravitational lensing is one of the most important probes of the nature of dark matter and dark energy. In order to extract cosmological information from next-generation weak lensing surveys (e.g., Euclid, Roman, LSST, and CSST) as much as possible, accurate measurements of weak lensing shear are required. Aims. There are existing algorithms to measure the weak lensing shear on imaging data, which have been successfully applied in previous surveys. In the meantime, machine learning (ML) has been widely recognized in various astrophysics applications in modeling and observations. In this work, we present a fully deep-learning-based approach to measuring weak lensing shear accurately. Methods. Our approach comprises two modules. The first one contains a convolutional neural network (CNN) with two branches for taking galaxy images and point spread function (PSF) simultaneously, and the output of this module includes the galaxy’s magnitude, size, and shape. The second module includes a multiple-layer neural network (NN) to calibrate weak-lensing shear measurements. We name the program FORKLENS and make it publicly available online. Results. Applying FORKLENS to CSST-like mock images, we achieve consistent accuracy with traditional approaches (such as moment-based measurement and forward model fitting) on the sources with high signal-to-noise ratios (S/N > 20). For the sources with S/N < 10, FORKLENS exhibits an ~36% higher Pearson coefficient on galaxy ellipticity measurements. Conclusions. After adopting galaxy weighting, the shear measurements with FORKLENS deliver accuracy levels to 0.2%. The whole procedure of FORKLENS is automated and costs about 0.7 milliseconds per galaxy, which is appropriate for adequately taking advantage of the sky coverage and depth of the upcoming weak lensing surveys.

Machine learning to predict myocardial injury and death after non-cardiac surgery.

Myocardial injury due to ischaemia within 30 days of non-cardiac surgery is prognostically relevant. We aimed to determine the discrimination, calibration, accuracy, sensitivity and specificity of single-layer and multiple-layer neural networks for myocardial injury and death within 30 postoperative days. We analysed data from 24,589 participants in the Vascular Events in Non-cardiac Surgery Patients Cohort Evaluation study. Validation was performed on a randomly selected subset of the study population. Discrimination for myocardial injury by single-layer vs. multiple-layer models generated areas (95%CI) under the receiver operating characteristic curve of: 0.70 (0.69-0.72) vs. 0.71 (0.70-0.73) with variables available before surgical referral, p < 0.001; 0.73 (0.72-0.75) vs. 0.75 (0.74-0.76) with additional variables available on admission, but before surgery, p < 0.001; and 0.76 (0.75-0.77) vs. 0.77 (0.76-0.78) with the addition of subsequent variables, p < 0.001. Discrimination for death by single-layer vs. multiple-layer models generated areas (95%CI) under the receiver operating characteristic curve of: 0.71 (0.66-0.76) vs. 0.74 (0.71-0.77) with variables available before surgical referral, p = 0.04; 0.78 (0.73-0.82) vs. 0.83 (0.79-0.86) with additional variables available on admission but before surgery, p = 0.01; and 0.87 (0.83-0.89) vs. 0.87 (0.85-0.90) with the addition of subsequent variables, p = 0.52. The accuracy of the multiple-layer model for myocardial injury and death with all variables was 70% and 89%, respectively.

Mechanical assessment of defects in welded joints: morphological classification and data augmentation

We develop a methodology for classifying defects based on their morphology and induced mechanical response. The proposed approach is fairly general and relies on morphological operators (Angulo and Meyer in 9th international symposium on mathematical morphology and its applications to signal and image processing, pp. 226-237, 2009) and spherical harmonic decomposition as a way to characterize the geometry of the pores, and on the Grassman distance evaluated on FFT-based computations (Willot in C. R., Méc. 343(3):232–245, 2015), for the predicted elastic response. We implement and detail our approach on a set of trapped gas pores observed in X-ray tomography of welded joints, that significantly alter the mechanical reliability of these materials (Lacourt et al. in Int. J. Numer. Methods Eng. 121(11):2581–2599, 2020). The space of morphological and mechanical responses is first partitioned into clusters using the “k-medoids” criterion and associated distance functions. Second, we use multiple-layer perceptron neural networks to associate a defect and corresponding morphological representation to its mechanical response. It is found that the method provides accurate mechanical predictions if the training data contains a sufficient number of defects representing each mechanical class. To do so, we supplement the original set of defects by data augmentation techniques. Artificially-generated pore shapes are obtained using the spherical harmonic decomposition and a singular value decomposition performed on the pores signed distance transform. We discuss possible applications of the present method, and how medoids and their associated mechanical response may be used to provide a natural basis for reduced-order models and hyper-reduction techniques, in which the mechanical effects of defects and structures are decorrelated (Ryckelynck et al. in C. R., Méc. 348(10–11):911–935, 2020).

A novel autonomous learning framework to enhance sEMG-based hand gesture recognition using depth information

Hand gesture recognition using surface electromyography (sEMG) has been one of the most efficient motion analysis techniques in human–computer interaction in the last few decades. In particular, multichannel sEMG techniques have achieved stable performance in hand gesture recognition. However, the general solution of collecting and labeling large data manually leads to time-consuming implementation. A novel learning method is therefore needed to facilitate efficient data collection and preprocessing. In this paper, a novel autonomous learning framework is proposed to integrate the benefits of both depth vision and EMG signals, which automatically label the class of collected EMG data using depth information. It then utilizes a multiple layer neural network (MNN) classifier to achieve real-time recognition of the hand gestures using only the sEMG. The overall framework is demonstrated in an augmented reality application by the recognition of 10 hand gestures using the Myo armband and an HTC VIVE PRO. The results show prominent performance by introducing depth information for real-time data labeling.

Evolving interval-based representation for multiple classifier fusion

Designing an ensemble of classifiers is one of the popular research topics in machine learning since it can give better results than using each constituent member. Furthermore, the performance of ensemble can be improved using selection or adaptation. In the former, the optimal set of base classifiers, meta-classifier, original features, or meta-data is selected to obtain a better ensemble than using the entire classifiers and features. In the latter, the base classifiers or combining algorithms working on the outputs of the base classifiers are made to adapt to a particular problem. The adaptation here means that the parameters of these algorithms are trained to be optimal for each problem. In this study, we propose a novel evolving combining algorithm using the adaptation approach for the ensemble systems. Instead of using numerical value when computing the representation for each class, we propose to use the interval-based representation for the class. The optimal value of the representation is found through Particle Swarm Optimization. During classification, a test instance is assigned to the class with the interval-based representation that is closest to the base classifiers’ prediction. Experiments conducted on a number of popular dataset confirmed that the proposed method is better than the well-known ensemble systems using Decision Template and Sum Rule as combiner, L2-loss Linear Support Vector Machine, Multiple Layer Neural Network, and the ensemble selection methods based on GA-Meta-data, META-DES, and ACO.

Voltage Sag Estimation in Sparsely Monitored Power Systems Based on Deep Learning and System Area Mapping

This paper proposes a voltage sag estimation approach based on a deep convolutional neural network. The proposed approach estimates the sag magnitude at unmonitored buses regardless of the system operating conditions and fault location and characteristics. The concept of system area mapping is also introduced via the use of bus matrix, which maps different patches in input matrix to various areas in the power system network. In this way, relevant features are extracted at various local areas in the power system and used in the analysis for higher level feature extraction, before feeding into a fully-connected multiple layer neural network for sag classification. The approach has been tested on the IEEE 68-bus test network and it has been demonstrated that the various sag categories can be identified accurately regardless of the operating condition under which the sags occur.

Efficient and self-adaptive in-situ learning in multilayer memristor neural networks

Memristors with tunable resistance states are emerging building blocks of artificial neural networks. However, in situ learning on a large-scale multiple-layer memristor network has yet to be demonstrated because of challenges in device property engineering and circuit integration. Here we monolithically integrate hafnium oxide-based memristors with a foundry-made transistor array into a multiple-layer neural network. We experimentally demonstrate in situ learning capability and achieve competitive classification accuracy on a standard machine learning dataset, which further confirms that the training algorithm allows the network to adapt to hardware imperfections. Our simulation using the experimental parameters suggests that a larger network would further increase the classification accuracy. The memristor neural network is a promising hardware platform for artificial intelligence with high speed-energy efficiency.

Application of deep autoencoder model for structural condition monitoring

Damage detection in structures is performed via vibration based structural identification. Modal information, such as frequencies and mode shapes, are widely used for structural damage detection to indicate the health conditions of civil structures. The deep learning algorithm that works on a multiple layer neural network model termed as deep autoencoder is proposed to learn the relationship between the modal information and structural stiffness parameters. This is achieved via dimension reduction of the modal information feature and a non-linear regression against the structural stiffness parameters. Numerical tests on a symmetrical steel frame model are conducted to generate the data for the training and validation, and to demonstrate the efficiency of the proposed approach for vibration based structural damage detection.

Hybrid privacy-preserving clinical decision support system in fog–cloud computing

In this paper, we propose a framework for hybrid privacy-preserving clinical decision support system in fog–cloud computing, called HPCS. In HPCS, a fog server uses a lightweight data mining method to securely monitor patients’ health condition in real-time. The newly detected abnormal symptoms can be further sent to the cloud server for high-accuracy prediction in a privacy-preserving way. Specifically, for the fog servers, we design a new secure outsourced inner-product protocol for achieving secure lightweight single-layer neural network. Also, a privacy-preserving piecewise polynomial calculation protocol allows cloud server to securely perform any activation functions in multiple-layer neural network. Moreover, to solve the computation overflow problem, a new protocol called privacy-preserving fraction approximation protocol is designed. We then prove that the HPCS achieves the goal of patient health status monitoring without privacy leakage to unauthorized parties by balancing real-time and high-accurate prediction using simulations.

A novel logical neuron model

The problem of representing and processing logical knowledge using a neural network is discussed. A novel neuron model, a triple-valued logical neuron (TLN) is presented. There are two kinds of TLNs, triple-valued “logical and” neuron (TLN-AND) and triple-valued “logical or” neuron (TLN-OR). A multiple-layer neural network (TLNN) made up of TLNs can implement a triple-valued logical inference system. TLNN can thus form a base for representing logical knowledge using neural networks. ©1999 John Wiley & Sons, Inc.

TMLNN: triple-valued or multiple-valued logic neural network

We discuss the problem of representing and processing triple-valued or multiple-valued logic knowledge using neural network in this paper. A novel neuron model, triple-valued or multiple-valued logic neuron (TMLN), is presented. Each triple-valued or multiple-valued logic neuron can represent a triple-valued or multiple-valued logic rule by itself. We will show that there are two triple-valued or multiple-valued logic neurons: TMLN-AND (triple-valued or multiple-valued "logic and" neuron) and TMLN-OR (triple-valued or multiple-valued "logic or" neuron). TMLN-AND can realize triple-valued or multiple-valued "logic and" while TMLN-OR can realize triple-valued or multiple-valued "logic or." Two simplified triple-valued or multiple-valued logic neuron models are also presented. We can show that a multiple-layer neural network (TMLNN) made up of triple-valued or multiple-valued logic neurons can implement a triple-valued or multiple-valued logic inference system. The training algorithm for TMLNN is presented and can be shown to converge. In our model, triple-valued or multiple-valued logic rules can be extracted from TMLNN with ease. TMLNN can thus form a base for representing logic knowledge using neural network.

Performance Improvement of the Fourier Transform Method in Particle Image Velocimetry Using a Neural Network.

In this paper, we present a new Fourier transform method with improved PIV measurement accuracy and processing time. A multiple-layer neural network is used in order to suppress obviously incorrect velocity vectors and to reduce the processing time. The information input to the network is the complex phase extracted from the Fourier transforms of two images obtained in a short time interval, and the output is the mean spatial shift of the image pattern in a sampling window. The learning of the correlation between the input and the output is performed using an error back propagation method, and performance tests are carried out using numerically simulated particle images and images obtained experimentally. The test results show that the present method is more robust against velocity fluctuation than the original one and the CPU time is reduced to about 75% of that of the original method.

マシンビジョンによる果菜の等級判定に関する研究（第１報） 階層型ニューラルネットワークによる形状判定のソフトウェアと選別システム

1. 本研究は果菜の等級選別のために, 機能性に優れ, 複数品目の果菜にも対応可能な汎用的選別システムを開発するもので, 第一段階としてマシンビジョンによる画像処理技術と階層型ニューラルネットワーク理論を導入した果菜の形状判定に関する基本的研究を行った.2. 果菜の形状判定は定形である工業製品の形状判定に比べて, あいまいさがみられることから困難である.よって, マシンビジョンによって適切な判定をするには果菜の形状特徴の表現法が重要な要素となる.3. 本研究では果菜の形状特徴は太さ, 長さ, 曲りの三要素で表現することを提案し, その選別評価として, 階層型ニューラルネットワーク理論を使った選別システムソフトウェアと選別システムを構築した.4. 研究の第一歩として, 果菜の太さ, 長さおよび曲りを表現できる階層型ニューラルネットワークによる果菜の形状特徴抽出法と特徴量を導入した.5. 作成したソフトウェアを検証するために, 学習と判定のシミュレーションを行い, 学習が成功したことを確認した.これにより, 果菜の等級選別システムの自動化の可能性を認めた

128 ニューラルネットによるフーリエ変換法の性能改善

In this paper, a multiple-layer neural network model has been applied to the Fourier transform method in 2-D Particle Image Velocimetry to improve the measurement accuracy. The input information of the neural network is the complex phase that is extracted from the Fourier transforms of two images captured in a short time interval, and the output is the spatial shift of the pattern on the images. The learning is performed by a conventional error back propagation method. The performance test shows that the present method is robust against velocity fluctuation and the computing time can be reduced to about 75% of that of the original Fourier transform method.

Making a mobile robot learn to determine its location using neural networks

The extraction of a meaningful relationship between a robot and its environment from sensed data is a key component of all intelligent robot task. In many instances, the relationship between the sensed data and the desired output from the robot is so highly complex that it becomes very difficult to specify algorithms for extracting the desired relationships. In such situations, the use of a nonalgorithmic approach through the procedural learning exhibited by the feedforward multiple-layer neural networks appears very attractive. Motivated by this, we present in this paper the details of applying such networks to the task of robot localization. The robot localization scheme presented here uses an ultrasonic ring sensor to capture the environment and is meant for indoor applications. Experimental results simulated and real sensor data are presented to show the usefulness of the proposed scheme.

206 ニューラルネットによる学習型流速場測定法

This paper provides a new Particle Image Velocimetry (PIV) with learning function by Neural Network. This method is based on the Fourier transform method and detects a special shift by using a multiple-layer Neural Network, on purpose to improve the precision of measurement. The network learns the input-output patterns in one-dimensional problem by Error Back Propagation method.Performance test is carried out in two-dimensional problem and the test results show that the present method induces less errors for the image patterns with velocity fluctuation than the original Fourier transform method.

Neural Networks for Selective Vehicle Routing Heuristics

In the vehicle routing problem (VRP), there is a collection of stops with known demands for service, and a fleet of vehicles with known capacities available to serve the stops. The problem is to assign the stops to vehicles, and specify routing sequences for each vehicle so that total distance is minimized. Many heuristic algorithms for the VRP have been developed over the last 25 years. Given a particular vehicle routing problem instance, we address here the issue of selecting which heuristic algorithm to apply to the problem. A modular system of neural networks is the knowledge base that is used to make the selection. Three types of multiple-layer neural networks are involved, and all are trained with the back-propagation learning paradigm. After training, the neural network system was applied to a collection of new problems not used in training. The system was very successful in identifying the optimal algorithm to apply to the new problems. We conclude that neural networks have considerable promise for use in vehicle routing model management systems. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.