Appropriate Number Of Hidden Units Research Articles

Dissolved Gas Analysis of Insulating Oil for Power Transformer Fault Diagnosis with Bayesian Neural Network

Dissolved gas analysis is widely used for preventative maintenance techniques and fault diagnoses of oil-immersed power transformers. There are also various conventional methods of dissolved gas analysis for insulating oil in power transformers including methods of Doernenburg ratios, Rogers ratios and Duval’s triangle. The Bayesian techniques have been developed over many years and applied to a range of different fields including the problem of training in artificial neural networks. In particular, the Bayesian approach can solve the problem of over-fitting of artificial neural networks after being trained. The Bayesian framework can be also utilised to compare and rank different architectures and types of artificial neural networks. This research aims at deploying a detailed procedure of training artificial neural networks with the Bayesian inference, also known as Bayesian neural networks, to classify power transformer faults based on Doernenburg and Rogers gas ratios. In this research, the IEC TC 10 databases were used to form training and test data sets. The results obtained from the performance of trained Bayesian neural networks show that despite the limitation of the available dissolved gas analysis data, Bayesian neural networks with an appropriate number of hidden units can successfully classify power transformer faults with accuracy rates greater than 80%.

A Hybrid of Deep CNN and Bidirectional LSTM for Automatic Speech Recognition

Abstract Deep neural networks (DNNs) have been playing a significant role in acoustic modeling. Convolutional neural networks (CNNs) are the advanced version of DNNs that achieve 4–12% relative gain in the word error rate (WER) over DNNs. Existence of spectral variations and local correlations in speech signal makes CNNs more capable of speech recognition. Recently, it has been demonstrated that bidirectional long short-term memory (BLSTM) produces higher recognition rate in acoustic modeling because they are adequate to reinforce higher-level representations of acoustic data. Spatial and temporal properties of the speech signal are essential for high recognition rate, so the concept of combining two different networks came into mind. In this paper, a hybrid architecture of CNN-BLSTM is proposed to appropriately use these properties and to improve the continuous speech recognition task. Further, we explore different methods like weight sharing, the appropriate number of hidden units, and ideal pooling strategy for CNN to achieve a high recognition rate. Specifically, the focus is also on how many BLSTM layers are effective. This paper also attempts to overcome another shortcoming of CNN, i.e. speaker-adapted features, which are not possible to be directly modeled in CNN. Next, various non-linearities with or without dropout are analyzed for speech tasks. Experiments indicate that proposed hybrid architecture with speaker-adapted features and maxout non-linearity with dropout idea shows 5.8% and 10% relative decrease in WER over the CNN and DNN systems, respectively.

Convolutional Neural Networks for Automatic Cognitive Radio Waveform Recognition

Cognitive radio technology is an important branch in the field of wireless communication, and automatic identification is a major part of cognitive radio technology. Convolutional neural network (CNN) is an advanced neural network, which is the forefront of application in the digital image recognition area. In this paper, we explore CNN in an automatic system to recognize the cognitive radio waveforms. Excitedly, it is a more effective model with high ratio of successful recognition (RSR) under high power background noise. The system can identify eight kinds of signals, including binary phase shift keying (Barker codes modulation) linear frequency modulation, Costas codes, Frank code, and polytime codes (T1, T2, T3, and T4). The recognition part includes a CNN classifier. First, we determine the appropriate architecture to make CNN effective for proposed system. Specifically, we focus on how many convolutional layers are needed, what appropriate number of hidden units is, and what the best pooling strategy is. Second, we research how to obtain the image features into CNN that based on Choi–Williams time-frequency distribution. Finally, by means of the simulations, the results of classification are demonstrated. Simulation results show the overall RSR is 93.7% when the signal-to-noise ratio is −2dB.

Deep Convolutional Neural Networks for Large-scale Speech Tasks

Convolutional Neural Networks (CNNs) are an alternative type of neural network that can be used to reduce spectral variations and model spectral correlations which exist in signals. Since speech signals exhibit both of these properties, we hypothesize that CNNs are a more effective model for speech compared to Deep Neural Networks (DNNs). In this paper, we explore applying CNNs to large vocabulary continuous speech recognition (LVCSR) tasks. First, we determine the appropriate architecture to make CNNs effective compared to DNNs for LVCSR tasks. Specifically, we focus on how many convolutional layers are needed, what is an appropriate number of hidden units, what is the best pooling strategy. Second, investigate how to incorporate speaker-adapted features, which cannot directly be modeled by CNNs as they do not obey locality in frequency, into the CNN framework. Third, given the importance of sequence training for speech tasks, we introduce a strategy to use ReLU+dropout during Hessian-free sequence training of CNNs. Experiments on 3 LVCSR tasks indicate that a CNN with the proposed speaker-adapted and ReLU+dropout ideas allow for a 12%–14% relative improvement in WER over a strong DNN system, achieving state-of-the art results in these 3 tasks.

Weighted least squares support vector machines: robustness and sparse approximation

Least squares support vector machines (LS-SVM) is an SVM version which involves equality instead of inequality constraints and works with a least squares cost function. In this way, the solution follows from a linear Karush–Kuhn–Tucker system instead of a quadratic programming problem. However, sparseness is lost in the LS-SVM case and the estimation of the support values is only optimal in the case of a Gaussian distribution of the error variables. In this paper, we discuss a method which can overcome these two drawbacks. We show how to obtain robust estimates for regression by applying a weighted version of LS-SVM. We also discuss a sparse approximation procedure for weighted and unweighted LS-SVM. It is basically a pruning method which is able to do pruning based upon the physical meaning of the sorted support values, while pruning procedures for classical multilayer perceptrons require the computation of a Hessian matrix or its inverse. The methods of this paper are illustrated for RBF kernels and demonstrate how to obtain robust estimates with selection of an appropriate number of hidden units, in the case of outliers or non-Gaussian error distributions with heavy tails.

位置制御メカニズムの解明を目指した肘関節トルク‐角度 筋活動度関係の解析

It is well known that contracting muscles have elastic and viscous properties, which change widely with the level of muscle activation. Zeffiro reported that the torque generated by contraction of the triceps brachii muscle in a monkey increased with an increase in elbow angle, just as does an elastic spring. Recently we have investigated torque-angle relations in flexor and extensor muscles of the human elbow under static conditions. The properties of the extensor were similar to Zeffiro’s results. The properties of the flexor, however, did not show a simple elastic property; torque increased and then decreased with increasing elbow angle.The viscous property of the muscle could be explained in terms of a force-velocity relation. We estimated the torque-angular velocity relation in elbow extensor muscles; torque decreased with increasing angular velocity of extension, which was in agreement with common force-velocity relations of the muscle.There were few previous investigations of torque-angle relations of the flexor muscle in voluntary flexion movements. The relation would be a key for understanding the control mechanism of upper arm posture.The purpose of the present study was to obtain and to examine the relations with the constant muscle activation. It is almost impossible for the subject to maintain constant muscle activation during flexion movements. We have utilized a new artificial network technique to overcome this difficulty. In the present study, the torque-angle relations of the elbow flexor muscles showed very fascinating aspects; the stiffness (torque/angle) was negative at the zero velocity and positive at non-zero velocity of flexion.The experiments were performed with three normal subjects (male, aged 22-25). The task was isovelocity flexion of the elbow joint in a horizontal plane at the height of the shoulder. The subject was asked to hold the elbow joint at a fully extended position (elbow angle was almost zero) against the load torque, and then to flex the elbow joint at a constant velocity to about 120 degrees. In the holding experiments, he was asked to hold the forearm at the desired angle against the load torque. Surface electromyograms (EMGs), elbow joint angle, and torque were measured. Applied torques were approximately 0, 5, 10, and 15% of MVC (maximum voluntary contraction at an angle of about 90 degrees). The flexion velocities were 30, 60, and 90 deg/s. The measurements were repeated at least 10 times for one experimental condition. EMGs were recorded from six muscles: brachialis, caput longum bicipitis brachii, caput breve bicipitis brachii, brachioradialis, caput laterale triceps brachii, and caput longum triceps brachii. A pair of Ag-AgCl surface electrodes (10 mmφ) were put on the skin over each muscle. EMGs were full-wave rectified and then low-pass filtered with a cutoff frequency of 35 Hz. In order to obtain IEMG, this filtered signal was further running averaged over the time span during which the joint angle changed approximately 0.5 degree. Note that in the present study, steady state behaviors were examined.A three-layer artificial neural network was constructed with inputs of the elbow joint angle, flexion velocity, and six-channel IEMGs, and the output was elbow joint torque. The activation function of the input units was linear, and that of the hidden and output units was sigmoid. The appropriate number of hidden units was determined by varying both the number and the initial connection weight with back propagation learning.(View PDF for the rest of the abstract.)

Function approximation-fast-convergence neural approach based on spectral analysis

We propose a constructive approach to building single-hidden-layer neural networks for nonlinear function approximation using frequency domain analysis. We introduce a spectrum-based learning procedure that minimizes the difference between the spectrum of the training data and the spectrum of the network's estimates. The network is built up incrementally during training and automatically determines the appropriate number of hidden units. This technique achieves similar or better approximation with faster convergence times than traditional techniques such as backpropagation.

The role of hidden layers in learning motor control in autonomous systems: [formula omitted]. Mechanical Engineering Department, Massachusetts Institute of Technology, 77 Massachusetts Avenue Room 3-449, Cambridge, MA 02139

Convolutional Neural Networks (CNNs) are an alternative type of neural network that can be used to reduce spectral variations and model spectral correlations which exist in signals. Since speech signals exhibit both of these properties, we hypothesize that CNNs are a more effective model for speech compared to Deep Neural Networks (DNNs). In this paper, we explore applying CNNs to large vocabulary continuous speech recognition (LVCSR) tasks. First, we determine the appropriate architecture to make CNNs effective compared to DNNs for LVCSR tasks. Specifically, we focus on how many convolutional layers are needed, what is an appropriate number of hidden units, what is the best pooling strategy. Second, investigate how to incorporate speaker-adapted features, which cannot directly be modeled by CNNs as they do not obey locality in frequency, into the CNN framework. Third, given the importance of sequence training for speech tasks, we introduce a strategy to use ReLU+dropout during Hessian-free sequence training of CNNs. Experiments on 3 LVCSR tasks indicate that a CNN with the proposed speaker-adapted and ReLU+dropout ideas allow for a 12%–14% relative improvement in WER over a strong DNN system, achieving state-of-the art results in these 3 tasks.