Comparison Of Neural Network Architectures Research Articles

Comparison of neural network architectures for feature extraction from binary black hole merger waveforms

We evaluate several neural-network architectures, both convolutional and recurrent, for gravitational-wave time-series feature extraction by performing point parameter estimation on noisy waveforms from binary-black-hole mergers. We build datasets of 100 000 elements for each of four different waveform models (or approximants) in order to test how approximant choice affects feature extraction. Our choices include SEOBNRv4P and IMRPhenomPv3, which contain only the dominant quadrupole emission mode, alongside IMRPhenomPv3HM and NRHybSur3dq8, which also account for high-order modes. Each dataset element is injected into detector noise corresponding to the third observing run of the LIGO-Virgo-KAGRA (LVK) collaboration. We identify the temporal convolutional network architecture as the overall best performer in terms of training and validation losses and absence of overfitting to data. Comparison of results between datasets shows that the choice of waveform approximant for the creation of a dataset conditions the feature extraction ability of a trained network. Hence, care should be taken when building a dataset for the training of neural networks, as certain approximants may result in better network convergence of evaluation metrics. However, this performance does not necessarily translate to data which is more faithful to numerical relativity simulations. We also apply this network on actual signals from LVK runs, finding that its feature-extracting performance can be effective on real data.

A fair experimental comparison of neural network architectures for latent representations of multi-omics for drug response prediction

BackgroundRecent years have seen a surge of novel neural network architectures for the integration of multi-omics data for prediction. Most of the architectures include either encoders alone or encoders and decoders, i.e., autoencoders of various sorts, to transform multi-omics data into latent representations. One important parameter is the depth of integration: the point at which the latent representations are computed or merged, which can be either early, intermediate, or late. The literature on integration methods is growing steadily, however, close to nothing is known about the relative performance of these methods under fair experimental conditions and under consideration of different use cases.ResultsWe developed a comparison framework that trains and optimizes multi-omics integration methods under equal conditions. We incorporated early integration, PCA and four recently published deep learning methods: MOLI, Super.FELT, OmiEmbed, and MOMA. Further, we devised a novel method, Omics Stacking, that combines the advantages of intermediate and late integration. Experiments were conducted on a public drug response data set with multiple omics data (somatic point mutations, somatic copy number profiles and gene expression profiles) that was obtained from cell lines, patient-derived xenografts, and patient samples. Our experiments confirmed that early integration has the lowest predictive performance. Overall, architectures that integrate triplet loss achieved the best results. Statistical differences can, overall, rarely be observed, however, in terms of the average ranks of methods, Super.FELT is consistently performing best in a cross-validation setting and Omics Stacking best in an external test set setting.ConclusionsWe recommend researchers to follow fair comparison protocols, as suggested in the paper. When faced with a new data set, Super.FELT is a good option in the cross-validation setting as well as Omics Stacking in the external test set setting. Statistical significances are hardly observable, despite trends in the algorithms’ rankings. Future work on refined methods for transfer learning tailored for this domain may improve the situation for external test sets. The source code of all experiments is available under https://github.com/kramerlab/Multi-Omics_analysis

A comparison of neural network architectures for data-driven reduced-order modeling

The popularity of deep convolutional autoencoders (CAEs) has engendered new and effective reduced-order models (ROMs) for the simulation of large-scale dynamical systems. Despite this, it is still unknown whether deep CAEs provide superior performance over established linear techniques or other network-based methods in all modeling scenarios. To elucidate this, the effect of autoencoder architecture on its associated ROM is studied through the comparison of deep CAEs against two alternatives: a simple fully connected autoencoder, and a novel graph convolutional autoencoder. Through benchmark experiments, it is shown that the superior autoencoder architecture for a given ROM application is highly dependent on the size of the latent space and the structure of the snapshot data, with the proposed architecture demonstrating benefits on data with irregular connectivity when the latent space is sufficiently large.

A comparison of neural network architectures for the prediction of MRR in EDM

The aim of the research work is to predict the material removal rate of a work-piece in electrical discharge machining (EDM). Here, an effort has been made to predict the material removal rate through back-propagation neural network (BPN) and radial basis function neural network (RBFN) for a work-piece of AISI D2 steel. The input parameters for the architecture are discharge-current (Ip), pulse-duration (Ton), and duty-cycle (τ) taken for consideration to obtained the output for material removal rate of the work-piece. In the architecture, it has been observed that radial basis function neural network is comparatively faster than back-propagation neural network but logically back-propagation neural network results more real value. Therefore BPN may consider as a better process in this architecture for consistent prediction to save time and money for conducting experiments.

Comparison of neural network architectures and inputs for radar rainfall adjustment for typhoon events

This work presents a radar rainfall adjustment approach that uses two neural network architectures, support vector regression and the radial basis function neural network. The proposed approach can increase the accuracy of radar rainfall estimates that are underestimated, especially in mountainous regions. Hourly rainfall data observed at 126 raingauges in typhoon events provide the ground-truth information for adjusting radar rainfall estimates. Various inputs to the adjustment model are variable combinations of the radar rainfall, the coordinates, the elevation and the distance to the radar station. Simulation results and their intercomparison indicate that including additional topographic variables in the input vector can enhance the model performance. Validation results pertaining to three typhoon events further demonstrate that the adjustment models can reduce radar rainfall errors. Moreover, the support vector regression outperforms the radial basis function neural network in terms of radar rainfall adjustment. The spatial rainfall distribution of adjusted radar rainfall is also presented, as well as the model calibration and validation by two sets of gauges to show the generality of the method.

Performance comparison of neural network architectures for speaker-independent phoneme recognition

We applied several types of time-delay neural networks (TDNNs), generally used for speaker-dependent and multispeaker speech recognition, to speaker-independent speech recognition and compared their performance. Six or 12 speakers were used to train each network, and recognition experiments for voiced stops /b, d, g/ were performed in open speaker mode. The best recognition rates were 91.3 percent and 93.6 percent, using six and 12 training speakers, respectively. We found that constructing modular networks, such as modular TDNN with each network corresponding to a speaker, is effective in terms of decreasing the number of training iterations needed, showing slightly better performance than with a single TDNN with a comparable network capacity. This is because the modular networks make use of limited capacity effectively. On the other hand, a single TDNN with an increased number of hidden units showed a recognition rate comparable to that of the modular TDNN.