Use Of Recurrent Networks Research Articles

Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker.

Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance. First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances. 1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features. The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks. The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application.

End-systole and end-diastole detection in short axis cine MRI using a fully convolutional neural network with dilated convolutions.

The correct assessment and characterization of heart anatomy and functionality is usually done through inspection of magnetic resonance image cine sequences. In the clinical setting it is especially important to determine the state of the left ventricle. This requires the measurement of its volume in the end-diastolic and end-systolic frames within the sequence trough segmentation methods. However, the first step required for this analysis before any segmentation is the detection of the end-systolic and end-diastolic frames within the image acquisition. In this work we present a fully convolutional neural network that makes use of dilated convolutions to encode and process the temporal information of the sequences in contrast to the more widespread use of recurrent networks that are usually employed for problems involving temporal information. We trained the network in two different settings employing different loss functions to train the network: the classical weighted cross-entropy, and the weighted Dice loss. We had access to a database comprising a total of 397 cases. Out of this dataset we used 98 cases as test set to validate our network performance. The final classification on the test set yielded a mean frame distance of 0 for the end-diastolic frame (i.e.: the selected frame was the correct one in all images of the test set) and 1.242 (relative frame distance of 0.036) for the end-systolic frame employing the optimum setting, which involved training the neural network with the Dice loss. Our neural network is capable of classifying each frame and enables the detection of the end-systolic and end-diastolic frames in short axis cine MRI sequences with high accuracy.

Fertility-GRU: Identifying Fertility-Related Proteins by Incorporating Deep-Gated Recurrent Units and Original Position-Specific Scoring Matrix Profiles.

Protein function prediction is one of the well-known problems in proteome research, attracting the attention of numerous researchers. However, the implementation of deep neural networks, which helps to increase the protein function prediction, still poses a big challenge. This study proposes a deep learning approach namely Fertility-GRU that incorporates gated recurrent units and position-specific scoring matrix profiles to predict the function of fertility-related protein, which is a highly crucial biological function. Fertility-related proteins also have been proven to be important in many biological entities (i.e., bone marrow and peripheral blood, postnatal mammalian ovary) and parameters (i.e., daily sperm production). As a result, our model can achieve a cross-validation accuracy of 85.8% and an independent accuracy of 91.1%. We also solve the problem of overfitting in the data set by adding dropout layers in the deep learning model. The independent testing results showed sensitivity, specificity, and Matthews correlation coefficient (MCC) values of 90.5%, 91.7%, and 0.82, respectively. Fertility-GRU demonstrates superiority in performance against the state-of-the-art predictor on the same data set. In our proposed study, we provided a method that enables more proteins to be discovered, especially proteins associated with fertility. Moreover, our achievement could promote the use of recurrent networks and gated recurrent units in proteome research. The source code and data set are freely accessible via https://github.com/khanhlee/fertility-gru .

On Neural Fuzzy Systems

Neural fuzzy system (NFS) is basically a fuzzy system that has been equipped with learning capability adapted from the learning idea used in neural networks. Due to their outstanding system modeling capability, NFS have been widely employed in various applications. In this article, we intend to discuss several ideas regarding the learning of NFS for modeling systems. The first issue discussed here is about structure learning techniques. Various ideas used in the literature are introduced and discussed. The second issue is about the use of recurrent networks in NFS to model dynamic systems. The discussion about the performance of such systems will be given. It can be found that such a delay feedback can only bring one order to the system not all possible order as claimed in the literature. Finally, the mechanisms and relative learning performance of with the use of the recursive least squares (RLS) algorithm are reported and discussed. The analyses will be on the effects of interactions among rules. Two kinds of systems are considered. They are the strict rules and generalized rules and have difference variances for membership functions. With those observations in our study, several suggestions regarding the use of the RLS algorithm in NFS are presented.

Connectionism coming of age: legacy and future challenges.

Connectionism coming of age: legacy and future challenges.

Recurrent Neural Networks for Narrowband Signal Detection in the Time-Frequency Domain

A recurrent neural network was trained to detect the time-frequency domain signature of narrowband radio signals against a background of astronomical noise. The objective was to investigate the use of recurrent networks for signal detection in the Search for Extra-Terrestrial Intelligence, though the problem is closely analogous to the detection of some classes of Radio Frequency Interference in radio astronomy.

Financial volatility trading using recurrent neural networks

We simulate daily trading of straddles on financial indexes. The straddles are traded based on predictions of daily volatility differences in the indexes. The main predictive models studied are recurrent neural nets (RNN). Such applications have often been studied in isolation. However, due to the special character of daily financial time-series, it is difficult to make full use of RNN representational power. Recurrent networks either tend to overestimate noisy data, or behave like finite-memory sources with shallow memory; they hardly beat classical fixed-order Markov models. To overcome data nonstationarity, we use a special technique that combines sophisticated models fitted on a larger data set, with a fixed set of simple-minded symbolic predictors using only recent inputs. Finally, we compare our predictors with the GARCH family of econometric models designed to capture time-dependent volatility structure in financial returns. GARCH models have been used to trade volatility. Experimental results show that while GARCH models cannot generate any significantly positive profit, by careful use of recurrent networks or Markov models, the market makers can generate a statistically significant excess profit, but then there is no reason to prefer RNN over much more simple and straightforward Markov models. We argue that any report containing RNN results on financial tasks should be accompanied by results achieved by simple finite-memory sources combined with simple techniques to fight nonstationarity in the data.

Neural networks as a tool for modelling the retention behaviour of dihydropyridines in micellar liquid chromatography

Artificial neural networks have been applied for modelling the retention behaviour of a group of dihydropyridines in Micellar Liquid Chromatography and their capabilities have been compared with classical statistical methods (multiple regression). Some parameters concerning the architecture of the nets have been evaluated. Thus, type of transfer function in the hidden layer, number of neurons in the hidden layer and the use of recurrent networks and transformations of the output data have been optimized. These studies have been carried out with the retention data for 27 dihydropyridines in an octyl silica column using micellar mobile phases containing hexadecyl trimethylammonium bromide (CTAB) or sodium dodecyl sulphate (SDS) as surfactants and n-propanol or n-butanol as organic modifiers. From the results obtained some considerations can be drawn. The selection of the transfer function is very important to get good results, the logarithmoid function being the best one among the different functions studied. Also, the least errors were obtained when three nodes in the hidden layer and the reciprocal of retention factors (as the output variable) were employed.

The relative order and inverses of recurrent networks

Differential geometry is used to investigate the structure of neural-network-based control systems. The key aspect is relative order—an invariant property of dynamic systems. Finite relative order allows the specification of a minimal architecture for a recurrent network. Any system with finite relative order has a left inverse. It is shown that a recurrent network with finite relative order has a local inverse that is also a recurrent network with the same weights. The results have implications for the use of recurrent networks in the inverse-model-based control of nonlinear systems.