Inclusion Of Temporal Information Research Articles

Polyp detection on video colonoscopy using a hybrid 2D/3D CNN.

Colonoscopy is the gold standard for early diagnosis and pre-emptive treatment of colorectal cancer by detecting and removing colonic polyps. Deep learning approaches to polyp detection have shown potential for enhancing polyp detection rates. However, the majority of these systems are developed and evaluated on static images from colonoscopies, whilst in clinical practice the treatment is performed on a real-time video feed. Non-curated video data remains a challenge, as it contains low-quality frames when compared to still, selected images often obtained from diagnostic records. Nevertheless, it also embeds temporal information that can be exploited to increase predictions stability. A hybrid 2D/3D convolutional neural network architecture for polyp segmentation is presented in this paper. The network is used to improve polyp detection by encompassing spatial and temporal correlation of the predictions while preserving real-time detections. Extensive experiments show that the hybrid method outperforms a 2D baseline. The proposed architecture is validated on videos from 46 patients and on the publicly available SUN polyp database. A higher performance and increased generalisability indicate that real-world clinical implementations of automated polyp detection can benefit from the hybrid algorithm and the inclusion of temporal information.

Improve automatic detection of animal call sequences with temporal context.

Many animals rely on long-form communication, in the form of songs, for vital functions such as mate attraction and territorial defence. We explored the prospect of improving automatic recognition performance by using the temporal context inherent in song. The ability to accurately detect sequences of calls has implications for conservation and biological studies. We show that the performance of a convolutional neural network (CNN), designed to detect song notes (calls) in short-duration audio segments, can be improved by combining it with a recurrent network designed to process sequences of learned representations from the CNN on a longer time scale. The combined system of independently trained CNN and long short-term memory (LSTM) network models exploits the temporal patterns between song notes. We demonstrate the technique using recordings of fin whale (Balaenoptera physalus) songs, which comprise patterned sequences of characteristic notes. We evaluated several variants of the CNN + LSTM network. Relative to the baseline CNN model, the CNN + LSTM models reduced performance variance, offering a 9–17% increase in area under the precision–recall curve and a 9–18% increase in peak F1-scores. These results show that the inclusion of temporal information may offer a valuable pathway for improving the automatic recognition and transcription of wildlife recordings.

Temporal Encoding is Required for Categorization, But Not Discrimination.

Core auditory cortex (AC) neurons encode slow fluctuations of acoustic stimuli with temporally patterned activity. However, whether temporal encoding is necessary to explain auditory perceptual skills remains uncertain. Here, we recorded from gerbil AC neurons while they discriminated between a 4-Hz amplitude modulation (AM) broadband noise and AM rates >4Hz. We found a proportion of neurons possessed neural thresholds based on spike pattern or spike count that were better than the recorded session's behavioral threshold, suggesting that spike count could provide sufficient information for this perceptual task. A population decoder that relied on temporal information outperformed a decoder that relied on spike count alone, but the spike count decoder still remained sufficient to explain average behavioral performance. This leaves open the possibility that more demanding perceptual judgments require temporal information. Thus, we asked whether accurate classification of different AM rates between 4 and 12Hz required the information contained in AC temporal discharge patterns. Indeed, accurate classification of these AM stimuli depended on the inclusion of temporal information rather than spike count alone. Overall, our results compare two different representations of time-varying acoustic features that can be accessed by downstream circuits required for perceptual judgments.

Instance-Level Microtubule Tracking.

We propose a new method of instance-level microtubule (MT) tracking in time-lapse image series using recurrent attention. Our novel deep learning algorithm segments individual MTs at each frame. Segmentation results from successive frames are used to assign correspondences among MTs. This ultimately generates a distinct path trajectory for each MT through the frames. Based on these trajectories, we estimate MT velocities. To validate our proposed technique, we conduct experiments using real and simulated data. We use statistics derived from real time-lapse series of MT gliding assays to simulate realistic MT time-lapse image series in our simulated data. This data set is employed as pre-training and hyperparameter optimization for our network before training on the real data. Our experimental results show that the proposed supervised learning algorithm improves the precision for MT instance velocity estimation drastically to 71.3% from the baseline result (29.3%). We also demonstrate how the inclusion of temporal information into our deep network can reduce the false negative rates from 67.8% (baseline) down to 28.7% (proposed). Our findings in this work are expected to help biologists characterize the spatial arrangement of MTs, specifically the effects of MT-MT interactions.

The Importance of Quasi-4D Path-Integrated Dose Accumulation for More Accurate Risk Estimation in Stereotactic Liver Radiotherapy.

Intrafraction organ deformation may be accounted for by inclusion of temporal information in dose calculation models. In this article, we demonstrate a quasi-4-dimensional method for improved risk estimation. Conventional 3-dimensional and quasi-4-dimensional calculations employing dose warping for dose accumulation were undertaken for patients with liver metastases planned for 42 Gy in 6 fractions of stereotactic body radiotherapy. Normal tissue complication probabilities and stochastic risks for radiation-induced carcinogenesis and cardiac complications were evaluated for healthy peripheral structures. Hypothetical assessments of other commonly employed dose/fractionation schedules on normal tissue complication probability estimates were explored. Conventional 3-dimensional dose computation may result in significant under- or overestimation of doses to organ at risk. For instance, doses differ (on average) by 17% (σ = 14%) for the left kidney, by 14% (σ = 7%) for the right kidney, by 7% (σ = 9%) for the large bowel, and by 10% (σ = 14%) for the duodenum. Discrepancies in the excess relative risk range up to about 30%. The 3-dimensional approach was shown to result in cardiac complication risks underestimated by >20%. For liver stereotactic body radiotherapy, we have shown that conventional 3-dimensional dose calculation may significantly over-/underestimate dose to organ at risk (90%-120% of the 4-dimensional estimate for the mean dose and 20%-150% for D2%). Providing dose estimates that most closely represent the actual dose delivered will provide valuable information to improve our understanding of the dose response for partial volume irradiation using hypofractionated schedules. Excess relative risks of radiocarcinogenesis were shown to range up to approximately excess relative risk = 4 and the prediction thereof depends greatly on the use of either 3-dimensional or 4-dimensional methods (with corresponding results differing by tens of percent).

Robot-assisted intermanual transfer of handwriting skills

We examined whether intermanual transfer of fine motor skills can be facilitated by training in a virtual environment. We focused on three types of assistance: visual – subjects could see a reference template on a computer screen – and two variants of haptic assistance. Subjects held a planar robot manipulandum and were required to write isolated cursive letters of an approximate size of 5cm. Therefore, the task was similar to writing on a horizontal blackboard. The robot generated forces that were directed either towards the reference template (path guidance) or towards the reference trajectory (trajectory guidance). The training protocol consisted of three assisted exercise sessions on three consecutive days. Performance on the following day was tested to assess retention. After training, the improvement in trajectory shape was only significant in trajectory guidance and, to a lesser extent, visual guidance. Path guidance exhibited no significant improvement. These effects were substantially retained one day after the end of training. Similar effects were observed in shape variability. Furthermore, all training modalities caused a reduction in movement duration, but no significant differences were observed among groups. These results suggest that robot assistance may be beneficial for improving intermanual transfer, but inclusion of temporal information in the guidance strategy is essential for learning to take place.

Activity recognition using temporal evidence theory

The ability to identify the behavior of people in a home is at the core of Smart Home functionality. Such environments are equipped with sensors that unobtrusively capture information about the occupants. Reasoning mechanisms transform the technical, frequently noisy data of sensors into meaningful interpretations of occupant activities. Time is a natural human way to reason about activities. Peoples' activities in the home often have an identifiable routine; activities take place at distinct times throughout the day and last for predicable lengths of time. However, the inclusion of temporal information is still limited in the domain of activity recognition. Evidence theory is gaining increasing interest in the field of activity recognition, and is suited to the incorporation of time related domain knowledge into the reasoning process. In this paper, an evidential reasoning framework that incorporates temporal knowledge is presented. We evaluate the effectiveness of the framework using a third party published smart home dataset. An improvement in activity recognition of 70% is achieved when time patterns and activity durations are included in activity recognition. We also compare our approach with Naive Bayes classifier and J48 Decision Tree, with temporal evidence theory achieving higher accuracies than both classifiers.

Standardized Shrinking LORETA-FOCUSS (SSLOFO): A New Algorithm for Spatio-Temporal EEG Source Reconstruction

This paper presents a new algorithm called Standardized Shrinking LORETA-FOCUSS (SSLOFO) for solving the electroencephalogram (EEG) inverse problem. Multiple techniques are combined in a single procedure to robustly reconstruct the underlying source distribution with high spatial resolution. This algorithm uses a recursive process which takes the smooth estimate of sLORETA as initialization and then employs the re-weighted minimum norm introduced by FOCUSS. An important technique called standardization is involved in the recursive process to enhance the localization ability. The algorithm is further improved by automatically adjusting the source space according to the estimate of the previous step, and by the inclusion of temporal information. Simulation studies are carried out on both spherical and realistic head models. The algorithm achieves very good localization ability on noise-free data. It is capable of recovering complex source configurations with arbitrary shapes and can produce high quality images of extended source distributions. We also characterized the performance with noisy data in a realistic head model. An important feature of this algorithm is that the temporal waveforms are clearly reconstructed, even for closely spaced sources. This provides a convenient way to estimate neural dynamics directly from the cortical sources.