Spatio-temporal Feature Extraction Research Articles

A Wearable System for Gait Training in Subjects with Parkinson’s Disease

In this paper, a system for gait training and rehabilitation for Parkinson's disease (PD) patients in a daily life setting is presented. It is based on a wearable architecture aimed at the provision of real-time auditory feedback. Recent studies have, in fact, shown that PD patients can receive benefit from a motor therapy based on auditory cueing and feedback, as happens in traditional rehabilitation contexts with verbal instructions given by clinical operators. To this extent, a system based on a wireless body sensor network and a smartphone has been developed. The system enables real-time extraction of gait spatio-temporal features and their comparison with a patient's reference walking parameters captured in the lab under clinical operator supervision. Feedback is returned to the user in form of vocal messages, encouraging the user to keep her/his walking behavior or to correct it. This paper describes the overall concept, the proposed usage scenario and the parameters estimated for the gait analysis. It also presents, in detail, the hardware-software architecture of the system and the evaluation of system reliability by testing it on a few subjects.

Spatiotemporal contrast enhancement and feature extraction in the bat auditory midbrain and cortex

Navigating on the wing in complete darkness is a challenging task for echolocating bats. It requires the detailed analysis of spatial and temporal information gained through echolocation. Thus neural encoding of spatiotemporal echo information is a major function in the bat auditory system. In this study we presented echoes in virtual acoustic space and used a reverse-correlation technique to investigate the spatiotemporal response characteristics of units in the inferior colliculus (IC) and the auditory cortex (AC) of the bat Phyllostomus discolor. Spatiotemporal response maps (STRMs) of IC units revealed an organization of suppressive and excitatory regions that provided pronounced contrast enhancement along both the time and azimuth axes. Most IC units showed either spatially centralized short-latency excitation spatiotemporally imbedded in strong suppression, or the opposite, i.e., central short-latency suppression imbedded in excitation. This complementary arrangement of excitation and suppression was very rarely seen in AC units. In contrast, STRMs in the AC revealed much less suppression, sharper spatiotemporal tuning, and often a special spatiotemporal arrangement of two excitatory regions. Temporal separation of excitatory regions ranged up to 25 ms and was thus in the range of temporal delays occurring in target ranging in bats in a natural situation. Our data indicate that spatiotemporal processing of echo information in the bat auditory midbrain and cortex serves very different purposes: Whereas the spatiotemporal contrast enhancement provided by the IC contributes to echo-feature extraction, the AC reflects the result of this processing in terms of a high selectivity and task-oriented recombination of the extracted features.

Obtaining scalable and accurate classification in large-scale spatio-temporal domains

We present an approach for learning models that obtain accurate classification of data objects, collected in large-scale spatio-temporal domains. The model generation is structured in three phases: spatial dimension reduction, spatio-temporal features extraction, and feature selection. Novel techniques for the first two phases are presented, with two alternatives for the middle phase. We explore model generation based on the combinations of techniques from each phase. We apply the introduced methodology to data-sets from the Voltage-Sensitive Dye Imaging (VSDI) domain, where the resulting classification models successfully decode neuronal population responses in the visual cortex of behaving animals. VSDI is currently the best technique enabling simultaneous high spatial (10,000 points) and temporal (10 ms or less) resolution imaging from neuronal population in the cortex. We demonstrate that not only our approach is scalable enough to handle computationally challenging data, but it also contributes to the neuroimaging field of study with its decoding abilities. The effectiveness of our methodology is further explored on a data-set from the hurricanes domain, and a promising direction, based on the preliminary results of hurricane severity classification, is revealed.

Continuous Arabic Sign Language Recognition in User Dependent Mode

Arabic Sign Language recognition is an emerging field of research. Previous attempts at automatic vision-based recog-nition of Arabic Sign Language mainly focused on finger spelling and recognizing isolated gestures. In this paper we report the first continuous Arabic Sign Language by building on existing research in feature extraction and pattern recognition. The development of the presented work required collecting a continuous Arabic Sign Language database which we designed and recorded in cooperation with a sign language expert. We intend to make the collected database available for the research community. Our system which we based on spatio-temporal feature extraction and hidden Markov models has resulted in an average word recognition rate of 94%, keeping in the mind the use of a high perplex-ity vocabulary and unrestrictive grammar. We compare our proposed work against existing sign language techniques based on accumulated image difference and motion estimation. The experimental results section shows that the pro-posed work outperforms existing solutions in terms of recognition accuracy.

Content-based video sequence interpretation

Video sequence interpretation attempts to make sense of events that occur when they have been captured on video. When such scenario understanding can be employed in real-time or near-instantaneously, it can be applied to situations where constant watch would otherwise be needed. The potential for consumer applications is wide-ranging. Examples include safety monitoring of unattended elderly persons, children, pets and security surveillance. In this paper, we present a content-based video sequence interpretation algorithm for confined or semi-confined spaces. The algorithm comprises spatio-temporal video object (VO) segmentation, feature extraction, and scene analysis. Application of the proposed algorithm is illustrated in the context of security monitoring. Our system activates an alarm to alert the user of a suspicious event virtually instantaneously.

A space-time delay neural network for motion recognition and its application to lipreading.

Motion recognition has received increasing attention in recent years owing to heightened demand for computer vision in many domains, including the surveillance system, multimodal human computer interface, and traffic control system. Most conventional approaches classify the motion recognition task into partial feature extraction and time-domain recognition subtasks. However, the information of motion resides in the space-time domain instead of the time domain or space domain independently, implying that fusing the feature extraction and classification in the space and time domains into a single framework is preferred. Based on this notion, this work presents a novel Space-Time Delay Neural Network (STDNN) capable of handling the space-time dynamic information for motion recognition. The STDNN is unified structure, in which the low-level spatiotemporal feature extraction and high-level space-time-domain recognition are fused. The proposed network possesses the spatiotemporal shift-invariant recognition ability that is inherited from the time delay neural network (TDNN) and space displacement neural network (SDNN), where TDNN and SDNN are good at temporal and spatial shift-invariant recognition, respectively. In contrast to multilayer perceptron (MLP), TDNN, and SDNN, STDNN is constructed by vector-type nodes and matrix-type links such that the spatiotemporal information can be accurately represented in a neural network. Also evaluated herein is the performance of the proposed STDNN via two experiments. The moving Arabic numerals (MAN) experiment simulates the object's free movement in the space-time domain on image sequences. According to these results, STDNN possesses a good generalization ability with respect to the spatiotemporal shift-invariant recognition. In the lipreading experiment, STDNN recognizes the lip motions based on the inputs of real image sequences. This observation confirms that STDNN yields a better performance than the existing TDNN-based system, particularly in terms of the generalization ability. In addition to the lipreading application, the STDNN can be applied to other problems since no domain-dependent knowledge is used in the experiment.

Hardware implementation of a visual-motion pixel using oriented spatiotemporal neural filters

A pixel for measuring two-dimensional (2-D) visual motion with two one-dimensional (1-D) detectors has been implemented in very large scale integration. Based on the spatiotemporal feature extraction model of Adelson and Bergen, the pixel is realized using a general-purpose analog neural computer and a silicon retina. Because the neural computer only offers sum-and-threshold neurons, the Adelson and Bergen's model is modified. The quadratic nonlinearity is replaced with a full-wave rectification, while the contrast normalization is replaced with edge detection and thresholding. Motion is extracted in two dimensions by using two 1-D detectors with spatial smoothing orthogonal to the direction of motion. Analysis shows that our pixel, although it has some limitations, has much lower hardware complexity compared to the full 2-D model. It also produces more accurate results and has a reduced aperture problem compared to the two 1-D model with no smoothing. Real-time velocity is represented as a distribution of activity of the 18 X and 18 Y velocity-tuned neural filters.