Band-limited Multiple Fourier Linear Combiner Research Articles

Real-Time Image-Based Vibration Extraction with Memory-Efficient Optical Flow and Block-Based Adaptive Filter

In this paper, we propose a real-time vibration extraction system, which extracts vibration component within a given frequency range from videos in real time, for realizing tremor suppression used in microsurgery assistance systems. To overcome the problems in our previous system based on the mean Lucas-Kanade (LK) optical flow of the whole frame, we have introduced a new architecture combining dense optical flow calculated with simple feature matching and block-based band-pass filtering using band-limited multiple Fourier linear combiner (BMFLC). As a feature of optical flow calculation, we use the simplified rotation-invariant histogram of oriented gradients (RIHOG) based on a gradient angle quantized to 1, 2, or 3 bits, which greatly reduces the usage of memory resources for a frame buffer. An obtained optical flow map is then divided into multiple blocks, and BMFLC is applied to the mean optical flow of each block independently. By using the L1-norm of adaptive weight vectors in BMFLC as a criterion, blocks belonging to vibrating objects can be isolated from background at low cost, leading to better extraction accuracy compared to the previous system. The whole system for 480p and 720p resolutions can be implemented on a single Xilinx Zynq-7000 XC7Z020 FPGA without any external memory, and can process a video stream supplied directly from a camera at 60fps.

Time-Frequency Analysis of Non-Stationary Biological Signals with Sparse Linear Regression Based Fourier Linear Combiner.

It is often difficult to analyze biological signals because of their nonlinear and non-stationary characteristics. This necessitates the usage of time-frequency decomposition methods for analyzing the subtle changes in these signals that are often connected to an underlying phenomena. This paper presents a new approach to analyze the time-varying characteristics of such signals by employing a simple truncated Fourier series model, namely the band-limited multiple Fourier linear combiner (BMFLC). In contrast to the earlier designs, we first identified the sparsity imposed on the signal model in order to reformulate the model to a sparse linear regression model. The coefficients of the proposed model are then estimated by a convex optimization algorithm. The performance of the proposed method was analyzed with benchmark test signals. An energy ratio metric is employed to quantify the spectral performance and results show that the proposed method Sparse-BMFLC has high mean energy (0.9976) ratio and outperforms existing methods such as short-time Fourier transfrom (STFT), continuous Wavelet transform (CWT) and BMFLC Kalman Smoother. Furthermore, the proposed method provides an overall 6.22% in reconstruction error.

Evolutionary Algorithm Based Feature Optimization for Multi-Channel EEG Classification.

The most BCI systems that rely on EEG signals employ Fourier based methods for time-frequency decomposition for feature extraction. The band-limited multiple Fourier linear combiner is well-suited for such band-limited signals due to its real-time applicability. Despite the improved performance of these techniques in two channel settings, its application in multiple-channel EEG is not straightforward and challenging. As more channels are available, a spatial filter will be required to eliminate the noise and preserve the required useful information. Moreover, multiple-channel EEG also adds the high dimensionality to the frequency feature space. Feature selection will be required to stabilize the performance of the classifier. In this paper, we develop a new method based on Evolutionary Algorithm (EA) to solve these two problems simultaneously. The real-valued EA encodes both the spatial filter estimates and the feature selection into its solution and optimizes it with respect to the classification error. Three Fourier based designs are tested in this paper. Our results show that the combination of Fourier based method with covariance matrix adaptation evolution strategy (CMA-ES) has the best overall performance.

Accelerometer-based estimation and modal velocity feedback vibration control of a stress-ribbon bridge with pneumatic muscles

Lightweight footbridges are very elegant but also prone to vibration. By employing active vibration control, smart footbridges could accomplish not only the architectural concept but also the required serviceability and comfort. Inertial sensors such as accelerometers allow the estimation of nodal velocities and displacements. A Kalman filter together with a band-limited multiple Fourier linear combiner (BMFLC) is applied to enable a drift-free estimation of these signals for the quasi-periodic motion under pedestrian excitation without extra information from other kinds of auxiliary sensors. The modal velocities of the structure are determined by using a second Kalman filter with the known applied actuator forces as inputs and the estimated nodal displacement and velocities as measurements. The obtained multi-modal velocities are then used for feedback control. An ultra-lightweight stress-ribbon footbridge built in the Peter-Behrens- Halle at the Technische Universitat Berlin served as the research object. Using two inertial sensors in optimal points we can estimate the dominant modal characteristics of this bridge. Real-time implementation and evaluation results of the proposed estimator will be presented in comparison to signals derived from classical displacement encoders. The real-time estimated modal velocities were applied in a multi-modal velocity feedback vibration control scheme with lightweight pneumatic muscle actuators. Experimental results demonstrate the feasibility of using inertial sensors for active vibration control of lightweight footbridges.

Spatial filter and feature selection optimization based on EA for multi-channel EEG.

The EEG signals employed for BCI systems are generally band-limited. The band-limited multiple Fourier linear combiner (BMFLC) with Kalman filter was developed to obtain amplitude estimates of the EEG signal in a pre-fixed frequency band in real-time. However, the high-dimensionality of the feature vector caused by the application of BMFLC to multi-channel EEG based BCI deteriorates the performance of the classifier. In this work, we apply evolutionary algorithm (EA) to tackle this problem. The real-valued EA encodes both the spatial filter and the feature selection into its solution and optimizes it with respect to the classification error. Three BMFLC based BCI configurations are proposed. Our results show that the BMFLC-KF with covariance matrix adaptation evolution strategy (CMAES) has the best overall performance.

Time-frequency analysis of band-limited EEG with BMFLC and Kalman filter for BCI applications

BackgroundTime-Frequency analysis of electroencephalogram (EEG) during different mental tasks received significant attention. As EEG is non-stationary, time-frequency analysis is essential to analyze brain states during different mental tasks. Further, the time-frequency information of EEG signal can be used as a feature for classification in brain-computer interface (BCI) applications.MethodsTo accurately model the EEG, band-limited multiple Fourier linear combiner (BMFLC), a linear combination of truncated multiple Fourier series models is employed. A state-space model for BMFLC in combination with Kalman filter/smoother is developed to obtain accurate adaptive estimation. By virtue of construction, BMFLC with Kalman filter/smoother provides accurate time-frequency decomposition of the bandlimited signal.ResultsThe proposed method is computationally fast and is suitable for real-time BCI applications. To evaluate the proposed algorithm, a comparison with short-time Fourier transform (STFT) and continuous wavelet transform (CWT) for both synthesized and real EEG data is performed in this paper. The proposed method is applied to BCI Competition data IV for ERD detection in comparison with existing methods.ConclusionsResults show that the proposed algorithm can provide optimal time-frequency resolution as compared to STFT and CWT. For ERD detection, BMFLC-KF outperforms STFT and BMFLC-KS in real-time applicability with low computational requirement.

Adaptive sliding bandlimited multiple fourier linear combiner for estimation of pathological tremor

Pathological tremor is a roughly sinusoidal movement and impacts individuals’ daily living activities. Biomechanical loading is employed as a potential method for tremor suppression with the accurate estimation of amplitude and frequency of tremor signals. In this paper, a study on tremor is conducted and the characteristics of tremor signals are analyzed. An adaptive sliding Bandlimited multiple Fourier linear combiner (ASBMFLC) algorithm is proposed to estimate the different desired signals with sliding frequency band and zero-phase lag. This method incorporates digital filter, Weighted-Frequency Fourier Linear Combiner (WFLC) and Bandlimited Multiple Fourier Linear Combiner (BMFLC) with modification of fundamental frequency and limitation of frequency range. Based on the experimental tremor signals, WFLC, BMFLC and the proposed algorithm are evaluated, respectively. The experimental results show that the developed algorithm could adapt to the unknown dominant frequency for determining the interesting frequency band without prior information. Furthermore, the improved method could provide higher accurate estimation of tremor and extract the voluntary components from measured signals, compared with WFLC and BMFLC, respectively.

A time-sequence-based fuzzy support vector machine adaptive filter for tremor cancelling for microsurgery

Hand tremors may cause some blemishes in precision and stability of a minimally invasive surgery (MIS). To track the tremor signals accurately, there are two main problems left to be settled. First, it is not practical to collect the sample data of tremor in large scale in practical applications. To deal with the hand tremors, a learning method based on small samples sizes and high dimensional input space is needed. Second, the hand tremors have time-varying characteristics. This fact is neglected by traditional learning methods, which could lead to imprecision and instability of a MIS. In this work, a time-sequence-based fuzzy support vector machine adaptive filter (TSF-SVMAF) for tremor cancelling is proposed. The proposed method is based on support vector machine and time series. It is suitable for solving the problem that the inputs are time-varying and the samples are small-scale. To cancel the time-varying hand tremors, different learning-weight-functions are designed for tremor signals with different frequencies. From the simulation results, compared with the existing methods such as back propagation (BP), weighted-frequency Fourier combiner (WFLC) and bandlimited multiple Fourier linear combiner (BMFLC), the proposed method has better performance when learning the time-varying hand tremors with small sample sizes.

An Enhanced Intelligent Handheld Instrument with Visual Servo Control for 2-DOF Hand Motion Error Compensation

The intelligent handheld instrument, ITrem2, enhances manual positioning accuracy by cancelling erroneous hand movements and, at the same time, provides automatic micromanipulation functions. Visual data is acquired from a high speed monovision camera attached to the optical surgical microscope and acceleration measurements are acquired from the inertial measurement unit (IMU) on board ITrem2. Tremor estimation and canceling is implemented via Band-limited Multiple Fourier Linear Combiner (BMFLC) filter. The piezoelectric actuated micromanipulator in ITrem2 generates the 3D motion to compensate erroneous hand motion. Preliminary bench-top 2-DOF experiments have been conducted. The error motions simulated by a motion stage is reduced by 67% for multiple frequency oscillatory motions and 56.16% for pre-conditioned recorded physiological tremor.

Adaptive estimation of EEG for subject-specific reactive band identification and improved ERD detection

The event-related desynchronization (ERD) is a magnitude decrease phenomenon which can be found in electroencephalogram (EEG) mu-rhythm in a certain narrow frequency band (reactive band) during different sensorimotor tasks and stimuli. The success of ERD detection depends on proper identification of subject specific reactive band. An adaptive algorithm band limited multiple Fourier linear combiner (BMFLC) is employed in this paper for identification of subject specific reactive band for real-time ERD detection. With the time–frequency mapping obtained with BMFLC, a procedure is formulated for reactive band identification. Improved classification is obtained by applying this method to a standard BCI data set compared to traditional ERD detection methods. Study conducted with 8 subjects drawn from BCI Competition IV data set show a 22% increase in ERD and 10% improvement in classification with the proposed method compared to standard ERD based classification.

A novel control architecture for physiological tremor compensation in teleoperated systems

Telesurgery delivers surgical care to a 'remote' patient by means of robotic manipulators. When accurate positioning of the surgeon's tool is required, as in microsurgery, physiological tremor causes unwanted imprecision during a surgical operation. Accurate estimation/compensation of physiological tremor in teleoperation systems has been shown to improve performance during telesurgery. A new control architecture is proposed for estimation and compensation of physiological tremor in the presence of communication time delays. This control architecture guarantees stability with satisfactory transparency. In addition, the proposed method can be used for applications that require modifications in transmitted signals through communication channels. Stability of the bilateral tremor-compensated teleoperation is preserved by extending the bilateral teleoperation to the equivalent trilateral Dual-master/Single-slave teleoperation. The bandlimited multiple Fourier linear combiner (BMFLC) algorithm is employed for real-time estimation of the operator's physiological tremor. Two kinds of stability analysis are employed. In the model-base controller, Llewellyn's Criterion is used to analyze the teleoperation absolute stability. In the second method, a nonmodel-based controller is proposed and the stability of the time-delayed teleoperated system is proved by employing a Lyapunov function. Experimental results are presented to validate the effectiveness of the new control architecture. The tremorous motion is measured by accelerometer to be compensated in real time. In addition, a Needle-Insertion setup is proposed as a slave robot for the application of brachytherapy, in which the needle penetrates in the desired position. The slave performs the desired task in two classes of environments (free motion of the slave and in the soft tissue). Experiments show that the proposed control architecture effectively compensates the user's tremorous motion and the slave follows only the master's voluntary motion in a stable manner.

Vision Aided Active Error Canceling in Handheld Microsurgical Instrument

The intelligent handheld instrument, ITrem2, enhances manual positioning accuracy by cancelling erroneous hand movements and, at the same time, provides automatic micromanipulation functions. Visual data are acquired from a high speed monovision camera attached to the optical surgical microscope and acceleration measurements are acquired from the inertial measurement unit (IMU) on board ITrem2. Tremor estimation and canceling is implemented via Band-limited Multiple Fourier Linear Combiner (BMFLC) filter. The piezoelectric actuated micromanipulator in ITrem2 generates the 3D motion to compensate erroneous hand motion. Experiments were conducted for cancellation of oscillatory motions and error amplitude was reduced by 67% in 1-DOF test.

Drift-Free Position Estimation of Periodic or Quasi-Periodic Motion Using Inertial Sensors

Position sensing with inertial sensors such as accelerometers and gyroscopes usually requires other aided sensors or prior knowledge of motion characteristics to remove position drift resulting from integration of acceleration or velocity so as to obtain accurate position estimation. A method based on analytical integration has previously been developed to obtain accurate position estimate of periodic or quasi-periodic motion from inertial sensors using prior knowledge of the motion but without using aided sensors. In this paper, a new method is proposed which employs linear filtering stage coupled with adaptive filtering stage to remove drift and attenuation. The prior knowledge of the motion the proposed method requires is only approximate band of frequencies of the motion. Existing adaptive filtering methods based on Fourier series such as weighted-frequency Fourier linear combiner (WFLC), and band-limited multiple Fourier linear combiner (BMFLC) are modified to combine with the proposed method. To validate and compare the performance of the proposed method with the method based on analytical integration, simulation study is performed using periodic signals as well as real physiological tremor data, and real-time experiments are conducted using an ADXL-203 accelerometer. Results demonstrate that the performance of the proposed method outperforms the existing analytical integration method.

Estimation and filtering of physiological tremor for real‐time compensation in surgical robotics applications

Physiological tremor is the main cause of imprecision in microsurgical procedures/robotics applications. Existing methods, such as weighted-frequency Fourier linear combiner (WFLC), rely on estimating the tremor under the assumption that it has a single dominant frequency. This paper focuses on developing a new algorithm for accurate tremor filtering in real time. A study conducted on several novice subjects and microsurgeons showed the tremor to contain several dominant frequencies in a band, rather than a single dominant frequency. Based on the tremor characteristics, a new algorithm band-limited multiple Fourier linear combiner (BMFLC) has been developed to estimate a band of signals with multiple dominant frequencies. A separation procedure to separate the intended motion/drift from the tremor portion is also discussed. A simulation study was first conducted to validate the theoretical development on recorded tremor data. The experimental set-up was designed to study the real-time performance of the proposed algorithm. Tremor sensing using accelerometers is also discussed, with the proposed algorithm. Our experiments showed that the developed BMFLC algorithm had an average tremor compensation of 64% compared to 43% for the WFLC algorithm in real-time for one degree of freedom (1-DOF) cancellation of tremor. The BMFLC algorithm can be applied for the three axes separately and 3-DOF cancellation of tremor can be achieved. Further research is required to deal with complex gestures involved during microsurgery.

Double adaptive bandlimited multiple Fourier linear combiner for real-time estimation/filtering of physiological tremor

Tremor is the root cause for human imprecision during microsurgery. Accurate filtering of physiological tremor is extremely important for compensation in robotics assisted microsurgical instruments/procedures. A study on several surgeons tremor is conducted and the characteristics of the tremor are analyzed. A double adaptive bandlimited multiple Fourier linear combiner is designed to estimate the modulated signals with multiple frequency components for filtering and compensation of tremor in real-time. A separation procedure to separate the intended motion/drift from the tremor portion is developed. The proposed methods are compared with the existing weighted-frequency Fourier linear combiner (WFLC) algorithm on the tremor data of surgeons/subjects. Critical validation of the algorithm is performed, experiments are conducted for 1-degree of freedom (DOF) cancellation of tremor. Our experiments showed that our newly developed algorithm has a tremor compensation of at least 65% compared to 46% for the WFLC algorithm.