Multispectral Image Sequences Research Articles

A multimodal assessment of cementless tibial baseplate fixation using radiography, radiostereometric analysis, and magnetic resonance imaging.

Fixation in cementless total knee arthroplasty is provided by osseous integration. Radiography, radiostereometric analysis (RSA), and magnetic resonance imaging (MRI) were used simultaneously to investigate fixation. Relationships between RSA-measured implant micromotions and MRI-evaluated osseous integration at the component-bone interface were assessed in 10 patients up to 6 months postoperation. Supine MRI (using multispectral imaging sequences) and RSA exams were performed to evaluate osseous integration and measure longitudinal migration, respectively. Inducible displacement was measured from standing RSA exams. Radiolucent lines were detected on conventional radiographs. Of 10 patients, 6 had fibrous membranes detected on MRI. No fluid or osteolytic interfaces were found, and no components were scored loose. Of 10 patients, 6 had radiolucent lines detected. Average maximum total point motion (MTPM) for longitudinal migration at 6 months was 0.816 mm (range 0.344-1.462 mm). Average MTPM for inducible displacement at 6 months was 1.083 mm (range 0.553-1.780 mm). Fictive points located in fibrous-classified baseplate quadrants had greater longitudinal migration than fictive points located in baseplate quadrants with normal interfaces at 2 weeks (p = 0.031), 6 weeks (p = 0.046), and 3 months (p = 0.047), and greater inducible displacements at 3 months (p = 0.011) and 6 months (p = 0.045). Greater early micromotion may be associated with the presence of fibrous membranes at the component-bone interface. Clinical significance: This multimodal imaging study contributes knowledge of the fixation of modern cementless TKA, supporting the notion that osseous integration is important for optimal implant fixation.

A high-efficiency acquisition method of LED multispectral images using Gray code based square wave frequency division modulation

LED illumination-based multispectral imaging (LEDMSI) is a fast, convenient, and effective multispectral image acquisition technique. The square wave frequency division modulation (SFDM) and the integral sampling of the camera can build an efficient match pattern. With regards to image quality decline problems due to the ringing effect caused by the drive module while square wave high-low level conversion in the SFDM-LEDMS system. This paper proposes a multispectral image acquisition method combining Gray code and square wave frequency modulation. The micro control unit was used to generate a series of frequency division square waves in Gray code mode followed by LEDs of different wavelengths modulated by SFDM-square waves as a light source. Subsequently, the camera was used to simultaneously collect a sequence of multi-spectral images followed by single-wavelength images using differential accumulation demodulation. Finally, the no-reference image quality assessment (NR-IQA) was used to evaluate the quality of each single-wavelength image obtained by Binary code and Gray code. We deployed five wavelengths as examples to verify the effectiveness of the proposed method. The accumulated results indicated an improvement of the image quality in terms of information and gradient using Gray code. In conclusion, the present study can provide an effective image acquisition alternative for household medical equipment.

Changes of the area of Spartina alterniflora and mangroves in Guangxi Shankou Mangrove National Nature Reserve from 1995 to 2019.

Given its high root regeneration ability and adaptation capacity, Spartina alterniflora would predominate the resource competition with other plant species. As an invasive alien species, it has caused serious damages to the coastal ecosystem of China. We explored the impacts of S. alterniflora invasion on the growth and expansion of mangroves around the coastal zones around Guangxi Shankou Mangrove National Nature Reserve (GSMNNR). The area of S. alterniflora and mangroves in the GSMNNR was analyzed based on a sequence of Landsat satellite multispectral images collected during 1995 to 2019. The results showed that S. alterniflora predominated in the competition with mangroves without human intervention. The area of S. alterniflora decreased under the conditions of cutting, rooting and other management measures, while the average annual growth rate of mangrove area correspondingly increased. It indicated that the invasion of S. alterniflora would inhibit the growth and expansion of mangroves. The S. alterniflora management measures, such as mowing and rooting, could weaken the negative impacts of S. alterniflora on mangroves. Mangroves could thus be effectively protected. This study could provide quantitative scientific data for S. alterniflora management and the protection of the mangrove ecosystem in the GSMNNR, and thus be a valuable reference to the prevention on a larger scale of S. alterniflora and the formulation of further protective measures for mangroves.

Tensor Decomposition Learning for Compression of Multidimensional Signals

Multidimensional signals like multispectral images and color videos are becoming ubiquitous in modern times, constantly introducing challenges in data storage and transfer, and therefore demanding efficient compression strategies. Such high dimensional observations can be naturally encoded as tensors, exhibiting significant redundancies across dimensions. This property is exploited by tensor decomposition techniques that are being increasingly used for compactly encoding large multidimensional arrays. While efficient, these methods are incapable of utilizing prior information present in training data. In this paper, a novel tensor decomposition learning method is proposed for the compression of high dimensional signals. Specifically, instead of extracting independent bases for each example, our method learns an appropriate basis for each dimension from a set of training samples by solving a constrained optimization problem. As such, each sample is quantized and encoded into a reduced-size core tensor of coefficients that corresponds to the multilinear combination of the learned basis matrices. Furthermore, the proposed method employs a symbol encoding dictionary for binarizing the decomposition outputs. Experimental results on synthetic data and real satellite multispectral image sequences demonstrate the efficacy of our method, surpassing competing compression methods while offering the flexibility to handle arbitrary high dimensional data structures.

ASRNet: Adversarial Segmentation and Registration Networks for Multispectral Fundus Images

Multispectral imaging (MSI) technique is often used to capture images of the fundus by illuminating it with different wavelengths of light. However, these images are taken at different points in time such that eyeball movements can cause misalignment between consecutive images. The multispectral image sequence reveals important information in the form of retinal and choroidal blood vessel maps, which can help ophthalmologists to analyze the morphology of these blood vessels in detail. This in turn can lead to a high diagnostic accuracy of several diseases. In this paper, we propose a novel semi-supervised end-to-end deep learning framework called “Adversarial Segmentation and Registration Nets” (ASRNet) for the simultaneous estimation of the blood vessel segmentation and the registration of multispectral images via an adversarial learning process. ASRNet consists of two subnetworks: (i) A segmentation module S that fulfills the blood vessel segmentation task, and (ii) A registration module R that estimates the spatial correspondence of an image pair. Based on the segmention-driven registration network, we train the segmentation network using a semi-supervised adversarial learning strategy. Our experimental results show that the proposed ASRNet can achieve state-of-the-art accuracy in segmentation and registration tasks performed with real MSI datasets.

EEG-based emotion recognition with deep convolutional neural networks.

The emotional state of people plays a key role in physiological and behavioral human interaction. Emotional state analysis entails many fields such as neuroscience, cognitive sciences, and biomedical engineering because the parameters of interest contain the complex neuronal activities of the brain. Electroencephalogram (EEG) signals are processed to communicate brain signals with external systems and make predictions over emotional states. This paper proposes a novel method for emotion recognition based on deep convolutional neural networks (CNNs) that are used to classify Valence, Arousal, Dominance, and Liking emotional states. Hence, a novel approach is proposed for emotion recognition with time series of multi-channel EEG signals from a Database for Emotion Analysis and Using Physiological Signals (DEAP). We propose a new approach to emotional state estimation utilizing CNN-based classification of multi-spectral topology images obtained from EEG signals. In contrast to most of the EEG-based approaches that eliminate spatial information of EEG signals, converting EEG signals into a sequence of multi-spectral topology images, temporal, spectral, and spatial information of EEG signals are preserved. The deep recurrent convolutional network is trained to learn important representations from a sequence of three-channel topographical images. We have achieved test accuracy of 90.62% for negative and positive Valence, 86.13% for high and low Arousal, 88.48% for high and low Dominance, and finally 86.23% for like-unlike. The evaluations of this method on emotion recognition problem revealed significant improvements in the classification accuracy when compared with other studies using deep neural networks (DNNs) and one-dimensional CNNs.

Ischemic stroke lesion segmentation using stacked sparse autoencoder

Automatic segmentation of ischemic stroke lesion volumes from multi-spectral Magnetic Resonance Imaging (MRI) sequences plays a vital role in quantifying and locating the lesion region. Most existing methods mainly rely on designing hand-crafted features followed by a classifier model for ischemic stroke lesion segmentation. Design of these features requires complex domain knowledge and often lacks the ability to differentiate between the stroke lesions and the normal classes. In this work, we propose an unsupervised featured learning approach based on stacked sparse autoencoder (SSAE) framework for automatically learning the features for accurate segmentation of stroke lesions from brain MR images. A deep architecture is designed using sparse autoencoder (SAE) layers, followed by support vector machine (SVM) classifier for classifying the patches into normal or lesions. We validated our approach on a publicly available Ischemic Stroke Lesion Segmentation (ISLES) 2015 dataset, with a mean precision of 0.968, mean dice coefficient (DC) of 0.943, mean recall of 0.924 and mean accuracy of 0.904. The experimental results show that our proposed approach significantly outperforms the state-of-the-art methods in terms of precision, DC, and recall. Quantitative evaluation was carried out and compared with the existing approaches, which demonstrates that the proposed method is 25.71%, 36.67%, and 16.96% higher in terms of precision, DC and recall values, respectively. The unsupervised features learned via SSAE framework performs better than the hand-crafted features and can be easily trained on large datasets.

Deblurring sequential ocular images from multi-spectral imaging (MSI) via mutual information.

Multi-spectral imaging (MSI) produces a sequence of spectral images to capture the inner structure of different species, which was recently introduced into ocular disease diagnosis. However, the quality of MSI images can be significantly degraded by motion blur caused by the inevitable saccades and exposure time required for maintaining a sufficiently high signal-to-noise ratio. This degradation may confuse an ophthalmologist, reduce the examination quality, or defeat various image analysis algorithms. We propose an early work specially on deblurring sequential MSI images, which is distinguished from many of the current image deblurring techniques by resolving the blur kernel simultaneously for all the images in an MSI sequence. It is accomplished by incorporating several a priori constraints including the sharpness of the latent clear image, the spatial and temporal smoothness of the blur kernel and the similarity between temporally-neighboring images in MSI sequence. Specifically, we model the similarity between MSI images with mutual information considering the different wavelengths used for capturing different images in MSI sequence. The optimization of the proposed approach is based on a multi-scale framework and stepwise optimization strategy. Experimental results from 22 MSI sequences validate that our approach outperforms several state-of-the-art techniques in natural image deblurring.

Measuring Spectral Inconsistency of Multispectral Images for Detection and Segmentation of Retinal Degenerative Changes

Multispectral imaging (MSI) creates a series of en-face fundus spectral sections by leveraging an extensive range of discrete monochromatic light sources and allows for an examination of the retina’s early morphologic changes that are not generally visible with traditional fundus imaging modalities. An Ophthalmologist’s interpretation of MSI images is commonly conducted by qualitatively analyzing the spectral consistency between degenerated areas and normal ones, which characterizes the image variation across different spectra. Unfortunately, an ophthalmologist’s interpretation is practically difficult considering the fact that human perception is limited to the RGB color space, while an MSI sequence contains typically more than ten spectra. In this paper, we propose a method for measuring the spectral inconsistency of MSI images without supervision, which yields quantitative information indicating the pathological property of the tissue. Specifically, we define mathematically the spectral consistency as an existence of a pixel-specific latent feature vector and a spectrum-specific projection matrix, which can be used to reconstruct the representative features of pixels. The spectral inconsistency is then measured using the number of latent feature vectors required to reconstruct the representative features in practice. Experimental results from 54 MSI sequences show that our spectral inconsistency measurement is potentially invaluable for MSI-based ocular disease diagnosis.

AN ALGORITHM FOR PEDESTRIAN DETECTION IN MULTISPECTRAL IMAGE SEQUENCES

Abstract. The growing interest for self-driving cars provides a demand for scene understanding and obstacle detection algorithms. One of the most challenging problems in this field is the problem of pedestrian detection. Main difficulties arise from a diverse appearances of pedestrians. Poor visibility conditions such as fog and low light conditions also significantly decrease the quality of pedestrian detection. This paper presents a new optical flow based algorithm BipedDetet that provides robust pedestrian detection on a single-borad computer. The algorithm is based on the idea of simplified Kalman filtering suitable for realization on modern single-board computers. To detect a pedestrian a synthetic optical flow of the scene without pedestrians is generated using slanted-plane model. The estimate of a real optical flow is generated using a multispectral image sequence. The difference of the synthetic optical flow and the real optical flow provides the optical flow induced by pedestrians. The final detection of pedestrians is done by the segmentation of the difference of optical flows. To evaluate the BipedDetect algorithm a multispectral dataset was collected using a mobile robot.

Groupwise registration of sequential images from multispectral imaging (MSI) of the retina and choroid.

Multispectral Imaging (MSI) produces a sequence of discrete spectral slices that penetrate different light-absorbing species or chromophores and is a noninvasive technology useful for the early detection of various retinal, optic nerve and choroidal diseases. However, eye movement during the image acquisition process may introduce spatial misalignment between MSI images. This potentially causes trouble in the manual/automatic interpretation of MSI, but still remains an unresolved problem to this date. To deal with this MSI misalignment problem, we present a method on the groupwise registration of sequential images from MSI of the retina and choroid. The advantage of our algorithm is at least threefold: 1) simultaneous estimation of landmark correspondences and a parametric motion model via quadratic programming, 2) enforcement of temporal smoothness on the estimated motion, and 3) inclusion of a robust matching cost function. As validated in our experiments with a database of 22 MSI sequences, our algorithm outperforms two state-of-the-art registration techniques proposed originally in other domains. Our algorithm is potentially invaluable in ophthalmologists' clinical practice regarding various eye diseases.

An Adaptive PHD Filter for Multitarget Tracking with Multispectral Data Fusion

In order to improve the detection and tracking performance of multiple targets from IR multispectral image sequences, the approach based on spectral fusion algorithm and adaptive probability hypothesis density (PHD) filter is proposed. Firstly, the nonstationary adaptive suppression method is proposed to remove the background clutter. Based on the multispectral image sequence, the spectral fusion method is used to detect the abnormal targets. Spectral fusion produces the appropriate binary detection model and the computational probability of detection. Secondly, the particle filtering-based adaptive PHD algorithm is developed to detect and track multiple targets. This algorithm can deal with the nonlinear measurement on target state. In addition, the calculated probability of detection substitutes the fixed detection probability in PHD filter. Finally, the synthetic data sets based on various actual background images were utilized to validate the effectiveness of the detection approach. The results demonstrate that the proposed approach outperforms the conventional sequential PHD filtering in terms of detection and tracking performances.

Active cell and ECM movements during development.

Dynamic imaging of the extracellular matrix (ECM) and cells can reveal how tissues are formed. Displacement differences between cells and the adjacent ECM scaffold can be used to establish active movements of mesenchymal cells. Cells can also generate large-scale tissue movements in which cell and ECM displacements are shared. We describe computational methods for analyzing multi-spectral time-lapse image sequences. The resulting data can distinguish between local "active" cellular motion versus large-scale, tissue movements, both of which occur during organogenesis. The movement data also provide the basis for construction of realistic biomechanical models and computer simulations of in vivo tissue formation.

Optimized imaging of the postoperative spine.

Few tasks in imaging are more challenging than that of optimizing evaluations of the instrumented spine. The authors describe how applying fundamental and more advanced principles to postoperative spine computed tomography and magnetic resonance examinations mitigates the challenges associated with metal implants and significantly improves image quality and consistency. Newer and soon-to-be-available enhancements should provide improved visualization of tissues and hardware as multispectral imaging sequences continue to develop.

Comparison of Multispectral Image Processing Techniques to Brain MR Image Classification

Brain Magnetic Resonance Imaging (MRI) has become a widely used modality because it produces multispectral image sequences that provide information of free water, proteinaceous fluid, soft tissue and other tissues with a variety of contrast. The abundance fractions of tissue signatures provided by multispectral images can be very useful for medical diagnosis compared to other modalities. Multiple Sclerosis (MS) is thought to be a disease in which the patient immune system damages the isolating layer of myelin around the nerve fibers. This nerve damage is visible in Magnetic Resonance (MR) scans of the brain. Manual segmentation is extremely time-consuming and tedious. Therefore, fully automated MS detection methods are being developed which can classify large amounts of MR data, and do not suffer from inter observer variability. In this paper we use standard fuzzy c-means algorithm (FCM) for multi-spectral images to segment patient MRI data. Geodesic Active Contours of Caselles level set is another method we implement to do the brain image segmentation jobs. And then we implement anther modified Fuzzy C-Means algorithm, where we call Bias-Corrected FCM as BCFCM, for bias field estimation for the same thing. Experimental results show the success of all these intelligent techniques for brain medical image segmentation.

Dim moving target detection in multispectral IR image sequence based on GLRT

The scope of this paper addressed the problem of detecting a dim moving target from a sequence of multispectral IR cubes.The detection problem was formulated in a general framework,assuming unknown target amplitude,position and velocity.This composite hypothesis testing problem was approached by means of the generalized likelihood ratio test(GLRT) theory.The detector structure and its actual implementation based on velocity filters were discussed in detail.Approximated expressions of the false alarm and detection probabilities were obtained and validated by means of simulation.To test the effectiveness of the detection algorithm,the detection results obtained on a set of synthetic multispectral IR image sequences were presented and discussed.These results indicate that the algorithm proposed can obtain a good performance on dim target detection with low SNR.

Multispectral imaging of the ocular fundus using light emitting diode illumination

We present an imaging system based on light emitting diode (LED) illumination that produces multispectral optical images of the human ocular fundus. It uses a conventional fundus camera equipped with a high power LED light source and a highly sensitive electron-multiplying charge coupled device camera. It is able to take pictures at a series of wavelengths in rapid succession at short exposure times, thereby eliminating the image shift introduced by natural eye movements (saccades). In contrast with snapshot systems the images retain full spatial resolution. The system is not suitable for applications where the full spectral resolution is required as it uses discrete wavebands for illumination. This is not a problem in retinal imaging where the use of selected wavelengths is common. The modular nature of the light source allows new wavelengths to be introduced easily and at low cost. The use of wavelength-specific LEDs as a source is preferable to white light illumination and subsequent filtering of the remitted light as it minimizes the total light exposure of the subject. The system is controlled via a graphical user interface that enables flexible control of intensity, duration, and sequencing of sources in synchrony with the camera. Our initial experiments indicate that the system can acquire multispectral image sequences of the human retina at exposure times of 0.05 s in the range of 500-620 nm with mean signal to noise ratio of 17 dB (min 11, std 4.5), making it suitable for quantitative analysis with application to the diagnosis and screening of eye diseases such as diabetic retinopathy and age-related macular degeneration.

A Generalized Likelihood Ratio Test for Detecting Land Mines Using Multispectral Images

In this letter, we address the problem of detecting anomalies in multispectral images that are due to the presence of land mines. The proposed detection scheme follows from a generalized likelihood ratio test (GLRT) involving a mixture of certain probability density functions. More specifically, the GLRT is based on the assumption that the pixel values of a subblock within a single spectral-plane image can be modeled as a mixture of two Gaussian density functions (model distribution of background pixels) and a uniform density function (model distribution of anomalous pixels when they are present). We extend the single spectral-plane GLRT to the multiple spectral-plane case where the subblocks are from multispectral images. To assess the proposed GLRT, which we call the Gaussian-uniform GLRT (GU-GLRT) algorithm, we applied the GU-GLRT and popular RX algorithms to images from a real multispectral image sequence and used receiver operating characteristic (ROC) curves as the figure of merit. In the experiments that we conducted, the GU-GLRT outperformed the RX algorithm in the sense that the area under the ROC curve was greatest for the GU-GLRT algorithm.

Automatic Registration and Mosaicking for Airborne Multispectral Image Sequences

Airborne remote sensing has important applications in agriculture monitoring because of the flexibility of system deployment. The major obstacle in practical use is its high cost. To reduce the cost, a multispectral system can be assembled by using individual cameras onboard a small aerial platform, such as a miniature unmanned aerial vehicle (miniUAV). In such a case, the cameras may have shifting and rotational misalignment, even after careful adjustment. Contiguous frames are captured as the platform flies. So multi-band registration within a single frame and frame-toframe mosaicking are necessary to obtain a co-registered multispectral image for the entire monitoring area before any commercial product can be generated to support practical decision-making. In this paper, we present automatic algorithms to achieve this goal. These algorithms are particularly useful to the image scenes where no distinctive features are available. Both automatic and manual evaluations confirm the effectiveness of the developed algorithms in multi-sensor data fusion for overall flat terrain without distinctive features.

Contextual reconstruction of cloud-contaminated multitemporal multispectral images

The frequent presence of clouds in passive remotely sensed imagery severely limits its regular exploitation in various application fields. Thus, the removal of cloud cover from this imagery represents an important preprocessing task consisting in the reconstruction of cloud-contaminated data. The intent of this study is to propose two novel general methods for the reconstruction of areas obscured by clouds in a sequence of multitemporal multispectral images. Given a cloud-contaminated image of the sequence, each area of missing measurements is reconstructed through an unsupervised contextual prediction process that reproduces the local spectro-temporal relationships between the considered image and an opportunely selected subset of the remaining temporal images. In the first method, the contextual prediction process is implemented by means of an ensemble of linear predictors, each trained over a local multitemporal region that is spectrally homogeneous in each temporal image of the selected subset. In order to obtain such regions, each temporal image is locally classified by an unsupervised classifier based on the expectation-maximization (EM) algorithm. In the second method, the local spectro-temporal relationships are reproduced by a single nonlinear predictor based on the support vector machines (SVM) approach. To illustrate the performance of the two proposed methods, an experimental analysis on a sequence of three temporal images acquired by the Landsat-7 Enhanced Thematic Mapper Plus sensor over a total period of four months is reported and discussed. It includes a detailed simulation study that aims at assessing with different reconstruction quality criteria the accuracy of the methods in different qualitative and quantitative cloud contamination conditions. Compared with two techniques based on compositing algorithms for cloud removal, the proposed methods show a clear superiority, which makes them a promising and useful tool in solving the considered problem, whose great complexity is commensurate with its practical importance.