Multiscale Morphological Operators Research Articles

Impact assessment of humanitarian demining using object-based peri-urban land cover classification and morphological building detection from VHR Worldview imagery

This study utilised very high-resolution (VHR) imagery to monitor and evaluate the impact of humanitarian demining activities in a peri-urban environment in Afghanistan. Identifying buildings and mapping the spatial distribution of different types of land cover is of great practical significance for demining organisations, such as the HALO Trust, to assess the impact of their mine-clearance activities, by quantifying change and the growth of a population in a specific area over time. This study had two main objectives: (i) to map the post-clearance land cover, and (ii) to detect and quantify the change in the number and area of buildings. Two independent workflows were implemented and evaluated. To map land cover, this study investigated the implementation of various machine learning algorithms in object-based image classification (OBIA) of VHR satellite imagery (Worldview-1,2,3). Image segmentation was carried out using the Large-Scale Mean-Shift (LSMS) algorithm, before classification was performed based on a machine learning Random Forests (RF) approach. Different parameters and spatial distribution of training samples were tested to analyse the model's performance. Further analysis determined that by using only the segments' mean value per spectral band (Red, Green, Blue), data redundancy in the training stage was eliminated. The final classified map had an overall accuracy of 90.67% and a total built-area of 643,660.28 m2 was detected in the 4.11 km2 study area. To detect and quantify buildings present in the study area, an alternative, automatic, unsupervised approach based on the morphological building index (MBI) was implemented using MATLAB. Two VHR (0.5 m) panchromatic images acquired by WorldView-1 in 2008 and 2018 were processed using a series of multi-scale and multi-directional morphological operators, before a series of post-processing thresholds were applied to refine the output. Parameters were systematically optimised for the datasets and their sensitivity analysed. By comparing the output to manually labelled reference maps, producer's and user's accuracy of 78.63/81.70% and 75.30/77.91% were attained for 2008 and 2018 imagery respectively. The built-area was found to increase from 58,705 to 611,920 m2 over the ten-year period, indicating a significant increase in the number of people resident in the area. Whilst OBIA was found to be more accurate than MBI for the 2018 pansharpened imagery, with producer's and user's accuracy of 91.00/90.60% versus 84.65/79.44% respectively, it must be considered that the MBI, an unsupervised method requiring no training, offers a fast solution if the sole objective is building detection.

Derivative and enhanced discrete analytic wavelet algorithm for rolling bearing fault diagnosis

In order to effectively analyze the fault signals of non-linear and non-stationary rolling bearings and obtain high-precision fault diagnosis results of rolling bearings, the derivative and enhanced discrete analytic wavelet algorithm for rolling bearing fault diagnosis is studied. The morphological spectrum of fault signal is obtained by multi-scale morphological opening operation. The morphological spectrum entropy is calculated from the morphological spectrum curve to describe the morphological characteristics of different signals of rolling bearing. The morphological features of the obtained signals are decomposed by approximate analytic complex wavelet transform, rearrangement of wavelet packet subspace and cross-combination of wavelet packet subspace. The derivative and enhanced discrete analytic wavelet algorithm is used to diagnose rolling bearing faults. The experimental results show that the algorithm can accurately diagnose the fault of rolling bearings, and can completely diagnose 10 specific faults of rolling bearings. The error of diagnosing the diameter of pits is less than 0.1%. It can be applied to the actual fault diagnosis of rolling bearings.

Inhomogeneous morphological PDEs for robust and adaptive image shock filters

Classical morphological filters suffer from well performing in a noisy environment, and intrinsic image structures are not taken into account. The authors propose here an alternative to overcome such weaknesses, by properly using robust shock filters and inhomogeneity. Thus, they obtain multiscale morphological operators by using image edge functions as local weights in inhomogeneous Hamiltonians in classical multiscale dilations/erosions formulated with partial differential equations (PDEs). They provide the equivalent sup-inf-based formulations, and derive sharpening/enhancement methods. In addition, they establish the PDE associated with the asymptotical iterations of the proposed robust and adaptive filters. The good behaviours of the proposed sup-inf and PDE-based methods are illustrated on synthetic, greyscale, and colour images; results are analysed both qualitatively and quantitatively.

Multi-Level Interpolation-Based Filter for Airborne LiDAR Point Clouds in Forested Areas

Over the past decades, plenty of filtering algorithms have been presented to distinguish ground and non-ground points from airborne LiDAR point clouds. However, with the existing methods, it is difficult to derive satisfactory filtering results on rugged terrains with dense vegetation due to the low-level penetration ability of laser pulses. Therefore, a multi-level interpolation-based filter is developed in this paper. The novelty of the algorithm lies within its usage of multi-scale morphological operations and robust z-score to correctly select ground seeds as more as possible, and a terrain-adaptive residual threshold to adapt to various terrain characteristics. Rural samples provided by International Society for Photogrammetry and Remote Sensing (ISPRS) were employed to assess the performance of the proposed method and its results were compared with 15 filtering algorithms developed in recent 5 years (2015-2019). Results show that the proposed method with the optimized parameters produces the best accuracy with the average total error and kappa coefficient of 1.89% and 87.88%, respectively. We further filtered high-density point clouds in six forested areas with different vegetation covers and terrain slopes. Results demonstrate that the proposed algorithm is more accurate than the well-known filtering methods including morphological-based filter, progressive TIN densification filter (PTD), improved PTD and cloth simulation filter, with the average total error decreased by 26.2%, 19.9%, 3.8% and 40.4%, respectively. Moreover, the DEMs of the proposed method have lower average root mean square errors than the four classical filters. Therefore, the proposed method can be considered as an effective ground filtering algorithm for airborne LiDAR point clouds in forested areas.

Segmentation of pulmonary vascular tree by incorporating vessel enhancement filter and variational region-growing.

Automatic segmentation of pulmonary vascular tree in the thoracic computed tomography (CT) image is a promising but challenging task with great clinical potential values. It is difficult to segment the whole vascular tree in reasonable time and acceptable accuracy. To develop a novel pulmonary vessel segmentation approach by incorporating vessel enhancement filters and the anisotropic diffusion filter with the variational region growing. First, the airway wall from the lung lobes is eliminated from CT images by using multi-scale morphological operations. Second, a Hessian-based multi-scale vesselness filter and medialness filter are applied to detect and enhance the potential vessel. Third, an anisotropic diffusion filter is used to remove noise and enhance the tube-like structures in CT images. Last, the vascular tree is segmented by applying variational region growing algorithm. Applying to the CT images collected from the entire dataset of VESSEL12 challenge, we achieved an average sensitivity of 92.9%, specificity of 91.6% and the area under the ROC curve of AUC = 0.972. This study demonstrated feasibility of segmenting the pulmonary vessel effectively by incorporating vessel enhancement filters and the anisotropic diffusion filter with the variational region growing algorithm. Our method cannot only segment both large and peripheral vessels, but also distinguish the vessels from the adjacent tissues, especially the airway walls.

A Multi-Scale Filtering Building Index for Building Extraction in Very High-Resolution Satellite Imagery

Building extraction plays a significant role in many high-resolution remote sensing image applications. Many current building extraction methods need training samples while it is common knowledge that different samples often lead to different generalization ability. Morphological building index (MBI), representing morphological features of building regions in an index form, can effectively extract building regions especially in Chinese urban regions without any training samples and has drawn much attention. However, some problems like the heavy computation cost of multi-scale and multi-direction morphological operations still exist. In this paper, a multi-scale filtering building index (MFBI) is proposed in the hope of overcoming these drawbacks and dealing with the increasing noise in very high-resolution remote sensing image. The profile of multi-scale average filtering is averaged and normalized to generate this index. Moreover, to fully utilize the relatively little spectral information in very high-resolution remote sensing image, two scenarios to generate the multi-channel multi-scale filtering index (MMFBI) are proposed. While no high-resolution remote sensing image building extraction dataset is open to the public now and the current very high-resolution remote sensing image building extraction datasets usually contain samples from the Northern American or European regions, we offer a very high-resolution remote sensing image building extraction datasets in which the samples contain multiple building styles from multiple Chinese regions. The proposed MFBI and MMFBI outperform MBI and the currently used object based segmentation method on the dataset, with a high recall and F-score. Meanwhile, the computation time of MFBI and MBI is compared on three large-scale very high-resolution satellite image and the sensitivity analysis demonstrates the robustness of the proposed method.

An Active Learning Method for DEM Extraction From Airborne LiDAR Point Clouds

Airborne Light Detection and Ranging (LiDAR) is a popular active remote sensing technology that has been developing very rapidly in recent years. To solve the problems of low filtering accuracy of airborne LiDAR point clouds in complex terrain environments and avoiding too much human intervention, this paper proposes a point cloud filtering method based on active learning. In the proposed method, the initial training samples are acquired and marked automatically by multi-scale morphological operations. In so doing, no training samples are selected and labeled manually, i.e., the training samples are added gradually according to the oracle used in active learning. In this paper, the oracle is set to a sigmoid function of residuals from the points to the fitted surface. Subsequently, the training model is revised progressively using the updated training samples. Finally, the classification results are further optimized by a slope-based method. Three datasets with different filtering challenges provided by the International Society for Photogrammetry and Remote Sensing (ISPRS) were used to test the proposed method. Comparing with the other ten famous filtering methods, the proposed method can achieve the smallest average total error (5.51%). Thus, it can be concluded that the proposed method performs very well toward different terrain environments.

Efficient implementation of morphological index for building/shadow extraction from remotely sensed images

Morphological building index (MBI) and morphological shadow index (MSI) are recently developed techniques that aim at automatically detect buildings/shadows using high-resolution remotely sensed imagery. The traditional mathematical morphology operations are usually time-consuming as they are based on the consideration of a wide range of image-object properties, such as brightness, contrast, shapes, sizes, and in the application of series of repeated transformations (e.g., classical opening and closing operators). In the case of MBI and MSI, the computational complexity is also increased due to the use of multiscale and multidirectional morphological operators. In this paper, we provide a computationally efficient implementation of MBI and MSI algorithms which is specifically developed for commodity graphic processing units using NVIDIA CUDA. We perform the evaluation of the parallel version of the algorithms using two different NVIDIA architectures and three widely used hyperspectral data sets. Experimental results show that the computational burden introduced when considering multidirectional morphological operators can be almost completely removed by the developed implementations.

Segmentation of the pulmonary vascular trees in 3D CT images using variational region-growing

ObjectivesThe long-term goal of this project is to quantify the aeration of lung parenchyma in 3D CT scans of patients with acute respiratory distress syndrome. This task requires lung delineation, as well as elimination of airways and vessels. The objective of this article was to present and evaluate the method used to segment out the vascular trees. Materials and methodsVascular trees are segmented by variational region growing. This process is performed within a lung mask, where the airways and bronchial walls were previously eliminated by adaptive multi-scale morphological operations. The region growth starts from seeds defined as the most salient points on a “vesselness” map. The “vesselness” function based on the eigenvalues of the Hessian matrix is also used in the region descriptor that controls variational region growing. The formulation of this descriptor, as well as the method used to eliminate the bronchial walls, are the original contributions of this work. The method was evaluated using the full set of 20 chest scans from the VESSEL12 challenge framework. ResultsOverall specificity of 0.938 and sensitivity of 0.772 were achieved. The method successfully differentiated vessels from bronchial walls (specificity, 0.848) but failed to detect the smallest vessels (sensitivity, 0.418). ConclusionTo the best of our knowledge, similar formulations of variational region growing have never been used to segment pulmonary vascular trees. The method seems to be suitable for the intended application, although its validation on actual images with acute respiratory distress syndrome remains to be done.

Super Resolution Image Reconstruction Through Bregman Iteration Using Morphologic Regularization

Multiscale morphological operators are studied extensively in the literature for image processing and feature extraction purposes. In this paper, we model a nonlinear regularization method based on multiscale morphology for edge-preserving super resolution (SR) image reconstruction. We formulate SR image reconstruction as a deblurring problem and then solve the inverse problem using Bregman iterations. The proposed algorithm can suppress inherent noise generated during low-resolution image formation as well as during SR image estimation efficiently. Experimental results show the effectiveness of the proposed regularization and reconstruction method for SR image.

Video-based person recognition using fovea intensity comparison code

This article proposes a feature extraction method for automatic person recognition in video. The method proposed by Viola and Jones (Viola, P. and Jones, M., 2001. Rapid object detection using a boosted cascade of simple features. In: IEEE international conference on computer vision and pattern recognition (CVPR 2001), Kauai Marriott, Hawaii, Vol. 1, 511–518) is used to detect the face region. Face region is processed in YC b C r colour space to determine the locations of the eyes. The centre of the mouth is determined relative to the locations of the eyes. Facial and mouth features are extracted using multiscale morphological erosion and dilation operations, respectively. The facial features are extracted relative to the locations of the eyes, and mouth features are extracted relative to the locations of the eyes and mouth. Fovea intensity comparison code and exclusive-OR operation for matching are used to recognise a person in video sequences. Projected fovea intensity comparison code (PFICC) and Euclidean distance for matching are also used to recognise a person in video sequences. The performance of the system using PFICC is evaluated in real time in the laboratory environment, and the system achieves a recognition rate (RR) of 99.0% and an equal error rate (EER) of about 0.84% for 50 subjects. The performance of the system is also evaluated for the eXtended Multi Modal Verification for Teleservices and Security (XM2VTS) database, and the system achieves an RR of 100% and an EER of about 0.23%.

Recognition of emotions from video using neural network models

In this paper, facial features from the video sequence are explored for characterizing the emotions. The emotions considered for this study are Anger, Fear, Happy, Sad and Neutral. For carrying out the proposed emotion recognition study, the required video data is collected from the studio, Center for Education Technology (CET), at Indian Institute of Technology (IIT) Kharagpur. The dynamic nature of the grey values of the pixels within the eye and mouth regions are used as the features to capture the emotion specific knowledge from the facial expressions. Multiscale morphological erosion and dilation operations are used to extract features from eye and mouth regions, respectively. The features extracted from left eye, right eye and mouth regions are used to develop the separate models for each emotion category. Autoassociative neural network (AANN) models are used to capture the distribution of the extracted features. The developed models are validated using subject dependent and independent emotion recognition studies. The overall performance of the proposed emotion recognition system is observed to be about 87%.

Real time face and mouth recognition using radial basis function neural networks

This paper presents a method for automatic real time face and mouth recognition using radial basis function neural networks (RBFNN). The proposed method uses the motion information to localize the face region, and the face region is processed in YC r C b color space to determine the locations of the eyes. The center of the mouth is determined relative to the locations of the eyes. Facial and mouth features are extracted using multiscale morphological erosion and dilation operations, respectively. The facial features are extracted relative to the locations of the eyes, and mouth features are extracted relative to the locations of the eyes and mouth. The facial and mouth features are given as input to radial basis function neural networks. The RBFNN is used to recognize a person in video sequences using face and mouth modalities. The evidence from face and mouth modalities are combined using a weighting rule, and the result is used for identification and authentication. The performance of the system using facial and mouth features is evaluated in real time in the laboratory environment, and the system achieves a recognition rate (RR) of 99.0% and an equal error rate (EER) of about 0.73% for 50 subjects. The performance of the system is also evaluated for XM2VTS database, and the system achieves a recognition rate (RR) of 100% an equal error rate (EER) of about 0.25% for 50 subjects.

Multiscale Morphological Operators and Geodesic Distance applied to Computed Axial Tomography Segmentation

Lately, several diagnostic and therapeutic procedures for mediastinal tumors and masses are done by videothoraxcoscopic techniques. Consequently, Computed Axial Tomography (C.A.T.) become a fundamental tool in tumors and masses diagnostic, especially to locate and measure them with accuracy and precision. This paper presents a segmentation method in C.A.T. images through techniques based in Mathematical Morphology. A semi-automatic method for structure segmentation based on Multiscale Morphological Operators is introduced. In addition to filtering and enhancing, this method capture the interior of a closed simple curve by the geodesic distance. These curves represents the external boundary of the different mediastinal structures. This method allows to calculate exactly the external boundary of the segmented structure and also the delimitated area.

Multimodal person authentication using speech, face and visual speech

This paper presents a method for automatic multimodal person authentication using speech, face and visual speech modalities. The proposed method uses the motion information to localize the face region, and the face region is processed in YC r C b color space to determine the locations of the eyes. The system models the nonlip region of the face using a Gaussian distribution, and it is used to estimate the center of the mouth. Facial and visual speech features are extracted using multiscale morphological erosion and dilation operations, respectively. The facial features are extracted relative to the locations of the eyes, and visual speech features are extracted relative to the locations of the eyes and mouth. Acoustic features are derived from the speech signal, and are represented by weighted linear prediction cepstral coefficients (WLPCC). Autoassociative neural network (AANN) models are used to capture the distribution of the extracted acoustic, facial and visual speech features. The evidence from speech, face and visual speech models are combined using a weighting rule, and the result is used to accept or reject the identity claim of the subject. The performance of the system is evaluated for newsreaders in TV broadcast news data, and the system achieves an equal error rate (EER) of about 0.45% for 50 subjects.

Frontal face authentication using morphological elastic graph matching

A novel dynamic link architecture based on multiscale morphological dilation-erosion is proposed for frontal face authentication. Instead of a set of Gabor filters tuned to different orientations and scales, multiscale morphological operations are employed to yield a feature vector at each node of the reference grid. Linear projection algorithms for feature selection and automatic weighting of the nodes according to their discriminatory power succeed to increase the authentication capability of the method. The performance of the morphological dynamic link architecture is evaluated in terms of the receiver operating characteristic in the M2VTS face image database. The comparison with other frontal face authentication algorithms indicates that the morphological dynamic link architecture with discriminatory power coefficients is the best algorithm with respect to the equal error rate achieved.

Frontal face authentication using discriminating grids with morphological feature vectors

A novel elastic graph matching procedure based on multiscale morphological operations, the so called morphological dynamic link architecture, is developed for frontal face authentication. Fast algorithms for implementing mathematical morphology operations are presented. Feature selection by employing linear projection algorithms is proposed. Discriminatory power coefficients that weigh the matching error at each grid node are derived. The performance of morphological dynamic link architecture in frontal face authentication is evaluated in terms of the receiver operating characteristic on the M2VTS face image database. Preliminary results for face recognition using the proposed technique are also presented.

Differential morphology and image processing

Image processing via mathematical morphology has traditionally used geometry to intuitively understand morphological signal operators and set or lattice algebra to analyze them in the space domain. We provide a unified view and analytic tools for morphological image processing that is based on ideas from differential calculus and dynamical systems. This includes ideas on using partial differential or difference equations (PDEs) to model distance propagation or nonlinear multiscale processes in images. We briefly review some nonlinear difference equations that implement discrete distance transforms and relate them to numerical solutions of the eikonal equation of optics. We also review some nonlinear PDEs that model the evolution of multiscale morphological operators and use morphological derivatives. Among the new ideas presented, we develop some general 2-D max/min-sum difference equations that model the space dynamics of 2-D morphological systems (including the distance computations) and some nonlinear signal transforms, called slope transforms, that can analyze these systems in a transform domain in ways conceptually similar to the application of Fourier transforms to linear systems. Thus, distance transforms are shown to be bandpass slope filters. We view the analysis of the multiscale morphological PDEs and of the eikonal PDE solved via weighted distance transforms as a unified area in nonlinear image processing, which we call differential morphology, and briefly discuss its potential applications to image processing and computer vision.

Implementing continuous-scale morphology via curve evolution

A new approach to digital implementation of continuous-scale mathematical morphology is presented. The approach is based on discretization of evolution equations associated with continuous multiscale morphological operations. Those equations, and their corresponding numerical implementation, can be derived either directly from mathematical morphology definitions or from curve evolution theory. The advantages of the proposed approach over the classical discrete morphology are demonstrated.

Measuring the Fractal Dimension of Signals: Morphological Covers and Iterative Optimization

Fractals can model many classes of time-series data. The fractal dimension is an important characteristic of fractals that contains information about their geometrical structure at multiple scales. The covering methods are a class of efficient approaches to measure the fractal dimension of an arbitrary fractal signal by creating multiscale covers around the signal’s graph. In this paper we develop a general method that uses multiscale morphological operations with varying structuring elements to unify and extend the theory and digital implementations of covering methods. It is theoretically established that, for the fractal dimension computation, covering one-dimensional signals with planar sets is equivalent to morphologically transforming the signal by one-dimensional functions, which reduces the computational complexity from quadratic in the signal’s length to linear. Then a morphological covering algorithm is developed and applied to discrete-time signals synthesized from Weierstrass functions, fractal interpolation functions, and fractional Brownian motion. Further, for deterministic parametric fractals depending on a single parameter related to their dimension, we develop an optimization method that starts from an initial estimate and iteratively con- verges to the true fractal dimension by searching in the parameter space and minimizing a distance between the original signal and all such signals from the same class. Experimental results are also provided to demonstrate the good performance of the developed methods.