Multiscale Shape Research Articles

Online Rain/Snow Removal From Surveillance Videos.

Video rain/snow removal from surveillance videos is an important task in the computer vision community since rain/snow existed in videos can severely degenerate the performance of many surveillance system. Various methods have been investigated extensively, but most only consider consistent rain/snow under stable background scenes. Rain/snow captured from practical surveillance camera, however, is always highly dynamic in time, and those videos also include occasionally transformed background scenes and background motions caused by waving leaves or water surfaces. To this issue, this paper proposes a novel rain/snow removal approach, which fully considers dynamic statistics of both rain/snow and background scenes taken from a video sequence. Specifically, the rain/snow is encoded as an online multi-scale convolutional sparse coding (OMS-CSC) model, which not only finely delivers the sparse scattering and multi-scale shapes of real rain/snow, but also well distinguish the components of background motion from rain/snow layer. The real-time ameliorated parameters in the model well encodes their temporally dynamic configurations. Furthermore, a transformation operator imposed on the background scenes is further embedded into the proposed model, which finely conveys the background transformations, such as rotations, scalings and distortions, inevitably existed in a real video sequence. The approach so constructed can naturally better adapt to the dynamic rain/snow as well as background changes, and also suitable to deal with the streaming video attributed its online learning mode. The proposed model is formulated in a concise maximum a posterior (MAP) framework and is readily solved by the alternating direction method of multipliers (ADMM). Compared with the state-of-the-art online and offline video rain/snow removal methods, the proposed method achieves best performance on synthetic and real videos datasets both visually and quantitatively. Specifically, our method can be implemented in relatively high efficiency, showing its potential to real-time video rain/snow removal. The code page is at: https://github.com/MinghanLi/OTMSCSC_matlab_2020.

Concurrent multi-scale shape optimization for micro and macro shape design of structures

Concurrent multi-scale shape optimization for micro and macro shape design of structures

Morphological Expression Model for Geographic Elements Based on the Shape-Tree Principle

In recent years, with the continuous development of online geographic information services, the multiscale online expression of geographic elements and topological information has become a hot research topic. Currently, a large amount of research focuses mainly on the compression of vector line elements. The main shortcoming of current algorithms is that vector data compression does not consider the optimality of line simplification. In the face of terabytes or even petabytes of data, problems arise such as low processing efficiency and long transmission time. Based on the shape-tree principle, the proposed method aims to reconstruct the organizational structure of a geographic element and establish a hierarchical representation model of the element based on its structure, realizing multiscale shape recognition and matching and searching geographic elements. Finally, the proposed method is compared with a model based on the wavelet transform and discrete Fourier transform. The results demonstrate the applicability and effectiveness of the proposed method in morphological expression with geospatial big data.

How the forest interacts with the trees: Multiscale shape integration explains global and local processing.

Hierarchical stimuli have been widely used to study global and local processing. Two classic phenomena have been observed using these stimuli: the global advantage effect (we identify the global shape faster) and an interference effect (we identify shape slower when the global and local shapes are different). Because these phenomena have been observed during shape categorization tasks, it is unclear whether they reflect the categorical judgment or the underlying shape representation. Understanding the underlying shape representation is also critical because both global and local processing are modulated by stimulus properties.We performed two experiments to investigate these issues. In Experiment 1, we show that these phenomena can be observed in a same-different task, and that participants show systematic variation in response times across image pairs. We show that the response times to any pair of images can be accurately predicted using two factors: their dissimilarity and their distinctiveness relative to other images. In Experiment 2, we show that these phenomena can also be observed in a visual search task where participant did not have to make any categorical shape judgments. Here too, participants showed highly systematic variations in response time that could be explained as a linear sum of shape comparisons across global and local scales. Finally, the dissimilarity and distinctiveness factors estimated from the same-different task were systematically related to the search dissimilarities observed during visual search.In sum, our results show that global and local processing phenomena are properties of a systematic shape representation governed by simple rules.

Characterizing the spatial and temporal variation of the land surface temperature hotspots in Wuhan from a local scale

ABSTRACT Land Surface Temperature (LST) derived from space-borne Thermal-infrared (TIR) sensors is a key parameter of urban climate studies. Current studies are inefficient to capture the spatial and temporal variations of LST for only one snapshot adopted at one time. Focusing on the characterization of the spatial and temporal of LST variations at local scales, the latent patterns, and morphological characteristics are extracted in this study. Technically, sixteen MODerate-resolution Imaging Spectroradiometer (MODIS) eight-day synthesized LST products (MYD11A2) in 2002, 2007, 2012, and 2017 are employed. First, the non-parametric Multi-Task Gaussian Process Model (MTGP) is used to extract the smooth and continuous Latent LST (LLST) patterns using one LST subset and its temporally adjacent images. Second, the Multi-Scale Shape Index (MSSI) is then applied to quantify the morphological characteristics at the optimal scale. Then, the LLST patterns and MSSI maps are clustered into multiple spatial categories. The specific clusters with the highest LLST and MSSI values are considered as local LLST hotspots. The Hotspots Weighted Mean Center (HSWMC) and standard deviation ellipse are adopted to further investigate the spatiotemporal change of hotspots orientation, direction, and trajectories. Results revealed that Impervious Surfaces (IS) composition is the most significant external forcing of local LST anomalies. The configuration factors (e.g., shape index, aggregation index) also have a noticeable local warming effect. This study represents a latent pattern and morphology-based framework for LST hotspots spatial and temporal variations characterization, catering to the zoning and grading strategies in urban planning.

Interacting with Self-Similarity

Shape similarity is a fundamental problem in geometry processing, enabling applications such as surface correspondence, segmentation, and edit propagation. For example, a user may paint a stroke on one finger of a model and desire the edit to propagate to all fingers. Automatic approaches have difficulty matching user expectations, either due to an algorithm’s inability to guess the scale at which the user is intending to edit or due to underlying deficiencies in the similarity metric (e.g., semantic information not present in the geometry).We propose an approach to interactively design self-similarity maps. We investigate two primitive operations, useful in a variety of scenarios: region and curve similarity. Users select example similar and dissimilar regions. Starting with an automatically generated multi-scale shape signature, our approach solves for a scale parameter and thresholds that group the example regions as specified. We propose a new Smooth Shape Diameter Signature (SSDS) as a more efficient alternative to the Heat or Wave Kernel Signature. If no such parameters can be found, our approach modifies the shape signature itself. Given a curve drawn on the surface, we perform hybrid discrete/continuous optimization to find similar curves elsewhere.We apply our approach for interactive editing scenarios: propagating mesh geometry, patterns duplication, and segmentation.

A deep learning AlexNet model for classification of red blood cells in sickle cell anemia

Sickle cell anemia (SCA) is a serious hematological disorder, where affected patients are frequently hospitalized throughout a lifetime and even can cause death. The manual method of detecting and classifying abnormal cells of SCA patient blood film through a microscope is time-consuming, tedious, prone to error, and require a trained hematologist. The affected patient has many cell shapes that show important biomechanical characteristics. Hence, having an effective way of classifying the abnormalities present in the SCA disease will give a better insight into managing the concerned patient's life. This work proposed algorithm in two-phase firstly, automation of red blood cells (RBCs) extraction to identify the RBC region of interest (ROI) from the patient’s blood smear image. Secondly, deep learning AlexNet model is employed to classify and predict the abnormalities presence in SCA patients. The study was performed with (over 9,000 single RBC images) taken from 130 SCA patient each class having 750 cells. To develop a shape factor quantification and general multiscale shape analysis. We reveal that the proposed framework can classify 15 types of RBC shapes including normal in an automated manner with a deep AlexNet transfer learning model. The cell's name classification prediction accuracy, sensitivity, specificity, and precision of 95.92%, 77%, 98.82%, and 90% were achieved, respectively.

A generalized phase field multiscale finite element method for brittle fracture

SUMMARYA generalized multiscale finite element method is introduced to address the computationally taxing problem of elastic fracture across scales. Crack propagation is accounted for at the microscale utilizing phase field theory. Both the displacement‐based equilibrium equations and phase field state equations at the microscale are mapped on a coarser scale. The latter is defined by a set of multinode coarse elements, where solution of the governing equations is performed. Mapping is achieved by employing a set of numerically derived multiscale shape functions. A set of representative benchmark tests is used to verify the proposed procedure and assess its performance in terms of accuracy and efficiency compared with the standard phase field finite element implementation.

Multi-Scale Shape Index-Based Local Binary Patterns for Texture Classification

To enhance the weakness of Local Binary Pattern (LBP) and its state-of-the-art variants, this letter presents a new variant of the local concave microstructure pattern (LCvMSP). The proposed multi-scale shape index based texture descriptor is named as SI-LCvMSP. Contrarily to the original LBP and LCvMSP, SI-LCvMSP uses the shape index instead of the original texture image in the kernel function. The shape index is a differential calculation and it can be calculated from local second-order derivatives of texture images. It captures microstructure and macrostructure texture information mathematically. As textural features, we use multi-scale and multi-resolution shape index information as well as rotation-invariant uniform LBP. Thus, we obtain the discriminative feature representation schema to construct cross-scale joint coding. The proposed method has a high discriminability and is less sensitive to image transforms such as rotation and illumination. Experimental results show that the SI-LCvMSP descriptor can improve classification accuracy.

Numerical Upscaling of Perturbed Diffusion Problems

In this paper we study elliptic partial differential equations with rapidly varying diffusion coefficient that can be represented as a perturbation of a reference coefficient. We develop a numerical method for efficiently solving multiple perturbed problems by reusing local computations performed with the reference coefficient. The proposed method is based on the Petrov--Galerkin Localized Orthogonal Decomposition (PG-LOD) which allows for straightforward parallelization with low communcation overhead and memory consumption. We focus on two types of perturbations: local defects which we treat by recomputation of multiscale shape functions and global mappings of a reference coefficient for which we apply the domain mapping method. We analyze the proposed method for these problem classes and present several numerical examples.

A Unified Deep Metric Representation for Mesh Saliency Detection and Non-Rigid Shape Matching

In this paper, we propose a deep metric for unifying the representation of mesh saliency detection and non-rigid shape matching. While saliency detection and shape matching are two closely related and fundamental tasks in shape analysis, previous methods approach them separately and independently, failing to exploit their mutually beneficial underlying relationship. In view of the existing gap between saliency and matching, we propose to solve them together using a unified metric representation of surface meshes. We show that saliency and matching can be rigorously derived from our representation as the principal eigenvector and the smoothed Laplacian eigenvectors respectively. Learning the representation jointly allows matching to improve the deformation-invariance of saliency while allowing saliency to improve the feature localization of matching. To parameterize the representation from a mesh, we also propose a deep recurrent neural network (RNN) for effectively integrating multi-scale shape features and a soft-thresholding operator for adaptively enhancing the sparsity of saliency. Results show that by jointly learning from a pair of saliency and matching datasets, matching improves the accuracy of detected salient regions on meshes, which is especially obvious for small-scale saliency datasets, such as those having one to two meshes. At the same time, saliency improves the accuracy of shape matchings among meshes with reduced matching errors on surfaces.

Laser-Assisted Multiscale Fabrication of Configuration-Editable Supercapacitors with High Energy Density.

The construction of multidimensional, diversified microsupercapacitors (MSC) is urgently needed for fast-changing flexible and wearable microelectronics, which still meets the challenges of tedious construction and difficult integration. Herein, a laser direct writing strategy has been developed for the one-step preparation of multiscale MSCs from editable macro-supercapacitors. The microstructured supercapacitors with predefined multiscale shapes not only maintain the high capacitance performance and stability but also display the tensile properties in arbitrary direction. The heat-treated ion liquid-modified reduced graphene oxide guarantees the thermal stability of an electrode material during laser cutting, and its high ion-accessible surface area improves the capacitance performance of the supercapacitor. The as-fabricated MSC demonstrates a wide voltage window (0-3 V), high areal specific capacitance (27.4 mF cm-2), and high energy density (32.1 μW h cm-2), which are far higher than those of most reported articles. Notably, the editable supercapacitors can imitate the stereo paper cutting to achieve an arbitrary one-dimensional to three-dimensional configuration, promising for various portable, stretchable, and wearable devices.

Multi-scale shape optimisation of lattice structures: an evolutionary-based approach

This study deals with the problem of the least-weight design of a lattice structure subject to constraints of different nature. To face this problem, a general multi-scale optimisation procedure is proposed. This approach aims at optimising both global and local geometric parameters defining the shape of the representative volume element of the lattice at the mesoscopic scale. The optimisation procedure involves design requirements defined at different scales: geometric and manufacturing constraints are involved at the mesoscopic scale, whilst thermodynamic constraints on the positive definiteness of the stiffness tensor of the lattice (modelled as an equivalent homogeneous anisotropic medium) intervene at the macroscopic scale. Finally, since lattice structures usually undergo compressive loads, a requirement on the first local buckling load is considered too. The proposed approach is based on (a) the non-uniform rational basis splines curves theory to describe the shape of the struts composing the lattice, (b) the strain energy homogenisation technique of periodic media to perform the scale transition and (c) a special genetic algorithm to perform optimisation calculations. The optimised solutions provided by the presented method are characterised by a weight saving of about $$39\%$$ with slightly enhanced mechanical properties when compared to conventional octahedral lattice structures.

An automatic detection method for lung nodules based on multi-scale enhancement filters and 3D shape features

An automatic detection method for lung nodules based on multi-scale enhancement filters and 3D shape features

Multi-scale shape optimization method for designing a macrostructure with periodic-microstructures using inverse homogenization method and H<sup>1</sup> gradient method

Multi-scale shape optimization method for designing a macrostructure with periodic-microstructures using inverse homogenization method and H<sup>1</sup> gradient method

Finite element modeling of a multi-physics poro-elastic problem in multiscale media

We present an iterative algorithm for mathematical modeling of an elastic deformation process in a fluid-saturated fractured-porous medium. A three-dimensional multi-physics problem describes the coupled isothermal processes of the solid elastic deformation and slightly compressible fluid flow under external pressure. Mathematical models of these processes are connected via interface conditions for the pressure and density fields on the surface of a fractured-porous medium. For solving the multi-physics problem, a special multiscale procedure was developed. We use a heterogeneous multiscale finite element discretization on coarse polyhedral grids for the solid elastic deformation problem. Multiscale shape functions are constructed using special interface conditions for a hydrodynamic pressure on the surface of pores. We apply a discontinuous Galerkin method and a stabilized finite element discretization on fine tetrahedral grids for solving the hydrodynamics problem in fluid-saturated pores. In this case, we can realize an effective parallel procedure for solving the multi-physics problem. In each pore, hydrodynamics problems can be solved in parallel and independently. Verifications of the computational schemes are presented. We consider three-dimensional media with a different volume concentration of cracks and pores. Computational modeling results are presented. A time of solving the multi-physics problem using fine and coarse grids is shown.

An Efficient Image Retrieval System Based on Multi-Scale Shape Features

This paper describes a multi-scale feature integration framework using angular pattern (AP), binary AP (BAP) and sequential backward selection (SBS) algorithms. These angular descriptors are represented by multi-scale features from which the best subsets of the scales are chosen using five-fold cross-validation technique along with SBS algorithm for efficient image retrieval. The SBS algorithm reduces the dimensionality of feature space which in turn reduces the matching time complexity. The extracted AP and BAP features are represented in histograms and are compared by the Chi-square distance metric. The experimental analysis is performed on the MPEG-7 CE-1 Part-B dataset images to demonstrate the effectiveness of multi-scale feature integration using SBS algorithm. The image retrieval performance of this framework is compared with state-of-the-art shape descriptors. Being multi-scale global shape descriptors, the proposed framework captures complete information about the shape and are invariant to scaling and rotation transformations.

Multiscale shape–material modeling by composition

We propose a formal framework for modeling multiscale material structures by recursive composition of two-scale material structures. The framework comprises three components: (1) single scale shape–material models, supported by single scale queries, to represent the geometry and spatial distribution of material property on each coarse and fine scales, (2) mechanisms to link the scales by establishing an explicit relationship between shape–material properties at fine scale and material properties at the coarse scale, and (3) multiscale queries abstracting fundamental multiscale operations by recursive composition. While the first component is consistent with classical solid heterogeneous material modeling, the second component manifests itself as a pair of conceptually new upscaling and downscaling functions. We show that classical solid modeling queries, exemplified by point membership testing, distance computation, and material evaluation, generalize to the corresponding multiscale queries that support implicit representations of multiscale structures as a composition of distinct single scale solid material models. The concept of neighborhood is indispensable in all three components. The framework provides a formal and consistent extension of solid modeling framework that underlies most commercial systems in use today, encompasses the variety of different approaches to multiscale modeling, identifies open issues and research problems with existing two-scale modeling methods, and provides foundations for next-generation systems by identifying key objects, classes, representation schemes, and API queries.

Automated Chest X-Ray Screening: Can Lung Region Symmetry Help Detect Pulmonary Abnormalities?

Our primary motivator is the need for screening HIV+ populations in resource-constrained regions for exposure to Tuberculosis, using posteroanterior chest radiographs (CXRs). The proposed method is motivated by the observation that radiological examinations routinely conduct bilateral comparisons of the lung field. In addition, the abnormal CXRs tend to exhibit changes in the lung shape, size, and content (textures), and in overall, reflection symmetry between them. We analyze the lung region symmetry using multi-scale shape features, and edge plus texture features. Shape features exploit local and global representation of the lung regions, while edge and texture features take internal content, including spatial arrangements of the structures. For classification, we have performed voting-based combination of three different classifiers: Bayesian network, multilayer perception neural networks, and random forest. We have used three CXR benchmark collections made available by the U.S. National Library of Medicine and the National Institute of Tuberculosis and Respiratory Diseases, India, and have achieved a maximum abnormality detection accuracy (ACC) of 91.00% and area under the ROC curve (AUC) of 0.96. The proposed method outperforms the previously reported methods by more than 5% in ACC and 3% in AUC.

Characterizing the Spatio-Temporal Pattern of Land Surface Temperature through Time Series Clustering: Based on the Latent Pattern and Morphology

Land Surface Temperature (LST) is a critical component to understand the impact of urbanization on the urban thermal environment. Previous studies were inclined to apply only one snapshot to analyze the pattern and dynamics of LST without considering the non-stationarity in the temporal domain, or focus on the diurnal, seasonal, and annual pattern analysis of LST which has limited support for the understanding of how LST varies with the advancing of urbanization. This paper presents a workflow to extract the spatio-temporal pattern of LST through time series clustering by focusing on the LST of Wuhan, China, from 2002 to 2017 with a 3-year time interval with 8-day MODerate-resolution Imaging Spectroradiometer (MODIS) satellite image products. The Latent pattern of LST (LLST) generated by non-parametric Multi-Task Gaussian Process Modeling (MTGP) and the Multi-Scale Shape Index (MSSI) which characterizes the morphology of LLST are coupled for pattern recognition. Specifically, spatio-temporal patterns are discovered after the extraction of spatial patterns conducted by the incorporation of k -means and the Back-Propagation neural networks (BP-Net). The spatial patterns of the 6 years form a basic understanding about the corresponding temporal variances. For spatio-temporal pattern recognition, LLSTs and MSSIs of the 6 years are regarded as geo-referenced time series. Multiple algorithms including traditional k -means with Euclidean Distance (ED), shape-based k -means with the constrained Dynamic Time Warping ( c DTW) distance measure, and the Dynamic Time Warping Barycenter Averaging (DBA) centroid computation method ( k - c DBA) and k -shape are applied. Ten external indexes are employed to evaluate the performance of the three algorithms and reveal k - c DBA as the optimal time series clustering algorithm for our study. The study area is divided into 17 geographical time series clusters which respectively illustrate heterogeneous temporal dynamics of LST patterns. The homogeneous geographical clusters correspond to the zoning custom of urban planning and design, and thus, may efficiently bridge the urban and environmental systems in terms of research scope and scale. The proposed workflow can be utilized for other cities and potentially used for comparison among different cities.