Cloud Texture Research Articles

Attention Guide Axial Sharing Mixed Attention (AGASMA) Network for Cloud Segmentation and Cloud Shadow Segmentation

Segmenting clouds and their shadows is a critical challenge in remote sensing image processing. The shape, texture, lighting conditions, and background of clouds and their shadows impact the effectiveness of cloud detection. Currently, architectures that maintain high resolution throughout the entire information-extraction process are rapidly emerging. This parallel architecture, combining high and low resolutions, produces detailed high-resolution representations, enhancing segmentation prediction accuracy. This paper continues the parallel architecture of high and low resolution. When handling high- and low-resolution images, this paper employs a hybrid approach combining the Transformer and CNN models. This method facilitates interaction between the two models, enabling the extraction of both semantic and spatial details from the images. To address the challenge of inadequate fusion and significant information loss between high- and low-resolution images, this paper introduces a method based on ASMA (Axial Sharing Mixed Attention). This approach establishes pixel-level dependencies between high-resolution and low-resolution images, aiming to enhance the efficiency of image fusion. In addition, to enhance the effective focus on critical information in remote sensing images, the AGM (Attention Guide Module) is introduced, to integrate attention elements from original features into ASMA, to alleviate the problem of insufficient channel modeling of the self-attention mechanism. Our experimental results on the Cloud and Cloud Shadow dataset, the SPARCS dataset, and the CSWV dataset demonstrate the effectiveness of our method, surpassing the state-of-the-art techniques for cloud and cloud shadow segmentation.

Triple line-scan camera measurement for efficient and accurate narrow-space 3-D scanning with motion estimation.

Vision-based three-dimensional (3-D) shape measurement plays a crucial role in the inspection of vehicles and trains in the field of transportation. Consequently, the demand for more comprehensive narrow-space inspection has become an inevitable necessity and presents a great challenge to conventional vision methods. We propose the following efficient and accurate narrow-space 3-D scanning method based on triple line-scan cameras. First, the structure of coplanar dual line-scan cameras with a narrow reflector is proposed, making it easy for the optical path to enter a narrow space and obtain 3-D shape information. Efficient in-motion measurement with a large field of view is thereby achieved without the worries that often accompany narrow-space constraints. Secondly, an additional line-scan camera is attached to the coplanar dual cameras and creates a time-space constraint in point cloud stitching direction through the triple line-scan camera structure. With an efficient algorithm template including matching and motion estimation, accurate point cloud stitching is ensured. Lastly, point cloud texture mapping and light source optimization are realized. Our experimental results prove that the method realizes low-distortion in-motion reconstruction in narrow space despite motion variation.

Assessing the quality of 3D point clouds using descriptors for color and geometry texture

Since the mid-20th century, the use of digital formats for visual content has revolutionized communication in society. The Internet and digital broadcasting systems, which became widely available in the 1990s, led to an incredible expansion of multimedia consumption among the public. As a result, telecommunication networks and providers were pushed to their limits to meet the growing demand for multimedia content. Traditional electronic imaging systems, such as TV broadcasting systems, were designed based on subjective quality analysis for defining parameters like the number of lines in a video. However, with the emergence of new digital visual content services that require faster and more affordable methods of evaluating human perceived quality, there has been a need for automatic quality assessment. In the past few decades, numerous visual quality models based on algorithms running on digital computers have been proposed. While these existing models are highly advanced for 2D digital imagery, a new generation of immersive media has emerged with different data structures that are not applicable to traditional 2D methods and require novel quality assessment metrics. These emerging immersive media formats provide a 3D visual representation of real objects and scenes. In this new visual format, objects can be captured, compressed, transmitted, an visualized in real-time as 3D content, allowing consumers to select free-viewpoints to view such media. One of the most popular formats for immersive media is Point Cloud (PC), which is composed of points with 3D geometry coordinates and color information, and sometimes other attributes such as reflectance and transparency. This research focuses on the quality assessment of 3D Point Clouds using novel color and geometric texture statistics. Since distortions in both color and geometry attributes of 3D visual content can affect the perceived visual quality, this work proposes using both color-based and geometry-based texture descriptors for Point Clouds to obtain visual degradation information through their statistics. Four novel Point Cloud texture descriptors are introduced in this work, three of which are color-based and one which is geometry-based. Additionally, a new voxelization method is proposed, which converts points to voxels (volume elements), and improves the performance of the color-based texture descriptors. The proposed Point Cloud quality assessment method demonstrates high performance, comparable to the state-of-the-art methods, while also being flexible and extensible to adapt to different types of distortions.

Neural Network-Based Identification of Cloud Types from Ground-Based Images of Cloud Layers

Clouds are a significant factor in regional climates and play a crucial role in regulating the Earth’s water cycle through the interaction of sunlight and wind. Meteorological agencies around the world must regularly observe and record cloud data. Unfortunately, the current methods for collecting cloud data mainly rely on manual observation. This paper presents a novel approach to identifying ground-based cloud images to aid in the collection of cloud data. However, there is currently no publicly available dataset that is suitable for this research. To solve this, we built a dataset of surface-shot images of clouds called the SSC, which was overseen by the Macao Meteorological Society. Compared to previous datasets, the SSC dataset offers a more balanced distribution of data samples across various cloud genera and provides a more precise classification of cloud genera. This paper presents a method for identifying cloud genera based on cloud texture, using convolutional neural networks. To extract cloud texture effectively, we apply Gamma Correction to the images. The experiments were conducted on the SSC dataset. The results show that the proposed model performs well in identifying 10 cloud genera, achieving an accuracy rate of 80% for the top three possibilities.

AICCA: AI-Driven Cloud Classification Atlas

Clouds play an important role in the Earth’s energy budget, and their behavior is one of the largest uncertainties in future climate projections. Satellite observations should help in understanding cloud responses, but decades and petabytes of multispectral cloud imagery have to date received only limited use. This study describes a new analysis approach that reduces the dimensionality of satellite cloud observations by grouping them via a novel automated, unsupervised cloud classification technique based on a convolutional autoencoder, an artificial intelligence (AI) method good at identifying patterns in spatial data. Our technique combines a rotation-invariant autoencoder and hierarchical agglomerative clustering to generate cloud clusters that capture meaningful distinctions among cloud textures, using only raw multispectral imagery as input. Cloud classes are therefore defined based on spectral properties and spatial textures without reliance on location, time/season, derived physical properties, or pre-designated class definitions. We use this approach to generate a unique new cloud dataset, the AI-driven cloud classification atlas (AICCA), which clusters 22 years of ocean images from the Moderate Resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua and Terra instruments—198 million patches, each roughly 100 km × 100 km (128 × 128 pixels)—into 42 AI-generated cloud classes, a number determined via a newly-developed stability protocol that we use to maximize richness of information while ensuring stable groupings of patches. AICCA thereby translates 801 TB of satellite images into 54.2 GB of class labels and cloud top and optical properties, a reduction by a factor of 15,000. The 42 AICCA classes produce meaningful spatio-temporal and physical distinctions and capture a greater variety of cloud types than do the nine International Satellite Cloud Climatology Project (ISCCP) categories—for example, multiple textures in the stratocumulus decks along the West coasts of North and South America. We conclude that our methodology has explanatory power, capturing regionally unique cloud classes and providing rich but tractable information for global analysis. AICCA delivers the information from multi-spectral images in a compact form, enables data-driven diagnosis of patterns of cloud organization, provides insight into cloud evolution on timescales of hours to decades, and helps democratize climate research by facilitating access to core data.

GEOMATICS SUPPORTING KNOWLEDGE OF CULTURAL HERITAGE AIMED AT RECOVERY AND RESTORATION

Abstract. The study presented aims a practical contribution to the use of the Terrestrial Laser Scanner and the Close Range Photogrammetry geomatic techniques and to their integration for the knowledge and development of the historical-architectural heritage, both in the step of planning a restorative conservation project, and in the subsequent step of the restoration works. In particular, these techniques and their integration were applied at the process of study, planning and execution of the restoration of the San Giovanni Battista church in Fonni (Sardinia, Italy), founded in the 16th century. The building, due to structural problems and humidity infiltrations, was subject to a serious intervention for consolidation and conservative restoration.In the first phase of the work, preliminary to the design, a TLS survey was made that allowed to obtain an accurate 3D model of the church. In a second phase, during the restoration work, CRP surveys were carried out and the accuracy of this technique was verified through a comparison with TLS surveys. The integration of the two techniques to improve the texture of point clouds detected with TLS was also tested.

UATNet: U-Shape Attention-Based Transformer Net for Meteorological Satellite Cloud Recognition

Cloud recognition is a basic task in ground meteorological observation. It is of great significance to accurately identify cloud types from long-time-series satellite cloud images for improving the reliability and accuracy of weather forecasting. However, different from ground-based cloud images with a small observation range and easy operation, satellite cloud images have a wider cloud coverage area and contain more surface features. Hence, it is difficult to effectively extract the structural shape, area size, contour shape, hue, shadow and texture of clouds through traditional deep learning methods. In order to analyze the regional cloud type characteristics effectively, we construct a China region meteorological satellite cloud image dataset named CRMSCD, which consists of nine cloud types and the clear sky (cloudless). In this paper, we propose a novel neural network model, UATNet, which can realize the pixel-level classification of meteorological satellite cloud images. Our model efficiently integrates the spatial and multi-channel information of clouds. Specifically, several transformer blocks with modified self-attention computation (swin transformer blocks) and patch merging operations are used to build a hierarchical transformer, and spatial displacement is introduced to construct long-distance cross-window connections. In addition, we introduce a Channel Cross fusion with Transformer (CCT) to guide the multi-scale channel fusion, and design an Attention-based Squeeze and Excitation (ASE) to effectively connect the fused multi-scale channel information to the decoder features. The experimental results demonstrate that the proposed model achieved 82.33% PA, 67.79% MPA, 54.51% MIoU and 70.96% FWIoU on CRMSCD. Compared with the existing models, our method produces more precise segmentation performance, which demonstrates its superiority on meteorological satellite cloud recognition tasks.

Notice of Violation of IEEE Publication Principles: Ground-Based Cloud Image Recognition System Based on Multi-CNN and Feature Screening and Fusion

The recognition of ground-based cloud images has rich application prospects in many aspects such as weather prediction, astronomical site selection, and meteorological observation. Affected by factors such as rotation and illumination, the traditional feature extraction method is difficult to accurately describe the features of cloud images, resulting in low accuracy of ground-based cloud image recognition, and cannot meet the requirements of practical applications. With the popularity of convolutional neural networks in image processing, ground-based cloud image recognition algorithms based on convolutional neural network have become a research focus. However, the features of the ground-based cloud image are relatively shallow, and the cloud texture and other features are seriously lost in the convolution process, and it is difficult to achieve a good recognition effect. This paper proposes a ground-based cloud image recognition system based on multi-scale convolutional neural network (Multi-CNN) and multilayer perceptron neural networks (MLP). The multi-level and multi-scale convolution feature extraction is performed through convolution layers of Multi-CNN, and the local features with strong resolving power are selected through the feature screening algorithm based on DP clustering. Finally, the local features are encoded and fused for cloud image classification based on MLP. Filed test results show that our method was superior to other tested network models in terms of the recognition accuracy of 94.8% under 9 classification. In addition, ablation experiments show that the multi-scale feature extraction, screening and local feature coding in this paper have a significant effect on improving the algorithm’s ability to distinguish different cloud images.

Application of Statistical Models of Image Texture and Physical Parameters of Clouds for Their Classification on MODIS Satellite Images

Modifications of algorithms for the classification of single-layer, vertical development clouds, and multilayer clouds based on a probabilistic neural network and a neuro-fuzzy classifier are proposed. Clouds are classified into 16 types according to the meteorological standard, including the combined subtypes of stratus, altocumulus, cirrus, and cirrostratus clouds. The article uses a description of clouds based on information about the texture of their images on satellite images from MODIS and its products with data on the physical parameters of clouds. The structure of classification algorithms is described. The results of the use of statistical models of image texture and physical parameters of clouds for initializing membership functions for a neural-fuzzy classifier are presented. Systems of effective classification characteristics for different classification algorithms are formed based on the GRAD modified truncated search method. The recognition results of single-layer, vertical development clouds, and multilayer clouds based on the corresponding test samples and full-size sets of MODIS satellite data with different spatial resolution are discussed.

THE ONE TO MULTIPLE AUTOMATIC HIGH ACCURACY REGISTRATION OF TERRESTRIAL LIDAR AND OPTICAL IMAGES

Abstract. The registration of ground laser point cloud and close-range image is the key content of high-precision 3D reconstruction of cultural relic object. In view of the requirement of high texture resolution in the field of cultural relic at present, The registration of point cloud and image data in object reconstruction will result in the problem of point cloud to multiple images. In the current commercial software, the two pairs of registration of the two kinds of data are realized by manually dividing point cloud data, manual matching point cloud and image data, manually selecting a two – dimensional point of the same name of the image and the point cloud, and the process not only greatly reduces the working efficiency, but also affects the precision of the registration of the two, and causes the problem of the color point cloud texture joint. In order to solve the above problems, this paper takes the whole object image as the intermediate data, and uses the matching technology to realize the automatic one-to-one correspondence between the point cloud and multiple images. The matching of point cloud center projection reflection intensity image and optical image is applied to realize the automatic matching of the same name feature points, and the Rodrigo matrix spatial similarity transformation model and weight selection iteration are used to realize the automatic registration of the two kinds of data with high accuracy. This method is expected to serve for the high precision and high efficiency automatic 3D reconstruction of cultural relic objects, which has certain scientific research value and practical significance.

Ground-based cloud classification by learning stable local binary patterns

Feature selection and extraction is the first step in implementing pattern classification. The same is true for ground-based cloud classification. Histogram features based on local binary patterns (LBPs) are widely used to classify texture images. However, the conventional uniform LBP approach cannot capture all the dominant patterns in cloud texture images, thereby resulting in low classification performance. In this study, a robust feature extraction method by learning stable LBPs is proposed based on the averaged ranks of the occurrence frequencies of all rotation invariant patterns defined in the LBPs of cloud images. The proposed method is validated with a ground-based cloud classification database comprising five cloud types. Experimental results demonstrate that the proposed method achieves significantly higher classification accuracy than the uniform LBP, local texture patterns (LTP), dominant LBP (DLBP), completed LBP (CLTP) and salient LBP (SaLBP) methods in this cloud image database and under different noise conditions. And the performance of the proposed method is comparable with that of the popular deep convolutional neural network (DCNN) method, but with less computation complexity. Furthermore, the proposed method also achieves superior performance on an independent test data set.

Measure for the Difference Between LBP Features Extracted From Original and Resized Cloud Images With Varying Resolutions

Currently, ground-based cloud images taken by using a whole-sky imager are especially popular in the field of meteorology because of their high resolution and accurate cloud information. Cloud images are natural texture images, and thus texture features based on local binary patterns (LBPs) are widely used to analyze texture images. However, the high-computation cost of extracting LBP features from high-resolution cloud texture images may make this technique unacceptable in practical image processing. A commonly adopted method is to resize the original image to an appropriate version with a decreased resolution. But this process will inevitably result in information loss. Accordingly, a measure based on the Kullback-Leibler (KL) divergence of the difference between LBP histogram features extracted from the original and resized images with varying resolutions is reported in this letter. Furthermore, a confidence interval technique is introduced to validate the significance of such difference. Experiments based on real ground-based cloud images show the measurement results of KL divergence in LBP features extracted from original and resized images. The experimental results indicate that images should be resized with caution when performing image processing.

Revealing the potential of spectral and textural predictor variables in a neural network-based rainfall retrieval technique

ABSTRACTEstimating rainfall areas and rates from geostationary satellite images has the opportunity of both, a high spatial and a high temporal resolution which cannot be achieved by other satellite-based systems until now. Most recent retrieval techniques are solely based on spectral channels of the satellites. These retrievals can be classified as ‘purely pixel-based’ because no information about the neighbourhood pixels is included. Assuming that precipitation is highly correlated with cloud processes and therefore with cloud texture, textural information derived from the neighbourhood of a pixel might give valuable information about the cloud type and hence about a respective probability of the rainfall rate. To study the potential of textural variables to improve optical rainfall retrieval techniques, rainfall areas and rainfall rates were estimated over Germany for the year 2010 using a neural network approach. In addition to the spectral predictor variables from Meteosat Second Generation (MSG), different Grey Level Co-occurance Matrix (GLCM) based textural variables were calculated from all MSG channels. Using recursive feature selection, models were trained and their performance was compared to spectral-only models. Contrary to the expectations, the performance of the models did not increase when textural information was included.

DESIGN A FILTER TO DETECT AND REMOVE VEGETATION FROM ULTRA-CAM-X AERIAL IMAGES’ POINT CLOUD TO PRODUCE AUTOMATICALLY DIGITAL ELEVATION MODEL

Abstract. Digital elevation model is one of the most important spatial information for displaying bare earth. Because of existing objects on the ground, manual editing is unavoidable. Aerial images’ point clouds produced by advanced matching methods are good resources for generating DEM. In this paper, the purpose is design a filter for detect and eliminate vegetation from point clouds. For this purpose, point clouds’ texture is used for finding vegetation. Texture of point clouds is segmented by Otsu method. In the next step, segmented image is added to raster of elevation and vegetation elevation is detected. Results is showing that point clouds’ texture is a good data for filtering vegetation and generating DEM automatically.

Texture Feature Extraction Method for Ground Nephogram Based on Hilbert Spectrum of Bidimensional Empirical Mode Decomposition

Abstract It is important to recognize the type of cloud for automatic observation by ground nephoscope. Although cloud shapes are protean, cloud textures are relatively stable and contain rich information. In this paper, a novel method is presented to extract the nephogram feature from the Hilbert spectrum of cloud images using bidimensional empirical mode decomposition (BEMD). Cloud images are first decomposed into several intrinsic mode functions (IMFs) of textural features through BEMD. The IMFs are converted from two- to one-dimensional format, and then the Hilbert–Huang transform is performed to obtain the Hilbert spectrum and the Hilbert marginal spectrum. It is shown that the Hilbert spectrum and the Hilbert marginal spectrum of different types of cloud textural images can be divided into three different frequency bands. A recognition rate of 87.5%–96.97% is achieved through random cloud image testing using this algorithm, indicating the efficiency of the proposed method for cloud nephogram.

Model relating humidity and intensity of satellite infrared imagery for global precipitation nowcasting

The satellite imagery-based hydro image processing (SIHIP) model elucidates a new precipitation nowcasting methodology by relating humidity and intensity of satellite infrared image. With this relation, the relative humidity can be quickly and reliably estimated in near real-time to eliminate the need for site-specific radiosonde. The algorithm uses clustering technique to separate the cloud texture from the ground surface textures and Haar wavelet to obtain its mean wavelength. This wavelength is used to obtain a relation between a real time entity and the observed entity, which affects the precipitation. SIHIP focuses on convective and precipitating clouds over the Indian subcontinent region for the periods June–September 2012 and 2013. It has undergone a rigorous test for validation on real time data from NASA Global Precipitation Measurement mission. The results of the SIHIP model reflect a significant improvement over existing global satellite precipitation nowcasting algorithms with a success rate of 95.14%.

Real-Time Simulation of Dynamic Cloud Based on Fractal

Cloud is a common, widespread natural phenomenon. Cloud modeling and rendering plays a great important role in virtual environments. This article based on texture splatting on particles method, and considers the clouds as a grade model. A method based on fractal is chosen to generate cloud texture and then texture is mapping to the facets. The method introduced in this article is simple and can improve the cloud modeling velocity. The results show that the method can meet the common requirements in scene simulation and have a good performance in real-time and reality.

A Study on Cloud Texture Feature Quantities of the SatelliteVIS Imagery with Solar Eclipse Shadow

Solar eclipse would result in an obvious shadow on satellite VIS imagery. In the shadow, the type and the shape of cloud can not be distinguished correctly, which would have impact on analyzing and applying the information of the satellite VIS imagery in practice, so it is very important to eliminate the eclipse shadow. In this paper, the theory of solar eclipse is given at first and then the radiant revisal function, which the satellite can receive the radiation from the earth, is given. Using the radiant revisal function, the eclipse shadow can be eliminated. Based on the point of image manipulation, using the gray level coocurrence matrix and cloud texture feature quantities, such as contrast, entropy and correlation, the effect of the solar eclipse elimination can be estimated. The results show that the method is more effective, the alteration of the gray level coocurrence matrix and the contrast and the entropy indicated that the shadow has been eliminated primarily, the texture and structure of cloud system are trenchanter than before , while the correlation of cloud system keeps unchanged.

Rainfall Estimation from a Combination of TRMM Precipitation Radar and GOES Multispectral Satellite Imagery through the Use of an Artificial Neural Network

This paper describes the development of a satellite precipitation algorithm designed to generate rainfall estimates at high spatial and temporal resolutions using a combination of Tropical Rainfall Measuring Mission (TRMM) precipitation radar (PR) data and multispectral Geostationary Operational Environmental Satellite (GOES) imagery. Coincident PR measurements were matched with four-band GOES image data to form the training dataset for a neural network. Statistical information derived from multiple GOES pixels was matched with each precipitation measurement to incorporate information on cloud texture and rates of change into the estimation process. The neural network was trained for a region of Brazil and used to produce half-hourly precipitation estimates for the periods 8-31 January and 10-25 February 1999 at a spatial resolution of 0.12 degrees. These products were validated using PR and gauge data. Instantaneous precipitation estimates demonstrated correlations of ~0.47 with independent validation data, exceeding those of an optimized GOES Precipitation Index method locally calibrated using PR data. A combination of PR and GOES data thus may be used to generate precipitation estimates at high spatial and temporal resolutions with extensive spatial and temporal coverage, independent of any surface instrumentation.

On the Spatial Distribution of Cloud Particles

Recent studies have led to the statistical characterization of the spatial (temporal) distributions of cloud (precipitation) particles as a doubly stochastic Poisson process. This paper arrives at a similar conclusion (larger-than-Poissonian variance) via the more fundamental route of statistical physics and significantly extends previous findings in several ways. The focus is on the stochastic structure in the spatial distribution of cloud particles. A new approach for exploring the stochastic structure of clouds is proposed using a direct relation between number density variance and the pair correlation function. In addition, novel counting diagrams, particularly useful for analyzing counts at low data rates, demonstrate droplet clustering and striking deviations from Poisson randomness on small (centimeter) scales. These findings are shown to agree with pair correlation functions calculated for droplet counts obtained from an aircraft-mounted cloud probe. Time series of the arrival of each droplet are used to bin the data evenly so as to avoid corruption of the statistics through the operations of multiplication and division. Furthermore, it is shown that statistically homogeneous series of particle counts exhibit super-Poissonian variance. Since it is not always practical or feasible to obtain such direct measurements, the possibility of studying cloud texture using a revival of the idea of coherent microwave scatter from cloud droplets is discussed, including a more complete interpretation of Bragg scatter that seems to explain some recent observations in clouds. Finally, the appearance of clustering and the subsequent geometric distribution of droplet counts are interpreted using basic considerations of turbulence.