Cloud Images Research Articles

Detection and characterization of spike architecture based on deep learning and X-ray computed tomography in barley

BackgroundSpike is the grain-bearing organ in cereal crops, which is a key proxy indicator determining the grain yield and quality. Machine learning methods for image analysis of spike-related phenotypic traits not only hold the promise for high-throughput estimating grain production and quality, but also lay the foundation for better dissection of the genetic basis for spike development. Barley (Hordeum vulgare L.) is one of the most important crops globally, ranking as the fourth largest cereal crop in terms of cultivated area and total yield. However, image analysis of spike-related traits in barley, especially based on CT-scanning, remains elusive at present.ResultsIn this study, we developed a non-invasive, high-throughput approach to quantitatively measuring the multitude of spike architectural traits in barley through combining X-ray computed tomography (CT) and a deep learning model (UNet). Firstly, the spikes of 11 barley accessions, including 2 wild barley, 3 landraces and 6 cultivars were used for X-ray CT scanning to obtain the tomographic images. And then, an optimized 3D image processing method was used to point cloud data to generate the 3D point cloud images of spike, namely ‘virtual’ spike, which is then used to investigate internal structures and morphological traits of barley spikes. Furthermore, the virtual spike-related traits, such as spike length, grain number per spike, grain volume, grain surface area, grain length and grain width as well as grain thickness were efficiently and non-destructively quantified. The virtual values of these traits were highly consistent with the actual value using manual measurement, demonstrating the accuracy and reliability of the developed model. The reconstruction process took 15 min approximately, 10 min for CT scanning and 5 min for imaging and features extraction, respectively.ConclusionsThis study provides an efficient, non-invasive and useful tool for dissecting barley spike architecture, which will contribute to high-throughput phenotyping and breeding for high yield in barley and other crops.

Drilling Parameters Multi-Objective Optimization Method Based on PSO-Bi-LSTM

The increasing exploration and development of complex oil and gas fields pose challenges to drilling efficiency and safety due to the presence of formations with varying hardness, abrasiveness, and rigidity. Consequently, there is a growing demand for drilling parameter optimization and speed-up technologies. However, existing models based on expert experience can only achieve single-objective optimization with limited accuracy, making real-time adaptation to changing drilling conditions and formation environments challenging. The emergence of artificial intelligence provides a new approach for optimizing drilling parameters. In this study, we introduce the Bi-directional Long Short-Term Memory (Bi-LSTM) deep learning algorithm with the attention mechanism to predict the rate of penetration (ROP). This algorithm improves the ROP prediction accuracy to 98.33%, ensuring reliable subsequent optimization results. Additionally, we propose a coupling optimization algorithm that combines Bi-LSTM with the particle swarm optimization algorithm (PSO) to enhance drilling efficiency through parameter optimization. Our approach aims to maximize drilling footage while maintaining the highest ROP. The optimal solutions obtained are verified through multi-parameter cloud image analysis, yielding consistent results. The application of our approach demonstrates an 81% increase in drilling speed and a 28% reduction in drill bit energy losses. Moreover, the real-time optimization results effectively guide field operations.

Distributed photovoltaic power output prediction based on satellite cloud map video frames

To address the challenge of predicting distributed photovoltaic (PV) power output for improved system integration and stability, this study proposes a novel method. Given the expanding scale of distributed PV systems and their economic constraints, accurate power output prediction becomes pivotal. Conventional prediction methods are hindered by the lack of meteorological stations at most distributed PV stations. In response, we present a dynamic convolutional generative adversarial network (DC-GAN) approach coupled with satellite cloud map video frames. By extracting shading features from satellite cloud images and utilizing DC-GAN, our method forecasts short-term cloud shading effects on future radiation. We further integrate radiation data from centralized PV stations, spatial correlations of distributed PV stations, and cloud shading characteristics. This information is used to construct a predictive model combining Convolutional Neural Networks (CNN) and Long short-term memory (LSTM), enhancing prediction accuracy. Comparative experiments confirm the superiority of our proposed method over traditional approaches, substantiating its effectiveness and practicality. Our method achieves notable accuracy improvements, establishing its value in predicting distributed PV power output. This research’s findings offer a valuable contribution to the field of renewable energy integration. In numerical assessments, our method demonstrates a significant increase in prediction accuracy, outperforming conventional techniques by 3.3%. This underscores the practical relevance and efficiency of our proposed approach in enhancing distributed PV power output prediction.

Novel solar forecasting scheme modelled by mixer dual path network and based on sky images

The prediction of global horizontal irradiance has become an effective technique to address the intermittence issue of photovoltaic (PV) power generation. This article proposes a novel deep neural network(DNN), named Mixer Dual Path Network (Mixer-DPN), for promising solar forecasting. It shares common features of cloud images and maintains the flexibility to explore new features through dual-path architecture by combining the Mixer layer and Dual Path Network. Therefore, the proposed model can provide more accurate prediction results compared to the classical DNN-based predictors. Moreover, the proposed model shows a faster convergence speed and smaller model size, which makes it suitable for a practical global horizontal irradiance. The merits of the proposed model are verified by testing it with the data from National Renewable Energy Laboratory comparing it with other DNN-based prediction models. Studies have shown that the new model has achieved excellent results in MSE, MAE and other indicators, and the R2 prediction accuracy rate has increased by 14% compared with the baseline model.

A Prior-Knowledge-Based Generative Adversarial Network for Unsupervised Satellite Cloud Image Restoration

High-quality satellite cloud images are of great significance for weather diagnosis and prediction. However, many of these images are often degraded due to relative motion, atmospheric turbulence, instrument noise, and other factors. In the satellite imaging process, the degradation also cannot be completely corrected. Therefore, it is necessary to further improve the satellite cloud image quality for real applications. In this study, we propose an unsupervised image restoration model with a two-stage network, in which the first stage, named the Prior-Knowledge-based Generative Adversarial Network (PKKernelGAN), aims to learn the blur kernel, and the second stage, named the Zero-Shot Deep Residual Network (ZSResNet), aims to improve the image quality. In PKKernelGAN, we propose a satellite cloud imaging loss function, which is a novel objective function that brings optimization of a generative model into the prior-knowledge domain. In ZSResNet, we build a dataset which contains the original satellite cloud image as high-quality images (HQ) paired with low-quality images (LQ) generated by the blur kernel learning from PKKernelGAN. The above innovations lead to a more efficient local structure in satellite cloud image restoration. The original dataset of our experiment is from the Sunflower 8 satellite provided by the Japan Meteorological Agency. This dataset is divided into training and testing sets to train and test PKKernelGAN. Then, ZSResNet is trained by the “LQ–HQ” image pairs generated by PKKernelGAN. Compared with other supervised and unsupervised deep learning models for image restoration, our model has a better performance. Extensive experiments have demonstrated that our proposed model can achieve better performance on different datasets.

Privacy-Preserving Image Retrieval Based on Disordered Local Histograms and Vision Transformer in Cloud Computing

Frequent data breaches in the cloud environment have seriously affected cloud subscribers and providers. Privacy-preserving image retrieval methods can improve the security of cloud image retrieval; however, existing methods have limited accuracy on dynamically updated image databases and mobile lightweight devices. In this study, we propose a privacy-preserving image retrieval method based on disordered local histograms and vision transformer in cloud computing, by designing a multiple encryption method and transformer-based feature model to better mine the local feature value of encrypted images. Specifically, the user performs different value substitution, position substitution, and color substitution on the subblocks of the image to protect the image information. The cloud server extracts the unordered local histogram from the encrypted image and generates retrievable features using transformer. Experiments show that compared with similar CNN schemes, the retrieval accuracy of this method is improved by 8.5%, and the retrieval efficiency is improved by 54.8%.

Recognition of Severe Convective Cloud Based on the Cloud Image Prediction Sequence from FY-4A

Severe convective weather is hugely destructive, causing significant loss of life and social and economic infrastructure. Based on the U-Net network with the attention mechanism, the recurrent convolution, and the residual module, a new model is proposed named ARRU-Net (Attention Recurrent Residual U-Net) for the recognition of severe convective clouds using the cloud image prediction sequence from FY-4A data. The characteristic parameters used to recognize severe convective clouds in this study were brightness temperature values TBB9, brightness temperature difference values TBB9−TBB12 and TBB12−TBB13, and texture features based on spectral characteristics. This method first input five satellite cloud images with a time interval of 30 min into the ARRU-Net model and predicted five satellite cloud images for the next 2.5 h. Then, severe convective clouds were segmented based on the predicted image sequence. The root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and correlation coefficient (R2) of the predicted results were 5.48 K, 35.52 dB, and 0.92, respectively. The results of the experiments showed that the average recognition accuracy and recall of the ARRU-Net model in the next five moments on the test set were 97.62% and 83.34%, respectively.

A Convolutional Neural Network (CNN) Classification Model for Web Page: A Tool for Improving Web Page Category Detection Accuracy

Game and Online Video Streaming are the most viewed web pages. Users who spend too much time on these types of web pages may suffer from internet addiction. Access to Game and Online Video Streaming web pages should be restricted to combat internet addiction. A tool is required to recognise the category of web pages based on the text content of the web pages. Due to the unavailability of a matrix representation that can handle long web page text content, this study employs a document representation known as word cloud image to visualise the words extracted from the text content web page after data pre-processing. The most popular words are shown in large size and appear in the centre of the word cloud image. The most common words are the words that appear frequently in the text content web page and are related to describing what the web page content is about. The Convolutional Neural Network (CNN) recognises the pattern of words presented in the core portions of the word cloud image to categorise the category to which the web page belongs. The proposed model for web page classification has been compared with the other web page classification models. It shows the good result that achieved an accuracy of 85.6%. It can be used as a tool that helps to make identifying the category of web pages more accurate

CloudY-Net: A Deep Convolutional Neural Network Architecture for Joint Segmentation and Classification of Ground-Based Cloud Images

Ground-based cloud images contain a wealth of cloud information and are an important part of meteorological research. However, in practice, ground cloud images must be segmented and classified to obtain the cloud volume, cloud type and cloud coverage. Existing methods ignore the relationship between cloud segmentation and classification, and usually only one of these is studied. Accordingly, our paper proposes a novel method for the joint classification and segmentation of cloud images, called CloudY-Net. Compared to the basic Y-Net framework, which extracts feature maps from the central layer, we extract feature maps from four different layers to obtain more useful information to improve the classification accuracy. These feature maps are combined to produce a feature vector to train the classifier. Additionally, the multi-head self-attention mechanism is implemented during the fusion process to enhance the information interaction among features further. A new module called Cloud Mixture-of-Experts (C-MoE) is proposed to enable the weights of each feature layer to be automatically learned by the model, thus improving the quality of the fused feature representation. Correspondingly, experiments are conducted on the open multi-modal ground-based cloud dataset (MGCD). The results demonstrate that the proposed model significantly improves the classification accuracy compared to classical networks and state-of-the-art algorithms, with classification accuracy of 88.58%. In addition, we annotate 4000 images in the MGCD for cloud segmentation and produce a cloud segmentation dataset called MGCD-Seg. Then, we obtain a 96.55 mIoU on MGCD-Seg, validating the efficacy of our method in ground-based cloud imagery segmentation and classification.

Scene- and pixel-level analysis of Landsat cloud coverage and image acquisition probability in South and Southeast Asia

Scene- and pixel-level analysis of Landsat cloud coverage and image acquisition probability in South and Southeast Asia

Cooperative Perception With V2V Communication for Autonomous Vehicles

Occlusion is a critical problem in the Autonomous Driving System. Solving this problem requires robust collaboration among autonomous vehicles traveling on the same roads. However, transferring the entirety of raw sensors' data among autonomous vehicles is expensive and can cause a delay in communication. This paper proposes a method called Realtime Collaborative Vehicular Communication based on Bird's-Eye-View (BEV) map. The BEV map holds the accurate depth information from the point cloud image while its 2D representation enables the method to use a novel and well-trained image-based backbone network. Most importantly, we encode the object detection results into the BEV representation to reduce the volume of data transmission and make real-time collaboration between autonomous vehicles possible. The output of this process, the BEV map, can also be used as direct input to most route planning modules. Numerical results show that this novel method can increase the accuracy of object detection by cross-verifying the results from multiple points of view. Thus, in the process, this new method also reduces the object detection challenges that stem from occlusion and partial occlusion. Additionally, different from many existing methods, this new method significantly reduces the data needed for transfer between vehicles, achieving a speed of 21.92 Hz for both the object detection process and the data transmission process, which is sufficiently fast for a real-time system.

Development of 3‐D point cloud imaging and 2‐D image processing methods for high‐throughput phenotyping of perennial ryegrass spikes

AbstractAdvances in technology have increased the adoption of high‐throughput phenotyping (HTP) methodologies, potentially replacing laborious and time‐consuming measurements and data recording. One promising HTP tool for fine‐featured and small‐sized characteristics is the three‐dimensional (3‐D) scanning and imaging system, but the utility of present two‐dimensional (2‐D) technology has not been fully explored for this purpose. The objective of this work was to reduce assessment and field evaluation time by developing 2‐D photogrammetric and 3‐D point cloud imaging methods for HTP of spike characteristics in perennial ryegrass (Lolium perenne L.), with special attention to traits that might be associated with seed retention. These HTP imaging systems were compared with direct data capture by hand on spikes of 21 diverse global accessions of perennial ryegrass. The Fiji (ImageJ) open‐source imaging software was used for photogrammetric analysis of spike structure, including spike length, spikelet number, internode length, and 2‐D curvature of the spike. An original approach to nondestructive HTP for crop characterization was developed using the commercially available Artec Space Spider. This 3‐D optical sensor was used to generate dense 3‐D point clouds to measure spike length, spikelet number, internode length, spikelet length, spikelet angle, and 3‐D curvature of the spike. Both methods were found to accurately characterize the subject; the 3‐D method was slower than 2‐D but was more precise (p ≤ 0.01) than 2‐D image analysis with a linear measurement deviation of only 0.17%. Fiji was effectively used for post‐processing image analysis, and the Space Spider can be used directly in the field to support HTP data collection. This non‐destructive field measurement system facilitates HTP in perennial ryegrass spikes and likely in other applications.

Estimation of Tropical Cyclone Intensity via Deep Learning Techniques from Satellite Cloud Images

Estimating the intensity of tropical cyclones (TCs) is usually involved as a critical step in studies on TC disaster warnings and prediction. Satellite cloud images (SCIs) are one of the most effective and preferable data sources for TC research. Despite the great achievements in various SCI-based studies, accurate and efficient estimation of TC intensity still remains a challenge. In recent years, machine learning (ML) techniques have gained fast development and shown significant potential in dealing with big data, particularly with images. This study focuses on the objective estimation of TC intensity based on SCIs via a comprehensive usage of some advanced deep learning (DL) techniques and smoothing methods. Two estimation strategies are proposed and examined which, respectively, involve one and two functional stages. The one-stage strategy uses Vision Transformer (ViT) or Deep Convolutional Neutral Network (DCNN) as the regression model for directly identifying TC intensity, while the second strategy involves a classification stage that aims to stratify SCI samples into a few intensity groups and a subsequent regression stage that specifies the TC intensity. Further efforts are made to improve the estimation accuracy by using smoothing manipulations (via four specific smoothing techniques) in the scenarios of the aforementioned two strategies and their fusion. Results show that DCNN performs better than ViT in the one-stage strategy, while using ViT as the classification model and DCNN as the regression model can result in the best performance in the two-stage strategy. It is interesting that although the strategy of singly using DCNN wins out over any concerned two-stage strategy, the fusion of the two strategies outperforms either the one-stage strategy or the two-stage strategy. Results also suggest that using smoothing techniques are beneficial for the improvement of estimation accuracy. Overall, the best performance is achieved by using a hybrid strategy that consists of the one-stage strategy, the two-stage strategy and smoothing manipulation. The associated RMSE and MAE values are 9.81 kt and 7.51 kt, which prevail over those from most existing studies.

Ultrafast control over chiral sum-frequency generation.

We introduce an ultrafast all-optical approach for efficient chiral recognition that relies on the interference between two low-order nonlinear processes that are ubiquitous in nonlinear optics: sum-frequency generation and third-harmonic generation. In contrast to traditional sum-frequency generation, our approach encodes the medium's handedness in the intensity of the emitted harmonic signal, rather than in its phase, and it enables full control over the enantiosensitive response. We show how, by sculpting the sub-optical-cycle oscillations of the driving laser field, we can force one molecular enantiomer to emit bright light while its mirror twin remains dark, thus reaching the ultimate efficiency limit of chiral sensitivity via low-order nonlinear light-matter interactions. Our work paves the way for ultrafast and highly efficient imaging and control of the chiral electronic clouds of chiral molecules using lasers with moderate intensities, in all states of matter: from gases to liquids to solids, with molecular specificity and on ultrafast time scales.

Determination of extensibility and toughness of wheat-flour dough based on bubbles blown by sheeted dough and airflow-3D imaging technique

The resistance strength (toughness) and extensibility of dough are closely related to the quality of the dough product. In this study, the toughness and extensibility of dough were detected using an airflow-structured light 3D imaging technique combine with bubbles blown out of sheeted dough. The spatial deformation characteristics and point cloud features of each point cloud image were determined using the point cloud data processing algorithms. The frame, height, volume, surface area, projected area of three planes, and three point cloud features corresponding to the maximum height of the bubble were taken as the extracted features. A total of eight features were selected, which were consequently concentrated in the first three principal components (PCs). The first three PCs were fed into three different prediction models. The ELM model achieved a relatively higher prediction accuracy than the other two models. For toughness and extensibility, Rp were 9092 and 0.8950, respectively.

CloudDeepLabV3+: a lightweight ground-based cloud segmentation method based on multi-scale feature aggregation and multi-level attention feature enhancement

ABSTRACT The segmentation of ground-based cloud images is the basis for obtaining numerous cloud parameters. To achieve high-precision adaptive cloud image segmentation requirements, this study designs a lightweight ground-based cloud image adaptive segmentation method named CloudDeepLabV3+ that integrates multi-scale features aggregation and multi-level attention feature enhancement. Firstly, a novel lightweight EfficientNetV2-S is designed as a feature extraction backbone to reduce network parameters. Secondly, a heterogeneous receptive field splicing atrous spatial pyramid pooling module is designed. It enhances the correlation of information between layers, and realizes multiscale information fusion. The feature enhancement module based on the self-attention mechanism intensifies the representation of local and global features. Thirdly, the feature alignment module based on the attention mechanism is constructed to pull deep and shallow features for alignment. Finally, we implement ablation study on the key components of method and comparison experiment with other advanced methods using five evaluation metrics. Results show that the key components play an important role in multiscale information fusion. It promotes the accuracy of cloud image feature extraction while reducing the loss of detailed features. Generalization performance verification indicates the excellent performance of the proposed model in advanced cloud feature extraction and cloud-mask generation.

Cultural relic reconstruction device based on ORB-SLAM

ORB-SLAM, an advanced visual SLAM technology, has excellent positioning and mapping capabilities [1]. In the reconstruction of cultural relics, ORB-SLAM technology can achieve autonomous perception and modeling of the surrounding environment of cultural relics by fusing and analyzing the images of monocular depth cameras, thereby reconstructing the 3D model of cultural relics, providing new possibilities for the protection and inheritance of cultural relics. Point cloud images are a digital technology used for 3D modeling, which can save point data on object surfaces and their relationships as 3D point clouds [2]. In terms of cultural relic reconstruction, point cloud images can restore and reconstruct the shape, size, texture, and other features of cultural relics by processing the collected point data on the surface of cultural relics into a 3D model. The purpose of this device is to reconstruct cultural relics by generating point cloud images based on ORB-SLAM technology. Combined with the designed handheld pan tilt device, it can achieve multi-angle point cloud mapping of cultural relics scenes in complex terrain. This paper will introduce the handheld gimbal design of the device, the realization of point cloud mapping based on the ORB-SLAM3 algorithm [3], and the application of point cloud scenes. Through the research in this article, it is expected to provide a new and efficient method for cultural relic reconstruction in the field of cultural relic protection and make contributions to the protection and inheritance of cultural heritage.

Automatic segment-wise restoration for wide irregular stripe noise in SDGSAT-1 multispectral data using side-slither data

The raw orbital images captured by the new launched SDGSAT-1 Multispectral Image for Inshore (MII) are plagued by wide-irregular stripe noise, due to inconsistent unit response. This paper proposed a new method for destriping wide-irregular stripe noise in MII using the characteristics of side-slither data. Firstly, the raw side-slither data was standardized using line detection to guarantee that each row observed the same ground object. Then, the whole orbital side-slither image was segmented into blocks of equal length, and it was found that the response of wide-irregular stripe noise is consistent within a certain length. The Inverse Distance Weight was used to interpolate the DN values of striped pixels as referenced values, and the segmented length was determined by calculating Pearson correlation coefficient between the original and referenced DN values. Thirdly, the Random Sample Consensus (RANSAC) algorithm was used to find the inliers and calculate the correction parameters, after it was discovered that the original and referenced DN values had a linear correlation. The proposed method, SIR (consists of image segmentation, pixel interpolation and RANSAC fitting), can directly destripe the raw orbital image. One orbital side-slither data and six ordinary orbital data were selected for verification. Twelve state-of-the-art methods were chosen for comparison with SIR. The accuracy scores of SIR on three assessment indexes were higher than those of twelve other methods. The destriping outcomes for the images of city, cloud, forest, and river demonstrated the effectiveness of SIR in correcting wide-irregular stripe noise in MII images.

Terahertz 3D point cloud imaging for complex targets.

The reconstruction of complex targets using terahertz technology is often hindered by diffraction and interference of electromagnetic waves, leading to the loss of fine target details. In this research article, we have introduced a terahertz synthetic aperture radar (SAR) imaging method that integrates an iterative closest point (ICP) algorithm, referred to as SAR-ICP, to achieve accurate reconstruction of intricate target structures. To accomplish this, multiple sets of point cloud data are acquired by varying the illumination viewpoint. The ICP algorithm is then employed to align and fuse these datasets, resulting in the generation of high-quality three-dimensional (3D) images. The experimental results validate the effectiveness of the proposed SAR-ICP method. The information entropy of the reconstructed 3D image using the SAR-ICP is approximately 0.05 times that of the conventional SAR method, indicating a superior image quality. In the future, we anticipate the widespread application of this method in areas such as security inspection, non-destructive testing, and other complex scenarios.

Automatic quantitative recognition method for vertical concealed cracks in asphalt pavement based on feature pixel points and 3D reconstructions

Non-destructive detection and characterisation of concealed cracks in asphalt pavements is a key aspect of ground-penetrating radar (GPR) road maintenance. This paper proposes a method based on feature pixel points to quantify and calculate the vertical height of cracks. Firstly, the team used gprMax to conduct numerical simulations to study the GPR image characteristics of concealed cracks in asphalt pavement with varying lengths and widths. Subsequently, the relationship between the pixel value of the crack area and the two-way travel time was established to obtain the relationship between the vertical height of the crack and the pixel. This study combined with the deep learning model (YOLOv5) allows for the calculation of the vertical height of a crack while simultaneously recognizing it, with the minimum error being only 1.3 %. Finally, concealed cracks were visualized in three-dimensional(3D) by slicing the results of vertical crack simulations of asphalt pavement and observing cloud images, and the principle of computed tomography(CT) was employed to reconstruct 3D models of cracked asphalt pavement and estimate the vertical height of the cracks. This method achieved a minimum error of only 2.9 %. The research presents a theoretical framework for recognizing and more intuitive characterizing concealed cracks in asphalt pavement accurately, enabling it to be used in practical engineering applications.