RGB Images Research Articles

Integration of weighted majority voting in machine learning algorithms to enhance pine tree crown mapping on UAV imagery

Abstract. The shape and area of the crown of each tree are among the most influential parameters for identifying and controlling the processes of photosynthesis, respiration, transpiration and its management. In such a way that various physiographic functions, such as carbon dioxide absorption, light energy absorption, oxygen release and transpiration, which are vital for the growth and development of the tree, are done in the crown. In this research, the RGB image of the UAV with a spatial resolution of 2 cm was resampled to three pixel sizes of 10, 30 and 50 cm. Then, each image was classified separately by SVM, ANN and MLC algorithms, which are all part of Ensemble. In the next step, each of the obtained crowns was compared with the digitization of the same crown, and based on the area of the crown obtained from each classification and normalization method, the weight was obtained specifically for the same crown. Finally, by using the weighted majority voting method, classifications were fusioned at the decision level. The results showed that the ANN method gives better results in all pixel sizes compared to MLC and SVM. Also, the combination of different classification methods with the weighted majority voting method based on the weight assigned to the same crown based on each classification method has significantly increased the classification accuracy of the tree crown in all the sizes of the analyzed pixels.

Wheat growth stage identification method based on multimodal data

Accurate identification of crop growth stages is a crucial basis for implementing effective cultivation management. With the development of deep learning techniques in image understanding, research on intelligent real-time recognition of crop growth stages based on RGB images has garnered significant attention. However, the small differences and high similarity in crop morphological characteristics during the transition between adjacent growth stages pose challenges for accurate identification. To address this issue, this study proposes a multi-scale convolutional neural network model, termed MultiScalNet-Wheat (MSN-W), which enhances the algorithm's ability to learn complex features by utilizing multi-scale convolution and attention mechanisms. This model extracts key information from redundant data to identify winter wheat growth stages in complex field environments. Experimental results show that the MSN-W model achieves a recognition accuracy of 97.6 %, outperforming typical convolutional neural network models such as VGG19, ResNet50, MobileNetV3, and DenseNet. To further address the difficulty in recognizing growth stages during transition periods, where canopy morphological features are highly similar and show small differences, this paper introduces an innovative approach by incorporating sequential environmental data related to wheat growth stages. By extracting these features and performing multi-modal collaborative inference, a multi-modal feature-based wheat growth stage recognition model, termed MultiModalNet-Wheat (MMN-W), is constructed on the basis of the MSN-W model. Experimental results indicate that the MMN-W model achieves a recognition accuracy of 98.5 %, improving by 0.9 % over the MSN-W model. Both the MSN-W and MMN-W models provide accurate methods for observing wheat growth stages, thereby supporting the scientific management of winter wheat at different growth stages.

Crop Classification from Drone Imagery Based on Lightweight Semantic Segmentation Methods

Technological advances have dramatically improved precision agriculture, and accurate crop classification is a key aspect of precision agriculture (PA). The flexibility and real-time nature of UAVs have led them to become an important tool for acquiring agricultural data and enabling precise crop classification. Currently, crop identification relies heavily on complex high-precision models that often struggle to provide real-time performance. Research on lightweight models specifically for crop classification is also limited. In this paper, we propose a crop classification method based on UAV visible-light images based on PP-LiteSeg, a lightweight model proposed by Baidu. To improve the accuracy, a pyramid pooling module is designed in this paper, which integrates adaptive mean pooling and CSPC (Convolutional Spatial Pyramid Pooling) techniques to handle high-resolution features. In addition, a sparse self-attention mechanism is employed to help the model pay more attention to locally important semantic regions in the image. The combination of adaptive average pooling and the sparse self-attention mechanism can better handle different levels of contextual information. To train the model, a new dataset based on UAV visible-light images including nine categories such as rice, soybean, red bean, wheat, corn, poplar, etc., with a time span of two years was created for accurate crop classification. The experimental results show that the improved model outperforms other models in terms of accuracy and prediction performance, with a MIoU (mean intersection ratio joint) of 94.79%, which is 2.79% better than the original model. Based on the UAV RGB images demonstrated in this paper, the improved model achieves a better balance between real-time performance and accuracy. In conclusion, the method effectively utilizes UAV RGB data and lightweight deep semantic segmentation models to provide valuable insights for crop classification and UAV field monitoring.

Diagnosing Fungal Infection in Wheat Kernels by Integrating Spectroscopic Technology and Digital Color Imaging System: Artificial Neural Network, Principal Component Analysis and Correlation Feature Selection Techniques

ABSTRACTContamination of cereal grain, especially wheat, with fungal infections can cause significant economic impacts and it endangers the health of humans and livestock. This study aims to appraise the UV/VIS–NIR and digital color (RGB) imaging systems and spectroscopic methodology to detect wheat kernels infected by fungi such as Penicillium expansum and Fusarium graminearum. NIR spectra of 190–1100 nm at 10 nm intervals, visible color reflectance images and non‐visible reflectance images of wheat kernels in the ultraviolet and near‐infrared ranges were applied to develop the multi‐layer perceptron (MLP) artificial neural network model. The optimum wavelengths were selected by application of the principal component analysis (PCA) after preprocessing the raw spectra. A confusion matrix was used in the correlation feature selection method (CFS) for the decision tree classifier of selected features. The results showed that the four UV wavelengths of 310, 330, 400, and 410 nm were the best wavelengths using PCA to distinguish healthy and unhealthy wheat kernels. Considering the intensity of the wavelengths as the neural network inputs, samples were classified into healthy and unhealthy categories with an accuracy of 90.9%. Also, 18 features of color images in RGB, LAB, HSV, HSI, YCbCr, and YIQ spaces provided the highest average accuracy of 44.4% in classifying healthy and infected wheat kernels by using a CCD Proline camera in the ultraviolet range. In contrast, other cameras in the visible and invisible range showed low accuracy. Furthermore, the best classification accuracy of the healthy and infected samples by the use of the CFS method was obtained at 88.1%. Based on the findings, spectroscopic methodology proved to be highly effective for detecting, classifying and automatic cleaning of various agricultural seeds, with a particular emphasis on wheat kernals.

Investigation of radiogenic heat production in granites of the Goiás Tin Province, Central Brazil

This study defines the radiogenic heat production of A-type granites in the Goiás Tin Province (GTP), Central Brazil, using airborne gamma-ray spectrometry. Pedra Branca Massif and Serra Dourada Granite are rich in tin, rare earth elements, and they exhibit anomalous radiogenic heat (5.5–15 µW/m³). They are therefore classified as high heat production granites (HHPG). By integrating radiogenic heat data, RGB imaging, magnetometry, density model and geological information, we associated anomalous radiogenic heat with mineralized regions present in the granites found in the GTP. Our methodology was validated using geological information, density model and other granites worldwide. It proved to be effective for targeting HHP granites.

A perception-guided CNN for grape bunch detection

Precision Viticulture (PV) is becoming an active and interdisciplinary research field since it requires solving interesting research issues to concretely answer the demands of specific use cases. A challenging problem in this context is the development of automatic methods for yield estimation. Computer vision methods can contribute to the accomplishment of this task, especially those that can replicate what winemakers do manually. In this paper, an automatic artificial intelligence method for grape bunch detection from RGB images is presented. A customized Convolutional Neural Network (CNN) is employed for pointwise classification of image pixels and the dependence of classification results on the type of input color channels and grapes color properties are studied. The advantage of using additional perception-based input features, such as luminance and visual contrast, is also evaluated, as well as the dependence of the method on the choice of the training set in terms of the amount of labeled data. The latter point has a significant impact on the practical use of the method on-site, its usability by non-expert users, and its adaptability to individual vineyards. Experimental results show that a properly trained CNN can discriminate and detect grape bunches even under uncontrolled acquisition conditions and with limited computational load, making the proposed method implementable on smart devices and suitable for on-site and real-time applications.

Salient Object Detection From Arbitrary Modalities.

Toward desirable saliency prediction, the types and numbers of inputs for a salient object detection (SOD) algorithm may dynamically change in many real-life applications. However, existing SOD algorithms are mainly designed or trained for one particular type of inputs, failing to be generalized to other types of inputs. Consequentially, more types of SOD algorithms need to be prepared in advance for handling different types of inputs, raising huge hardware and research costs. Differently, in this paper, we propose a new type of SOD task, termed Arbitrary Modality SOD (AM SOD). The most prominent characteristics of AM SOD are that the modality types and modality numbers will be arbitrary or dynamically changed. The former means that the inputs to the AM SOD algorithm may be arbitrary modalities such as RGB, depths, or even any combination of them. While, the latter indicates that the inputs may have arbitrary modality numbers as the input type is changed, e.g. single-modality RGB image, dual-modality RGB-Depth (RGB-D) images or triple-modality RGB-Depth-Thermal (RGB-D-T) images. Accordingly, a preliminary solution to the above challenges, i.e. a modality switch network (MSN), is proposed in this paper. In particular, a modality switch feature extractor (MSFE) is first designed to extract discriminative features from each modality effectively by introducing some modality indicators, which will generate some weights for modality switching. Subsequently, a dynamic fusion module (DFM) is proposed to adaptively fuse features from a variable number of modalities based on a novel Transformer structure. Finally, a new dataset, named AM-XD, is constructed to facilitate research on AM SOD. Extensive experiments demonstrate that our AM SOD method can effectively cope with changes in the type and number of input modalities for robust salient object detection. Our code and AM-XD dataset will be released on https://github.com/nexiakele/AMSODFirst.

On-the-go table grape ripeness estimation via proximal snapshot hyperspectral imaging

The monitoring of grapes for ripeness estimation is a practice that enables fruit harvesting at the optimal time. Hyperspectral Imaging (HSI) represents a non-destructive and high-throughput alternative to traditional laboratory analyses. Current literature approaches perform hyperspectral measurements using line scan sensors or low-resolution static snapshot cameras, which hinder a fast per-bunch ripeness characterization. We propose a framework for on-the-go collection and processing of proximal snapshot hyperspectral images to estimate single bunch ripeness parameters. Focusing on table grapes (Vitis vinifera L. cv. Red Globe), we collected images under natural illumination with a hyperspectral camera (500–900 nm) mounted on a moving vehicle in an experimental block sited in Piacenza, Italy. We investigated images collected in August and September 2021 representing two ripening stages. The composition of the imaged grape bunches was determined through laboratory chemical analyses to predict Total Soluble Solids (TSS) and anthocyanin concentration. The images were pre-processed via multimodal image registration to correct the unalignment of bands due to the vehicle motion, and the single bunches were automatically identified on false RGB images through a Mask Region-Convolutional Neural Network (Mask R-CNN) instance segmentation network. The mean spectra of the bunches were used as input features of a Partial Least Squares Regression (PLSR) model to predict the chemical parameters at single bunch and whole vine scales. The regression model of TSS had an R2 (10-fold nested cross-validation) of 0.75 and 0.85 on a per-bunch and per-vine basis, respectively. The regression model of anthocyanin had an R2 of 0.68 and 0.49 on a per-bunch and per-vine basis, respectively. The results suggest the potential of using snapshot hyperspectral images for high-throughput analysis of a per-bunch grape ripeness estimation. The method described in this study could give valuable information to improve grape ripening monitoring and management of harvest operations and even allow for precise and automated robotic harvesting.

Assessment of Drought Response in Kimchi Cabbage, Radish, and Lettuce Seedlings Using RGB Image Analysis

Assessment of Drought Response in Kimchi Cabbage, Radish, and Lettuce Seedlings Using RGB Image Analysis

Multimodal Deep Learning Integration of Image, Weather, and Phenotypic Data Under Temporal Effects for Early Prediction of Maize Yield

Maize (Zea mays L.) has been shown to be sensitive to temperature deviations, influencing its yield potential. The development of new maize hybrids resilient to unfavourable weather is a desirable aim for crop breeders. In this paper, we showcase the development of a multimodal deep learning model using RGB images, phenotypic, and weather data under temporal effects to predict the yield potential of maize before or during anthesis and silking stages. The main objective of this study was to assess if the inclusion of historical weather data, maize growth captured through imagery, and important phenotypic traits would improve the predictive power of an established multimodal deep learning model. Evaluation of the model performance when training from scratch showed its ability to accurately predict ~89% of hybrids with high-yield potential and demonstrated enhanced explanatory power compared with previously published models. Shapley Additive explanations (SHAP) analysis indicated the top influential features include plant density, hybrid placement in the field, date to anthesis, parental line, temperature, humidity, and solar radiation. Including weather historical data was important for model performance, significantly enhancing the predictive and explanatory power of the model. For future research, the use of the model can move beyond maize yield prediction by fine-tuning the model on other crop data, serving as a potential decision-making tool for crop breeders to determine high-performing individuals from diverse crop types.

Multi-modal scene graph inspired policy for visual navigation

Visual navigation needs the agent locate the given target with visual perception. To enable robots to effectively execute tasks, combining large language models (LLMs) with multi-modal inputs in navigation is necessary. While LLMs offer rich semantic knowledge, they lack specific real-world information and real-time interaction capabilities. This paper introduces a Multi-modal Scene Graph (MMSG) navigation framework that aligns LLMs with visual perception models to predict next steps. Firstly, a multi-modal scene dataset is constructed, containing triplets of object-relations-target words. We provide target words and lists of existing objects in the scene to generate a large number of instructions and corresponding action plans for GPT-\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$-$$\\end{document}3.5. The generated data is then utilized for pre-train LLM for path planning. During inference, we discover objects in the scene by extending the DETR visual object detector to multi-view RGB image collected from different reachable positions. Experimental results show that path planning generated from MMSG outperforms state-of-the-art methods, indicating its feasibility in complex environments. We evaluate our methods on the ProTHOR dataset and show superior navigation performance.

Using a Yolov8-Based Object Detection Model for an Automatic Garbage Sorting System

This study presents the development of an automatic garbage sorting system based on computer vision techniques and the You Only Look Once version 8 (YOLOv8) algorithm. RGB images of garbage, including cans and plastic bottles, were collected by a camera. The YOLOv8 model, trained on a dataset of both RGB images of cans and plastic bottles, acts as the core for detection and classification. During real-time sorting, the classification results and picking points from the YOLOv8 model are used by a gantry robot. The gantry robot is developed and controlled by a Programmable Logic Controller (PLC) to place each type of garbage into its respective categories. A conveyor tracking algorithm was implemented to pick up garbage moving on the conveyor. The system's performance is evaluated using 20 garbage samples for the can and the plastic bottle category. The accuracy, precision, recall, and F1 score were 92.5%, 90.5%, 95%, and 92.7%, respectively. The system proposed in this study can be modified to sort other garbage types.

A dataset of drone-captured, segmented images for oil spill detection in port environments

The high incidence of oil spills in port areas poses a serious threat to the environment, prompting the need for efficient detection mechanisms. Utilizing automated drones for this purpose can significantly improve the speed and accuracy of oil spill detection. Such advancements not only expedite cleanup operations, reducing environmental harm but also enhance polluter accountability, potentially deterring future incidents. Currently, there’s a scarcity of datasets employing RGB images for oil spill detection in maritime settings. This paper presents a unique, annotated dataset aimed at addressing this gap, leveraging a neural network for analysis on both desktop and edge computing platforms. The dataset, captured via drone, comprises 1268 images categorized into oil, water, and other, with a convolutional neural network trained using an Unet model architecture achieving an F1 score of 0.71 for oil detection. This underscores the dataset’s practicality for real-world applications, offering crucial resources for environmental conservation in port environments.

Embedded System‐Based Malaria Detection From Blood Smear Images Using Lightweight Deep Learning Model

ABSTRACTThe disease of malaria, transmitted by female Anopheles mosquitoes, is highly contagious, resulting in numerous deaths across various regions. Microscopic examination of blood cells remains one of the most accurate methods for malaria diagnosis, but it is time‐consuming and can produce inaccurate results occasionally. Due to machine learning and deep learning advances in medical diagnosis, improved diagnostic accuracy can now be achieved while costs can be reduced compared to conventional microscopy methods. This work utilizes an open‐source dataset with 26 161 blood smear images in RGB for malaria detection. Our preprocessing resized the original dimensions of the images into 64 × 64 due to the limitations in computational complexity in developing embedded systems‐based malaria detection. We present a novel embedded system approach using 119 154 trainable parameters in a lightweight 17‐layer SqueezeNet model for the automatic detection of malaria. Incredibly, the model is only 1.72 MB in size. An evaluation of the model's performance on the original NIH malaria dataset shows that it has exceptional accuracy, precision, recall, and F1 scores of 96.37%, 95.67%, 97.21%, and 96.44%, respectively. Based on a modified dataset, the results improved further to 99.71% across all metrics. Compared to current deep learning models, our model significantly outperforms them for malaria detection, making it ideal for embedded systems. This model has also been rigorously tested on the Jetson Nano B01 edge device, demonstrating a rapid single image prediction time of only 0.24 s. The fusion of deep learning with embedded systems makes this research a crucial step toward improving malaria diagnosis. In resource‐constrained settings, the model's lightweight architecture and accuracy enhancements hold great promise for addressing the critical challenge of malaria detection.

Feature fusion and context interaction for RGB-D indoor semantic segmentation

Depth information in indoor scenes is crucial to understanding the spatial and occluding relationships of objects. Depth information contains rich geometric details and is unaffected by lighting, texture, and color, significantly enhancing traditional color images. RGB images and depth images have been widely applied in various image analysis. However, effectively leveraging the complementary information from both modalities remains a challenge. To address this issue, we propose a unified and effective feature fusion and context interaction network (FCINet). It includes three modules. Firstly, we construct a multi-modal feature fusion module (MFFM), which can achieve the aggregation of two modalities along spatial and channel dimensions. Secondly, we construct a global and local information interaction context module (GLCM) to encode rich semantic information. Finally, we construct a feature alignment and fusion module (FAFM), which integrates upsampled low-level features with high-level features to mitigate spatial information loss. Experimental results indicate that the model achieved favorable outcomes on both the NYU Depth v2 dataset and the more complex SUN RGB-D dataset.

Estimation of Strawberry Canopy Volume in Unmanned Aerial Vehicle RGB Imagery Using an Object Detection-Based Convolutional Neural Network.

Estimating canopy volumes of strawberry plants can be useful for predicting yields and establishing advanced management plans. Therefore, this study evaluated the spatial variability of strawberry canopy volumes using a ResNet50V2-based convolutional neural network (CNN) model trained with RGB images acquired through manual unmanned aerial vehicle (UAV) flights equipped with a digital color camera. A preprocessing method based on the You Only Look Once v8 Nano (YOLOv8n) object detection model was applied to correct image distortions influenced by fluctuating flight altitude under a manual maneuver. The CNN model was trained using actual canopy volumes measured using a cylindrical case and small expanded polystyrene (EPS) balls to account for internal plant spaces. Estimated canopy volumes using the CNN with flight altitude compensation closely matched the canopy volumes measured with EPS balls (nearly 1:1 relationship). The model achieved a slope, coefficient of determination (R2), and root mean squared error (RMSE) of 0.98, 0.98, and 74.3 cm3, respectively, corresponding to an 84% improvement over the conventional paraboloid shape approximation. In the application tests, the canopy volume map of the entire strawberry field was generated, highlighting the spatial variability of the plant's canopy volumes, which is crucial for implementing site-specific management of strawberry crops.

Malware Detection and Classification using Generative Adversarial Network

The Generative Adversarial Networks (GANs) are playing a crucial role in deep-learning-based malware classification to overcome the dataset imbalance and unseen malware. The Generative AI is preferably used in many applications, such as improving image resolution and generating audio, video, and text. The cybercriminals are also using the Generative AI for generating the malware and deepfake videos to harm the targeted person or device. By generating the synthetic data, it makes the deep learning model more robust to detect such types of unseen and adversarial attacks. This work utilizes GANs for generating adversarial malware samples to train a classification and detection model, improving the model’s ability to identify sophisticated malware variants. The performance of the proposed Conditional Generative Adversarial Network (CGAN) model is evaluated on a multiclass malware grayscale image dataset and a binary class malware RGB image dataset. The performance of proposed model is compared with current state-of-the-art. Results indicate a significant improvement in classification accuracy and a reduction in training time and false positives, showcasing GAN’s potential in the dynamic cybersecurity landscape.

Polarization remote sensing of ships in sea haze

ABSTRACT Sea haze reduces the visibility of ships and makes the optical imaging system blind, which increases the difficulty of ship remote sensing. In this paper, we propose a new method for polarization remote sensing of ships in sea haze. The method takes advantage of image haze removal and exposure correction to improve the visibility of ships in sea haze. Different from the existing polarization-based image haze removal methods, the proposed method does not depend on any assumptions about the polarization properties of incident light. The exposure correction method is proposed based on the Retinex model and Bayes theorem, which can improve the visibility of ships in haze removal images. To evaluate the effectiveness of the proposed method, we implemented an outdoor experiment on polarization remote sensing of ships in sea haze weather. The experimental results indicate that the method can improve the visibility of ships significantly. In addition, the method outperforms the Stokes parameter-based ship remote sensing method in terms of the image quality metric fog aware density evaluator and can be easily generalized to RGB image haze removal.

Fine-tuning GAN Models with Unpaired Aerial Images for RGB to NDVI Translation in Vegetation Index Estimation

Calculating the Normalized Difference Vegetation Index (NDVI) requires expensive multispectral cameras, posing challenges such as high costs and the need for technical expertise. This research aims to develop a method to transform RGB images obtained from Unmanned Aerial Vehicle (UAV) into NDVI images. Our proposed method leverages the CycleGAN model, an unsupervised image-to-image translation framework, to learn the intricate mapping between RGB values and NDVI. The model was trained on unpaired datasets of RGB and NDVI images, sourced from paddy fields located in Gresik and Yogyakarta, Indonesia. This process successfully encapsulated the complex correlation between these two modalities. Various training strategies were systematically investigated, including weight initialization schemes, fine-tuning procedures, and learning rate policies, to optimize the model's performance. The fine-tuned CycleGAN demonstrated superior performance in creating synthetic NDVI images from unpaired dataset, surpassing other methods in terms of fidelity, quality, and structural coherence. The results were impressive, with a Normalized Root Mean Square Error (NRMSE) of 0.327, a Peak Signal-to-Noise Ratio (PSNR) of 16.330, an Oriented FAST and Rotated BRIEF (ORB) score of 0.859, and a Structural Similarity Index (SSIM) of 0.757. The best performing CycleGAN model was then deployed on a low-spec microcomputer device, specifically the Raspberry Pi 4B with an average computation time of 21.0077 seconds. Raspberry Pi 4B was chosen for its lightweight, compact dimensions, and compatibility with efficient battery power connections, making it suitable for drone deployment.

Mapping Forest Parameters to Model the Mobility of Terrain Vehicles

This study aims to evaluate the feasibility of using non-contact data collection methods—specifically, UAV (unmanned aerial vehicle)-based and terrestrial laser scanning technologies—to assess forest stand passability, which is crucial for military operations. The research was conducted in a mixed forest stand in the Březina military training area, where the position of trees and their DBHs (Diameter Breast Heights) were recorded. The study compared the effectiveness of different methods, including UAV RGB imaging, UAV-LiDAR, and handheld mobile laser scanning (HMLS), in detecting tree positions and estimating DBH. The results indicate that HMLS data provided the highest number of detected trees and the most accurate positioning relative to the reference measurements. UAV-LiDAR showed better tree detection compared to UAV RGB imaging, though both aerial methods struggled with canopy penetration in densely structured forests. The study also found significant variability in DBH estimation, especially in complex forest stands, highlighting the challenges of accurate tree detection in diverse environments. The findings suggest that while current non-contact methods show promise, further refinement and integration of data sources are necessary to improve their applicability for assessing forest passability in military or rescue contexts.