RGB Images Research Articles

Multimodal Data Fusion for Precise Lettuce Phenotype Estimation Using Deep Learning Algorithms

Effective lettuce cultivation requires precise monitoring of growth characteristics, quality assessment, and optimal harvest timing. In a recent study, a deep learning model based on multimodal data fusion was developed to estimate lettuce phenotypic traits accurately. A dual-modal network combining RGB and depth images was designed using an open lettuce dataset. The network incorporated both a feature correction module and a feature fusion module, significantly enhancing the performance in object detection, segmentation, and trait estimation. The model demonstrated high accuracy in estimating key traits, including fresh weight (fw), dry weight (dw), plant height (h), canopy diameter (d), and leaf area (la), achieving an R2 of 0.9732 for fresh weight. Robustness and accuracy were further validated through 5-fold cross-validation, offering a promising approach for future crop phenotyping.

LIDeepDet: Deepfake Detection via Image Decomposition and Advanced Lighting Information Analysis

The proliferation of AI-generated content (AIGC) has empowered non-experts to create highly realistic Deepfake images and videos using user-friendly software, posing significant challenges to the legal system, particularly in criminal investigations, court proceedings, and accident analyses. The absence of reliable Deepfake verification methods threatens the integrity of legal processes. In response, researchers have explored deep forgery detection, proposing various forensic techniques. However, the swift evolution of deep forgery creation and the limited generalizability of current detection methods impede practical application. We introduce a new deep forgery detection method that utilizes image decomposition and lighting inconsistency. By exploiting inherent discrepancies in imaging environments between genuine and fabricated images, this method extracts robust lighting cues and mitigates disturbances from environmental factors, revealing deeper-level alterations. A crucial element is the lighting information feature extractor, designed according to color constancy principles, to identify inconsistencies in lighting conditions. To address lighting variations, we employ a face material feature extractor using Pattern of Local Gravitational Force (PLGF), which selectively processes image patterns with defined convolutional masks to isolate and focus on reflectance coefficients, rich in textural details essential for forgery detection. Utilizing the Lambertian lighting model, we generate lighting direction vectors across frames to provide temporal context for detection. This framework processes RGB images, face reflectance maps, lighting features, and lighting direction vectors as multi-channel inputs, applying a cross-attention mechanism at the feature level to enhance detection accuracy and adaptability. Experimental results show that our proposed method performs exceptionally well and is widely applicable across multiple datasets, underscoring its importance in advancing deep forgery detection.

Fusion of UAV-Acquired Visible Images and Multispectral Data by Applying Machine-Learning Methods in Crop Classification

The sustainable development of agriculture is closely related to the adoption of precision agriculture techniques, and accurate crop classification is a fundamental aspect of this approach. This study explores the application of machine learning techniques to crop classification by integrating RGB images and multispectral data acquired by UAVs. The study focused on five crops: rice, soybean, red bean, wheat, and corn. To improve classification accuracy, the researchers extracted three key feature sets: band values and vegetation indices, texture features extracted from a grey-scale co-occurrence matrix, and shape features. These features were combined with five machine learning models: random forest (RF), support vector machine (SVM), k-nearest neighbour (KNN) based, classification and regression tree (CART) and artificial neural network (ANN). The results show that the Random Forest model consistently outperforms the other models, with an overall accuracy (OA) of over 97% and a significantly higher Kappa coefficient. Fusion of RGB images and multispectral data improved the accuracy by 1–4% compared to using a single data source. Our feature importance analysis showed that band values and vegetation indices had the greatest impact on classification results. This study provides a comprehensive analysis from feature extraction to model evaluation, identifying the optimal combination of features to improve crop classification and providing valuable insights for advancing precision agriculture through data fusion and machine learning techniques.

Accurately Segmenting/Mapping Tobacco Seedlings Using UAV RGB Images Collected from Different Geomorphic Zones and Different Semantic Segmentation Models

The tobacco seedling stage is a crucial period for tobacco cultivation. Accurately extracting tobacco seedlings from satellite images can effectively assist farmers in replanting, precise fertilization, and subsequent yield estimation. However, in complex Karst mountainous areas, it is extremely challenging to accurately segment tobacco plants due to a variety of factors, such as the topography, the planting environment, and difficulties in obtaining high-resolution image data. Therefore, this study explores an accurate segmentation model for detecting tobacco seedlings from UAV RGB images across various geomorphic partitions, including dam and hilly areas. It explores a family of tobacco plant seedling segmentation networks, namely, U-Net, U-Net++, Linknet, PSPNet, MAnet, FPN, PAN, and DeepLabV3+, using the Hill Seedling Tobacco Dataset (HSTD), the Dam Area Seedling Tobacco Dataset (DASTD), and the Hilly Dam Area Seedling Tobacco Dataset (H-DASTD) for model training. To validate the performance of the semantic segmentation models for crop segmentation in the complex cropping environments of Karst mountainous areas, this study compares and analyzes the predicted results with the manually labeled true values. The results show that: (1) the accuracy of the models in segmenting tobacco seedling plants in the dam area is much higher than that in the hilly area, with the mean values of mIoU, PA, Precision, Recall, and the Kappa Coefficient reaching 87%, 97%, 91%, 85%, and 0.81 in the dam area and 81%, 97%, 72%, 73%, and 0.73 in the hilly area, respectively; (2) The segmentation accuracies of the models differ significantly across different geomorphological zones; the U-Net segmentation results are optimal for the dam area, with higher values of mIoU (93.83%), PA (98.83%), Precision (93.27%), Recall (96.24%), and the Kappa Coefficient (0.9440) than those of the other models; in the hilly area, the U-Net++ segmentation performance is better than that of the other models, with mIoU and PA of 84.17% and 98.56%, respectively; (3) The diversity of tobacco seedling samples affects the model segmentation accuracy, as shown by the Kappa Coefficient, with H-DASTD (0.901) > DASTD (0.885) > HSTD (0.726); (4) With regard to the factors affecting missed segregation, although the factors affecting the dam area and the hilly area are different, the main factors are small tobacco plants (STPs) and weeds for both areas. This study shows that the accurate segmentation of tobacco plant seedlings in dam and hilly areas based on UAV RGB images and semantic segmentation models can be achieved, thereby providing new ideas and technical support for accurate crop segmentation in Karst mountainous areas.

Abstract 4131796: Non-Contact Biometric System for Early Detection of Hypertension and Diabetes Using AI and RGB Imaging

Introduction: In recent years, the development of wearable devices and bioinformation analysis applications utilizing smartphone sensors has seen significant advancements. Despite these technological strides, their adoption and sustained use remain limited to health-conscious individuals. The general populace, particularly those indifferent to health management, stands to miss out on the potential benefits of wearable technologies due to a lack of engagement. This disconnect underscores the necessity for more accessible health monitoring solutions. Research Questions and Goals: This study aims to bridge the gap in health monitoring accessibility by introducing a system capable of non-invasive, non-contact acquisition of biometric information. Such a system seeks to democratize the early detection and prevention of diseases across various demographics. Methods and Results: A prospective cohort clinical study was conducted in a hospital setting with approximately 200 consenting patients and healthy controls. Images of the subjects' faces and hands were captured with a high-speed camera, and blood pressure measurements were taken simultaneously for correlation. A machine learning algorithm was developed to analyze skin blood flow and spectral features from these images to identify potential diseases. The study specifically focused on early detection of hypertension and diabetes. Hypertension was categorized for individuals with systolic blood pressure ≥115 mmHg or diastolic blood pressure ≥75 mmHg. Diabetes was identified in subjects with HbA1c levels ≥6.5 or those previously diagnosed. The algorithm's performance in detecting hypertension showed a 94.2% in alignment with the AHA guidelines for stage1 hypertension, based on pulse wave features. For a subset including 40% early hypertension cases, accuracy rates were 86.2% for 30-second data segments and 80.9% for 5-second segments. For diabetes detection, the application of the algorithm to video data achieved a 75.3%, utilizing blood flow patterns as markers. Hypertension and diabetes were analyzed using different features. Conclusions: The integration of AI algorithms with video data analysis of skin and blood vessels demonstrates a promising avenue for the non-contact, early detection of hypertension and diabetes. This approach not only offers a viable non-contact monitoring technology, but also represents a significant leap towards inclusive, accessible health care prevention and management strategies.

Submarine Landslide Identification Based on Improved DeepLabv3 with Spatial and Channel Attention

As one of the most destructive, hazardous, and frequent marine geohazards, correctly recognizing submarine landslides holds substantial importance for regional risk assessment, disaster prevention, and marine resource development. Many conventional approaches to prediction and mapping necessitate the involvement of expert insights, oversight, and extensive field investigations, which can result in significant time and effort invested in the prediction process. This paper focuses on employing a deep neural network semantic segmentation technique to detect submarine landslides to replace previous methods, such as numerical analysis and physical modeling, to predict and identify the landslide areas quickly. The peripheral zone of the western Iberian Sea is selected as the study area. Since the neural network image recognition task usually requires RGB images as input data, factors such as slope, hillshade, and elevation extracted from digital elevation model (DEM) data are used to synthesize RGB images through band synthesis methods, and the number and diversity of data are increased utilizing data enhancement. Based on the classical semantic segmentation model DeepLabV3, this paper proposes an improved deep learning method, which strengthens the ability of model feature extraction for complex situations by adding an attention mechanism module, improving the spatial pyramid pooling module, and improving the landslide intersection over union metric from 0.4257 to 0.5219 and the F1-score metric from 0.609 to 0.6631 to achieve effective identification of submarine landslides.

LiDAR-360 RGB Camera-360 Thermal Camera Targetless Calibration for Dynamic Situations

Integrating multiple types of sensors into autonomous systems, such as cars and robots, has become a widely adopted approach in modern technology. Among these sensors, RGB cameras, thermal cameras, and LiDAR are particularly valued for their ability to provide comprehensive environmental data. However, despite their advantages, current research primarily focuses on the one or combination of two sensors at a time. The full potential of utilizing all three sensors is often neglected. One key challenge is the ego-motion compensation of data in dynamic situations, which results from the rotational nature of the LiDAR sensor, and the blind spots of standard cameras due to their limited field of view. To resolve this problem, this paper proposes a novel method for the simultaneous registration of LiDAR, panoramic RGB cameras, and panoramic thermal cameras in dynamic environments without the need for calibration targets. Initially, essential features from RGB images, thermal data, and LiDAR point clouds are extracted through a novel method, designed to capture significant raw data characteristics. These extracted features then serve as a foundation for ego-motion compensation, optimizing the initial dataset. Subsequently, the raw features can be further refined to enhance calibration accuracy, achieving more precise alignment results. The results of the paper demonstrate the effectiveness of this approach in enhancing multiple sensor calibration compared to other ways. In the case of a high speed of around 9 m/s, some situations can improve the accuracy about 30 percent higher for LiDAR and Camera calibration. The proposed method has the potential to significantly improve the reliability and accuracy of autonomous systems in real-world scenarios, particularly under challenging environmental conditions.

SYMPATHIQUE: image-based tracking of symptoms and monitoring of pathogenesis to decompose quantitative disease resistance in the field

BackgroundQuantitative disease resistance (QR) is a complex, dynamic trait that is most reliably quantified in field-grown crops. Traditional disease assessments offer limited potential to disentangle the contributions of different components to overall QR at critical crop developmental stages. Yet, a better functional understanding of QR could greatly support a more targeted, knowledge-based selection for QR and improve predictions of seasonal epidemics. Image-based approaches together with advanced image processing methodologies recently emerged as valuable tools to standardize relevant disease assessments, increase measurement throughput, and describe diseases along multiple dimensions.ResultsWe present a simple, affordable, and easy-to-operate imaging set-up and imaging procedure for in-field acquisition of wheat leaf image sequences. The development of Septoria tritici blotch and leaf rusts was monitored over time via robust methods for symptom detection and segmentation, spatial alignment of images, symptom tracking, and leaf- and symptom characterization. The average accuracy of the spatial alignment of images in a time series was approximately 5 pixels (~ 0.15 mm). Leaf-level symptom counts as well as individual symptom property measurements revealed stable patterns over time that were generally in excellent agreement with visual impressions. This provided strong evidence for the robustness of the methodology to variability typically inherent in field data. Contrasting patterns in the number of lesions resulting from separate infection events and lesion expansion dynamics were observed across wheat genotypes. The number of separate infection events and average lesion size contributed to different degrees to overall disease intensity, possibly indicating distinct and complementary mechanisms of QR.ConclusionsThe proposed methodology enables rapid, non-destructive, and reproducible measurement of several key epidemiological parameters under field conditions. Such data can support decomposition and functional understanding of QR as well as the parameterization, fine-tuning, and validation of epidemiological models. Details of pathogenesis can translate into specific symptom phenotypes resolvable using time series of high-resolution RGB images, which may improve biological understanding of plant-pathogen interactions as well as interactions in disease complexes.

Development of a Drone-Based Phenotyping System for European Pear Rust (Gymnosporangium sabinae) in Orchards

Computer vision techniques offer promising tools for disease detection in orchards and can enable effective phenotyping for the selection of resistant cultivars in breeding programmes and research. In this study, a digital phenotyping system for disease detection and monitoring was developed using drones, object detection and photogrammetry, focusing on European pear rust (Gymnosporangium sabinae) as a model pathogen. High-resolution RGB images from ten low-altitude drone flights were collected in 2021, 2022 and 2023. A total of 16,251 annotations of leaves with pear rust symptoms were created on 584 images using the Computer Vision Annotation Tool (CVAT). The YOLO algorithm was used for the automatic detection of symptoms. A novel photogrammetric approach using Agisoft’s Metashape Professional software ensured the accurate localisation of symptoms. The geographic information system software QGIS calculated the infestation intensity per tree based on the canopy areas. This drone-based phenotyping system shows promising results and could considerably simplify the tasks involved in fruit breeding research.

Impact of RAW vs. RGB images on low-light object detection in autonomous driving

Abstract. In the field of autonomous driving, object detection in low-light conditions presents a critical challenge. Traditional detection algorithms, reliant on RGB imagery, often suffer from diminished performance under inadequate illumination. In contrast, RAW images, which retain a higher level of intrinsic image information, have the potential to enhance detection accuracy. Therefore, the paper aims to assess whether models trained on RAW images exhibit enhanced object detection performance in low-light scenarios. To this end, experiments are conducted utilizing the Low-light Object Detection (LOD) dataset, which includes paired RAW and RGB images, with variations in lighting simulated through different exposure times. Two distinct datasets are constructed: one consisting of RAW-normal and RAW-dark images, and another comprising RGB-normal and RGB-dark images. The models evaluated in this research include YOLOv8, Faster R-CNN (Region-based Convolutional Neural Network), and EfficientDet, selected to ensure robustness and generalizability of the findings. The results demonstrate that models trained on RAW images significantly outperform those trained on RGB images, thus exhibiting higher accuracy both in general and under low-light conditions. The preservation of detailed image information in RAW format facilitates enhanced feature extraction, thereby improving detection accuracy and resilience under low-light conditions. These findings provide novel insights into advancing object detection methodologies for autonomous driving systems operating in challenging lighting environments.

Adaptive Depth Estimation via SoftHebbLayer in a Hybrid ResNet-UNet Architecture

Abstract. Depth estimation is a key task in computer vision, critical for applications such as autonomous navigation and augmented reality. This paper introduces a novel hybrid neural network that combines ResNet and UNet architectures with a SoftHebbLayer, inspired by Hebbian learning principles, to improve depth estimation from RGB images. The ResNet backbone extracts robust hierarchical features, while the UNet decoder reconstructs fine-grained depth maps. The SoftHebbLayer dynamically adjusts feature connections based on co-activation, enhancing the models adaptability to diverse environments. This approach addresses common challenges in depth estimation, including poor generalization and computational inefficiency. We evaluated the model on the DIODE dataset, achieving strong results in Mean Squared Error (MSE), which is 0.0800 and Root Mean Squared Error (RMSE), which is 0.2805, demonstrating improved accuracy in both indoor and outdoor scenes. While the model excels in precision, further refinement is needed to reduce computational overhead and improve performance in challenging environments. This research paves the way for more efficient, adaptable depth estimation models, with potential applications in mobile robotics and real-time edge computing systems.

Image splicing detection using integrated LBP and DCT features

Abstract. Image splicing is one of the most common techniques used for picture manipulation and forgery. With the advent of user-friendly photo editing software, image splicing has become more prevalent and increasingly difficult to detect. This paper proposes a passive photo splicing detection approach based on Local Binary Patterns (LBP) and Discrete Cosine Transform (DCT) to identify splicing forgeries. The input RGB images are first converted to the YCbCr color space. Subsequently, the chrominance channels, Cb and Cr, are divided into overlapping blocks. Each block's LBP code is then transformed into the DCT domain. For each block, the standard deviation of each DCT coefficient is computed and used as a feature. Support Vector Machine (SVM) is employed as the classifier in a predictive model to determine whether the images have been spliced. To evaluate the proposed approach, two benchmark datasets for photo tampering were utilized. Experimental results indicate that the proposed method outperforms traditional splicing detection techniques in terms of detection accuracy and performance. This enhanced detection capability underscores the potential of combining LBP and DCT features with SVM classification for robust image splicing detection, paving the way for improved digital forensics tools in combating image manipulation.

Evaluation of coarse aggregate properties in hardened concrete based on segment anything model (SAM)

The applicability of the segment anything model (SAM) algorithm, a novel technique for analyzing the characteristics of coarse aggregate from RGB images of concrete cross-sections, was evaluated. Unlike traditional deep learning method, the SAM algorithm effectively and clearly separates coarse aggregate and mortar without requiring a large database, extensive pre-training, conversion of RGB images to grayscale, or manual threshold setting. Additionally, a new technique was applied to separate the pores recognized as aggregates by using the image difference according to the change in vertical and side lighting. Consequently, the area ratio of coarse aggregate, after the separation and removal of pores from the aggregate area, exhibited an error of approximately 7.0 % when compared to the actual volume ratio of the aggregate. However, in contrast to the calculation of the area ratio of coarse aggregate, the SAM algorithm demonstrated significant errors in size analysis, leading to distortions in the sieve curve. Nevertheless, the SAM algorithm indicates potential for application in various concrete-related fields of concrete in the future.

Multi-View Fusion-Based Automated Full-Posture Cattle Body Size Measurement

Cattle farming is an important part of the global livestock industry, and cattle body size is the key indicator of livestock growth. However, traditional manual methods for measuring body sizes are not only time-consuming and labor-intensive but also incur significant costs. Meanwhile, automatic measurement techniques are prone to being affected by environmental conditions and the standing postures of livestock. To overcome these challenges, this study proposes a multi-view fusion-driven automatic measurement system for full-attitude cattle body measurements. Outdoors in natural light, three Zed2 cameras were installed covering different views of the channel. Multiple images, including RGB images, depth images, and point clouds, were automatically acquired from multiple views using the YOLOv8n algorithm. The point clouds from different views undergo multiple denoising to become local point clouds of the cattle body. The local point clouds are coarsely and finely aligned to become a complete point cloud of the cattle body. After detecting the 2D key points on the RGB image created by the YOLOv8x-pose algorithm, the 2D key points are mapped onto the 3D cattle body by combining the internal parameters of the camera and the depth values of the corresponding pixels of the depth map. Based on the mapped 3D key points, the body sizes of cows in different poses are automatically measured, including height, length, abdominal circumference, and chest circumference. In addition, support vector machines and Bézier curves are employed to rectify the missing and deformed circumference body sizes caused by environmental effects. The automatic body measurement system measured the height, length, abdominal circumference, and chest circumference of 47 Huaxi Beef Cattle, a breed native to China, and compared the results with manual measurements. The average relative errors were 2.32%, 2.27%, 3.67%, and 5.22%, respectively, when compared with manual measurements, demonstrating the feasibility and accuracy of the system.

Convolutional-free Malware Image Classification using Self-attention Mechanisms

Malware analysis is a critical aspect of cybersecurity, aiming to identify and differentiate malicious software from benign programmes to protect computer systems from security threats. Despite advancements in cybersecurity measures, malware continues to pose significant risks in cyberspace, necessitating accurate and rapid analysis methods. This paper introduces an innovative approach to malware classification using image analysis, involving three key phases: converting operation codes into RGB image data, employing a Generative Adversarial Network (GAN) for synthetic oversampling, and utilising a simplified Vision Transformer (ViT)-based classifier for image analysis. The method enhances feature richness and explainability through visual imagery data and addresses imbalanced classification using GAN-based oversampling techniques. The proposed framework combines the strengths of convolutional autoencoders, hybrid classifiers, and adapted ViT models to achieve a balance between accuracy and computational efficiency. As shown in the experiments, our convolutional-free approach possesses excellent accuracy and precision compared with convolutional models and outperforms CNN models on two datasets, thanks to the multi-head attention mechanism. On the Big2015 dataset, our model outperforms other CNN models with an accuracy of 0.8369 and an AUC of 0.9791. Specifically, our model reaches an accuracy of 0.9697 and an F1 score of 0.9702 on MALIMG, which is extraordinary.

Novel Steganographic Method Based on Hermitian Positive Definite Matrix and Weighted Moore–Penrose Inverses

In this paper, we describe the concept of a new data-hiding technique for steganography in RGB images where a secret message is embedded in the blue layer of specific bytes. For increasing security, bytes are chosen randomly using a random square Hermitian positive definite matrix, which is a stego-key. The proposed solution represents a very strong key since the number of variants of positive definite matrices of order 8 is huge. Implementing the proposed steganographic method consists of splitting a color image into its R, G, and B channels and implementing two segments, which take place in several phases. The first segment refers to embedding a secret message in the carrier (image or text) based on the unique absolute elements values of the Hermitian positive definite matrix. The second segment refers to extracting a hidden message based on a stego-key generated based on the Hermitian positive definite matrix elements. The objective of the data-hiding technique using a Hermitian positive definite matrix is to embed confidential or sensitive data within cover media (such as images, audio, or video) securely and imperceptibly; by doing so, the hidden data remain confidential and tamper-resistant while the cover media’s visual or auditory quality is maintained.

Optimizing pulmonary chest x-ray classification with stacked feature ensemble and swin transformer integration

This research presents an integrated framework designed to automate the classification of pulmonary chest x-ray images. Leveraging convolutional neural networks (CNNs) with a focus on transformer architectures, the aim is to improve both the accuracy and efficiency of pulmonary chest x-ray image analysis. A central aspect of this approach involves utilizing pre-trained networks such as VGG16, ResNet50, and MobileNetV2 to create a feature ensemble. A notable innovation is the adoption of a stacked ensemble technique, which combines outputs from multiple pre-trained models to generate a comprehensive feature representation. In the feature ensemble approach, each image undergoes individual processing through the three pre-trained networks, and pooled images are extracted just before the flatten layer of each model. Consequently, three pooled images in 2D grayscale format are obtained for each original image. These pooled images serve as samples for creating 3D images resembling RGB images through stacking, intended for classifier input in subsequent analysis stages. By incorporating stacked pooling layers to facilitate feature ensemble, a broader range of features is utilized while effectively managing complexities associated with processing the augmented feature pool. Moreover, the study incorporates the Swin Transformer architecture, known for effectively capturing both local and global features. The Swin Transformer architecture is further optimized using the artificial hummingbird algorithm (AHA). By fine-tuning hyperparameters such as patch size, multi-layer perceptron (MLP) ratio, and channel numbers, the AHA optimization technique aims to maximize classification accuracy. The proposed integrated framework, featuring the AHA-optimized Swin Transformer classifier utilizing stacked features, is evaluated using three diverse chest x-ray datasets-VinDr-CXR, PediCXR, and MIMIC-CXR. The observed accuracies of 98.874%, 98.528%, and 98.958% respectively, underscore the robustness and generalizability of the developed model across various clinical scenarios and imaging conditions.

Using Multi-Sensor Data Fusion Techniques and Machine Learning Algorithms for Improving UAV-Based Yield Prediction of Oilseed Rape

Accurate and timely prediction of oilseed rape yield is crucial in precision agriculture and field remote sensing. We explored the feasibility and potential for predicting oilseed rape yield through the utilization of a UAV-based platform equipped with RGB and multispectral cameras. Genetic algorithm–partial least square was employed and evaluated for effective wavelength (EW) or vegetation index (VI) selection. Additionally, different machine learning algorithms, i.e., multiple linear regression (MLR), partial least squares regression (PLSR), least squares support vector machine (LS-SVM), back propagation neural network (BPNN), extreme learning machine (ELM), and radial basis function neural network (RBFNN), were developed and compared. With multi-source data fusion by combining vegetation indices (color and narrow-band VIs), robust prediction models of yield in oilseed rape were built. The performance of prediction models using the combination of VIs (RBFNN: Rpre = 0.8143, RMSEP = 171.9 kg/hm2) from multiple sensors manifested better results than those using only narrow-band VIs (BPNN: Rpre = 0.7655, RMSEP = 188.3 kg/hm2) from a multispectral camera. The best models for yield prediction were found by applying BPNN (Rpre = 0.8114, RMSEP = 172.6 kg/hm2) built from optimal EWs and ELM (Rpre = 0.8118, RMSEP = 170.9 kg/hm2) using optimal VIs. Taken together, the findings conclusively illustrate the potential of UAV-based RGB and multispectral images for the timely and non-invasive prediction of oilseed rape yield. This study also highlights that a lightweight UAV equipped with dual-image-frame snapshot cameras holds promise as a valuable tool for high-throughput plant phenotyping and advanced breeding programs within the realm of precision agriculture.

Continuous Growth Monitoring and Prediction with 1D Convolutional Neural Network Using Generated Data with Vision Transformer.

Crop growth information is collected through destructive investigation, which inevitably causes discontinuity of the target. Real-time monitoring and estimation of the same target crops can lead to dynamic feedback control, considering immediate crop growth. Images are high-dimensional data containing crop growth and developmental stages and image collection is non-destructive. We propose a non-destructive growth prediction method that uses low-cost RGB images and computer vision. In this study, two methodologies were selected and verified: an image-to-growth model with crop images and a growth simulation model with estimated crop growth. The best models for each case were the vision transformer (ViT) and one-dimensional convolutional neural network (1D ConvNet). For shoot fresh weight, shoot dry weight, and leaf area of lettuce, ViT showed R2 values of 0.89, 0.93, and 0.78, respectively, whereas 1D ConvNet showed 0.96, 0.94, and 0.95, respectively. These accuracies indicated that RGB images and deep neural networks can non-destructively interpret the interaction between crops and the environment. Ultimately, growers can enhance resource use efficiency by adapting real-time monitoring and prediction to feedback environmental controls to yield high-quality crops.

Real-Time Leaves Segmentation in RGB Images with Deep Learning in a Single-Board Computer

Abstract. This work proposed and evaluated methods for real-time leaf segmentation using a single-board computer. The main aim was to explore the state-of-the-art techniques based on the YOLO algorithm for real-time operation. For this purpose, the available variants of YOLOv8 and YOLOv9 were evaluated, and a semi-automatic labelling method based on the Segment Anything Model (SAM) algorithm was used. Given the need to delimit the leaf contour for labelling, it was possible to create a larger and more accurate dataset compared to the purely manual procedure. In addition, the cost-benefit of the applied algorithms and methods were assessed, considering the computational demand required, as well as the accuracy, recall, and precision delivered by these techniques. In this study, both quantitative analysis of the trained architectures' metrics and qualitative examination through direct observation of images were conducted to identify crucial aspects. The experiments were conducted with a post-processed dataset and the suitability for real-time applications was based on the elapsed time for segmentation. We concluded that the YOLOv8n architecture is the best one among those tested, presenting a precision and recall of 0.9064 and 0.7233, respectively. This architecture represents the best cost-benefit ratio between computational cost and real-time performance, being able to perform segmentation in 310 ms with the NVIDIA Jetson Nano board. Furthermore, when computational cost is not a problem or even when segmentation time can be higher, the YOLO8m network may be recommended when the recall metric is more important than precision. This network presented a precision and recall of 0.8556 and 0.7726, respectively, and presented a better performance in segmenting leaves located in more complex parts of the image and with a higher recall.