Uneven Lighting Research Articles

Robot Localization Method Based on Multi-Sensor Fusion in Low-Light Environment

When robots perform localization in indoor low-light environments, factors such as weak and uneven lighting can degrade image quality. This degradation results in a reduced number of feature extractions by the visual odometry front end and may even cause tracking loss, thereby impacting the algorithm’s positioning accuracy. To enhance the localization accuracy of mobile robots in indoor low-light environments, this paper proposes a visual inertial odometry method (L-MSCKF) based on the multi-state constraint Kalman filter. Addressing the challenges of low-light conditions, we integrated Inertial Measurement Unit (IMU) data with stereo vision odometry. The algorithm includes an image enhancement module and a gyroscope zero-bias correction mechanism to facilitate feature matching in stereo vision odometry. We conducted tests on the EuRoC dataset and compared our method with other similar algorithms, thereby validating the effectiveness and accuracy of L-MSCKF.

Text Detection on Industrial Barrel Label with Convolutional Attention and Dual‐Branch Feature Network

Industrial barrel labels generally have low visual contrast, uneven lighting, and cluttered background, making it challenging to accurately locate text regions. This paper proposes a text detection network to solve the inaccurate localization problem based on DBNet. First, a convolutional attention mechanism is applied to the feature extraction network to get more valuable text feature maps. Then, a dual‐branch convolutional feature module is proposed in the feature pyramid to enrich contextual information. Besides, during the probability map generation stage, using a feature remodeling enhancement module to further distinguish text and text boundaries. This paper designs comparative experiments on ILTD, ICDAR2015 and MSRA‐TD500 datasets, achieve F‐measure of 92.3%, 86.0% and 84.1%, which are 2.2%, 2.3%, and 1.9% higher than DBNet, respectively. They demonstrate that our proposed method exhibits competitive performance and strong robustness. © 2024 Institute of Electrical Engineers of Japan and Wiley Periodicals LLC.

Research on Temporal–Spatial Partition Control Strategies for Luminous and Thermal Environment in High Space of Gymnasiums

The lighting design of large-space buildings in gymnasiums can impact the indoor luminous and thermal environment, resulting in an uneven light and thermal distribution. This paper investigates the luminous and thermal environment control strategies for high spaces in gymnasiums, by simulating the luminous and thermal environment under different lighting forms and establishing a comprehensive evaluation model. The results show that the weights of the indoor luminous environment, thermal environment, and comprehensive energy consumption change with season and time under different lighting forms, which provides a basis for developing a temporal–spatial partition control strategy. The temporal–spatial partition control strategy is proposed for summer and winter, including the shading angle control under the lighting forms of south-facing side windows, west-facing side windows, and top skylights. Under summer conditions, the south-facing side windows have no shading from 8:00 to 10:00 and 14:00 to 16:00, and the shading angle is 0° from 10:00 to 14:00; the west-facing side windows have no shading from 8:00 to 14:00, the shading angle is 0° from 14:00 to 15:00, and the shading angle is 15° from 15:00 to 16:00; and the top skylight has a shade angle of 15° from 8:00 to 9:00, 30° from 9:00 to 11:00, 45° from 11:00 to 13:00, and no shade from 13:00 to 16:00. Under winter conditions, the south-facing side windows have no shading all day; the west-facing side windows have no shading from 8:00 to 14:00, and the shading angle is 30° from 14:00 to 16:00; and the top skylight has no shading from 8:00 to 13:00, a shading angle of 45° from 13:00 to 15:00, and a shading angle of 75° from 15:00 to 16:00. This paper provides a set of scientific and reasonable luminous and thermal environment regulation strategies for large-space buildings, which can help optimize the building energy consumption and improve the indoor environment quality.

Growing Tomato Seedlings Suitable for Mechanical Grafting under Regulated Light Regime

Abstract: The uniformity of growth and mechanical properties of grafted seedlings affect the quality of mechanical grafting operations. The growth uniformity of grafted seedlings in a greenhouse will be poor due to the uneven and unstable light and temperature conditions. Plant factories can cultivate grafted seedlings in the most suitable environment by regulating environmental parameters such as light and temperature. The aim of this study was to investigate the impact of the light conditions on tomato seedlings in plant factory and to develop an optimal cultivation light formula. The effects of light intensity (50, 100, 150, 200, and 250 μmol m−2 s−2) and photoperiod (10, 12, 14, 16, and 18 h a day(h/d)) on the morphological and mechanical properties of tomato seedlings were experimentally investigated. Orthogonal experiments were conducted involving light quality (R:B = 75:25, R:B = 50:50, and R:B = 25:75), light intensity (150 μmol m−2 s−2, 200 μmol m−2 s−2, and 250 μmol m−2 s−2), and photoperiod (14, 16, and 18 h/d) as independent variables to determine the optimal combination. Finally, a comparative grafting experiment was conducted between the seedlings cultivated using the optimal light formula and commercially available seedlings. The result showed that increasing light intensity inhibited hypocotyl length and promoted seedling stem growth, and excessive light intensity decreased seedling mechanical properties. The optimal light intensity for rootstocks is 200 μmol m−2 s−2, and the optimal light intensity for scions is 250 μmol m−2 s−2. Shortening the photoperiod would promote hypocotyl growth and inhibit seedling stem elongation. Different photoperiods had a significant impact on the mechanical properties of tomato seedlings. The most suitable photoperiod for rootstocks was 18 h/d and for scions was 16 h/d. The most suitable light formula was R:B = 50:50, 250 μmol m−2 s−2, 18 h/d. By analyzing the experimental results, the mechanical properties of seedlings grown by the regulated light environment were better than those of commercially available seedlings, and the success rate of mechanical grafting was 7% higher. Overall, in plant factories compared to commercially available tomato seedlings, tomato seedlings cultivated by the regulated light environment were more suitable for mechanical grafting. This research result provides theoretical support for subsequent research on grafting machinery.

CP-RDM: a new object detection algorithm for casting and pouring robots

Abstract Automating the casting sector heavily relies on pivotal technology for object detection in pouring robots. A sophisticated algorithm designed to identify and locate target pouring holes in intricate casting workshops is crucial for advancing the intelligence of the casting process. However, the workshop environment for pouring is generally challenging, with uneven lighting, varying sizes of pouring holes, and significant occlusion in the target area, all impacting the accuracy of target detection tasks.
To overcome these challenges, this paper proposes enhancing the YOLOv8s algorithm for object detection in pouring robots. Firstly, to address the issue of different scales in pouring holes, a Multi-Scale Residual Channel and Spatial Information Fusion Module (MRCS) is designed to aggregate channel and spatial information, thereby enhancing the feature extraction capability of the model. The proposed enhancement is validated on the Pascal VOC dataset. Secondly, a SimAM attention mechanism is added at the end of the backbone network to focus the object detection network more on the positional region of the pouring hole. Importantly, this addition does not introduce extra parameters or computational burden to the model.
Finally, in the detection part of the model, the detection head from the RT-DETR model is introduced. This combination of real-time detection capability from YOLO and deep feature extraction capability from RT-DETR enhances the detection accuracy of the model while ensuring real-time performance. Experimental results on the updated pouring hole dataset reveal that, with only a slight increase in parameters, the proposed model achieves a 2.5% and 3.5% improvement in mAP@0.5 and F1-Score, respectively, compared to the baseline algorithm YOLOv8s. Precision (P) is enhanced by 1.8%, recall (R) by 3.5%, and PFS reaches 110, meeting the requirements for real-time pouring in pouring robots.

Tracking and automatic behavioral analysis of group-housed pigs based on YOLOX+BoT-SORT-slim

Automatic analysis of pig behavior is crucial for assessing their health and welfare status in group-housed pig farms. Currently, computer vision technology has been widely applied to recognize and track pig behavior. However, these applications lack the capability for automatic behavior analysis. Additionally, execution speed is crucial for long-term tracking, as the tracking target can easily be lost due to the complex background of pig farms, uneven lighting, and the similar appearance of pigs. To tackle these challenges, we propose a lightweight multi-object tracking (MOT) and behavioral analysis method: YOLOX+BoT-SORT-Slim. Firstly, the YOLOX detector detects pig targets and recognizes the pigs’ four behaviors including “stand”, “lie”, “eat”, and “other”. Secondly, the BoT-SORT-Slim algorithm tracks the detected pigs and their behaviors at high speed. Finally, we devise a behavior automatic analysis algorithm that integrates the behavioral information and the tracking results to complete pigs’ behavior analysis. To evaluate the proposed method, we conducted experiments on 1-minute, 10-minute, and 1-hour datasets. The experimental results show that on the 1-minute public and private test sets, the proposed method achieves 6.4 and 3.6 times the frame rate (FPS) of the original algorithm, with 68.05 FPS and 71.27 FPS, respectively. It also achieves a higher-order tracking accuracy (HOTA) of 84.7 % and 78.7 %, multi-object tracking accuracy (MOTA) of 98.3 % and 97.0 %, and an identification F1 score (IDF1) of 98.4 % and 93.4 %. The proposed method obtains the best results on all test sets compared to five other MOT algorithms including DeepSORT, StrongSORT, ByteTrack, OC-SORT, and BoT-SORT. In the 10-minute and 1-hour test sets, the proposed method achieves 70.28 and 72.24, 63.3 % and 76.4 %, 94.1 % and 99.6 %, and 72.5 % and 90.2 % for FPS, HOTA, MOTA, and IDF1, respectively. In terms of behavior analysis, three types of pigs’ behavior analysis are plotted for each test video to assess their health status. The experimental results indicate that the proposed method performs excellently in behavior analysis and tracking, which can reliably monitor and manage pig behavior in real time, providing automated management technical support.

A Distorted Light Field Image Correction Method Based on Improved Hough Transform

Introduction: In using a camera to take photos, due to environmental limitations, uneven lighting can cause uneven distribution of the image light field, resulting in distortion of the image background and target, blurring of details, and distorted light field images. Method: In view of this, research is conducted on the correction of distorted light field images based on the Hough transform. First, the improved Hough transform is utilized to locate the four coordinates, the matrix information of the normal image is applied to calculate the corresponding parameter amount, and then the low-frequency part of the image spectrum is removed. Finally, it uses the Gaussian function for difference, inputs the original data, and gets the correction result of the distorted light field image. Result: The research results indicate that in the practical application of the distorted light field image correction algorithm based on the Hough transform, the improved Hough transform algorithm is superior to the traditional one. Conclusion: In comparative experiments, the research algorithm outperforms the other three algorithms, with an average color restoration of 93.76% and an average signal-to-noise ratio of 54.22dB. The superiority of the research algorithm has been verified, indicating that the research method can perfectly correct distorted light field images and achieve good correction results.

Research on Detection Method of Chaotian Pepper in Complex Field Environments Based on YOLOv8.

The intelligent detection of chili peppers is crucial for achieving automated operations. In complex field environments, challenges such as overlapping plants, branch occlusions, and uneven lighting make detection difficult. This study conducted comparative experiments to select the optimal detection model based on YOLOv8 and further enhanced it. The model was optimized by incorporating BiFPN, LSKNet, and FasterNet modules, followed by the addition of attention and lightweight modules such as EMBC, EMSCP, DAttention, MSBlock, and Faster. Adjustments to CIoU, Inner CIoU, Inner GIoU, and inner_mpdiou loss functions and scaling factors further improved overall performance. After optimization, the YOLOv8 model achieved precision, recall, and mAP scores of 79.0%, 75.3%, and 83.2%, respectively, representing increases of 1.1, 4.3, and 1.6 percentage points over the base model. Additionally, GFLOPs were reduced by 13.6%, the model size decreased to 66.7% of the base model, and the FPS reached 301.4. This resulted in accurate and rapid detection of chili peppers in complex field environments, providing data support and experimental references for the development of intelligent picking equipment.

In-field grading and sorting technology of apples: A state-of-the-art review

Apple is one of the most popular fruits. In-field grading and sorting of apples would enhance growers’ economic benefits by lowering production costs. This study reviews the key components and progress of the quality inspection algorithm for in-field grading and sorting of apples. Four key components (e.g., conveyor, imaging chamber, sorting actuator, and bin filler) are presented in detail, followed by summarizing the shortcomings of these components. The apple’s external (color, size, and defects) and internal quality inspection technologies, such as optical technologies of visible light, near-infrared (NIR), hyperspectral/multispectral imaging (HSI/MSI), and structured illumination (SI) were presented. Despite the excellent detection performance of emerging technologies (e.g., HSI, MSI, and SI), visible light is still dominantly used for in-filed grading. Challenges in getting information on the whole surface area of the apple, uneven lighting, machine size, throughput, and associated costs hamper the commercialization of apple in-field grading and sorting equipment. At present, more efforts should be devoted to internal quality inspection, by developing reliable, fast, and accurate detection equipment and algorithms. With the advancement of sensors and automation algorithms, as well as the emergence of mechanical systems that are suitable for in-field use, it is anticipated that the apple in-field grading and sorting equipment that inspects both external and internal quality will be realized and commercialized in the near future.

RGBD-based method for segmenting apparent pores within bridge towers

Abstract As a crucial technology in computer vision, image semantic segmentation is extensively applied to tasks such as detecting apparent defects in concrete, identifying structural cracks, and interpreting facility scenes within infrastructure settings. Challenges such as uneven lighting inside bridge towers and the similarity in color and texture between concrete pore structures and their surrounding areas frequently result in lower segmentation accuracy. This article introduces a multimodal semantic segmentation model incorporating depth information to tackle these challenges. By integrating depth and RGB images as inputs, the model constructs an interactive space and utilizes a cross-attention mechanism along with global context for guided learning, thus enabling precise feature extraction and segmentation. The experimental results demonstrate that this segmentation network excels on a custom-made concrete pore dataset, with a precision rate of 90.88%, recall rate of 87.48%, intersection over union (IoU) of 80.42%, and F1 and Dice coefficient of 89.10% and 89.15%, respectively. These outcomes affirm the network’s effectiveness in segmenting apparent defects in concrete and offer technical support and solutions for image semantic segmentation tasks in settings like bridge towers.

Unveiling sequential layered heterogeneity in multi-spectral transmission images clustering through iterative terrace compression with adaptive neighborhoods analysis

Multi-spectral transmission imaging looks forward to an early breast cancer screening with cost-effectiveness, safety, and ease of use. However, the biological tissue of the breast owns scattering and strong absorption properties, resulting in transmission images with a low signal-to-noise ratio and prior investigation focused on detection heterogeneity within a single layer, but the different depths in heterogeneity due to the spherical shape of the breast causing the challenges. Accurately detecting heterogeneity in different layers is crucial due to low contrast, veiling layers, and unclear boundaries, which can be difficult to identify in clustering and segmentation. The breast heterogeneity identification can be handled by combining image processing techniques; nevertheless, considering labeling limits and data accessibility, the clustering algorithm provides an acceptable substitute. Therefore, the paper proposes a novel methodology integrating the iterative terrace compression method with adaptive neighborhoods analysis to identify sequential multi-layered heterogeneity and enhance the clustering of multi-spectral transmission images. Our experimental setup involved the collection of low grayscale images across six wavelengths, followed by advanced image processing techniques, including frame accumulation to improve image signal-to-noise ratio and then polynomial surface fitting to eliminate trend items from uneven light. After that, iterative terrace compression with adaptive neighborhoods analysis is applied to reduce redundancy and unveil heterogeneity in different layers and window functions to remove unnecessary gray information. Finally, clustering combined with pseudo-color is used for clustering analysis and visual representation for aiding in heterogeneity detection. Results indicate a significant improvement in detection metrics, with our method outpacing existing techniques regarding the Dice coefficient, F1 score, and Jaccard. This innovative approach not only enhances the characterization of breast tissue heterogeneities in different layers but also promises practical clinical applications in breast cancer screening.

Rethinking of Underwater Image Restoration Based on Circularly Polarized Light

Polarimetric imaging technology plays a crucial role in de-scattering, particularly in the field of underwater image restoration. Circularly polarized light (or the underlying circular polarization memory effect) has been proven to better preserve the polarization characteristics of detected light. Utilizing circularly polarized light as illumination can further enhance the effectiveness of polarization de-scattering techniques. After rethinking the advantages of circularly polarized light, this paper proposes a new method for underwater polarimetric imaging restoration that leverages the pre-processing of polarized sub-images and the correlation of polarization characteristics (i.e., the angle of polarization and degree of polarization). Additionally, to address the challenge of selecting scattering light parameters due to uneven light fields in target scenes, an intensity adjustment factor search algorithm is designed. This algorithm eliminates the need for the manual selection of scattering light parameters, effectively solving the problem of uneven illumination in restoration results. A series of experiments demonstrate that, compared to traditional algorithms, the proposed method offers superior detail restoration and higher robustness.

Flotation froth image enhancement based on region decomposition and guided filtering

Extraction of information from froth images is important for automatic control of froth flotation. However, images captured by cameras often suffer from severe uneven lighting, which significantly reduces the quality of froth images. Low-quality images hinder the accurate extraction of froth information, thereby affecting the control of the froth flotation system. Hence, we propose an image enhancement method based on region decomposition and guided filtering to improve the quality of images. Initially, we separate the image into regions with sufficient and insufficient illumination based on reflectance. In regions with insufficient illumination, the guided filter is applied to direct pixels to acquire information from brighter points in their neighborhood. Conversely, in other regions, we regulate the magnitude of pixel variations to prevent overexposure. Finally, a detail enhancement method is proposed based on a multi-scale Gaussian pyramid and texture fusion to improve clarity and naturalness. The experiments show that the method we proposed surpasses several state-of-the-art algorithms on public datasets. In the field of flotation, our method effectively enhances the image quality. Compared to other enhancement methods under the same segmentation strategy, our method significantly improves segmentation accuracy, demonstrating its strong practical value. In addition, our method also shows advantages in terms of computational speed.

Trans-DCN: A High-Efficiency and Adaptive Deep Network for Bridge Cable Surface Defect Segmentation

Cables are vital load-bearing components of cable-stayed bridges. Surface defects can lead to internal corrosion and fracturing, significantly impacting the stability of the bridge structure. The detection of surface defects from bridge cable images faces numerous challenges, including shadow disturbances due to uneven lighting and difficulties in addressing multiscale defect features. To address these challenges, this paper proposes a novel and cost-effective deep learning segmentation network, named Trans-DCN, to detect defects in the surface of the bridge cable. The network leverages an efficient Transformer-based encoder and integrates multiscale features to overcome the limitations associated with local feature inadequacy. The decoder implements an atrous Deformable Convolution (DCN) pyramid and dynamically fuses low-level feature information to perceive the complex distribution of defects. The effectiveness of Trans-DCN is evaluated by comparing it with state-of-the-art segmentation baseline models using a dataset comprising cable bridge defect images. Experimental results demonstrate that our network outperforms the state-of-the-art network SegFormer, achieving a 27.1% reduction in GFLOPs, a 1.2% increase in mean Intersection over Union, and a 1.5% increase in the F1 score. Ablation experiments confirmed the effectiveness of each module within our network, further substantiating the significant validity and advantages of Trans-DCN in the task of bridge cable defect segmentation. The network proposed in this paper provides an effective solution for downstream cable bridge image analysis.

Lightweight citrus leaf disease detection model based on ARMS and cross-domain dynamic attention

In citrus cultivation, Anthracnose, Scab, and Greasy Spot significantly impact yield and quality. Facing challenges in detecting small targets against complex orchard backgrounds with uneven lighting and obstructions, existing models suffer from low detection accuracy. This study introduces the YOLOv8n-CDDA citrus leaf disease detection model. The Cross-Domain Dynamic Attention (CDDA) mechanism deconstructs the backbone network’s input feature maps into sections, dynamically assigning spatial and channel attention weights to reconstruct critical information and capture the variations and weak semantic features of disease textures. The proposed Adaptive Random Mix-Cut Splicing (ARMS) image augmentation technique blends diseased leaf images with healthy citrus backgrounds, enhancing the diversity and number of background targets. To reduce computational and memory consumption, the network is streamlined through channel pruning; to compensate for the loss in accuracy from pruning, a teacher–assistant–student network format is used for knowledge distillation, where the student network learns from soft knowledge to improve disease recognition accuracy. Finally, Grad-CAM++ technology generates heatmaps of the detections, facilitating the visualization of effective features and deepening understanding of the model’s focus areas. Experimental results demonstrate that the YOLOv8n-CDDA model achieves an average accuracy of 90.89% in disease detection, with an average recall rate of 81.12%, and a mean Average Precision (mAP50) of 88.36%. Compared to the original YOLO v8n and current mainstream detection models such as YOLOv5s, SSD, and Faster-RCNN, the improvements in average accuracy are respectively 2.95%, 4.78%, 14.22%, and 21.01%; in average recall, 2.36%, 3.09%, 15.74%, and 23.27%; and in mAP50, 2.38%, 3.13%, 13.45%, and 20.91%. After pruning and distillation for lightweight adaptation, the YOLOv8n-CDDA model has a parameter size of 0.8M, requires 4.2 GFLOPs, weighs 2.0 MB, and operates at 45 fps. Compared to YOLOv8n, this represents a reduction of 2.2M in parameters, 3.9 GFLOPs, and 4 MB in model weight, with an increase of 7 fps in speed. This model exhibits exceptional performance in the complex environment of citrus leaf disease detection, providing robust technical support for citrus growth monitoring studies, and offering insights for disease detection in other crops as well.

RpTrack: Robust Pig Tracking with Irregular Movement Processing and Behavioral Statistics

Pig behavioral analysis based on multi-object tracking (MOT) technology of surveillance videos is vital for precision livestock farming. To address the challenges posed by uneven lighting scenes and irregular pig movements in the MOT task, we proposed a pig MOT method named RpTrack. Firstly, RpTrack addresses the issue of lost tracking caused by irregular pig movements by using an appropriate Kalman Filter and improved trajectory management. Then, RpTrack utilizes BIoU for the second matching strategy to alleviate the influence of missed detections on the tracking performance. Finally, the method utilizes post-processing on the tracking results to generate behavioral statistics and activity trajectories for each pig. The experimental results under conditions of uneven lighting and irregular pig movements show that RpTrack significantly outperforms four other state-of-the-art MOT methods, including SORT, OC-SORT, ByteTrack, and Bot-SORT, on both public and private datasets. The experimental results demonstrate that RpTrack not only has the best tracking performance but also has high-speed processing capabilities. In conclusion, RpTrack effectively addresses the challenges of uneven scene lighting and irregular pig movements, enabling accurate pig tracking and monitoring of different behaviors, such as eating, standing, and lying. This research supports the advancement and application of intelligent pig farming.

SFMnet: Edge detection of HABs based on spatial feature mapping encoder-decoder network

Achieving high-accuracy real-time recognition of marine red tide algae images is crucial in implementing red tide algae early warning systems. However, due to uneven lighting and the deposition of unknown suspended particles in seawater, real-time collected harmful algal blooms (HABs) microscopic images in marine engineering suffer from poor clarity and blurred cell contours, leading to lower accuracy of current edge detection models for microscopic images. Firstly, to enhance the quality of degraded images in the original HABs dataset, we design an image enhancement algorithm based on underwater image restoration models, which involves precise depth estimation of seawater scenes and fusion of transmission maps. This algorithm effectively eliminates chromatic aberration and enhances image edge details and contrast. Secondly, we propose a deep-learning model based on the VGG16 network. This model integrates multi-scale features through the Spatial Feature Mapping module to improve edge detection accuracy. Experimental results using real-time HABs microscopic image data show that the proposed enhancement method achieves the best results regarding contrast, image enhancement metrics, and clarity gradient. Moreover, the edge detection method achieves an ODS of 0.636 and an OIS of 0.701. The results demonstrate that performing edge detection after the image enhancement preprocess significantly improves detection accuracy.

3D Camera and Single-Point Laser Sensor Integration for Apple Localization in Spindle-Type Orchard Systems.

Accurate localization of apples is the key factor that determines a successful harvesting cycle in the automation of apple harvesting for unmanned operations. In this regard, accurate depth sensing or positional information of apples is required for harvesting apples based on robotic systems, which is challenging in outdoor environments because of uneven light variations when using 3D cameras for the localization of apples. Therefore, this research attempted to overcome the effect of light variations for the 3D cameras during outdoor apple harvesting operations. Thus, integrated single-point laser sensors for the localization of apples using a state-of-the-art model, the EfficientDet object detection algorithm with an mAP@0.5 of 0.775 were used in this study. In the experiments, a RealSense D455f RGB-D camera was integrated with a single-point laser ranging sensor utilized to obtain precise apple localization coordinates for implementation in a harvesting robot. The single-point laser range sensor was attached to two servo motors capable of moving the center position of the detected apples based on the detection ID generated by the DeepSORT (online real-time tracking) algorithm. The experiments were conducted under indoor and outdoor conditions in a spindle-type apple orchard artificial architecture by mounting the combined sensor system behind a four-wheel tractor. The localization coordinates were compared between the RGB-D camera depth values and the combined sensor system under different light conditions. The results show that the root-mean-square error (RMSE) values of the RGB-D camera depth and integrated sensor mechanism varied from 3.91 to 8.36 cm and from 1.62 to 2.13 cm under 476~600 lx to 1023~1100 × 100 lx light conditions, respectively. The integrated sensor system can be used for an apple harvesting robotic manipulator with a positional accuracy of ±2 cm, except for some apples that were occluded due to leaves and branches. Further research will be carried out using changes in the position of the integrated system for recognition of the affected apples for harvesting operations.

A multi-branch dual attention segmentation network for epiphyte drone images

Acquiring images of epiphytes growing on trees using Unmanned Aerial Vehicles (UAVs) enables botanists to efficiently collect data on these important plant species. Despite the advantages offered by UAVs, challenges such as complex backgrounds, uneven lighting inside the tree canopy, and accessibility issues hinder the acquisition of quality images, resulting in acquiring images datasets of heterogenous quality. AI/Deep Learning algorithms can be used to segment target plants in these images for selecting sampling locations. Existing DL models require large volume of data for training, and they tend to prioritize local features over global ones, impacting segmentation accuracy, particularly on smaller, heterogeneous quality image datasets. To overcome these limitations, we propose a multi-branch dual attention segmentation network designed to effectively handle small datasets with heterogeneous quality. The proposed network incorporates dedicated branches for extracting both global and local features, utilizing spatial and channel attention mechanisms to focus on important regions. Through a fusion process and a decoder with crossed fusion technique, this network effectively combines and enhances features from multiple branches, resulting in improved segmentation performance. Output obtained from the trained model demonstrated major improvements in predicting the boundary regions and class labels, even in close-range, low-light, and zoomed/cropped images. The average Intersection over Union (IoU) scores of the trained model was 5% higher for images acquired close range, 48% higher for images in low-light conditions, and 68% higher for zoomed/cropped images when compared to those obtained from TransUnet, a state-of-the-art vision transformer model trained on epiphyte dataset. The proposed network can be used for segmenting epiphytes in images of heterogeneous quality as well as identifying targets in images acquired in domains such as agriculture and forestry.

An accurate semantic segmentation model for bean seedlings and weeds identification based on improved ERFnet

In agricultural production activities, the growth of crops always accompanies the competition of weeds for nutrients and sunlight. In order to mitigate the adverse effects of weeds on yield, we apply semantic segmentation techniques to differentiate between seedlings and weeds, leading to precision weeding. The proposed EPAnet employs a loss function coupled with Cross-entropy loss and Dice loss to enhance attention to feature information. A multi-Decoder cooperative module based on ERFnet is designed to enhance information transfer during feature mapping. The SimAM is introduced to enhance position recognition. DO-CONV is used to replace the traditional convolution Feature Pyramid Networks (FPN) connection layer to integrate feature information, improving the model’s performance on leaf edge processing, and is named FDPN. Moreover, the Overall Accuracy has been improved by 0.65%, the mean Intersection over Union (mIoU) by 1.91%, and the Frequency-Weighted Intersection over Union (FWIoU) by 1.19%. Compared to other advanced methods, EPAnet demonstrates superior image segmentation results in complex natural environments with uneven lighting, leaf interference, and shadows.