Illumination Changes Research Articles

Toward real-time and accurate dense 3D mapping of crop fields for combine harvesters using a stereo camera.

Fast and accurate 3D scene perception is a crucial prerequisite for the autonomous navigation and harvesting of combine harvesters. However, crop field scenarios pose severe challenges for vision-based perception systems due to repetitive scenes, illumination changes and real-time constraints on embedded computing platforms. In this paper, we propose a feature-based, two-stage approach for real-time dense 3D mapping for combine harvesters. In the first stage, our approach constructs a sparse 3D map using reliable feature matching, which provides prior knowledge about the environment. In the second stage, our method formulates per-pixel disparity calculation as probabilistic inference. The key to our approach is the ability to compute dense 3D maps by combining Bayesian estimation with efficient and discriminative point cues from images, exhibiting tolerance against visual measurement uncertainties due to repetitive textures and uneven lighting in crop fields. We validate the performance of the proposed method using real crop field data, and the results demonstrate that our dense 3D maps provide detailed spatial metric information while maintaining a balance between accuracy and efficiency. This makes our approach highly valuable for online perception in combine harvesters operating with resource-limited systems.

Regional feature learning using attribute structural analysis in bipartite attention framework for vehicle re-identification

<p><span lang="EN-US">Vehicle re-identification identifies target vehicles using images obtained by numerous non-overlapping real-time surveillance cameras. The effectiveness of re-identification is further challenging because of illumination changes, pose differences of captured images, and resolution. Fine-grained appearance changes in vehicles are recognized in addition to the coarse-grained characteristics like color of the vehicle along with model, and other custom features like logo stickers, annual service signs, and hangings to overcome these challenges. To prove the efficiency of our proposed bipartite attention framework, a novel dataset called Attributes27 which has 27 labelled attributes for each class are created. Our framework contains three major sections: The first section where the overall and semantic characteristics of every individual vehicle image are extracted by a double branch convolutional neural network (CNN) layer. Secondly, to identify the region of interests (ROIs) each branch has a self-attention block linked to it. Lastly to extract the regional features from the obtained ROIs, a partition-alignment block is deployed. The results of our proposed system’s evaluation on the Attributes27 and VeRi-776 datasets has highlighted significant regional attributes of each vehicle and improved the accuracy. Attributes27 and VeRi-776 datasets exhibits 98.5% and 84.3% accuracy respectively which are comparatively higher than the existing methods with 78.6% accuracy.</span></p>

LCE-Calib: Automatic LiDAR-Frame/Event Camera Extrinsic Calibration With a Globally Optimal Solution

The combination of light detection and rangings (LiDARs) and cameras enables a mobile robot to perceive environments with multimodal data, becoming a key factor in achieving robust perception. Traditional frame cameras are sensitive to changing illumination conditions, motivating us to introduce novel event cameras to make LiDAR-camera fusion more complete and robust. However, to jointly exploit these sensors, the challenging extrinsic calibration problem should be addressed. This article proposes an automatic checkerboard-based approach to calibrate extrinsics between a LiDAR and a frame/event camera, where the following four contributions are presented: 1) we present an automatic feature extraction and checkerboard tracking method from LiDAR's point clouds; 2) we reconstruct realistic frame images from event streams, applying traditional corner detectors to event cameras; 3) we propose an initialization-refinement procedure to estimate extrinsics using point-to-plane and point-to-line constraints in a coarse-to-fine manner; 4) we introduce a unified and globally optimal solution to address two optimization problems in calibration. Our approach has been validated with extensive experiments on 19 simulated and real-world datasets and outperforms the state-of-the-art.

Keypoint Detection and Description through Deep Learning in Unstructured Environments

Feature extraction plays a crucial role in computer vision and autonomous navigation, offering valuable information for real-time localization and scene understanding. However, although multiple studies investigate keypoint detection and description algorithms in urban and indoor environments, far fewer studies concentrate in unstructured environments. In this study, a multi-task deep learning architecture is developed for keypoint detection and description, focused on poor-featured unstructured and planetary scenes with low or changing illumination. The proposed architecture was trained and evaluated using a training and benchmark dataset with earthy and planetary scenes. Moreover, the trained model was integrated in a visual SLAM (Simultaneous Localization and Maping) system as a feature extraction module, and tested in two feature-poor unstructured areas. Regarding the results, the proposed architecture provides a mAP (mean Average Precision) in a level of 0.95 in terms of keypoint description, outperforming well-known handcrafted algorithms while the proposed SLAM achieved two times lower RMSE error in a poor-featured area with low illumination, compared with ORB-SLAM2. To the best of the authors’ knowledge, this is the first study that investigates the potential of keypoint detection and description through deep learning in unstructured and planetary environments.

A Circular Box-Based Deep Learning Model for the Identification of Signet Ring Cells from Histopathological Images.

Signet ring cell (SRC) carcinoma is a particularly serious type of cancer that is a leading cause of death all over the world. SRC carcinoma has a more deceptive onset than other carcinomas and is mostly encountered in its later stages. Thus, the recognition of SRCs at their initial stages is a challenge because of different variants and sizes and illumination changes. The recognition process of SRCs at their early stages is costly because of the requirement for medical experts. A timely diagnosis is important because the level of the disease determines the severity, cure, and survival rate of victims. To tackle the current challenges, a deep learning (DL)-based methodology is proposed in this paper, i.e., custom CircleNet with ResNet-34 for SRC recognition and classification. We chose this method because of the circular shapes of SRCs and achieved better performance due to the CircleNet method. We utilized a challenging dataset for experimentation and performed augmentation to increase the dataset samples. The experiments were conducted using 35,000 images and attained 96.40% accuracy. We performed a comparative analysis and confirmed that our method outperforms the other methods.

6D Object Pose Estimation Based on Cross-Modality Feature Fusion.

The 6D pose estimation using RGBD images plays a pivotal role in robotics applications. At present, after obtaining the RGB and depth modality information, most methods directly concatenate them without considering information interactions. This leads to the low accuracy of 6D pose estimation in occlusion and illumination changes. To solve this problem, we propose a new method to fuse RGB and depth modality features. Our method effectively uses individual information contained within each RGBD image modality and fully integrates cross-modality interactive information. Specifically, we transform depth images into point clouds, applying the PointNet++ network to extract point cloud features; RGB image features are extracted by CNNs and attention mechanisms are added to obtain context information within the single modality; then, we propose a cross-modality feature fusion module (CFFM) to obtain the cross-modality information, and introduce a feature contribution weight training module (CWTM) to allocate the different contributions of the two modalities to the target task. Finally, the result of 6D object pose estimation is obtained by the final cross-modality fusion feature. By enabling information interactions within and between modalities, the integration of the two modalities is maximized. Furthermore, considering the contribution of each modality enhances the overall robustness of the model. Our experiments indicate that the accuracy rate of our method on the LineMOD dataset can reach 96.9%, on average, using the ADD (-S) metric, while on the YCB-Video dataset, it can reach 94.7% using the ADD-S AUC metric and 96.5% using the ADD-S score (<2 cm) metric.

Detection of attack behaviour of pig based on deep learning

Attack behaviour detection of the pig is a valid method to protect the health of pig. Due to the farm conditions and the illumination changes of the piggery, the images of the pig in the videos are often being overlapped, which lead to difficulties in recognizing pig attack behaviour. We propose an improved YOLOX target detection model to overcome these difficulties. The improvements of the proposed model are: (1) the normalization attention mechanism is adopted to gain global information in the last block of the neck network and (2) the loss function IoU in YOLOX is replaced by DIoU to improve the detection accuracy. The pig attack behaviour considered in this paper includes the ear biting, the tail biting, the head to head collision and the head to body collision. The dataset is builded from the artificially observed attack video segments by using the inter-frame difference method. In the pig attack behaviour detection experiments, the improved YOLOX model achieves 93.21% precision which is 5.30% higher than the YOLOX model. The experiment results show that the improved YOLOX can realize pig attack behaviour detection with high precision.

Co-occurrence spatial–temporal model for adaptive background initialization in high-dynamic complex scenes

Background information is an important aspect of pre-processing for advanced applications in computer vision. The literature has made rapid progress in background initialization. However, background initialization still suffers from high-dynamic complex scenes, such as illumination change, background motion, or camera jitter. Therefore, this study presents a novel Co-occurrence Spatial–Temporal (CoST) model for background initialization in high-dynamic complex scenes. CoST achieves a spatial–temporal model through a co-occurrence pixel-block structure. The proposed approach extracts the spatial–temporal information of pixels to self-adaptively generate the background without the influence of high-dynamic complex scenes. The efficiency of CoST is verified through experimental results compared with state-of-the-art algorithms. The source code of CoST is available online at: https://github.com/HelloMrDeng/CoST.git.

Lightweight and interpretable convolutional neural network for real-time heart rate monitoring using low-cost video camera under realistic conditions

Recent research has shown that a person's heart rate (HR) can be estimated using video data through remote photoplethysmography (rPPG). However, this approach is faced with various challenges, including the inability to prepare training data that encompasses all realistic conditions, the impact of complex inference models on reasoning speed, and the lack of interpretability that hinders medical and healthcare applications. To tackle these issues, a lightweight and interpretable convolutional neural network is proposed for real-time HR monitoring using a low-cost video camera under realistic conditions. The Mediapipe framework is leveraged to construct a facial detection and tracking pipeline that is robust to head movements and illumination changes. Empirical mode decomposition (EMD) is then combined with a channel-wise attention-based convolutional neural network (CNN) for HR inference. Additionally, a temporal long-term peak merge method is proposed as a post-processing step to further enhance the accuracy of the neural network inference. The results of linear regression and Bland-Altman analysis demonstrate consistency between the estimated HR values and the ground truth. Moreover, experimental outcomes show no significant difference in the inference times of the proposed method running with or without a GPU, with a reasoning speed on mobile CPU remaining within 100 ms, ensuring real-time HR monitoring. Furthermore, this study provides pioneering empirical evidence to open the black box of neural networks in HR monitoring using rPPG signals.

Drone Based RGBT Tracking with Dual-Feature Aggregation Network

In the field of drone-based object tracking, utilization of the infrared modality can improve the robustness of the tracker in scenes with severe illumination change and occlusions and expand the applicable scene of the drone object tracking task. Inspired by the great achievements of Transformer structure in the field of RGB object tracking, we design a dual-modality object tracking network based on Transformer. To better address the problem of visible-infrared information fusion, we propose a Dual-Feature Aggregation Network that utilizes attention mechanisms in both spatial and channel dimensions to aggregate heterogeneous modality feature information. The proposed algorithm has achieved better performance by comparing with the mainstream algorithms in the drone-based dual-modality object tracking dataset VTUAV. Additionally, the algorithm is lightweight and can be easily deployed and executed on a drone edge computing platform. In summary, the proposed algorithm is mainly applicable to the field of drone dual-modality object tracking and the algorithm is optimized so that it can be deployed on the drone edge computing platform. The effectiveness of the algorithm is proved by experiments and the scope of drone object tracking is extended effectively.

Analyzing the impact of indoor environmental quality on physiological responses and work performance: Implications for IEQ control strategies

This study aims to analyze the effects of indoor environmental quality (IEQ) factors with simultaneous changes in illuminance and noise levels on building occupants' physiological response and work performance. To this end, four scenarios with changes in illuminance and noise levels were set up, and brain wave indices and work performance measured through a Stroop test were conducted during the scenarios. First, the findings revealed that noise and the interaction between illuminance and noise affect the building occupants' physiological response. Second, the illuminance was 150 lux, and the noise level was 40 dB, there was a correlation between the relative theta (0.685; p < .05) at the F3 location on the scalp and the average reaction time. Lastly, a comparison was made between psychologically preferred IEQ, IEQ when the physiological response is excellent, and IEQ when work performance is excellent. IEQ when the physiological response is excellent showed a high correlation with psychologically preferred IEQ, and it had a higher correlation with IEQ when work performance is excellent than psychologically preferred IEQ. Hence, an IEQ control strategy based on the physiological responses is necessary to improve work performance. It is expected to enhance building occupants’ work performance, health, and comfort.

A Novel FDLSR-Based Technique for View-Independent Vehicle Make and Model Recognition.

Vehicle make and model recognition (VMMR) is an important aspect of intelligent transportation systems (ITS). In VMMR systems, surveillance cameras capture vehicle images for real-time vehicle detection and recognition. These captured images pose challenges, including shadows, reflections, changes in weather and illumination, occlusions, and perspective distortion. Another significant challenge in VMMR is the multiclass classification. This scenario has two main categories: (a) multiplicity and (b) ambiguity. Multiplicity concerns the issue of different forms among car models manufactured by the same company, while the ambiguity problem arises when multiple models from the same manufacturer have visually similar appearances or when vehicle models of different makes have visually comparable rear/front views. This paper introduces a novel and robust VMMR model that can address the above-mentioned issues with accuracy comparable to state-of-the-art methods. Our proposed hybrid CNN model selects the best descriptive fine-grained features with the help of Fisher Discriminative Least Squares Regression (FDLSR). These features are extracted from a deep CNN model fine-tuned on the fine-grained vehicle datasets Stanford-196 and BoxCars21k. Using ResNet-152 features, our proposed model outperformed the SVM and FC layers in accuracy by 0.5% and 4% on Stanford-196 and 0.4 and 1% on BoxCars21k, respectively. Moreover, this model is well-suited for small-scale fine-grained vehicle datasets.

Fast multi-feature tracking method based on tightly coupled sensors

In contrast to conventional methods of optimizing feature-matching results at the data-processing level, we utilize the tightly coupled mode to fuse the information collected by the camera and inertial sensor at the data-gathering level, which contributed to better accuracy and robustness in the feature-matching process. Specifically, the proposed visual-inertial feature tracking method that combines inertial measurement unit (IMU) calibration, feature matching, and prediction algorithms. The approach includes a vision-aided multi-level IMU systemic calibration method and an inertial-aided image feature prediction algorithm, which effectively processes and utilizes information from multiple sensors. Our method addresses not only the influence of image distortion and blur resulting from illumination changes and fast camera motion but also the problem of measurement errors that can arise from the long-term operation of the inertial sensors. Extensive experiments have been conducted to demonstrate that the efficiency of it is superior to the state-of-the-art methods: With the accuracy rate remains the same level, the speed of feature matching is improved by 41.8%. Additionally, when it is applied to simultaneous localization and mapping systems, its localization performances are better than that of the VINS-mono method by 8.1%.

Performance improvement of reconfiguration strategies in photovoltaic array by replacement of blocking diodes with MOSFETs

ABSTRACT The power from photovoltaic array (PVA) is affected by temperature, solar irradiation, dust concentration, wind velocity, photovoltaic array configuration (PVAC) and shading pattern. The problem of partial shading (PS) is recognized as a major concern which decreases the PVA output power and represents multiple peak power points in the PV characteristic. Different PVACs respond differently with regard to their mismatch power loss (MPL) and fill factor (FF) under partial shaded condition (PSC). The conventional PVACs, such as SP-diode (Series-Parallel), TCT-diode (Total-Cross-Tied), BL-diode (Bridge-Link) and HC-diode (Honey-Comb), use blocking diodes (B-diodes) to block the reverse current. On the other hand, the proposed PVACs, including SP-MOSFET, TCT-MOSFET, BL-MOSFET and HC-MOSFET, replace the B-diodes with Blocking P-MOSFETs (BP-MOSFETs). The improvement in power output of the PVA is examined for four different configurations with B-diode and BP-MOSFET under changing illumination conditions. It is observed that TCT-MOSFET configuration shows better performance with least power losses and maximum efficiency under PSCs.

Quasi‐Z based adaptive sliding mode control for three‐phase photovoltaic grid‐connected system

AbstractConsidering the non‐linear characteristics of both the input and output of photovoltaic (PV) modules and quasi‐Z‐source inverters, as well as the unpredictable natural factors such as large disturbances caused by changes in illumination and temperature, an average state model for the PV quasi‐Z‐source inverter is established. This paper uses the output current of photovoltaic module, DC‐link capacitor voltage and its integral as state variables, then an adaptive reaching law with second‐order sliding mode characteristics is designed by combining with power reaching law and variable speed reaching law. By using the sliding mode controller based on the adaptive reaching law to control the stability of the DC‐link voltage of the quasi‐Z‐source inverter and the small capacitor voltage ripple, the photovoltaic system can operate stably at the maximum power point when the temperature and illumination conditions change abruptly, and the grid‐connected effect is improved. Through SIMULINK and RT‐LAB real‐time simulation, the effectiveness of the proposed adaptive sliding mode control strategy is verified when the environment changes, which effectively improves the dynamic response speed and grid‐connected effect of the system, and ensures the global robustness of the grid‐connected system.

Improving Remote Photoplethysmography Performance through Deep-Learning-Based Real-Time Skin Segmentation Network

In recent years, health-monitoring systems have become increasingly important in the medical and safety fields, including patient and driver monitoring. Remote photoplethysmography is an approach that captures blood flow changes due to cardiac activity by utilizing a camera to measure transmitted or reflected light through the skin, but it has limitations in its sensitivity to changes in illumination and motion. Moreover, remote photoplethysmography signals measured from nonskin regions are unreliable, leading to inaccurate remote photoplethysmography estimation. In this study, we propose Skin-SegNet, a network that minimizes noise factors and improves pulse signal quality through precise skin segmentation. Skin-SegNet separates skin pixels and nonskin pixels, as well as accessories such as glasses and hair, through training on facial structural elements and skin textures. Additionally, Skin-SegNet reduces model parameters using an information blocking decoder and spatial squeeze module, achieving a fast inference time of 15 ms on an Intel i9 CPU. For verification, we evaluated Skin-SegNet using the PURE dataset, which consists of heart rate measurements from various environments. When compared to other skin segmentation methods with similar inference speeds, Skin-SegNet demonstrated a mean absolute percentage error of 1.18%, showing an improvement of approximately 60% compared to the 4.48% error rate of the other methods. The result even exhibits better performance, with only 0.019 million parameters, in comparison to DeepLabV3+, which has 5.22 million model parameters. Consequently, Skin-SegNet is expected to be employed as an effective preprocessing technique for facilitating efficient remote photoplethysmography on low-spec computing devices.

Acceleration of inferred neural responses to oddball targets in an individual with bilateral amygdala lesion compared to healthy controls

Detecting unusual auditory stimuli is crucial for discovering potential threat. Locus coeruleus (LC), which coordinates attention, and amygdala, which is implicated in resource prioritization, both respond to deviant sounds. Evidence concerning their interaction, however, is sparse. Seeking to elucidate if human amygdala affects estimated LC activity during this process, we recorded pupillary responses during an auditory oddball and an illuminance change task, in a female with bilateral amygdala lesions (BG) and in n = 23 matched controls. Neural input in response to oddballs was estimated via pupil dilation, a reported proxy of LC activity, harnessing a linear-time invariant system and individual pupillary dilation response function (IRF) inferred from illuminance responses. While oddball recognition remained intact, estimated LC input for BG was compacted to an impulse rather than the prolonged waveform seen in healthy controls. This impulse had the earliest response mean and highest kurtosis in the sample. As a secondary finding, BG showed enhanced early pupillary constriction to darkness. These findings suggest that LC-amygdala communication is required to sustain LC activity in response to anomalous sounds. Our results provide further evidence for amygdala involvement in processing deviant sound targets, although it is not required for their behavioral recognition.

Illuminant metamerism between natural teeth and zirconia restorations evaluated with a chromatic adaptation transform

Little is known about the effect of illuminant metamerism between natural teeth and zirconia restorations, despite their increasing clinical popularity. The purpose of this in vitro study was to compare illuminant metamerism between pairs of natural teeth and layered zirconia restorations and pairs of natural teeth and monolithic zirconia restorations under 10 different illuminants and analyze their metameric potential. Spectral reflectance factors were obtained from 10 pairs of extracted natural teeth and layered zirconia restorations and 28 pairs of extracted natural teeth and monolithic multilayer zirconia restorations. Each pair showed a color match that was within the visual threshold for clinical acceptability (CIEDE2000≤1.8). A special index of metamerism for the change of illuminant (Milm) was calculated from the CIEDE2000 color difference equation. Descriptive statistics and the one-sample t test were used to analyze the results for the Milm and for both groups of layered and monolithic zirconia restorations (α=.05). Layered zirconia restorations reached a mean ±standard deviation value for Milm=0.3 ±0.2 and Milm=0.5 ±0.4 for monolithic zirconia restorations (P<.01). The effect of illuminant metamerism between natural teeth and zirconia crowns was weak and generally within the clinical acceptability limit.

Fast and Accurate Visual Tracking with Group Convolution and Pixel-Level Correlation

Visual object trackers based on Siamese networks perform well in visual object tracking (VOT); however, degradation of the tracking accuracy occurs when the target has fast motion, large-scale changes, and occlusion. In this study, in order to solve this problem and enhance the inference speed of the tracker, fast and accurate visual tracking with a group convolution and pixel-level correlation based on a Siamese network is proposed. The algorithm incorporates multi-layer feature information on the basis of Siamese networks. We designed a multi-scale feature aggregated channel attention block (MCA) and a global-to-local-information-fused spatial attention block (GSA), which enhance the feature extraction capability of the network. The use of a pixel-level mutual correlation operation in the network to match the search region with the template region refines the bounding box and reduces background interference. Comparing our work with the latest algorithms, the precision and success rates on the UAV123, OTB100, LaSOT, and GOT10K datasets were improved, and our tracker was able to run at 40FPS, with a better performance in complex scenes such as those with occlusion, illumination changes, and fast-motion situations.

Start-of-day oculomotor screening demonstrates the effects of fatigue and rest during a total immersion training program

BackgroundInvestigating changes in sleep and fatigue metrics during intensive surgical and trauma skills training, this study explored the dynamic association between oculomotor metrics and fatigue. Specifically, alterations in these relations over extended stress exposure, the influence of time of day, and the impact of fatigue exposure on sleep metrics were examined. MethodsThirty-nine military medical students participated in 6 days of immersion, hyper-realistic, and high-stress experiential casualty training. Participants completed surveys assessing the state of sleepiness with oculomotor tests performed each morning and evening, analyzing eye movement and pupillary change to characterize fatigue. Participants wore Fitbit TM devices to measure overall time asleep and time in each sleep stage during the training. ResultsFitbit data showed increased average minutes in rapid eye movement, deep sleep, and less time in light sleep from day 1 to day 4. The microsaccade peak velocity-to-displacement ratio exhibited a morning decrease but not in afternoon sessions, indicating repeated but temporary effects of accumulated fatigue. There were no findings regarding pupil reactivity to illumination changes. ConclusionThis study describes characteristics of fatigue measured by rapid and individually calibrated oculomotor tests. It demonstrates oculomotor relationships to fatigue in start-of-day testing, providing a direction for timing for optimal fatigue testing. These data suggest that improved sleep could signal resilience to fatigue during afternoon testing. Further investigation with more participants and longer duration is warranted. A deeper understanding of the interrelationships between training, sleep, and fatigue could improve surgical and military fitness.