Modulation Of Perception Research Articles

Geometric information constraint 3D object detection from LiDAR point cloud for autonomous vehicles under adverse weather

3D object detection, as the core of the autonomous vehicle perception module, is essential for efficient transportation and comfortable experiences. However, the challenge of 3D object detection under adverse weather conditions hinders the advancement of autonomous vehicles to higher levels. Hence, achieving accurate 3D object detection under adverse weather conditions is increasingly crucial as it forms the foundation for trajectory planning and driving strategy making in autonomous vehicles, thereby revolutionizing transportation modes for both goods and passengers. Advances in Light Detection and Ranging (LiDAR) technology have facilitated the development of 3D object detection in the past few years. Adverse weather, which inevitably occurs in real-world driving scenarios, could degrade measurement accuracy and point density of LiDAR and lead to particle interference. Detecting accurate 3D bounding boxes from sparse, incomplete point clouds with particle interference is difficult. Therefore, this research presents a novel geometric information constraint network for 3D object detection tasks from LiDAR point clouds under adverse weather (GIC-Net). In this study, we focus on how to incorporate geometric location information and line geometric feature information into the network against adverse weather. Further, we propose a geometric location constrained backbone module (GLC) to reduce rain and snow particle interference and ensure sufficient receptive fields. Then, we design a line geometric feature constraint module (LGFC) to add line constraints of 3D bounding boxes into the training process. Finally, a line loss function is designed, and features from the GLC and LGFC modules are fed into the multi-task detection head for accurate 3D bounding box prediction. Experiments on the Canadian Adverse Driving Conditions (CADC) autonomous vehicle dataset demonstrate the superiority of our method over six other state-of-the-art methods under adverse weather, which is at least 13.32 %, 4.67 %, and 10.44 % mAP higher than the other compared methods in the car, truck, and pedestrian classes respectively. Also, we further verify the better generalization ability of our network compared to other methods.

ANYmal parkour: Learning agile navigation for quadrupedal robots.

Performing agile navigation with four-legged robots is a challenging task because of the highly dynamic motions, contacts with various parts of the robot, and the limited field of view of the perception sensors. Here, we propose a fully learned approach to training such robots and conquer scenarios that are reminiscent of parkour challenges. The method involves training advanced locomotion skills for several types of obstacles, such as walking, jumping, climbing, and crouching, and then using a high-level policy to select and control those skills across the terrain. Thanks to our hierarchical formulation, the navigation policy is aware of the capabilities of each skill, and it will adapt its behavior depending on the scenario at hand. In addition, a perception module was trained to reconstruct obstacles from highly occluded and noisy sensory data and endows the pipeline with scene understanding. Compared with previous attempts, our method can plan a path for challenging scenarios without expert demonstration, offline computation, a priori knowledge of the environment, or taking contacts explicitly into account. Although these modules were trained from simulated data only, our real-world experiments demonstrate successful transfer on hardware, where the robot navigated and crossed consecutive challenging obstacles with speeds of up to 2 meters per second.

Autonomous Behavior Selection For Self-driving Cars Using Probabilistic Logic Factored Markov Decision Processes

ABSTRACT We propose probabilistic logic factored Markov decision processes (PL-fMDPs) as a behavior selection scheme for self-driving cars. Probabilistic logic combines logic programming with probability theory to achieve clear, rule-based knowledge descriptions of multivariate probability distributions, and a flexible mixture of deductive and probabilistic inferences. Factored Markov decision processes (fMDPs) are widely used to generate reward-optimal action policies for stochastic sequential decision problems. For evaluation, we developed a simulated self-driving car with reliable modules for behavior selection, perception, and control. The behavior selection module is composed of a two-level structure of four action policies obtained from PL-fMDPs. Three main tests were conducted focused on the selection of the appropriate actions in specific driving scenarios, and the overtaking of static obstacle vehicles and dynamic obstacle vehicles. We performed 520 repetitions of these tests. The self-driving car completed its task without collisions in 99.2% of the repetitions. Results show the suitability of the overall self-driving strategy and PL-fMDPs to construct safe action policies for self-driving cars.

Weather Interaction-Aware Spatio-Temporal Attention Networks for Urban Traffic Flow Prediction

As the cornerstone of intelligent transportation systems, accurate traffic prediction can reduce the pressure of urban traffic, reduce the cost of residents’ travel time, and provide a reference basis for urban construction planning. Existing traffic prediction methods focus on spatio-temporal dependence modeling, ignoring the influence of weather factors on spatio-temporal characteristics, and the prediction task has complexity and an uneven distribution in different spatio-temporal scenarios and weather changes. In view of this, we propose a weather interaction-aware spatio-temporal attention network (WST-ANet), in which we integrate feature models and dynamic graph modules in the encoder and decoder, and use a spatio-temporal weather interaction perception module for prediction. Firstly, the contextual semantics of the traffic flows are fused using a feature embedding module to improve the adaptability to weather drivers; then, an encoder–decoder is constructed by combining the Dynamic Graph Module and the WSTA Block, to extract spatio-temporal aggregated correlations in the roadway network; finally, the feature information of the encoder was weighted and aggregated using the cross-focusing mechanism, and attention was paid to the hidden state of the encoding. Traffic flow was predicted using the PeMS04 and PeMS08 datasets and compared with multiple typical baseline models. It was learned through extensive experiments that the accuracy evaluation result is the smallest in WST-ANet, which demonstrated the superiority of the proposed model. This can more accurately predict future changes in traffic in different weather conditions, providing decision makers with a basis for optimizing scenarios.

Autonomous Driving System Driven by Artificial Intelligence Perception Fusion

Perception, as the information input module of the automatic driving system, determines the lower limit of the entire automatic driving system. Both autonomous driving perception and robot perception are constantly approaching the real physical world through digital methods, and this real physical world is only limited to the scope of human perception, such as lane lines, traffic lights, driving obstacles, and so on. The main premise of this process is that humans already know the categories or properties of the physical world, and only allow machines and systems to replicate human responses. Whether it is a pure visual route or a multi-source fusion route, the essence is the difference between the perceptual system schemes, one focusing on the vertical and the other on the horizontal. The pure vision solution represented by Tesla or the multi-source sensor fusion file represented by Waymo. In fact, the perception module of the automatic driving system usually has multiple sensors to achieve information redundancy and information complementarity through multiple dimensions, but there is the possibility of information conflict between different sensors. This paper aims at the advantages of perception-driven artificial intelligence to achieve breakthroughs in autonomous driving innovation, and analyzes how perception fusion drive is applied to the practical application of autonomous driving, so as to analyze the future development prospects of artificial intelligence.

Open-Pit Granite Mining Area Extraction Using UAV Aerial Images and the Novel GIPNet

The ability to rapidly and accurately delineate open-pit granite mining areas is pivotal for effective production planning and environmental impact assessment. Over the years, advancements in remote sensing techniques, including the utilization of satellite imagery, LiDAR technology and unmanned aerial vehicles, have revolutionized the way mining areas are monitored and managed. Simultaneously, in the context of the open-pit mining area extraction task, deep learning-based automatic recognition is gradually replacing manual visual interpretation. Leveraging the potential of unmanned aerial vehicles (UAVs) for real-time, low-risk remote sensing, this study employs UAV-derived orthophotos for mining area extraction. Central to the proposed approach is the novel Gather–Injection–Perception (GIP) module, designed to overcome the information loss typically associated with conventional feature pyramid modules during feature fusion. The GIP module effectively enriches semantic features, addressing a crucial information limitation in existing methodologies. Furthermore, the network introduces the Boundary Perception (BP) module, uniquely tailored to tackle the challenges of blurred boundaries and imprecise localization in mining areas. This module capitalizes on attention mechanisms to accentuate critical high-frequency boundary details in the feature map and synergistically utilizes both high- and low-dimensional feature map data for deep supervised learning. The suggested method demonstrates its superiority in a series of comparative experiments on a specially assembled dataset of research area images. The results are compelling, with the proposed approach achieving 90.67% precision, 92.00% recall, 91.33% F1-score, and 84.04% IoU. These figures not only underscore the effectiveness of suggested model in enhancing the extraction of open-pit granite mining areas but also provides a new idea for the subsequent application of UAV data in the mining scene.

Atom-ProteinQA: Atom-level protein model quality assessment through fine-grained joint learning

MotivationProtein model quality assessment (ProteinQA) is a fundamental task that is essential for biologically relevant applications, i.e., protein structure refinement, protein design, etc. Previous works aimed to conduct ProteinQA only on the global structure or per-residue level, ignoring potentially usable and precise cues from a fine-grained per-atom perspective. In this study, we propose an atom-level ProteinQA model, named Atom-ProteinQA, in which two innovative modules are designed to extract geometric and topological atom-level relationships respectively. Specifically, on the one hand, a geometric perception module exploits 3D sparse convolution to capture the geometric features of the input protein, generating fine-grained atom-level predictions. On the other hand, natural chemical bonds are utilized to construct an atom-level graph, then message passing from a topological perception module is applied to output residue-level predictions in parallel. Eventually, through a cross-model aggregation module, features from different modules mutually interact, enhancing performance on both the atom and residue levels. ResultsExtensive experiments show that our proposed Atom-ProteinQA outperforms previous methods by a large margin, regardless of residue-level or atom-level assessment. Concretely, we achieved state-of-the-art performance on CATH-2084, Decoy-8000, public benchmarks CASP13 & CASP14, and the CAMEO. AvailabilityThe repository of this project is released on: https://github.com/luyfcandy/Atom_ProteinQA.

HFENet: Hybrid feature encoder network for detecting salient objects in RGB-thermal images

Deep convolutional neural networks (CNNs) have gained prominence in computer vision applications, including RGB salient object detection (SOD), owing to the advancements in deep learning. Nevertheless, the majority of deep CNNs employ either VGGNet or ResNet as their backbone architecture for extracting image information. This approach may lead to the following problems. 1) Variations between imaging modalities during feature extraction across layers. Cross-modal features across layers are often fused in a single step, resulting in inadequate cross-modal feature extraction. 2) Feature long-range dependence problem in multilayer feature decoding. 3) Image boundary blurring. To address these issues, we initially leverage the advantages offered by the VGGNet and ResNet architectures. Additionally, we present a novel hybrid VGG–ResNet feature encoder for RGB-T SOD. Specifically, we introduce a geometry information aggregation module that effectively combines and enhances the VGGNet spatial features of the RGB-T modalities from the bottom to the top. Moreover, we propose a innovative global saliency perception module that progressively refines the ResNet semantic features from the top to the bottom by integrating both local and global information. Furthermore, we introduce a Pearson-gated module to tackle the challenge of long-range dependence between features. This module utilizes gating to merge features by calculating the Pearson correlation coefficients of the fused features at multiple levels. Lastly, we devise an edge-aware module to precisely learn the contours of salient objects, thereby enhancing the clarity of the object boundaries. Extensive experiments conducted on three RGB-T SOD benchmarks demonstrate that our proposed network surpasses the performance of state-of-the-art methods for SOD.

Reagent dosage inference based on graph convolutional memory perception network for zinc roughing flotation

In froth flotation, the precise presetting of reagent dosage is crucial for enhancing mineral recovery and flotation efficiency. Considering that the manual operation method is susceptible to some subjective factors, many machine vision-based intelligent reagent dosage presetting approaches are developed. However, prevailing presetting methods primarily rely on empirical knowledge, and ignore the potential intelligent reasoning processes, resulting in a suboptimal flotation performance. In this paper, a reagent dosage inference method based on a graph convolutional memory perception network is proposed, which can leverage the power of both empirical knowledge and neural network to deduce the optimal reagent dosages. This method constructs a memory unit stored with representative operation data, which can be served as a reservoir of empirical knowledge, facilitating dosage inference by referencing relevant data according to the current flotation condition. Subsequently, a graph convolutional neural network is developed to acquire the logical connections between reagent dosage and key parameters of the flotation process. Finally, with the guidance of empirical knowledge, a multi-head attention perception module is employed to deduce the optimal reagent dosage according to the current flotation condition and the acquired logical connection. The effectiveness of the proposed method has been validated in the experiment. Compared to other methods, the proposed method obtains more accurate inference results, and its inferred reagent dosage achieves more satisfactory flotation effect.

Blocking the dopaminergic receptors within the hippocampal dentate gyrus reduced analgesic responses induced by restraint stress in the formalin test

Previous studies have shown that various receptors, including dopamine receptors, are expressed in the hippocampal dentate gyrus (DG). Besides, indicatively, dopamine receptors play an essential role in the modulation of pain perception. On the other hand, stressful experiences can produce analgesia, termed stress-induced analgesia (SIA). The current study examined the probable role of dopamine receptors within the DG in antinociception induced by restraint stress (RS). Ninety-seven male albino Wistar rats were unilaterally implanted with a cannula in the DG. Animals received intra-DG microinjections of SCH23390 or Sulpiride (0.25, 1, and 4 μg/rat) as D1-and D2-like dopamine receptor antagonists, respectively, five minutes before RS. Ten minutes after the end of the induction of RS for three hours, 50 μl 2.5% formalin was injected subcutaneously into the plantar surface of the hind paw to induce persistent inflammatory pain. Pain scores were evaluated at 5-minute intervals for 60 minutes. These findings showed that; exposure to RS for three hours produced SIA in both phases of the formalin test, while this RS-induced analgesia was attenuated in the early and late phases of the formalin test by intra-DG microinjection of SCH23390 and Sulpiride. The results of the present study suggested that both D1- and D2-like dopamine receptors in the DG have a considerable role in the induced analgesia by RS.

RASNet: Renal automatic segmentation using an improved U-Net with multi-scale perception and attention unit

For the treatment of renal disease, the application of radioactive equipment has become one of the important methods. Accurate segmentation of renal contour plays an important role in clinical diagnosis. However, manual renal contour drawing is not only inefficient but also prone to inaccurate outlining results due to different manual proficiency and fatigue caused by long-term work. There is little research on automatic renal segmentation with renal dynamic imaging. To address this issue, an improved model based on a deep neural network called Renal Automatic Segmentation Network (RASNet) is proposed, to aid in the automatic segmentation of renal contours. Besides, a multi-scale spatial perception module and a decoding module with attention connection are introduced to enrich the semantic information and further improve the accuracy of network segmentation. Extensive experiments were conducted on a renal dynamic medical image database established in this paper. Analysis results show the superiority of the proposed RASNet to several existing segmentation frameworks.

Modulation of pain perception through transcranial alternating current stimulation and its nonlinear relationship with the simulated electric field magnitude.

Oscillatory activities observed in multiple regions are closely associated with the experience of pain. Specifically, oscillatory activities within the theta- and beta-frequency bands, observed in the left dorsolateral prefrontal cortex (DLPFC), have been implicated in pain perception among healthy individuals and those with chronic pain. However, their physiological significance remains unclear. We explored the modulation of pain perception in healthy individuals by theta- and beta-band transcranial alternating current stimulation (tACS) over the left DLPFC and examined the relationship between the modulation effect and magnitude of the electric field elicited by tACS in the left DLPFC using computational simulation. Our findings revealed that both theta- and beta-tACS increased the heat pain threshold during and after stimulation. Notably, the simulated electric field magnitude in the left DLPFC exhibited an inverted U-shaped relationship with the pain modulation effect for theta-tACS. Our study findings suggested that there would be an optimal electric field strength to produce a high analgesic effect for theta-tACS. The application of theta- and beta-tACS interventions targeting the left DLPFC might facilitate the treatment of chronic pain. Furthermore, the attainment of effective pain modulation via theta-tACS over the DLPFC warrants the use of optimal stimulus intensity.

Perceptual transitions between object rigidity and non-rigidity: Competition and cooperation among motion energy, feature tracking, and shape-based priors.

Why do moving objects appear rigid when projected retinal images are deformed non-rigidly? We used rotating rigid objects that can appear rigid or non-rigid to test whether shape features contribute to rigidity perception. When two circular rings were rigidly linked at an angle and jointly rotated at moderate speeds, observers reported that the rings wobbled and were not linked rigidly, but rigid rotation was reported at slow speeds. When gaps, paint, or vertices were added, the rings appeared rigidly rotating even at moderate speeds. At high speeds, all configurations appeared non-rigid. Salient features thus contribute to rigidity at slow and moderate speeds but not at high speeds. Simulated responses of arrays of motion-energy cells showed that motion flow vectors are predominantly orthogonal to the contours of the rings, not parallel to the rotation direction. A convolutional neural network trained to distinguish flow patterns for wobbling versus rotation gave a high probability of wobbling for the motion-energy flows. However, the convolutional neural network gave high probabilities of rotation for motion flows generated by tracking features with arrays of MT pattern-motion cells and corner detectors. In addition, circular rings can appear to spin and roll despite the absence of any sensory evidence, and this illusion is prevented by vertices, gaps, and painted segments, showing the effects of rotational symmetry and shape. Combining convolutional neural network outputs that give greater weight to motion energy at fast speeds and to feature tracking at slow speeds, with the shape-based priors for wobbling and rolling, explained rigid and non-rigid percepts across shapes and speeds (R2 = 0.95). The results demonstrate how cooperation and competition between different neuronal classes lead to specific states of visual perception and to transitions between the states.

Modulation of sleep perception through auditory closed-loop stimulation

Modulation of sleep perception through auditory closed-loop stimulation

Neural Decoding of the Speech Envelope: Effects of Intelligibility and Spectral Degradation.

During continuous speech perception, endogenous neural activity becomes time-locked to acoustic stimulus features, such as the speech amplitude envelope. This speech-brain coupling can be decoded using non-invasive brain imaging techniques, including electroencephalography (EEG). Neural decoding may provide clinical use as an objective measure of stimulus encoding by the brain-for example during cochlear implant listening, wherein the speech signal is severely spectrally degraded. Yet, interplay between acoustic and linguistic factors may lead to top-down modulation of perception, thereby complicating audiological applications. To address this ambiguity, we assess neural decoding of the speech envelope under spectral degradation with EEG in acoustically hearing listeners (n = 38; 18-35 years old) using vocoded speech. We dissociate sensory encoding from higher-order processing by employing intelligible (English) and non-intelligible (Dutch) stimuli, with auditory attention sustained using a repeated-phrase detection task. Subject-specific and group decoders were trained to reconstruct the speech envelope from held-out EEG data, with decoder significance determined via random permutation testing. Whereas speech envelope reconstruction did not vary by spectral resolution, intelligible speech was associated with better decoding accuracy in general. Results were similar across subject-specific and group analyses, with less consistent effects of spectral degradation in group decoding. Permutation tests revealed possible differences in decoder statistical significance by experimental condition. In general, while robust neural decoding was observed at the individual and group level, variability within participants would most likely prevent the clinical use of such a measure to differentiate levels of spectral degradation and intelligibility on an individual basis.

Heat Transfer-Inspired Network for Image Super-Resolution Reconstruction.

Image super-resolution (SR) is a critical image preprocessing task for many applications. How to recover features as accurately as possible is the focus of SR algorithms. Most existing SR methods tend to guide the image reconstruction process with gradient maps, frequency perception modules, etc. and improve the quality of recovered images from the perspective of enhancing edges, but rarely optimize the neural network structure from the system level. In this article, we conduct an in- depth exploration for the inner nature of the SR network structure. In light of the consistency between thermal particles in the thermal field and pixels in the image domain, we propose a novel heat-transfer-inspired network (HTI-Net) for image SR reconstruction based on the theoretical basis of heat transfer. With the finite difference theory, we use a second-order mixed-difference equation to redesign the residual network (ResNet), which can fully integrate multiple information to achieve better feature reuse. In addition, according to the thermal conduction differential equation (TCDE) in the thermal field, the pixel value flow equation (PVFE) in the image domain is derived to mine deep potential feature information. The experimental results on multiple standard databases demonstrate that the proposed HTI-Net has superior edge detail reconstruction effect and parameter performance compared with the existing SR methods. The experimental results on the microscope chip image (MCI) database consisting of realistic low-resolution (LR) and high-resolution (HR) images show that the proposed HTI-Net for image SR reconstruction can improve the effectiveness of the hardware Trojan detection system.

Overlap-Aware Hierarchical Decoder for point cloud registration

Extracting high-quality correspondences is a critical challenge in current feature-learning based point cloud registration methods. Recently, coarse-to-fine network structures have shown great potential in addressing this challenge. Inspired by such structures, we investigate the effectiveness of two-stage network optimization in matching and propose a non-keypoint registration model called Overlap-Aware Hierarchical Decoder (OAH-Net). The construction of this model focuses on reducing outliers in the matching results and improving the understanding of geometric transformation invariance. To achieve this goal, we propose a point-to-point perception module for encoding the paired point clouds and perceiving the overlapping regions, as well as a pyramid hierarchical decoder for decoding multi-level features. We design an optimal matching mechanism adapted to the pyramid structure to handle accurate correspondences from multi-level matching, thereby improving the accuracy of the final registration. Experimental evaluations conducted on indoor and outdoor benchmark tests validate the outstanding performance of OAH-Net, which not only ensures low parameter count and computational efficiency but also demonstrates accuracy and stability comparable to state-of-the-art models.

Cross-modality integration framework with prediction, perception and discrimination for video anomaly detection

Video anomaly detection is an important task for public security in the multimedia field. It aims to distinguish events that deviate from normal patterns. As important semantic representation, the textual information can effectively perceive different contents for anomaly detection. However, most existing methods primarily rely on visual modality, with limited incorporation of textual modality in anomaly detection. In this paper, a cross-modality integration framework (CIForAD) is proposed for anomaly detection, which combines both textual and visual modalities for prediction, perception and discrimination. Firstly, a feature fusion prediction (FUP) module is designed to predict the target regions by fusing the visual features and textual features for prompting, which can amplify the discriminative distance. Then an image-text semantic perception (ISP) module is developed to judge semantic consistency by associating the fine-grained visual features with textual features, where a strategy of local training and global inference is introduced to perceive local details and global semantic correlation. Finally, a self-supervised time attention discrimination (TAD) module is built to explore the inter-frame relation and further distinguish abnormal sequences from normal sequences. Extensive experiments on the three challenging benchmarks indicate that our CIForAD obtains state-of-the-art anomaly detection performance.

Time-aware and task-transferable adversarial attack for perception of autonomous vehicles

With rapid development of self-driving vehicles, recent work in adversarial machine learning started to study adversarial examples (AEs) for perception of autonomous driving (AD). However, generating practical AEs for the perception module remains a significant challenge. Traditional adversarial attacks tend to focus on a single computer vision task, making it difficult to compromise multiple perception tasks such as object detection and segmentation simultaneously. Additionally, the limited computational resources available on-board and the necessity for online operation pose further obstacles to deploying adversarial attacks on real autonomous driving platforms. To address the aforementioned issues, we propose the Time-aware Perception Attack (TPA), which is a real-time cross-task adversarial attack for perception of autonomous driving. In particular, we propose a novel backbone based adversarial attack method to modify input images to approach Lipschitz Constant Point (LCP), which results in erroneous inferences for all the sub-models in perception module. The novel part of this work is proposing an efficient yet effective LCP approaching algorithm. Comparing to conventional LCP based attacks, which consume significant amount of computation resources and can be only applied on small DNNs, TPA generates AEs on an intermediate layer of surrogate backbone, significantly enhancing the cross-task transferability and accelerates the attack process. Evaluation results on Berkeley Driving Dataset 100k (BDD100k) show that, comparing to the state-of-the-art baselines, the proposed TPA achieves higher attack effectiveness and faster processing speed and outperforms the baselines by a large margin.

Thermo-inspired model of self-propelled hard disk agents for heterogeneous bidirectional pedestrian flow

The aim of this work is to present a hybrid pedestrian model that describes the bidirectional walking of crowds in a corridor. The self-propelled hard disk model combines both perceptual as well as decision-making characteristics of situated agents. This model considers homogeneous and heterogeneous groups with relaxed, standard, and hurried pedestrian profiles according to their walking preferences. The situated agent perception module considers one radius in the base model, while the extended model considers three radii to improve the pedestrian evasion rules. We have explored the corresponding changes in the formation of spatial walking patterns. The findings include comparative variations in the formation of following lanes, the time-based evolution of crowds in both homogeneous and heterogeneous groups, and the transition from transient traffic jams to the appearance of segregation waves emerging from self-organization mechanisms. In this point, we propose a thermodynamics-inspired model associated with the collective behavior of pedestrians. A macroscopic ideal gas-type equation of state is established to define the dependence between volume, pressure, and temperature of pedestrian crowds in counterflow. To achieve this goal, the temperature, defined as T∼vα¯ where v is the walking speed of individual pedestrians, must possess an α parameter in the interval [1,2]. The experimental simulations include binary contact interactions and collisions of pedestrians with the corridor borders. The macroscopic description of the model inspired by thermodynamics can be complemented with the usual analysis of the fundamental diagram.