Visual Constraints Research Articles

Correctable Landmark Discovery via Large Models for Vision-Language Navigation.

Vision-Language Navigation (VLN) requires the agent to follow language instructions to reach a target position. A key factor for successful navigation is to align the landmarks implied in the instruction with diverse visual observations. However, previous VLN agents fail to perform accurate modality alignment especially in unexplored scenes, since they learn from limited navigation data and lack sufficient open-world alignment knowledge. In this work, we propose a new VLN paradigm, called COrrectable LaNdmark DiScOvery via Large ModEls (CONSOLE). In CONSOLE, we cast VLN as an open-world sequential landmark discovery problem, by introducing a novel correctable landmark discovery scheme based on two large models ChatGPT and CLIP. Specifically, we use ChatGPT to provide rich open-world landmark cooccurrence commonsense, and conduct CLIP-driven landmark discovery based on these commonsense priors. To mitigate the noise in the priors due to the lack of visual constraints, we introduce a learnable cooccurrence scoring module, which corrects the importance of each cooccurrence according to actual observations for accurate landmark discovery. We further design an observation enhancement strategy for an elegant combination of our framework with different VLN agents, where we utilize the corrected landmark features to obtain enhanced observation features for action decision. Extensive experimental results on multiple popular VLN benchmarks (R2R, REVERIE, R4R, RxR) show the significant superiority of CONSOLE over strong baselines. Especially, our CONSOLE establishes the new state-of-the-art results on R2R and R4R in unseen scenarios.

W-VSLAM: A Visual Mapping Algorithm for Indoor Inspection Robots.

In recent years, with the widespread application of indoor inspection robots, high-precision, robust environmental perception has become essential for robotic mapping. Addressing the issues of visual-inertial estimation inaccuracies due to redundant pose degrees of freedom and accelerometer drift during the planar motion of mobile robots in indoor environments, we propose a visual SLAM perception method that integrates wheel odometry information. First, the robot's body pose is parameterized in SE(2) and the corresponding camera pose is parameterized in SE(3). On this basis, we derive the visual constraint residuals and their Jacobian matrices for reprojection observations using the camera projection model. We employ the concept of pre-integration to derive pose-constraint residuals and their Jacobian matrices and utilize marginalization theory to derive the relative pose residuals and their Jacobians for loop closure constraints. This approach solves the nonlinear optimization problem to obtain the optimal pose and landmark points of the ground-moving robot. A comparison with the ORBSLAM3 algorithm reveals that, in the recorded indoor environment datasets, the proposed algorithm demonstrates significantly higher perception accuracy, with root mean square error (RMSE) improvements of 89.2% in translation and 98.5% in rotation for absolute trajectory error (ATE). The overall trajectory localization accuracy ranges between 5 and 17 cm, validating the effectiveness of the proposed algorithm. These findings can be applied to preliminary mapping for the autonomous navigation of indoor mobile robots and serve as a basis for path planning based on the mapping results.

Presaccadic Attention Enhances and Reshapes the Contrast Sensitivity Function Differentially around the Visual Field.

Contrast sensitivity (CS), which constrains human vision, decreases from fovea to periphery, from the horizontal to the vertical meridian, and from the lower vertical to the upper vertical meridian. It also depends on spatial frequency (SF), and the contrast sensitivity function (CSF) depicts this relation. To compensate for these visual constraints, we constantly make saccades and foveate on relevant objects in the scene. Already before saccade onset, presaccadic attention shifts to the saccade target and enhances perception. However, it is unknown whether and how it modulates the interplay between CS and SF, and if this effect varies around polar angle meridians. CS enhancement may result from a horizontal or vertical shift of the CSF, increase in bandwidth, or any combination. In addition, presaccadic attention could enhance CS similarly around the visual field, or it could benefit perception more at locations with poorer performance (i.e., vertical meridian). Here, we investigated these possibilities by extracting key attributes of the CSF of human observers. The results reveal that presaccadic attention (1) increases CS across SF, (2) increases the most preferred and the highest discernable SF, and (3) narrows the bandwidth. Therefore, presaccadic attention helps bridge the gap between presaccadic and postsaccadic input by increasing visibility at the saccade target. Counterintuitively, this CS enhancement was more pronounced where perception is better-along the horizontal than the vertical meridian-exacerbating polar angle asymmetries. Our results call for an investigation of the differential neural modulations underlying presaccadic perceptual changes for different saccade directions.

Anti-disturbance control strategy in capture stage for AUV dynamic base docking with optical guided constraints

This article investigates the challenge of potential optical guidance failure at the end of the underwater docking process due to relative attitude deviation between the Autonomous Underwater Vehicle (AUV) and the recovery platform, especially in the context of a moving base. A novel anti-disturbance control strategy is presented, accounting for environmental dynamics and optical guidance limitations. Firstly, we establish constraint models such as the effective field of view and the guidance beam associated with the optical guidance method. We then divide the recovery control strategy area based on the constraint model and propose the Line-of-sight Considering Visual Constraints (LOSCVC) docking control strategy. This strategy utilizes the line-of-sight method and the virtual leader method to make motion decisions based on the relative attitude offset and completes the docking operation by tracking the motion of the virtual leader. Furthermore, we consider the effect of ocean currents on the docking process of the moving base and design a disturbance compensation control law based on the Lyapunov redesign method. This strategy ensures the AUV maintains effective and continuous optical guidance during the capture phase, thereby achieving precise, safe, and rapid docking control. Simulation results validate the effectiveness and robustness of the proposed docking control strategy.

Lecture sur écran : apports et contraintes

The use of digital tools means that every day we have to deal with digital information that we need to see, identify and understand in order to use it properly. Current research on reading incorporates this constant and rapid evolution of digital technologies, in order to better understand the interactions between interfaces and users. The aim of this article is to provide a summary of current research examining the impact of screens on reading comprehension performance. After describing the mechanics of reading and the cognitive models used to formalise text comprehension, this article describes the characteristics of reading on digital media, identifying in particular the associated perceptual and visual constraints, and the way in which the type of medium influences performance in text comprehension, particularly in classroom situations.

Investigating the crowding effect on letters and symbols in deaf adults

Reading requires the transformation of a complex array of visual features into sounds and meaning. For deaf signers who experience changes in visual attention and have little or no access to the sounds of the language they read, understanding the visual constraints underlying reading is crucial. This study aims to explore a fundamental aspect of visual perception intertwined with reading: the crowding effect. This effect manifests as the struggle to distinguish a target letter when surrounded by flanker letters. Through a two-alternative forced choice task, we assessed the recognition of letters and symbols presented in isolation or flanked by two or four characters, positioned either to the left or right of fixation. Our findings reveal that while deaf individuals exhibit higher accuracy in processing letters compared to symbols, their performance falls short of that of their hearing counterparts. Interestingly, despite their proficiency with letters, deaf individuals didn’t demonstrate quicker letter identification, particularly in the most challenging scenario where letters were flanked by four characters. These outcomes imply the development of a specialized letter processing system among deaf individuals, albeit one that may subtly diverge from that of their hearing counterparts.

An Adaptive Simplification Method for Coastlines Using a Skeleton Line “Bridge” Double Direction Buffering Algorithm

Aiming at the problem that the current double direction buffering algorithm is easy to use to seal the “bottleneck” area when simplifying coastlines, an adaptive simplification method for coastlines using a skeleton line “bridge” double direction buffering algorithm is proposed. Firstly, from the perspective of visual constraints, the relationship between the buffer distance and the coastline line width and the minimum recognition distance of the human eye is theoretically derived and determined. Then, based on the construction of the coastline skeleton binary tree, the “bridge” skeleton line is extracted using the “source tracing” algorithm. Finally, the shoreline adaptive simplification is realized by constructing a visual buffer of “bridge” skeleton lines to bridge the original resulting coastline and the local details. The experimental results show that the proposed method can effectively solve the problem that the current double direction buffering algorithm has, which can significantly improve the quality of simplification.

LVI-Fusion: A Robust Lidar-Visual-Inertial SLAM Scheme

With the development of simultaneous positioning and mapping technology in the field of automatic driving, the current simultaneous localization and mapping scheme is no longer limited to a single sensor and is developing in the direction of multi-sensor fusion to enhance the robustness and accuracy. In this study, a localization and mapping scheme named LVI-fusion based on multi-sensor fusion of camera, lidar and IMU is proposed. Different sensors have different data acquisition frequencies. To solve the problem of time inconsistency in heterogeneous sensor data tight coupling, the time alignment module is used to align the time stamp between the lidar, camera and IMU. The image segmentation algorithm is used to segment the dynamic target of the image and extract the static key points. At the same time, the optical flow tracking based on the static key points are carried out and a robust feature point depth recovery model is proposed to realize the robust estimation of feature point depth. Finally, lidar constraint factor, IMU pre-integral constraint factor and visual constraint factor together construct the error equation that is processed with a sliding window-based optimization module. Experimental results show that the proposed algorithm has competitive accuracy and robustness.

Adversarial attacks on video quality assessment models

Most currently developed video quality assessment (VQA) algorithms have achieved excellent performance by using deep neural network (DNN). However, DNN is vulnerable to adversarial attacks, as an efficient surrogate for validating the model robustness, and there lack adversarial attack methods against VQA models. To this end, we propose a spatiotemporal attack network to generate adversarial examples for evaluating the robustness of VQA models that contains a spatial subnetwork and a temporal subnetwork. The proposed network, dubbed the Space-Time Quality Attack Network (STQA-Net1), first computes the just noticeable difference (JND) maps of a video sequence as the input of the spatial subnetwork. The spatial subnetwork encodes the computed maps as spatial features and feeds the spatial features to the temporal subnetwork. Then, the spatial features are fused with the output of the temporal subnetwork and the fused features are decoded as attack weight maps. A visual constraint is used to control the visibility of perturbations and guide the generation of perturbation maps by multiplying JND maps with attack weight maps. Finally, the generated perturbation maps are added to the original video to form an adversarial example. Further, we also try to design a two-branch network to generate two opposite examples in a targeted attack scenario. The proposed attack methods against six state-of-the-art VQA algorithms are thoroughly tested on three VQA databases. The experimental results show that the proposed attack methods are very effective for testing the robustness of VQA models.

A self-supervised monocular odometry with visual-inertial and depth representations

Background:Self-supervised monocular visual-odometry (VO) methods have demonstrated impressive results in relative pose estimation, but they encounter challenges when visual information is unreliable. To address this, we introduce a self-supervised visual-inertial odometry (VIO) system. Methods:Our method adopts a trinocular assumption, where three consecutive frames are treated as if captured simultaneously by three cameras. This approach expands the field of view and enhances robustness in visual constraints for 3D structures. Additionally, we incorporate Inertial Measurement Unit (IMU) data into the system. This integration, achieved by an attention-based fusion strategy, allows our visual-inertial odometry (VIO) system to adaptively select the most reliable representations for accurate relative pose estimation. Results:Experiments on public and self-collected datasets show that the proposed method is state-of-the-art in visual odometry. Conclusion:The encouraging results show that it is feasible to apply this learning-based VIO system in autonomous cars and robots.

Predation of the black-eared opossum (Didelphis aurita) on the great kiskadee (Pitangus sulphuratus) and the water rat (Nectomys squamipes)

Abstract Understanding the predatory behavior of a species is critical to its ecological role. Little is known about the predatory behavior of the marsupial Didelphis aurita. Here, we present two cases to shed light on its opportunistic nature: an opossum struggling to locate and attack an immobile juvenile bird, indicating olfactory detection and visual constraints, and an opossum carrying a weakened rat that was host of four blowfly larvae. These events reinforce D. aurita’s opportunistic behavior in attacking debilitated vertebrates.

Relationships Between Task Constraints, Visual Constraints, Joint Coordination and Football-Specific Performance in Talented Youth Athletes: An Ecological Dynamics Approach.

Individual performance in team sports is a multifactorial reflection of how well a player can cope and accomplish tasks in varied playing situations. Thus, performance analysis should not only focus on outcomes, but also on underlying mechanisms of those outcomes. We adopted principles of the ecological dynamics approach (EDA) to investigate the effect of introducing constraints on players' joint coordination responses for a football-specific performance drill outcome. Seventeen talented youth football (soccer) players performed a football-specific drill under different conditions: basic constraints, additional defender dummies, stroboscopic glasses, and a combination of the latter two constraints. We recorded these players' execution time, passing accuracy, and lower extremity joint kinematics. We calculated joint coordination for hip-knee, knee-ankle, and trunk-hip couplings. The added constraints negatively affected execution time and passing accuracy, and caused changes in joint coordination. Furthermore, we identified a relationship between execution time and joint coordination. This study serves as an example how the EDA can be adopted to investigate mechanisms that underlie individual performance in team sports.

Improved Edge Folding Algorithm for 3D Building Models Taking into Account the Visual Features

Simplifying 3D building models, effectively reducing model complexity and improving mapping efficiency, is an important part of 3D GIS. In order to address the problem that the simplification basis considered by most 3D building model data compression algorithms is mainly based on geometric features and fails to retain the visual features of the model, this paper proposed a half-folding simplification algorithm for 3D building models that took the visual features into account. The algorithm incorporated regional boundary constraints in the pre-processing stage to focus on preserving the boundary features of the model, and incorporated visual constraints, such as the vertex importance, edge length, and texture, in the calculation of the edge folding cost to construct a new error metric model. The simplification order was changed by redefining the edge folding cost, and a trade-off between compression rate and retention rate was made by using half-edge folding to maximise the retention of the visual features of the building. The experimental results showed that the algorithm could effectively reduce the number of elements drawn by the system and achieve a high level of detail information retention in the visual feature region of the model with a good visual representation.

Skyglow relieves a crepuscular bird from visual constraints on being active

Artificial light at night significantly alters the predictability of the natural light cycles that most animals use as an essential Zeitgeber for daily activity. Direct light has well-documented local impacts on activity patterns of diurnal and nocturnal organisms. However, artificial light at night also contributes to an indirect illumination of the night sky, called skyglow, which is rapidly increasing. The consequences of this wide-spread form of artificial night light on the behaviour of animals remain poorly understood, with only a few studies performed under controlled (laboratory) conditions. Using animal-borne activity loggers, we investigated daily and seasonal flight activity of a free-living crepuscular bird species in response to nocturnal light conditions at sites differing dramatically in exposure to skyglow. We find that flight activity of European Nightjars (Caprimulgus europaeus) during moonless periods of the night is four times higher in Belgium (high skyglow exposure) than in sub-tropical Africa and two times higher than in Mongolia (near-pristine skies). Moreover, clouds darken the sky under natural conditions, but skyglow can strongly increase local sky brightness on overcast nights. As a result, we find that nightjars' response to cloud cover is reversed between Belgium and sub-tropical Africa and between Belgium and Mongolia. This supports the hypothesis that cloudy nights reduce individual flight activity in a pristine environment, but increase it when the sky is artificially lit. Our study shows that in the absence of direct light pollution, anthropogenic changes in sky brightness relieve nightjars from visual constraints on being active. Individuals adapt daily activities to artificial night-sky brightness, allowing them more time to fly than conspecifics living under natural light cycles. This modification of the nocturnal timescape likely affects behavioural processes of most crepuscular and nocturnal species, but its implications for population dynamics and interspecific interactions remain to be investigated.

Acoustic recording complements camera traps for monitoring sensitive penguin populations

In the face of global biodiversity loss that threatens social–ecological sustainability, we must improve our knowledge of species' behaviours, life histories and populations. Enhanced understanding is especially critical for taxa like seabirds that live in remote habitats where monitoring is logistically challenging, and technology‐based approaches could lead to major advances. Considering conservation threats and breeding behaviours, penguins represent a promising taxon for further exploration of monitoring methods. Only five of 18 penguin species are currently considered species of Least Concern by the IUCN, and the sensitivity of penguins to ecological disturbances makes them important indicators of ecosystem dynamics. Penguins regularly congregate in dense breeding colonies and are visually and aurally conspicuous animals, making them relatively easy to monitor in some respects. In situ observations and visual remote sensing monitoring methods (i.e. camera traps, uncrewed aerial vehicles and satellite imagery) currently provide highly valuable information on penguin behaviour and population dynamics, but these methods have a number of shortcomings. Several rely on human presence in remote areas, some require clean lines of sight with no visual obstructions, and some offer limited spatial and temporal resolution. In this study we explore the use of passive acoustic monitoring (PAM) as a complementary remote sensing method to monitor penguin behaviour and populations at high spatial and temporal resolution without visual constraints or the need for continuous human presence. We conducted observations of vocalization rates and placed automated acoustic recorders in colonies of Magellanic Penguins Spheniscus magellanicus and Southern Rockhopper Penguins Eudyptes chrysocome in conjunction with camera traps. We found positive relationships between acoustic activity and counts of Magellanic Penguins in camera trap photos. We also identified clear diel patterns of acoustic activity that differed between breeding stages, and we found positive correlations between acoustic activity and estimates of colony density for various times of day and radii around recorders. While much work remains to improve this method and refine interpretation, PAM holds great promise as a complementary tool for monitoring the relative abundance and behavioural dynamics of penguins and other colonial animals, particularly those that burrow or nest in dense vegetation that impairs visual monitoring techniques.

Understanding the visual constraints on lexical processing: New empirical and simulation results.

Word identification is slower and less accurate outside central vision, but the precise relationship between retinal eccentricity and lexical processing is not well specified by models of either word identification or reading. In a seminal eye-movement study, Rayner and Morrison (1981) found that participants made remarkably accurate naming and lexical-decision responses to words displayed more than 3 degrees from the center of vision-even under conditions requiring fixed gaze. However, the validity of these findings is challenged by a range of methodological limitations. We report a series of gaze-contingent lexical-decision and naming experiments that replicate and extend Rayner and Morrison's study to provide a more accurate estimate of how visual constraints delimit lexical processing. Simulations were conducted using the E-Z Reader model (Reichle et al., 2012) to assess the implications for understanding eye-movement control during reading. Augmenting the model's assumptions about the impact of both eccentricity and visual crowding on the rate of lexical processing provided good fits to the observed data without impairing the model's ability to simulate benchmark eye-movement effects. The findings are discussed with a view toward the development of a complete model of reading. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Increased water temperature and turbidity act independently to alter social behavior in guppies (Poecilia reticulata).

Changes in environmental conditions can shift the costs and benefits of aggregation or interfere with the sensory perception of near neighbors. This affects group cohesion with potential impacts on the benefits of collective behavior such as reduced predation risk. Organisms are rarely exposed to one stressor in isolation, yet there are only a few studies exploring the interactions between multiple stressors and their effects on social behavior. Here, we tested the effects of increased water temperature and turbidity on refuge use and three measures of aggregation in guppies (Poecilia reticulata), increasing temperature and turbidity in isolation or in combination. When stressors were elevated in isolation, the distribution of fish within the arena as measured by the index of dispersion became more aggregated at higher temperatures but less aggregated when turbidity was increased. Another measure of cohesion at the global scale, the mean inter-individual distance, also indicated that fish were less aggregated in turbid water. This is likely due to turbidity acting as a visual constraint, as there was no evidence of a change in risk perception as refuge use was not affected by turbidity. Fish decreased refuge use and were closer to their nearest neighbor at higher temperatures. However, the nearest neighbor distance was not affected by turbidity, suggesting that local-scale interactions can be robust to the moderate increase in turbidity used here (5 NTU) compared with other studies that show a decline in shoal cohesion at higher turbidity (>100 NTU). We did not observe any significant interaction terms between the two stressors, indicating no synergistic or antagonistic effects. Our study suggests that the effects of environmental stressors on social behavior may be unpredictable and dependent on the metric used to measure cohesion, highlighting the need for mechanistic studies to link behavior to the physiology and sensory effects of environmental stressors.

Minimally Persistent Graph Generation and Formation Control for Multi-Robot Systems under Sensing Constraints

This paper presents a minimally persistent graph generation and formation control strategy for multi-robot systems with sensing constraints. Specifically, each robot has a limited field of view (FOV) and range sensing capability. To tackle this problem, one needs to construct an appropriate interaction topology, namely assign neighbors to each robot such that all their sensing constraints are satisfied. In addition, as a stringent yet reasonable guarantee for the visual constraints, it is also required that the prescribed neighbors always stay within its visual field during the formation evolution. To this end, given a set of feasible initial positions, we first present a depth-first-search (DFS)-based algorithm to generate a minimally persistent graph, which encodes the sensing constraints via its directed edges. Then, based on the resultant graph, by invoking the gradient-based control technique and control barrier function (CBF), we propose a class of distributed formation control laws, rendering not only the convergence to the desired formation but also the satisfaction of sensing constraints. Simulation and experimental results are presented to verify the effectiveness of the proposed approach.

Feasibility assessment of solar photovoltaic deployments on building surfaces with the constraint of visual impacts

As a major component of social acceptance, visual impact is often considered a significant constraint in solar applications. Visual impact assessment of solar applications, however, has been limited to pedestrians in previous studies. The extent to which PV systems can have visual impacts on occupants and whether it is necessary to include occupants in the measurement of visual impact remains uncertain. To fill this gap, we extended it from pedestrians to occupants and proposed a quantitative method to integrate pedestrians and occupants into a framework, combining the estimation of solar potential for the feasibility assessment of PV applications in a built environment. The concept is tested with a real case, located in Qingdao city, China, to present the technical flowchart for the feasibility assessment of solar PV deployments with the visual constraint. Building surfaces with qualified solar irradiation and low visibility were identified and compared in two cases, that is, with and without the inclusion of occupants as the visual constraint. The comparison results show that the change of suitable building surfaces for solar applications is 172,306 m2 (21% of suitable area) and 126 GWh (19% of yield energy) across the study area, indicating the significance of including occupants in the visibility assessment for the deployment of solar applications. The proposed method considers the visual constraint for the feasibility assessment of solar applications from the perspective of pedestrians and occupants, and it is helpful to identify the suitable surfaces for the large-scale deployment of solar applications at an early planning stage of solar city.

Visual field asymmetries in numerosity processing

A small number of objects can be rapidly and accurately enumerated, whereas a larger number of objects can only be approximately enumerated. These subitizing and estimation abilities, respectively, are both spatial processes relying on extracting information across spatial locations. Nevertheless, whether and how these processes vary across visual field locations remains unknown. Here, we examined if enumeration displays asymmetries around the visual field. Experiment 1 tested small number (1–6) enumeration at cardinal and non-cardinal peripheral locations while manipulating the spacing among the objects. Experiment 2 examined enumeration at cardinal locations in more detail while minimising crowding. Both experiments demonstrated a Horizontal-Vertical Asymmetry (HVA) where performance was better along the horizontal axis relative to the vertical. Experiment 1 found that this effect was modulated by spacing with stronger asymmetry at closer spacing. Experiment 2 revealed further asymmetries: a Vertical Meridian Asymmetry (VMA) with better enumeration on the lower vertical meridian than on the upper and a Horizontal Meridian Asymmetry (HMA) with better enumeration along the left horizontal meridian than along the right. All three asymmetries were evident for both subitizing and estimation. HVA and VMA have been observed in a range of visual tasks, indicating that they might be inherited from early visual constraints. However, HMA is observed primarily in mid-level tasks, often involving attention. These results suggest that while enumeration processes can be argued to inherit low-level visual constraints, the findings are, parsimoniously, consistent with visual attention playing a role in both subitizing and estimation.