Viewpoint Changes Research Articles

비평형의 생태인류학

The existing approaches to ecological anthropology-cultural ecology, cultural materialism, and environmental conservation studies-would suppose an equilibrium ecological thought for stability, balance, climax, and succession of nature. However, changing viewpoint from equilibrium to non-equilibrium ecology emphasized temporality, variability, externality and resilience. It is also attempted to explain nature-human relationships in new ways. Tracing issues of ecological anthropology-land degradation, the tragedy of common, indigenous people and knowledge and biodiversity conservation, I explored the change of research tendency reflecting non-equilibrium thought. The ethnographic studies of non-western society and constructing scientific knowledges overcame the nature/culture dichotomy and showed the entanglement of human and nonhuman networks. The non-equilibrium thinking could present describing ecological matters and practices more complex.

A Robust and Efficient Loop Closure Detection Approach for Hybrid Ground/Aerial Vehicles

Frequent and dramatic viewpoint changes make loop closure detection of hybrid ground/aerial vehicles extremely challenging. To address this issue, we present a robust and efficient loop closure detection approach based on the state-of-the-art simultaneous localization and mapping (SLAM) framework and pre-trained deep learning models. First, the outputs of the SuperPoint network are processed to extract both tracking features and additional features used in loop closure. Next, binary-encoded SuperPoint descriptors are applied with a method based on Bag of VisualWords (BoVW) to detect loop candidates efficiently. Finally, the combination of SuperGlue and SuperPoint descriptors provides correspondences of keypoints to verify loop candidates and calculate relative poses. The system is evaluated on the public datasets and a real-world hybrid ground/aerial vehicles dataset. The proposed approach enables reliable loop detection, even when the relative translation between two viewpoints exceeds 7 m or one of the Euler angles is above 50°.

An adaptive feature region-based line segment matching method for viewpoint-changed images with discontinuous parallax and poor textures

Many methods have been proposed to extract line segment (LS) correspondences for images with viewpoint variations. However, the matching performance on images with viewpoint variations is still limited due to the influence of discontinuous parallax and poor textures on images. Existing line segment matching (LSM) methods rarely consider the coexistence of these two situations. This study attempts to address this problem by proposing an adaptive feature region (FR)-based LSM method based on the observation that different FRs are suitable for different image scenes under image viewpoint change. In the proposed method, LSs are paired, and two types of FRs, symmetric FR centered on the line-pair intersection and asymmetric FR with the intersection as a vertex, are constructed for every line pair. The asymmetric FR of a line pair is a subregion of its symmetric FR. And the complement region of asymmetric FR in symmetric FR is called complement FR. To choose a suitable FR for the matching of each line pair adaptively, the asymmetric FR-based feature descriptor similarity and epipolar line-based constraints are combined to determine candidate line pair matches firstly. Then, the candidate matches with similar complement FR-based feature descriptors are considered initial matches and used to construct a topological constraint to check other candidates. Finally, we estimate an adaptive influence region-based local homography for each matched line pair to constrain the matching of unmatched individual LSs. The experimental results on the test images with viewpoint changes show that our approach outperforms the state-of-the-art methods in terms of Recall, matching precision (MP), and F-Measure (reflecting the overall matching performance of Recall and MP). Especially, it improves the average Recall and average F-Measure of the best method among the comparison methods by 45.20% and 25.83%, respectively.

GraffMatch: Global Matching of 3D Lines and Planes for Wide Baseline LiDAR Registration

Using geometric landmarks like lines and planes can increase navigation accuracy and decrease map storage requirements compared to commonly-used LiDAR point cloud maps. However, landmark-based registration for applications like loop closure detection is challenging because a reliable initial guess is not available. Global landmark matching has been investigated in the literature, but these methods typically use ad hoc representations of 3D line and plane landmarks that are not invariant to large viewpoint changes, resulting in incorrect matches and high registration error. To address this issue, we adopt the affine Grassmannian manifold to represent 3D lines and planes and prove that the distance between two landmarks is invariant to rotation and translation if a shift operation is performed before applying the Grassmannian metric. This invariance property enables the use of our graph-based data association framework for identifying landmark matches that can subsequently be used for registration in the least-squares sense. Evaluated on a challenging landmark matching and registration task using publicly-available LiDAR datasets, our approach yields a 1.7x and 3.5x improvement in successful registrations compared to methods that use viewpoint-dependent centroid and "closest point" representations, respectively.

Energy Savings in Buildings Based on Image Depth Sensors for Human Activity Recognition

A smart city is a city that binds together technology, society, and government to enable the existence of a smart economy, smart mobility, smart environment, smart living, smart people, and smart governance in order to reduce the environmental impact of cities and improve life quality. The first step to achieve a fully connected smart city is to start with smaller modules such as smart homes and smart buildings with energy management systems. Buildings are responsible for a third of the total energy consumption; moreover, heating, ventilation, and air conditioning (HVAC) systems account for more than half of the residential energy consumption in the United States. Even though connected thermostats are widely available, they are not used as intended since most people do not have the expertise to control this device to reduce energy consumption. It is commonly set according to their thermal comfort needs; therefore, unnecessary energy consumption is often caused by wasteful behaviors and the estimated energy saving is not reached. Most studies in the thermal comfort domain to date have relied on simple activity diaries to estimate metabolic rate and fixed values of clothing parameters for strategies to set the connected thermostat’s setpoints because of the difficulty in tracking those variables. Therefore, this paper proposes a strategy to save energy by dynamically changing the setpoint of a connected thermostat by human activity recognition based on computer vision preserving the occupant’s thermal comfort. With the use of a depth sensor in conjunction with an RGB (Red–Green–Blue) camera, a methodology is proposed to eliminate the most common challenges in computer vision: background clutter, partial occlusion, changes in scale, viewpoint, lighting, and appearance on human detection. Moreover, a Recurrent Neural Network (RNN) is implemented for human activity recognition (HAR) because of its data’s sequential characteristics, in combination with physiological parameters identification to estimate a dynamic metabolic rate. Finally, a strategy for dynamic setpoints based on the metabolic rate, predicted mean vote (PMV) parameter and the air temperature is simulated using EnergyPlus™ to evaluate the energy consumption in comparison with the expected energy consumption with fixed value setpoints. This work contributes with a strategy to reduce energy consumption up to 15% in buildings with connected thermostats from the successful implementation of the proposed method.

I3N: Intra- and Inter-Representation Interaction Network for Change Captioning

Change captioning aims to describe the disagreement of image pairs with a linguistic sentence. Compared with single image captioning, change captioning requires not only understanding the fine-grained information of each image, but also determining whether change occurs and further representing the differences of image pairs. Although much progress has been made, it remains a severe challenge of the precise difference representation in the distraction of viewpoint change, especially that of tiny difference. In this paper, we propose a novel Intra- and Inter-representation Interaction Network (I3N) to learn the fine difference representation and be immune to viewpoint change. In the Intra-representation Interaction stage, we design Geometry-Semantic Interaction Refining (GSIR) to explore the positional and semantic interactions of intra-image, which can be a prior knowledge of enduring viewpoint change and reinforce the cognition of semantic change. In the Inter-representation Interaction stage, to endow the model with the capability of pinpointing the latent difference in viewpoint change, Hierarchical Representation Interaction (HRI) models difference from coarse to fine representations through the Semantic Matcher and Change Amplifier module. The proposed approach outperforms the state-of-the-art methods with an encouraging performance on the existing change captioning benchmarks. Our code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/yueshengbin/I3N</uri> .

Viewpoint-Adaptive Representation Disentanglement Network for Change Captioning.

Change captioning is to describe the fine-grained change between a pair of images. The pseudo changes caused by viewpoint changes are the most typical distractors in this task, because they lead to the feature perturbation and shift for the same objects and thus overwhelm the real change representation. In this paper, we propose a viewpoint-adaptive representation disentanglement network to distinguish real and pseudo changes, and explicitly capture the features of change to generate accurate captions. Concretely, a position-embedded representation learning is devised to facilitate the model in adapting to viewpoint changes via mining the intrinsic properties of two image representations and modeling their position information. To learn a reliable change representation for decoding into a natural language sentence, an unchanged representation disentanglement is designed to identify and disentangle the unchanged features between the two position-embedded representations. Extensive experiments show that the proposed method achieves the state-of-the-art performance on the four public datasets. The code is available at https://github.com/tuyunbin/VARD.

An Image–Text Dual-Channel Union Network for Person Re-Identification

Multiple people can have similar appearances and portions of images can be occluded or have viewpoint changes in real scenarios, causing the increased difficulty of person re-identification (Re-ID). To address these problems, we propose a dual-channel person Re-ID algorithm that integrates person Re-ID image-text pairs into the classification network for end-to-end learning. We construct an image channel and a text channel, and subsequently extract visual information and text information using a convolutional neural network and a simple recurrent units network, respectively. The text information is used to assist in the learning of visual information, consequently improving the robustness of the visual information. In addition, the visual features are divided into two branches to calculate the global and local features. Global features focus on the overall appearance of a person, whereas local features provide more fine-grained detail. Text information is more accurate and reliable, and it is thus more robust to occlusion and viewpoint changes. Visual information complemented by text information can describe a person more accurately and reliably. Extensive experiments demonstrate our method achieves state-of-the-art performance.

Ambiguity-Aware Multi-Object Pose Optimization for Visually-Assisted Robot Manipulation

6D object pose estimation aims to infer the relative pose between the object and the camera using a single image or multiple images. Most works have focused on predicting the object pose without associated uncertainty under occlusion and structural ambiguity (symmetricity). However, these works demand prior information about shape attributes, and this condition is hardly satisfied in reality; even asymmetric objects may be symmetric under the viewpoint change. In addition, acquiring and fusing diverse sensor data is challenging when extending them to robotics applications. Tackling these limitations, we present an ambiguity-aware 6D object pose estimation network, PrimA6D++, as a generic uncertainty prediction method. The major challenges in pose estimation, such as occlusion and symmetry, can be handled in a generic manner based on the measured ambiguity of the prediction. Specifically, we devise a network to reconstruct the three rotation axis primitive images of a target object and predict the underlying uncertainty along each primitive axis. Leveraging the estimated uncertainty, we then optimize multi-object poses using visual measurements and camera poses by treating it as an object SLAM problem. The proposed method shows a significant performance improvement in T-LESS and YCB-Video datasets. We further demonstrate real-time scene recognition capability for visually-assisted robot manipulation.

Condition-Adaptive Graph Convolution Learning for Skeleton-Based Gait Recognition.

Graph convolutional networks have been widely applied in skeleton-based gait recognition. A key challenge in this task is to distinguish the individual walking styles of different subjects across various views. Existing state-of-the-art methods employ uniform convolutions to extract features from diverse sequences and ignore the effects of viewpoint changes. To overcome these limitations, we propose a condition-adaptive graph (CAG) convolution network that can dynamically adapt to the specific attributes of each skeleton sequence and the corresponding view angle. In contrast to using fixed weights for all joints and sequences, we introduce a joint-specific filter learning (JSFL) module in the CAG method, which produces sequence-adaptive filters at the joint level. The adaptive filters capture fine-grained patterns that are unique to each joint, enabling the extraction of diverse spatial-temporal information about body parts. Additionally, we design a view-adaptive topology learning (VATL) module that generates adaptive graph topologies. These graph topologies are used to correlate the joints adaptively according to the specific view conditions. Thus, CAG can simultaneously adjust to various walking styles and viewpoints. Experiments on the two most widely used datasets (i.e., CASIA-B and OU-MVLP) show that CAG surpasses all previous skeleton-based methods. Moreover, the recognition performance can be enhanced by simply combining CAG with appearance-based methods, demonstrating the ability of CAG to provide useful complementary information.

From Keypoints to Object Landmarks via Self-Training Correspondence: A novel approach to Unsupervised Landmark Discovery.

This paper proposes a novel paradigm for the unsupervised learning of object landmark detectors. Contrary to existing methods that build on auxiliary tasks such as image generation or equivariance, we propose a self-training approach where, departing from generic keypoints, a landmark detector and descriptor is trained to improve itself, tuning the keypoints into distinctive landmarks. To this end, we propose an iterative algorithm that alternates between producing new pseudo-labels through feature clustering and learning distinctive features for each pseudo-class through contrastive learning. With a shared backbone for the landmark detector and descriptor, the keypoint locations progressively converge to stable landmarks, filtering those less stable. Compared to previous works, our approach can learn points that are more flexible in terms of capturing large viewpoint changes. We validate our method on a variety of difficult datasets, including LS3D, BBCPose, Human3.6M and PennAction, achieving new state of the art results. Code and models can be found at https://github.com/malldimi1/KeypointsToLandmarks.

Neighborhood Contrastive Transformer for Change Captioning

Change captioning is to describe the semantic change between a pair of similar images in natural language. It is more challenging than general image captioning, because it requires capturing fine-grained change information while being immune to irrelevant viewpoint changes, and solving syntax ambiguity in change descriptions. In this paper, we propose a neighborhood contrastive transformer to improve the model's perceiving ability for various changes under different scenes and cognition ability for complex syntax structure. Concretely, we first design a neighboring feature aggregating to integrate neighboring context into each feature, which helps quickly locate the inconspicuous changes under the guidance of conspicuous referents. Then, we devise a common feature distilling to compare two images at neighborhood level and extract common properties from each image, so as to learn effective contrastive information between them. Finally, we introduce the explicit dependencies between words to calibrate the transformer decoder, which helps better understand complex syntax structure during training. Extensive experimental results demonstrate that the proposed method achieves the state-of-the-art performance on three public datasets with different change scenarios. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/tuyunbin/NCT</uri> .

Learning Spatial Regularization Correlation Filters With the Hilbert-Schmidt Independence Criterion in RKHS for UAV Tracking

Unmanned aerial vehicle (UAV) object tracking have been widely employed in military and civilian tasks and is very challenging due to factors such as aerial view, low resolution, background clutter, and small scale. Currently, many correlation filter (CF) methods for UAV object tracking introduce predefined regularization terms to improve the discriminative power. However, the predefined spatial constraints neither exploit the changing foreground and background information during tracking nor adapt to complex video attributes, such as background clutter, illumination variation, and viewpoint change. In this work, a novel CF tracking model is proposed to learn the spatial regularization correlation filters with the Hilbert-Schmidt independence criterion (HSIC_SRCF) in the reproducing kernel Hilbert space (RKHS) for UAV tracking. First, HSIC_SRCF learns the high confidence spatial weight coefficient for the filters, which encourages the filters to focus on the more reliable region. Second, the alternative direction method of multipliers (ADMM) can be used to optimize the proposed model, and the subproblems of the model have closed-form solutions. Third, the model update scheme uses RKHS and the peak-to-sidelobe ratio to obtain the confidence of the object bounding box and reduce the effect of background noise for the UAV object. The ablation study shows that the proposed approach achieves a precision of 0.699 and a success rate of 0.607, which are 2.8% higher and 2.2% higher, respectively, than the baseline AutoTrack on the UAV123@10fps. Experimental results on four popular UAV datasets, including VisDrone2018-test-dev, UAV123@10fps, UAV123, and UAVDT, show that our approach performs favorably compared with current state-of-the-art methods.

Geo-Localization via Ground-to-Satellite Cross-View Image Retrieval

The large variation of viewpoint and irrelevant content around the target always hinder accurate image retrieval and its subsequent tasks. In this paper, we investigate an extremely challenging task: given a ground-view image of a landmark, we aim to achieve cross-view geo-localization by searching out its corresponding satellite-view images. Specifically, the challenge comes from the gap between ground-view and satellite-view, which includes not only large viewpoint changes (some parts of the landmark may be invisible from front view to top view) but also highly irrelevant background (the target landmark tend to be hidden in other surrounding buildings), making it difficult to learn a common representation or a suitable mapping. To address this issue, we take advantage of drone-view information as a bridge between ground-view and satellite-view domains. We propose a Peer Learning and Cross Diffusion (PLCD) framework. PLCD consists of three parts: 1) a peer learning across ground-view and drone-view to find visible parts to benefit ground-drone cross-view representation learning; 2) a patch-based network for satellite-drone cross-view representation learning; 3) a cross diffusion between ground-drone space and satellite-drone space. Extensive experiments conducted on the University-Earth and University-Google datasets show that our method outperforms state-of-the-arts significantly.

Implicit Ray Transformers for Multiview Remote Sensing Image Segmentation

The mainstream CNN-based remote sensing (RS) image semantic segmentation approaches typically rely on massively labeled training data. Such a paradigm struggles with the problem of RS multi-view scene segmentation with limited labeled views due to the lack of consideration of 3D information within the scene. In this paper, we propose “Implicit Ray-Transformer (IRT)” based on Implicit Neural Representation (INR) for RS scene semantic segmentation with sparse labels (5% of the images being labeled). We explore a new way of introducing the multi-view 3D structure priors to the task for accurate and view-consistent semantic segmentation. The proposed method includes a two-stage learning process. In the first stage, we optimize a neural field to encode the color and 3D structure of the remote sensing scene based on multi-view images. In the second stage, we design a Ray Transformer to leverage the relations between the neural field 3D features and 2D texture features for learning better semantic representations. Different from previous methods that only consider 3D priors or 2D features, we incorporate additional 2D texture information and 3D priors by broadcasting CNN features to different point features along the sampled ray. To verify the effectiveness of the proposed method, we construct a challenging dataset containing six synthetic sub-datasets collected from the Carla platform and three real sub-datasets from Google Maps. Experiments show that the proposed method outperforms the CNN-based methods and the state-of-the-art INR-based segmentation methods in quantitative and qualitative metrics. The ablation study shows that under a limited number of fully annotated images, the combination of the 3D structure priors and 2D texture can significantly improve the performance and effectively complete missing semantic information in novel views. Experiments also demonstrate the proposed method could yield geometry-consistent segmentation results against illumination changes and viewpoint changes. Our data and code will be public.

Study on the effects of changes in VR environment and viewpoint on the physical behavior of vehicle occupants

Study on the effects of changes in VR environment and viewpoint on the physical behavior of vehicle occupants

Unsupervised Domain Adaptation for Person Re-Identification Via Individual-Preserving and Environmental-Switching Cyclic Generation

Unsupervised domain adaptation for person re-identification (Re-ID) suffers severe domain discrepancies between source and target domains. To reduce the domain shift caused by the changes of context, camera style, or viewpoint, existing methods in this field fine-tune and adapt the Re-ID model with augmented samples, either translating source samples to the target style or assigning pseudo labels to the target. The former methods may lose identity details but keep redundant source background during translation. In contrast, the latter techniques may give noisy labels when the model meets the unseen background and person pose. We mitigate the domain shift in the former translation direction by cyclically decoupling environment and identity-related features. We propose a novel individual-preserving and environmental-switching cyclic generation network (IPES-GAN). Our network has the following distinct features: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Decoupled features instead of fused features:</i> we encode the images into an individual part and an environmental part, which are proved beneficial to generation and adaptation; 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Cyclic generation instead of one-step adaptive generation</i> . We swap source and target environment features to generate cross-domain images with preserved identity-related features conditioned with source (target) background features and then changed again to generate back the input image so that cyclic generation runs in a self-supervised way. Experiments carried out on two significant benchmarks: Market-1501 and DukeMTMC-Reid, reveal state-of-the-art performance.

Nexus between the Ethical Leadership and Green Innovation in the Hospitality Industry of Pakistan

Despite the apparent use for management, the personal values and ethical leadership that employees demonstrate towards innovation and commitment have not been utilized sufficiently to meet corporate environmental goals. An increasing number of firms are using this unified approach to their own green initiatives. Nonetheless, achieving this integration requires a considerable change in viewpoint on the part of the hospitality industry towards ethical issues, as well as a transformation in the unsustainable behaviors that employees of all levels may have developed over decades. So, this research aims to determine the relationship between employee green commitment, green innovation, and ethical leadership in the hospitality and tourism industry. The data was collected from 460 respondents through convenience sampling, a non-probability sampling technique, and analyzed by the SPSS process Macro. The result showed a positive relationship between ethical leadership, green innovation, and green commitment, whereas the mediation of green commitment enhances the effect of ethical leadership and green innovation. The study provides substantial impact for company leaders and managers on how to leverage green innovation to improve environmental performance and gain a competitive edge in their industries. Because stakeholders are increasingly expecting and demanding that businesses "go green" in everything they do, we feel that investing in environmental management is a prudent approach for organizations seeking to improve their consumer reputations.

The Effects of a Triggering Instruction on Communication Strategy Transference

Research on communication strategies (CSs) has been drawing much attention in the area of psycholinguistics and applied linguistics. This study examined if triggering instruction can accelerate the cognitive aspects of CSs transference from L1 to L2. In the first part of the experiment, all participants, twenty-one Japanese university students, looked at pictures and were asked to say anything that came to their mind in English. In the second part, half of them (an experimental group) were instructed to begin the task by saying likes and dislikes of the picture. The other half (a control group) did not receive such an instruction. Participants also answered a questionnaire asking about self-evaluation of the task performance in order to examine their affective and cognitive state. Their utterances were classified as either objective descriptions of what was shown in the picture or subjective expressions of what they imagined or felt. The results of the experimental group clearly demonstrated the effect of instructional intervention; subjective expression increased and objective description decreased, and also their anxiety decreased from the first part of the experiment to the second. These findings suggest that more attention should be paid to the cognitive aspects of CSs such as change of viewpoint from objective to subjective, which we may call ‘self-expression switching,’ and its transference from L1.

VW-SC3D: A Sparse 3D CNN-Based Spatial–Temporal Network with View Weighting for Skeleton-Based Action Recognition

In recent years, human action recognition has received increasing attention as a significant function of human–machine interaction. The human skeleton is one of the most effective representations of human actions because it is highly compact and informative. Many recent skeleton-based action recognition methods are based on graph convolutional networks (GCNs) as they preserve the topology of the human skeleton while extracting features. Although many of these methods give impressive results, there are some limitations in robustness, interoperability, and scalability. Furthermore, most of these methods ignore the underlying information of view direction and rely on the model to learn how to adjust the view from training data. In this work, we propose VW-SC3D, a spatial–temporal model with view weighting for skeleton-based action recognition. In brief, our model uses a sparse 3D CNN to extract spatial features for each frame and uses a transformer encoder to obtain temporal information within the frames. Compared to GCN-based methods, our method performs better in extracting spatial–temporal features and is more adaptive to different types of 3D skeleton data. The sparse 3D CNN makes our model more computationally efficient and more flexible. In addition, a learnable view weighting module enhances the robustness of the proposed model against viewpoint changes. A test on two different types of datasets shows a competitive result with SOTA methods, and the performance is even better in view-changing situations.