Changes In Viewing Angle Research Articles

Digital image correlation calculation method for RGB-D camera multi-view matching using variable template

Multi-view matching is an indispensable method for digital image correlation 3D reconstruction. However, as the camera viewing angle changes, traditional DIC methods can lead to the accumulation of matching errors and an increase in mismatches, thereby affecting the 3D reconstruction quality. This paper proposes a digital image correlation method for multi-view RGB-D images based on variable template matching. Initial positioning is achieved by dynamically adjusting the size and sampling direction of the variable template. Sub-pixel registration is then performed using the inverse compositional algorithm, followed by multi-view point cloud fusion using the Iterative Closest Point (ICP) registration method. Experimental results show that the iteration speed of point cloud registration is increased by 30%, and the cumulative error is reduced by 62.5%. These improvements demonstrate that the proposed method is suitable for multi-viewpoint matching scenarios, achieving excellent results in multi-view feature point matching and multi-view point cloud fusion, significantly enhancing 3D reconstruction accuracy and efficiency.

Enhancing light extraction efficiency and color stability for OLED by truncated micro-cone array films

A newly truncated micro-cone array film as light extraction layer for organic light emitting diodes (OLEDs) has been developed, which was fabricated by process combined with photolithography, polydimethylsiloxane (PDMS) molding, self-assembled anti-adhesion, and ultraviolet (UV) forming techniques. In this study, by varying the sidewall angle and fill factor of this film, their influences on the optical properties and light extraction efficiency for OLED are fully investigated. As a result, device performance of OLED employed truncated micro-cone array film, with a high fill factor and sidewall angle of 76.49⁰, was found to be significantly enhanced up to 52% compared with the reference OLED. Notably, the proposed truncated micro-cone array film also stabilized emissive color stability while the viewing angle changes from 0° to 70°, achieving an excellent angular shift with CIE-indices of (0.0025, 0.0066). Aforementioned findings contribute to the development in advanced OLEDs for display and lighting applications.

A Fast Matching Method for the SAR Images with Large Viewing Angles Based on Inertial Navigation Information and Neighborhood Structure Consensus

In the field of multi-view SAR target location, the greater the difference in viewing angles, the higher the target location accuracy. However, this makes it difficult to match the same target between the SAR images with different viewing angles. The performance of traditional SAR image-matching algorithms will deteriorate or even fail to match the images correctly when the viewing angle is gradually increased. To solve this problem, a fast SAR matching method for the SAR images with large viewing angles based on inertial navigation information and neighborhood structure consensus (ININSC) is proposed in this paper. In this algorithm, the key targets are detected in the SAR images by using the maximum connected domain algorithm and the K-means clustering algorithm, and the connected domain centroid of the target is taken as a feature point. The approximate position of the key targets after the viewing angle change is found through inertial navigation information, and then accurate and fast matching is achieved by using the consensus of the neighborhood topological structure of the key targets. The measured data sets demonstrate that compared with traditional SAR image-matching algorithms, the proposed ININSC algorithm solves such a problem that SAR images cannot be accurately matched under the differences at large viewing angles and has better robustness and timeliness.

Illuminance Analysis according to View Angle Changes of Large-area Flat-type LED Lighting System to Improve Optical Characteristics

Illuminance Analysis according to View Angle Changes of Large-area Flat-type LED Lighting System to Improve Optical Characteristics

Simulating satellite urban land surface temperatures: sensitivity to sensor view angle and assumed landscape complexity

Urban land surface temperature (LST) from satellite earth observation (EO) varies with sensor view angle. Where these variations are not accounted for, urban LST products are inconsistent through time, limiting their use in urban weather and climate model evaluations and process studies (e.g. urban heat island, building energy balance, human thermal comfort). Obstacle-resolving numerical models (ORM) of urban form and radiation exchanges are being used to: (a) understand relations between EO view angle, the 3-dimensional urban surface, and the surface temperatures from urban land surface models, and (b) evaluate parameterisations (parametric models) that aim to account for LST angular effects for the next operational satellite products. Most ORM are limited to simplified buildings (e.g. cuboids) and surface temperatures by lack of datasets. Novelly, we use both a realistic urban form model and observed surface temperatures to assess the impact of simplifying the urban form and temperature on the modelled LST anisotropy. We test various sets of assumptions in central London by combining ground-based thermal camera observations and the state-of-the-art Discrete Anisotropic Radiative Transfer (DART) model. The high resolution realistic model (< 1 m) includes thermal (surface temperatures varying by sun-surface geometry, shadow history and materials every 30 min) and geometry (sloped roofs, chimneys and vegetation) data. These data are used to simulate brightness temperatures of EO pixels to quantify LST view angle variations.During daytime, a change of view angle of 47° off-nadir corresponds to a difference in LST of up to 5.1 K for the realistic building model. The intermediate-complexity landscape (easily obtainable building geometry/footprints and more idealised surface temperature distributions driven only by shadow patterns) gives the best agreement in simulated LST to the realistic landscape. The directional variations are still captured in total (daytime mean absolute error 0.44 K) when using an idealised ORM representation of the same landscape (cuboid buildings, simplified surface temperature) except for roofs which are near-isotropic. Results suggest that geometry assumptions used in current ORM are suited for evaluation of parametric models used to develop and verify operational LST sensor view angle corrections. Future work should consider more realistic materials and scattering processes including low emissivity glass and metals with challenging specular properties.

Visual search as an embodied process: The effects of perspective change and external reference on search performance.

Traditional visual search tasks in the laboratories typically involve looking for targets in 2D displays with exemplar views of objects. In real life, visual search commonly entails 3D objects in 3D spaces with nonperpendicular viewing and relative motions between observers and search array items, both of which lead to transformations of objects’ projected images in lawful but unpredicted ways. Furthermore, observers often do not have to memorize a target before searching, but may refer to it while searching, for example, holding a picture of someone while looking for them from a crowd. Extending the traditional visual search task, in this study, we investigated the effects of image transformation as a result of perspective change yielded by discrete viewing angle change (Experiment 1) or continuous rotation of the search array (Experiment 2) and of having external references on visual search performance. Results showed that when searching from 3D objects with a non-zero viewing angle, performance was similar to searching from 2D exemplar views of objects; when searching for 3D targets from rotating arrays in virtual reality, performance was similar to searching from stationary arrays. In general, discrete or continuous perspective change did not affect the search outcomes in terms of accuracy, response time, and self-rated confidence, or the search process in terms of eye movement patterns. Therefore, visual search does not require the exact match of retinal images. Additionally, being able to see the target during the search improved search accuracy and observers’ confidence. It increased search time because, as revealed by the eye movements, observers actively checked back on the reference target. Thus, visual search is an embodied process that involves real-time information exchange between the observers and the environment.

A Cross-View Gait Recognition Method Using Two-Way Similarity Learning

Gait recognition is a powerful tool for long-distance identification. However, gaits are influenced by walking environments and appearance changes. Therefore, the gait recognition rate declines sharply when the viewing angle changes. In this work, we propose a novel cross-view gait recognition method with two-way similarity learning. Focusing on the relationships between gait elements in three-dimensional space and the wholeness of human body movements, we design a three-dimensional gait constraint model that is robust to view changes based on joint motion constraint relationships. Different from the classic three-dimensional model, the proposed model characterizes motion constraints and action constraints between joints based on time and space dimensions. Next, we propose an end-to-end two-way gait network using long short-term memory and residual network 50 to extract the temporal and spatial difference features, respectively, of model pairs. The two types of difference features are merged at a high level in the network, and similarity values are obtained through the softmax layer. Our method is evaluated based on the challenging CASIA-B data set in terms of cross-view gait recognition. The experimental results show that the method achieves a higher recognition rate than the previously developed model-based methods. The recognition rate reaches 72.8%, and the viewing angle changes from 36° to 144° for normal walking. Finally, the new method also performs better in cases with large cross-view angles, illustrating that our model is robust to viewing angle changes and that the proposed network offers considerable potential in practical application scenarios.

Novel-view X-ray projection synthesis through geometry-integrated deep learning.

X-ray imaging is a widely used approach to view the internal structure of a subject for clinical diagnosis, image-guided interventions and decision-making. The X-ray projections acquired at different view angles provide complementary information of patient's anatomy and are required for stereoscopic or volumetric imaging of the subject. In reality, obtaining multiple-view projections inevitably increases radiation dose and complicates clinical workflow. Here we investigate a strategy of obtaining the X-ray projection image at a novel view angle from a given projection image at a specific view angle to alleviate the need for actual projection measurement. Specifically, a Deep Learning-based Geometry-Integrated Projection Synthesis (DL-GIPS) framework is proposed for the generation of novel-view X-ray projections. The proposed deep learning model extracts geometry and texture features from a source-view projection, and then conducts geometry transformationon the geometry features to accommodate the change of view angle.At the final stage, the X-ray projection in the target view is synthesized from the transformed geometry and the shared texture features via an image generator. The feasibility and potential impact of the proposed DL-GIPS model are demonstrated using lung imaging cases.The proposed strategy can be generalized to a general case of multiple projections synthesis from multiple input views and potentially provides a new paradigm for various stereoscopic and volumetric imaging with substantially reduced efforts in data acquisition.

The devil is in the face: Exploiting harmonious representations for facial expression recognition

Despite the recent effort from computer vision community, facial expression recognition (FER) remains a largely unsolved problem. This is because the appearance of people’s face undergoes dramatic changes due to changes in view angle, pose, illumination plus ambiguous facial expressions and low-quality facial images. In this work, we show the advantage of feature representation learning by dynamically graph message propagating subject to FER discriminative learning constraints and minimizing the distance of expression-agnostic transformed instance feature pairs. Specifically, we formulate a novel Harmonious Representation Learning (HRL) model for joint learning of landmark-guided graph message propagation, and spatially invariant feature learning using only generic matching metrics. Extensive comparative evaluations demonstrate the superiority of our proposed approach for FER over a variety of state-of-the-art methods on three major benchmark datasets including SFEW 2.0, RAF-DB, and CK+.

Processing of Airborne Video SAR Data Using the Modified Back Projection Algorithm

This article proposes an algorithm framework recently developed for video synthetic aperture radar (SAR), and presents the processing results of the airborne data collected by a W-band radar. This framework aims to accelerate time domain algorithm and produce SAR video that has no change of view angle, which is highly desired in target tracking. The presented framework includes a few modified processing algorithms, where the strategy of subaperture recursion is proposed. The modified acceleration algorithm is used to avoid the interpolation and to adapt to high squint angle without rotation of image grids. Additionally, the conventional autofocus method has been improved to deal with urban data that usually contain strong scatterers. The algorithm framework can achieve SAR video without rotation of view angle with a lower computational load compared to other existing time domain methods, which has been verified on airborne real data. Its processing efficiency becomes impressive if the apertures are highly overlapped in some cases that requires very high frame rate.

Novel-View X-Ray Projection Synthesis Through Geometry-Integrated Deep Learning

X-ray imaging is a widely used approach to view the internal structure of a subject for clinical diagnosis, image-guided interventions and decision-making. The X-ray projections acquired at different view angles provide complementary information of patient's anatomy and are required for stereoscopic or volumetric imaging of the subject. In reality, obtaining multiple-view projections inevitably increases radiation dose and complicates clinical workflow. Here we investigate a strategy of obtaining the X-ray projection image at a novel view angle from a given projection image at a specific view angle to alleviate the need for actual projection measurement. Specifically, we propose a deep learning-based geometry-integrated projection synthesis framework for generating novel-view X-ray projections.We for the first time investigate the novel-view projection synthesis problem for X-ray imaging. We propose a deep learning-based geometry-integrated projection synthesis model to generate novel-view X-ray projections through feature disentanglement and geometry transformation. The proposed deep learning model extracts geometry and texture features from a source-view projection, and then conducts geometry transformation on the geometry features to accommodate the change of view angle. Then the X-ray projection in target view is synthesized from the transformed geometry and shared texture features via an image generator. We validate the proposed approach by experimenting on one-to-one and multi-to-multi X-ray projection synthesis using lung imaging cases across various patients. We conduct experiments on a public dataset containing 1018 lung CT images, where 80% and 20% data are used for training and testing. The projection images are digitally produced from CT images using geometry consistent with a clinical on-board cone-beam CT system for radiation therapy.We deploy the trained model on the held-out testing set for novel-view projection synthesis. The generated results are compared with ground truth qualitatively and quantitatively. For AP-to-lateral and lateral-to-AP view synthesis, average NRMSE / SSIM / PSNR values over all testing data are 0.185 / 0.880 / 22.646 and 0.173 / 0.911 / 23.908, respectively. Promising results are also obtained for multi-to-multi view synthesis. By visualizing the generated projections, we observe the proposed model can generate images closely to the targets despite the various anatomic structures of different patients, indicating the potential of the proposed model for X-ray projection synthesis.This work investigates the synthesis of novel-view X-ray projections and presents a robust model combining the deep learning and geometry transformation. It is shown that the generated X-ray projections reveal the internal anatomy distribution from new viewpoints, which potentially provides a new paradigm for volumetric imaging with substantially reduced efforts in data acquisition.

A Climatological View of Horizontal Ice Plates in Clouds: Findings From Nadir and Off‐Nadir CALIPSO Observations

AbstractWe study horizontal ice plates in clouds using satellite lidar measurements of the Cloud‐Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO). This study investigates global microphysical and geographical properties of horizontal ice plates to obtain insights into ice‐plate climatology based on lidar’s long‐term measurements during 2006–2014. We then summarize the effects of the lidar's viewing angle change from 0.3° to 3.0° in 2007 on satellite particle phase/shape classifications. Using a classification algorithm developed in the previous studies, we show that the ice plate detection decreases by 81.7% due the tilting. With an updated version of this algorithm, 30.8% of these are recovered, although 50.8% remain undetected. Nevertheless, this study also shows that geographical characteristics of ice plates are still preserved during the off‐nadir period (within the remaining 50%), suggesting the undiscovered climatological information on ice plates in the post‐2007 observations. According to our analysis, the tilting mainly affects horizontal ice plate detection, while the impacts on water and randomly oriented ice detections are limited. The temperature of the ice plates ranges from −25.5°C and −7.5°C, with a mode temperature of −13.5°C, although the ice plates also occur ubiquitously across mid‐ to high‐latitudes between −20°C and −40°C, which is much colder than what found in previous nadir studies. This study offers a detailed discussion on the fundamental characteristics of horizontal ice plates that will provide robust information for the algorithm preparation for future satellite lidar observations such as the Earth, Clouds, Aerosol and Radiation Explorer (EarthCARE).

Things in the air: tagging wearable IoT information on drone videos

Drones have been applied to a wide range of security and surveillance applications recently. With drones, Internet of Things are extending to 3D space. An interesting question is: Can we conduct person identification (PID) in a drone view? Traditional PID technologies such as RFID and fingerprint/iris/face recognition have their limitations or require close contact to specific devices. Hence, these traditional technologies can not be easily deployed to drones due to dynamic change of view angle and height. In this work, we demonstrate how to retrieve IoT data from users’ wearables and correctly tag them on the human objects captured by a drone camera to identify and track ground human objects. First, we retrieve human objects from videos and conduct coordination transformation to handle the change of drone positions. Second, a fusion algorithm is applied to measure the correlation of video data and inertial data based on the extracted human motion features. Finally, we can couple human objects with their wearable IoT devices, achieving our goal of tagging wearable device data (such as personal profiles) on human objects in a drone view. Our experimental evaluation shows a recognition rate of 99.5% for varying walking paths, and 98.6% when the drone’s camera angle is within 37°. To the best of our knowledge, this is the first work integrating videos from drone cameras and IoT data from inertial sensors.

21.3: Research of visually sensitive colors as viewing angle change on TFT‐LCD displays

21.3: Research of visually sensitive colors as viewing angle change on TFT‐LCD displays

A Novel Human Action Recognition and Behaviour Analysis Technique using SWFHOG

In this paper, a new local feature, called, Salient Wavelet Feature with Histogram of Oriented Gradients (SWFHOG) is introduced for human action recognition and behaviour analysis. In the proposed approach, regions having maximum information are selected based on their entropies. The SWF feature descriptor is formed by using the wavelet sub-bands obtained by applying wavelet decomposition to selected regions. To improve the accuracy further, the SWF feature vector is combined with the Histogram of Oriented Gradient global feature descriptor to form the SWFHOG feature descriptor. The proposed algorithm is evaluated using publicly available KTH, Weizmann, UT Interaction, and UCF Sports datasets for action recognition. The highest accuracy of 98.33% is achieved for the UT interaction dataset. The proposed SWFHOG feature descriptor is tested for behaviour analysis to identify the actions as normal or abnormal. The actions from SBU Kinect and UT Interaction dataset are divided into two sets as Normal Behaviour and Abnormal Behaviour. For the application of behaviour analysis, 95% recognition accuracy is achieved for the SBU Kinect dataset and 97% accuracy is obtained for the UT Interaction dataset. Robustness of the proposed SWFHOG algorithm is tested against Camera view angle change and imperfect actions using Weizmann robustness testing datasets. The proposed SWFHOG method shows promising results as compared to earlier methods.

Recurrent matching networks of spatial alignment learning for person re-identification

Person re-identification (re-id) usually refers to matching people across disjoint camera views. Many existing methods focus on extracting discriminative features or learning distance metrics to make the intraclass distance smaller than interclass distances. These methods subconsciously assume that pedestrian images are well aligned. However, one major challenge in person re-id is the unconstrained spatial misalignment between image pairs due to view angle changes and pedestrian pose variations. To address this problem, in this paper, we propose Recurrent Matching Network of Spatial Alignment Learning (RMN-SAL) to simulate the human vision perception. Reinforcement learning is introduced to locate attention regions, since it provides a flexible learning strategy for sequential decision-making. A linear mapping is employed to convert the environment state into spatial constraint, comprising spatial alignment into feature learning. And recurrent models are used to extract information from a sequence of corresponding regions. Finally, person re-id is performed based on the global features and the features from the learned alignment regions. Our contributions are: 1) the recurrent matching network, which can subtly combine local feature learning and sequential spatial correspondence learning into an end-to-end framework; 2) the design of a location network, which is based on reinforcement learning and aims to learn task-specific sequential spatial correspondences for different image pairs through the local pairwise internal representation interactions. The proposed model is evaluated on three benchmarks, including Market-1501, DukeMTMC-reID and CUHK03, and achieves better performances than other methods.

Dynamic patterns based on the superposition of micropattern array and microlens array

Dynamic patterns have been studied theoretically and experimentally in this work. The microstructure consists of a base layer made up of micropattern array, and a superposed revealing layer made up of microlens array. Dynamic patterns will be observed directly on the revealing layer without visual facilities. When the viewing angle changes, displacement of dynamic patterns is produced. This technology is compared with traditional moiré techniques. In addition, analytical models to describe its structures are derived based on a vector method. There is a good agreement between the theoretical and experimental results, which indicates this study is helpful to design various kinds of dynamic patterns and dynamic effects, and this technology can be applied for many applications, especially in anti-counterfeiting field.

Robust Feature-Based Automated Multi-View Human Action Recognition System

Automated human action recognition has the potential to play an important role in public security, for example, in relation to the multiview surveillance videos taken in public places, such as train stations or airports. This paper compares three practical, reliable, and generic systems for multiview video-based human action recognition, namely, the nearest neighbor classifier, Gaussian mixture model classifier, and the nearest mean classifier. To describe the different actions performed in different views, view-invariant features are proposed to address multiview action recognition. These features are obtained by extracting the holistic features from different temporal scales which are modeled as points of interest which represent the global spatial-temporal distribution. Experiments and cross-data testing are conducted on the KTH, WEIZMANN, and MuHAVi datasets. The system does not need to be retrained when scenarios are changed which means the trained database can be applied in a wide variety of environments, such as view angle or background changes. The experiment results show that the proposed approach outperforms the existing methods on the KTH and WEIZMANN datasets.

Robust salient motion detection in non-stationary videos via novel integrated strategies of spatio-temporal coherency clues and low-rank analysis

Salient motion detection is vital for security surveillance, pattern and motion recognition, traffic control, human–computer interaction, etc. Although such a subject has been very well investigated for analysis of stationary videos, many technical challenges still prevail when correctly handling and analyzing non-stationary videos recorded by hand-hold and pan-tilt-zoom cameras. To ameliorate, this paper develops a novel and robust salient motion detection method (especially valuable for quantitative analysis of non-stationary videos) by employing new computational strategies, including low-rank analysis aided by the divide-and-conquer approach, and exploration of the space–time semantic coherency. The key idea in our new approach is to respectively conduct multi-purpose low-rank analysis over a temporal series of well-decomposed frame-batches that have relatively-consistent backgrounds. First, we conduct bilateral random projection (BRP)-based low-rank analysis to accurately keep track of short-term stable-background observations, which consist of frames with similar global appearance and small local variations. Then, to eliminate the side effects due to visual variations induced by view angle changes, we incorporate the low-rank background prior into previous short-term observation to guide robust principal component analysis (RPCA) low-rank revealing based robust salient motion detection over current short-term observation. Meanwhile, a series of saliency clues extracted from the stabilized short-term observations are leveraged to expedite the proper updating of the low-rank background information, which enables us to effectively combat several obstinate problems. Finally, we conduct comprehensive experiments on the public CD2014 benchmark and other five non-stationary videos recorded from the hand-hold camera, and make extensive and quantitative evaluations with six state-of-the-art methods. Experimental results indicate that our method not only outperforms all other methods in the case of non-stationary videos but also obtains outstanding performance for stationary videos.

Quantification model of proper curvature for large‐sized curved TVs

AbstractWe present a feasible modeling method to estimate the proper curvature for large‐sized curved TVs. We especially focused on two factors – preference and perceived distortion – to be considered to determine the proper curvature. The preference includes a number of advantages that people expect to fulfill the possibilities of a more immersive image and sense of realism from curved display. On the other hand, the perceived distortion means the side effects that people can notice uncomfortable feeling caused by the shape of a curved one. In order to find out how two factors would be different as a number of conditions like curvature, size, and viewing angle change, a series of subjective assessments were conducted. The evaluation results show that both the preference and perceived distortion vary with the conditions considerably. We performed the statistical analysis based on the results and proposed the quantification model of proper curvature, which has higher preference and less perceived distortion, for various‐sized curved TVs.