Typical Indoor Scenes Research Articles

基于超宽带冲激雷达的人体动作识别和心率估计数据集

Human action recognition and heart rate estimation for indoor human targets carry significant applications in many fields such as smart homes, health care, sports analysis, etc., and have attracted increasing attention in recent years. Impulse-Radio Ultra-Wideband (IR-UWB) radar, with its advantages in privacy protection, environmental requirements and performance, has found extensive applications in indoor human target monitoring. In view of the data demand for indoor human perception related research based on IR-UWB radar, and the issues with a lack of relevant public data and inadequacy in the authenticity of simulation data, this paper presents a dataset of human action recognition and heart rate estimation based on IR-UWB radar through typical indoor scene measurement experiments and data processing and annotation. The dataset collects 1,629 groups of action recognition data (i.e. unmanned scene data and 10 common actions such as walking, waving, squatting, etc.) from 10 human targets as well as 4,670 groups of heart rate estimation data. All data are presented in a standardized format with precise labeling, making it a valuable resource for research in relevant fields.

Sound Source Localization Indoors Based on Two-Level Reference Points Matching

A dense sample point layout is the conventional approach to ensure the positioning accuracy for fingerprint-based sound source localization (SSL) indoors. However, mass reference point (RPs) matching of online phases may greatly reduce positioning efficiency. In response to this compelling problem, a two-level matching strategy is adopted to shrink the adjacent RPs searching scope. In the first-level matching process, two different methods are adopted to shrink the search scope of the online phase in a simple scene and a complex scene. According to the global range of high similarity between adjacent samples in a simple scene, a greedy search method is adopted for fast searching of the sub-database that contains the adjacent RPs. Simultaneously, in accordance with the specific local areas’ range of high similarity between adjacent samples in a complex scene, the clustering method is used for database partitioning, and the RPs search scope can be compressed by sub-database matching. Experimental results show that the two-level RPs matching strategy can effectively improve the RPs matching efficiency for the two different typical indoor scenes on the premise of ensuring the positioning accuracy.

LIDAR-BASED INITIAL GLOBAL LOCALIZATION USING IMPERFECT ARCHITECTURAL SKELETON INFORMATION

Abstract. Initial global localization of a mobile robotic platform is the foundation for its navigation and mapping, especially when the platform enters into unknown environments. In GNSS-denied indoor scenes, LiDAR is widely used for robot localization, especially in indoor scenes with poor lighting. In most existing LiDAR-based initial global localization methods, it is necessary to build the point cloud reference map in advance, which costs a large quantity of manpower and time. For this reason, a LiDAR-based initial global localization method using imperfect architectural skeleton information is proposed in this work. Firstly, we propose a lightweight management scheme for collected imperfect architectural information, which is convenient for efficient registration with real scans. Secondly, we extract architectural skeletons (stable man-made structures such as walls and columns) from both architectural information and real scans, and design them as line pairs feature patterns like P-LP, V-LP and C-LP. Thirdly, we propose a matrix descriptor for line pairs feature patterns description and initial matching. Finally, we construct error equations to estimate the pose by initial matching line pairs, and acquire the optimal localization results with the highest hit ratio on architectural grid map. A mobile robotic platform with the 16 beam LiDAR is experimented in typical indoor scenes such as rooms, corridors and undergrounding parking lots. Experiments show that the success rate of initial global localization reaches 80%, the average position error is about 0.10m and the running time is about 400ms per 1000 scans, which meet the requirements of indoor autonomous driving.

A Low-Cost and Portable Indoor 3D Mapping Approach Using Biaxial Line Laser Scanners and a One-Dimension Laser Range Finder Integrated with Microelectromechanical Systems

Existing indoor 3D mapping solutions suffer from high cost and poor portability. In this article, a low-cost and portable indoor 3D mapping approach using biaxial line laser scanners and a one-dimension laser range finder integrated with microelectromechanical systems is proposed. A multiple-sensor calibration approach is presented to perform the extrinsic calibration of the integrated 3D mapping system. The 2D point cloud acquired by the horizontal laser scanner and the orientation information obtained by the microelectromechanical systems are used as inputs for a simultaneous localization and mapping framework to estimate the 2D poses. The height information acquired by the laser range finder is then fused to obtain the 3D pose, which is applied to restore the actual position and orientation of the 2D point cloud generated by the tilted laser scanner to reconstruct the 3D point cloud of the indoor environment. The experimental results—three typical indoor scenes—demonstrate that the proposed approach can achieve accuracies of 3 cm and 2°. Therefore, the proposed approach is a low-cost, portable, and accurate solution for indoor 3D mapping.

Dominant path determining method for millimeter wave propagation in indoor scenarios based on ant colony algorithm

A fast ray tracing method based on ant colony algorithm is proposed to analyze the indoor propagation characteristics of millimeter-wave more efficiently. This method uses ant colony distribution calculation and heuristic search principle to find the dominant ray path of radio wave propagation, which replaces the total ray path analysis to reduce the algorithm complexity and improve execution efficiency. Compared with the measured data results, the result of the simulation analysis of three frequency channel characteristics of 28 GHz, 32 GHz and 38 GHz in typical indoor scene shows better consistency, and the prediction error is less than 0.8 dB. The computational efficiency is improved by 378.9% compared with the traditional ray tracing method. The above-mentioned results show that the proposed method can effectively reduce the computational complexity under the premise of ensuring the prediction accuracy, and has the value of engineering application.

A COMPARATIVE STUDY OF POINT CLOUDS SEMANTIC SEGMENTATION USING THREE DIFFERENT NEURAL NETWORKS ON THE RAILWAY STATION DATASET

Abstract. Point cloud data have rich semantic representations and can benefit various applications towards a digital twin. However, they are unordered and anisotropically distributed, thus being unsuitable for a typical Convolutional Neural Networks (CNN) to handle. With the advance of deep learning, several neural networks claim to have solved the point cloud semantic segmentation problem. This paper evaluates three different neural networks for semantic segmentation of point clouds, namely PointNet++, PointCNN and DGCNN. A public indoor scene of the Amersfoort railway station is used as the study area. Unlike the typical indoor scenes and even more from the ubiquitous outdoor ones in currently available datasets, the station consists of objects such as the entrance gates, ticket machines, couches, and garbage cans. For the experiment, we use subsets from the data, remove the noise, evaluate the performance of the selected neural networks. The results indicate an overall accuracy of more than 90% for all the networks but vary in terms of mean class accuracy and mean Intersection over Union (IoU). The misclassification mainly occurs in the classes of couch and garbage can. Several factors that may contribute to the errors are analyzed, such as the quality of the data and the proportion of the number of points per class. The adaptability of the networks is also heavily dependent on the training location: the overall characteristics of the train station make a trained network for one location less suitable for another.

Improved Point Cloud Registration with Scale Invariant Feature Extracted

The efficiency and accuracy of point set registration is always a challenge to deal with. In this paper, we propose an improved algorithm using a scale invariant feature extracted. The larger transformation scale and rotation angle cause lower registration accuracy. The initial corresponding point set can be obtained using a scale invariant feature transform (SIFT) operator. In addition, the geometric features of the points are combined to remove the wrong points. The unit quaternion algorithm is used to estimate the best rigid body transformation matrix for precise registration. The experiments show that the registration accuracy increases by 7.99%, while the time consumption decreased by 6.24% in a typical indoor scene.

Precise 3-D Indoor Localization Based on Wi-Fi FTM and Built-In Sensors

More and more applications of location-based services lead to the development of indoor positioning technology. As a part of the Internet-of-Things ecosystem, most existing indoor positioning algorithms are applied to specific situations, e.g., pedestrian navigation and target detection. To meet the high-precision indoor localization requirement, IEEE 802.11 included the Wi-Fi fine-time measurement (FTM) protocol in 2016, which provides a novel approach for Wi-Fi ranging between the mobile terminal and Wi-Fi access point (AP). This article proposes a precise 3-D indoor localization algorithm based on Wi-Fi FTM and smartphone built-in sensors (3D-WFBS). The adaptive extended Kalman filter (AEKF) is used to estimate the pedestrian's real-time heading and walking speed, and the received signal strength indication and round-trip time collected from Wi-Fi APs are combined for proximity detection and providing more accurate ranging results. In addition, the unscented particle filter is applied to fuse the results of AEKF, proximity detection, and Wi-Fi ranging. The experimental results show that compared with the existing dead reckoning method and the other fusion methods, the proposed 3D-WFBS algorithm is proved to achieve meter-level indoor positioning accuracy in typical indoor scenes.

Integration and evaluation of SLAM-based backpack mobile mapping system

Mobile mapping is an efficient technology to acquire spatial data of the environment. As a supplement of vehicle-borne and air-borne methods, Backpack mobile mapping system (MMS) has a wide application prospect in indoor and underground space. High-precision positioning and attitude determination are the key to MMS. Usually, GNSS/INS integrated navigation system provides reliable pose information. However, in the GNSS-denied environments, there is no effective long-term positioning method. With the development of simultaneous localization and mapping (SLAM) algorithm, it provides a new solution for indoor mobile mapping. This paper develops a portable backpack mobile mapping system, which integrates multi-sensor such as LiDAR, IMU, GNSS and panoramic camera. The 3D laser SLAM algorithm is applied to the mobile mapping to realize the acquisition of geographic information data in various complex environments. The experimental results in typical indoor and outdoor scenes show that the system can achieve high-precision and efficient acquisition of 3D information, and the relative precision of point cloud is 2~4cm, which meets the requirements of scene mapping and reconstruction.

Scene-aware audio for 360° videos

Although 360° cameras ease the capture of panoramic footage, it remains challenging to add realistic 360° audio that blends into the captured scene and is synchronized with the camera motion. We present a method for adding scene-aware spatial audio to 360° videos in typical indoor scenes, using only a conventional mono-channel microphone and a speaker. We observe that the late reverberation of a room's impulse response is usually diffuse spatially and directionally. Exploiting this fact, we propose a method that synthesizes the directional impulse response between any source and listening locations by combining a synthesized early reverberation part and a measured late reverberation tail. The early reverberation is simulated using a geometric acoustic simulation and then enhanced using a frequency modulation method to capture room resonances. The late reverberation is extracted from a recorded impulse response, with a carefully chosen time duration that separates out the late reverberation from the early reverberation. In our validations, we show that our synthesized spatial audio matches closely with recordings using ambisonic microphones. Lastly, we demonstrate the strength of our method in several applications.

Drift-Aware Monocular Localization Based on a Pre-Constructed Dense 3D Map in Indoor Environments

Recently, monocular localization has attracted increased attention due to its application to indoor navigation and augmented reality. In this paper, a drift-aware monocular localization system that performs global and local localization is presented based on a pre-constructed dense three-dimensional (3D) map. In global localization, a pixel-distance weighted least squares algorithm is investigated for calculating the absolute scale for the epipolar constraint. To reduce the accumulative errors that are caused by the relative position estimation, a map interaction-based drift detection method is introduced in local localization, and the drift distance is computed by the proposed line model-based maximum likelihood estimation sample consensus (MLESAC) algorithm. The line model contains a fitted line segment and some visual feature points, which are used to seek inliers of the estimated feature points for drift detection. Taking advantage of the drift detection method, the monocular localization system switches between the global and local localization modes, which effectively keeps the position errors within an expected range. The performance of the proposed monocular localization system is evaluated on typical indoor scenes, and experimental results show that compared with the existing localization methods, the accuracy improvement rates of the absolute position estimation and the relative position estimation are at least 30.09% and 65.59%, respectively.

A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation.

Recent advances of 3D acquisition devices have enabled large-scale acquisition of 3D scene data. Such data, if completely and well annotated, can serve as useful ingredients for a wide spectrum of computer vision and graphics works such as data-driven modeling and scene understanding, object detection and recognition. However, annotating a vast amount of 3D scene data remains challenging due to the lack of an effective tool and/or the complexity of 3D scenes (e.g. clutter, varying illumination conditions). This paper aims to build a robust annotation tool that effectively and conveniently enables the segmentation and annotation of massive 3D data. Our tool works by coupling 2D and 3D information via an interactive framework, through which users can provide high-level semantic annotation for objects. We have experimented our tool and found that a typical indoor scene could be well segmented and annotated in less than 30 minutes by using the tool, as opposed to a few hours if done manually. Along with the tool, we created a dataset of over a hundred 3D scenes associated with complete annotations using our tool. Both the tool and dataset will be available at http://scenenn.net.

LOW COST MULTI-SENSOR ROBOT LASER SCANNING SYSTEM AND ITS ACCURACY INVESTIGATIONS FOR INDOOR MAPPING APPLICATION

Abstract. In order to solve the automation of 3D indoor mapping task, a low cost multi-sensor robot laser scanning system is proposed in this paper. The multiple-sensor robot laser scanning system includes a panorama camera, a laser scanner, and an inertial measurement unit and etc., which are calibrated and synchronized together to achieve simultaneously collection of 3D indoor data. Experiments are undertaken in a typical indoor scene and the data generated by the proposed system are compared with ground truth data collected by a TLS scanner showing an accuracy of 99.2% below 0.25 meter, which explains the applicability and precision of the system in indoor mapping applications.

Visual Map Construction Using RGB-D Sensors for Image-Based Localization in Indoor Environments

RGB-D sensors capture RGB images and depth images simultaneously, which makes it possible to acquire the depth information at pixel level. This paper focuses on the use of RGB-D sensors to construct a visual map which is an extended dense 3D map containing essential elements for image-based localization, such as poses of the database camera, visual features, and 3D structures of the building. Taking advantage of matched visual features and corresponding depth values, a novel local optimization algorithm is proposed to achieve point cloud registration and database camera pose estimation. Next, graph-based optimization is used to obtain the global consistency of the map. On the basis of the visual map, the image-based localization method is investigated, making use of the epipolar constraint. The performance of the visual map construction and the image-based localization are evaluated on typical indoor scenes. The simulation results show that the average position errors of the database camera and the query camera can be limited to within 0.2 meters and 0.9 meters, respectively.

EXTENSION AND EVALUATION OF THE AGAST FEATURE DETECTOR

Vision-aided inertial navigation is a navigation method which combines inertial navigation with computer vision techniques. It can provide a six degrees of freedom navigation solution from passive measurements without external referencing (e.g. GPS). Thus, it can operate in unknown environments without any prior knowledge. Such a system, called IPS (Integrated Positioning System) is developed by the German Aerospace Center (DLR). <br><br> For optical navigation applications, a reliable and efficient feature detector is a crucial component. With the publication of AGAST, a new feature detector has been presented, which is faster than other feature detectors. To apply AGAST to optical navigation applications, we propose several methods to improve its performance. Based on a new non-maximum suppression algorithm, automatic threshold adaption algorithm in combination with an image split method, the optimized AGAST provides higher reliability and efficiency than the original implementation using the Kanade Lucas Tomasi (KLT) feature detector. Finally, we compare the performance of the optimized AGAST with the KLT feature detector in the context of IPS. The presented approach is tested using real data from typical indoor scenes, evaluated on the accuracy of the navigation solution. The comparison demonstrates a significant performance improvement achieved by the optimized AGAST.

EXTENSION AND EVALUATION OF THE AGAST FEATURE DETECTOR

Abstract. Vision-aided inertial navigation is a navigation method which combines inertial navigation with computer vision techniques. It can provide a six degrees of freedom navigation solution from passive measurements without external referencing (e.g. GPS). Thus, it can operate in unknown environments without any prior knowledge. Such a system, called IPS (Integrated Positioning System) is developed by the German Aerospace Center (DLR). For optical navigation applications, a reliable and efficient feature detector is a crucial component. With the publication of AGAST, a new feature detector has been presented, which is faster than other feature detectors. To apply AGAST to optical navigation applications, we propose several methods to improve its performance. Based on a new non-maximum suppression algorithm, automatic threshold adaption algorithm in combination with an image split method, the optimized AGAST provides higher reliability and efficiency than the original implementation using the Kanade Lucas Tomasi (KLT) feature detector. Finally, we compare the performance of the optimized AGAST with the KLT feature detector in the context of IPS. The presented approach is tested using real data from typical indoor scenes, evaluated on the accuracy of the navigation solution. The comparison demonstrates a significant performance improvement achieved by the optimized AGAST.

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

We propose a real-time approach for indoor scene reconstruction. It is capable of producing a ready-to-use 3D geometric model even while the user is still scanning the environment with a consumer depth camera. Our approach features explicit representations of planar regions and nonplanar objects extracted from the noisy feed of the depth camera, via an online structure analysis on the dynamic, incomplete data. The structural information is incorporated into the volumetric representation of the scene, resulting in a seamless integration with KinectFusion's global data structure and an efficient implementation of the whole reconstruction process. Moreover, heuristics based on rectilinear shapes in typical indoor scenes effectively eliminate camera tracking drift and further improve reconstruction accuracy. The instantaneous feedback enabled by our on-the-fly structure analysis, including repeated object recognition, allows the user to selectively scan the scene and produce high-fidelity large-scale models efficiently. We demonstrate the capability of our system with real-life examples.

Automatic semantic modeling of indoor scenes from low-quality RGB-D data using contextual information

We present a novel solution to automatic semantic modeling of indoor scenes from a sparse set of low-quality RGB-D images. Such data presents challenges due to noise, low resolution, occlusion and missing depth information. We exploit the knowledge in a scene database containing 100s of indoor scenes with over 10,000 manually segmented and labeled mesh models of objects. In seconds, we output a visually plausible 3D scene, adapting these models and their parts to fit the input scans. Contextual relationships learned from the database are used to constrain reconstruction, ensuring semantic compatibility between both object models and parts. Small objects and objects with incomplete depth information which are difficult to recover reliably are processed with a two-stage approach. Major objects are recognized first, providing a known scene structure. 2D contour-based model retrieval is then used to recover smaller objects. Evaluations using our own data and two public datasets show that our approach can model typical real-world indoor scenes efficiently and robustly.

On Sensor Bias in Experimental Methods for Comparing Interest-Point, Saliency, and Recognition Algorithms

Most current algorithm evaluation protocols use large image databases, but give little consideration to imaging characteristics used to create the data sets. This paper evaluates the effects of camera shutter speed and voltage gain under simultaneous changes in illumination and demonstrates significant differences in the sensitivities of popular vision algorithms under variable illumination, shutter speed, and gain. These results show that offline data sets used to evaluate vision algorithms typically suffer from a significant sensor specific bias which can make many of the experimental methodologies used to evaluate vision algorithms unable to provide results that generalize in less controlled environments. We show that for typical indoor scenes, the different saturation levels of the color filters are easily reached, leading to the occurrence of localized saturation which is not exclusively based on the scene radiance but on the spectral density of individual colors present in the scene. Even under constant illumination, foreshortening effects due to surface orientation can affect feature detection and saliency. Finally, we demonstrate that active and purposive control of the shutter speed and gain can lead to significantly more reliable feature detection under varying illumination and nonconstant viewpoints.

Localization and noise in edge detection

Two aspects of edge detection are analyzed, namely accuracy of localization and sensitivity to noise. The detection of corners and trihedral vertices is analyzed for gradient schemes and zero-crossing schemes. It is shown that neither scheme correctly detects corners of trihedral vertices, but that the gradient schemes are less sensitive to noise. A simple but important conclusion is that the noise present in digital images of typical indoor scenes is small and the signal-to-noise ratio is high. The noise present in digital images is so small as to make the performances of a variety of filters almost indistinguishable. As a consequence small filters can be used and the exact shape of the filter is not critical. >