Cluttered Indoor Environments Research Articles

Indoor human action recognition based on millimeter-wave radar micro-doppler signature

Considering that millimeter-wave radar lacks sufficient data to support Transfer Learning (TL) and Human Action Recognition (HAR), we propose a Heterogeneous Multi-source Transfer Learning (HMTL) method and a data selection algorithm based on Categorical Probability to obtain Hash Coding (CPHC). CPHC is utilized to select data from multi-source datasets in the same domain for downscaling and matching to ensure the similarity between the selected data features and the target task features to obtain better performance on the target task. The experimental results show that the CPHC can downscale data more efficiently than the traditional algorithm. HMTL can effectively improve the classification accuracy of the network to the state-of-the-art (SOAT) level. RPME addresses that the radar Micro-Doppler Signature (MDS) is flooded by noise due to the cluttered indoor environment, making the MDS more obvious and thus improving the classification accuracy.

UTIL: An ultra-wideband time-difference-of-arrival indoor localization dataset

Ultra-wideband (UWB) time-difference-of-arrival (TDOA)-based localization has emerged as a promising, low-cost, and scalable indoor localization solution, which is especially suited for multi-robot applications. However, there is a lack of public datasets to study and benchmark UWB TDOA positioning technology in cluttered indoor environments. We fill in this gap by presenting a comprehensive dataset using Decawave’s DWM1000 UWB modules. To characterize the UWB TDOA measurement performance under various line-of-sight (LOS) and non-line-of-sight (NLOS) conditions, we collected signal-to-noise ratio (SNR), power difference values, and raw UWB TDOA measurements during the identification experiments. We also conducted a cumulative total of around 150 min of real-world flight experiments on a customized quadrotor platform to benchmark the UWB TDOA localization performance for mobile robots. The quadrotor was commanded to fly with an average speed of 0.45 m/s in both obstacle-free and cluttered environments using four different UWB anchor constellations. Raw sensor data including UWB TDOA, inertial measurement unit (IMU), optical flow, time-of-flight (ToF) laser altitude, and millimeter-accurate ground truth robot poses were collected during the flights. The dataset and development kit are available at https://utiasdsl.github.io/util-uwb-dataset/ .

Framework for Autonomous UAV Navigation and Target Detection in Global-Navigation-Satellite-System-Denied and Visually Degraded Environments

Autonomous Unmanned Aerial Vehicles (UAVs) have possible applications in wildlife monitoring, disaster monitoring, and emergency Search and Rescue (SAR). Autonomous capabilities such as waypoint flight modes and obstacle avoidance, as well as their ability to survey large areas, make UAVs the prime choice for these critical applications. However, autonomous UAVs usually rely on the Global Navigation Satellite System (GNSS) for navigation and normal visibility conditions to obtain observations and data on their surrounding environment. These two parameters are often lacking due to the challenging conditions in which these critical applications can take place, limiting the range of utilisation of autonomous UAVs. This paper presents a framework enabling a UAV to autonomously navigate and detect targets in GNSS-denied and visually degraded environments. The navigation and target detection problem is formulated as an autonomous Sequential Decision Problem (SDP) with uncertainty caused by the lack of the GNSS and low visibility. The SDP is modelled as a Partially Observable Markov Decision Process (POMDP) and tested using the Adaptive Belief Tree (ABT) algorithm. The framework is tested in simulations and real life using a navigation task based on a classic SAR operation in a cluttered indoor environment with different visibility conditions. The framework is composed of a small UAV with a weight of 5 kg, a thermal camera used for target detection, and an onboard computer running all the computationally intensive tasks. The results of this study show the robustness of the proposed framework to autonomously explore and detect targets using thermal imagery under different visibility conditions. Devising UAVs that are capable of navigating in challenging environments with degraded visibility can encourage authorities and public institutions to consider the use of autonomous remote platforms to locate stranded people in disaster scenarios.

Kernel-based diffusion approximated Markov decision processes for autonomous navigation and control on unstructured terrains

We propose a diffusion approximation method to the continuous-state Markov decision processes that can be utilized to address autonomous navigation and control in unstructured off-road environments. In contrast to most decision-theoretic planning frameworks that assume fully known state transition models, we design a method that eliminates such a strong assumption that is often extremely difficult to engineer in reality. We first take the second-order Taylor expansion of the value function. The Bellman optimality equation is then approximated by a partial differential equation, which only relies on the first and second moments of the transition model. By combining the kernel representation of the value function, we design an efficient policy iteration algorithm whose policy evaluation step can be represented as a linear system of equations characterized by a finite set of supporting states. We first validate the proposed method through extensive simulations in 2D obstacle avoidance and 2.5D terrain navigation problems. The results show that the proposed approach leads to a much superior performance over several baselines. We then develop a system that integrates our decision-making framework with onboard perception and conduct real-world experiments in both cluttered indoor and unstructured outdoor environments. The results from the physical systems further demonstrate the applicability of our method in challenging real-world environments.

Local trajectory planning for mobile robot in cluttered environment based on Model Predictive Control

The task of local trajectory planning for an autonomous wheeled robotic platform in cluttered indoor environment is considered. Such environment might include narrow passages, which width is less than the length of the platform. Therefore, it is not possible to apply standard approach, when the obstacles are inflated with the maximum radius of the platform. We propose a novel approach based on numerical solution of nonlinear model predictive control task. Oblong shape of the platform is approximated with a high-order ellipse. We define differentiable sigmoid po-tential function, which may be computed for any point of the workspace given position and orientation of the plat-form. This function is small far from the platform, and very high inside the platform; it increases when moving to-wards the robot. The value of this potential function is computed for the set of the support obstacle points and add-ed to the cost function. This function serve as a penalty for collision with obstacles or coming too close to them. We develop an algorithm for the mapping support points onto occupancy grid, which provide collision avoidance. We apply Acados open library, which implement numerical solution of nonlinear model predictive task with sequential quadratic programming. Our approach is implemented as a local planner for the collaborative mobile platform. The experiments were made in artificial maze and in real office environment with narrow passages. Proposed ap-proached allowed the robot to come through the passages that were 10-20 cm wider than the platform. Computa-tion time was around 20 milliseconds.

Real-Time Localization for Closed-Loop Control of Assistive Furniture

For people with limited mobility, navigating in cluttered indoor environment is challenging. In this work, we propose a mobile assistive furniture suite that is designed to ease the life of people with special needs in indoor movement. To enable intelligent coordination of this system, a key component is the localization of each mobile furniture. The challenge is to assess the state of an arbitrary living scenario so that the estimation can be used as a real-time feedback signal for autonomous closed-loop control of mobile furniture. We propose a perception pipeline that addresses these challenges. A machine learning model is designed and trained to jointly achieve multi-object semantic keypoint detection and classification in camera images. The synthetic data generation is employed to augment the training set and boost the model performance. A robust point cloud registration uses the detected semantic keypoints and depth information to estimate poses of the furniture. Tracking is applied to achieve smooth estimation. A high-performance accelerator that optimizes the efficiency of using heterogeneous devices is applied to achieve real-time performance. This visual perception pipeline is used in closed-loop control to steer the mobile furniture from initial to a desired location demonstrated in experiments on real hardware.

Optimal Beacon Deployment for Positioning in Cluttered Indoor Environments

Accurate indoor positioning is vital for many indoor autonomous systems. This article focuses on the influence of beacon placement on overall positioning quality in cluttered indoor environments. We first propose a metric based on the Fisher information matrix (FIM) to evaluate the performance of the positioning service provided by a specific beacon layout. After analyzing the characteristics of the FIM, we introduce an additional penalty term in the performance metric to handle singular cases. We then investigate the optimal beacon placement with respect to overall positioning quality. We derive the gradient of the optimization objective and propose an efficient optimization method called the quotient-space gradient-based particle swarm optimization (QGPSO), which combines the quotient-space PSO with the gradient ascent method, enabling the particle swarm to obtain more accurate optimization results locally while retaining the original global exploration capability. Numerical results validate our analysis and the superiority of the proposed optimization method over other baselines.

Multiagent Autonomous Source Search Using Submodularity and Branch-and-Bound

This paper is concerned with near-optimal source search problem using a multiagent system in cluttered indoor environments. The goal of the problem is to maximize the detection probability within the minimum search time. We propose a two-stage strategy to achieve this goal. In the first stage, a greedy approach is used to define a set of grid cells with the aim of maximizing the detection probability. In the second stage, an iterative branch-and-bound procedure is used to design the search paths of all agents so that all grid cells are visited by one agent and the largest search path among all agents is minimized. Simulation results show that the proposed search algorithm has better performance in terms of exploration time compared to other existing methods.

Design and Motion Planning for a Reconfigurable Robotic Base

A robotic platform for mobile manipulation needs to satisfy two contradicting requirements for many real-world applications: A compact base is required to navigate through cluttered indoor environments, while the support needs to be large enough to prevent tumbling or tip over, especially during fast manipulation operations with heavy payloads or forceful interaction with the environment. This paper proposes a novel robot design that fulfills both requirements through a versatile footprint. It can reconfigure its footprint to a narrow configuration when navigating through tight spaces and to a wide stance when manipulating heavy objects. Furthermore, its triangular configuration allows for high-precision tasks on uneven ground by preventing support switches. A model predictive control strategy is presented that unifies planning and control for simultaneous navigation, reconfiguration, and manipulation. It converts task-space goals into whole-body motion plans for the new robot. The proposed design has been tested extensively with a hardware prototype. The footprint reconfiguration allows to almost completely remove manipulation-induced vibrations. The control strategy proves effective in both lab experiment and during a real-world construction task.

FAST-LIO2: Fast Direct LiDAR-Inertial Odometry

This article presents FAST-LIO2: a fast, robust, and versatile LiDAR-inertial odometry framework. Building on a highly efficient tightly coupled iterated Kalman filter, FAST-LIO2 has two key novelties that allow fast, robust, and accurate LiDAR navigation (and mapping). The first one is directly registering raw points to the map (and subsequently update the map, i.e., mapping) without extracting features. This enables the exploitation of subtle features in the environment and, hence, increases the accuracy. The elimination of a hand-engineered feature extraction module also makes it naturally adaptable to emerging LiDARs of different scanning patterns; the second main novelty is maintaining a map by an incremental k-dimensional (k-d) tree data structure, incremental k-d tree ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ikd-Tree</i> ), that enables incremental updates (i.e., point insertion and delete) and dynamic rebalancing. Compared with existing dynamic data structures (octree, R <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\ast$</tex-math></inline-formula> -tree, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nanoflann</i> k-d tree), <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ikd-Tree</i> achieves superior overall performance while naturally supports downsampling on the tree. We conduct an exhaustive benchmark comparison in 19 sequences from a variety of open LiDAR datasets. FAST-LIO2 achieves consistently higher accuracy at a much lower computation load than other state-of-the-art LiDAR-inertial navigation systems. Various real-world experiments on solid-state LiDARs with small field of view are also conducted. Overall, FAST-LIO2 is computationally efficient (e.g., up to 100 Hz odometry and mapping in large outdoor environments), robust (e.g., reliable pose estimation in cluttered indoor environments with rotation up to 1000 deg/s), versatile (i.e., applicable to both multiline spinning and solid-state LiDARs, unmanned aerial vehicle (UAV) and handheld platforms, and Intel- and ARM-based processors), while still achieving a higher accuracy than existing methods. Our implementation of the system FAST-LIO2 and the data structure <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ikd-Tree</i> are both open-sourced on Github.

A Modular Vision Language Navigation and Manipulation Framework for Long Horizon Compositional Tasks in Indoor Environment.

In this paper we propose a new framework—MoViLan (Modular Vision and Language) for execution of visually grounded natural language instructions for day to day indoor household tasks. While several data-driven, end-to-end learning frameworks have been proposed for targeted navigation tasks based on the vision and language modalities, performance on recent benchmark data sets revealed the gap in developing comprehensive techniques for long horizon, compositional tasks (involving manipulation and navigation) with diverse object categories, realistic instructions and visual scenarios with non reversible state changes. We propose a modular approach to deal with the combined navigation and object interaction problem without the need for strictly aligned vision and language training data (e.g., in the form of expert demonstrated trajectories). Such an approach is a significant departure from the traditional end-to-end techniques in this space and allows for a more tractable training process with separate vision and language data sets. Specifically, we propose a novel geometry-aware mapping technique for cluttered indoor environments, and a language understanding model generalized for household instruction following. We demonstrate a significant increase in success rates for long horizon, compositional tasks over recent works on the recently released benchmark data set -ALFRED.

Finding the Right Place: Sensor Placement for UWB Time Difference of Arrival Localization in Cluttered Indoor Environments

Ultra-wideband (UWB) time difference of arrival (TDOA)-based localization has recently emerged as a promising indoor positioning solution. However, in cluttered environments, both the UWB radio positions and the obstacle-induced non-line-of-sight (NLOS) measurement biases significantly impact the quality of the position estimate. Consequently, the placement of the UWB radios must be carefully designed to provide satisfactory localization accuracy for a region of interest. In this work, we propose a novel algorithm that optimizes the UWB radio positions for a pre-defined region of interest in the presence of obstacles. The mean-squared error (MSE) metric is used to formulate an optimization problem that balances the influence of the geometry of the radio positions and the NLOS effects. We further apply the proposed algorithm to compute a minimal number of UWB radios required for a desired localization accuracy and their corresponding positions. In a real-world cluttered environment, we show that the designed UWB radio placements provide 47% and 76% localization root-mean-squared error (RMSE) reduction in 2D and 3D experiments, respectively, when compared against trivial placements.

Robust Structure Identification and Room Segmentation of Cluttered Indoor Environments From Occupancy Grid Maps

Identifying the environment’s structure, through detecting core components such as rooms and walls, can facilitate several tasks fundamental for the successful operation of indoor autonomous mobile robots, including semantic environment understanding. These robots often rely on 2D occupancy maps for core tasks such as localisation and motion and task planning. However, reliable identification of structure and room segmentation from 2D occupancy maps is still an open problem due to clutter (e.g., furniture and movable objects), occlusions, and partial coverage. We propose a method for the RObust StructurE identification and ROom SEgmentation (ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> ) of 2D occupancy maps that may be cluttered and incomplete. ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> identifies the main directions of walls and is resilient to clutter and partial observations, allowing to extract a clean, abstract geometrical floor-plan-like description of the environment, which is used to segment, i.e., to identify rooms in, the original occupancy grid map. ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> is tested in several real-world publicly available cluttered maps obtained in different conditions. The results show that it can robustly identify the environment structure in 2D occupancy maps suffering from clutter and partial observations, while significantly improving room segmentation accuracy. Thanks to the combination of clutter removal and robust room segmentation, ROSE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> consistently achieves higher performance than the state-of-the-art methods, against which it is compared.

Multi-agent robotic system (MARS) for UAV-UGV path planning and automatic sensory data collection in cluttered environments

There has been growing interest in increasing the application of robotic and automation technologies for building indoor inspection. However, much previous research on indoor robotic applications was limited to a single type of unmanned aerial/ground vehicle (UAV/UGV), each of which has certain limitations and constraints. Besides, the robotic systems suffer from inefficient control within cluttered indoor environments containing many obstacles. This paper presents a multi-agent robotic system (MARS) for automatic UAV-UGV path planning and indoor navigation to automate sensory data collection. The proposed MARS consists of a new system architecture that defines the attributes and data requirements for UAV and UGV indoor path planning. To improve indoor navigation in cluttered environments, an enhanced shunting short-term memory model is established to optimize the pathfinding of UAV/UGV for data collection. Assessment of indoor navigation is conducted with a simulation-based approach and LiDAR SLAM. A mediating agent, which harnesses a control algorithm and information exchange mechanism, is proposed to interoperate UAV and UGV for automated data collection. The proposed new MARS is examined in experiments, in which a single UAV, dual UAVs, and combined UAV-UGV are tested in a research laboratory. The result indicates that the MARS can support automated path planning and indoor navigation for 2D image and 3D point cloud data collection.

MICRO AND MACRO QUADCOPTER DRONES FOR INDOOR MAPPING TO SUPPORT DISASTER MANAGEMENT

Abstract. The use of drones to explore indoor spaces has gained attention and popularity for disaster management and indoor navigation applications. In this paper we present the operations and mapping techniques of two drones that are different in terms of size, the sensors deployed, and the positioning and mapping techniques used. The first drone is a low-cost commercial quadcopter microdrone, a Crazyflie, while the second drone is a relatively expensive research quadcopter macrodrone, called MAX. We investigated their feasibility in mapping areas where satellite positioning is not available, such as indoor spaces. We compared the point clouds obtained by a multi-ranger deck, a multi-layer LIDAR scanner and a stereo camera, and assessed each against ground truth obtained with a terrestrial laser scanner. Results showed that both drones are capable of mapping relatively cluttered indoor environments and can provide point clouds that are sufficient for a quick exploration. Furthermore, the LIDAR scanner-based system can handle a relatively large office environment with an accumulated drift less than 0.02% (1 cm) on the Z-axis and 0.77% (50 cm) on the X and Y axes over a length trajectory of about 65 m. Despite the limited features of the sensor configuration of the Crazyflie, its performance is promising for mapping indoor spaces, given the relatively low deviation from the ground truth: cloud-to-cloud distances measured were generally less than 20 cm.

IOT-5G USER TRACKING IN A 5G NETWORK USING 60 GHZ MM-WAVES BASED ON AN ABF-ED ALGORITHMS FOR A CLUTTERED INDOOR ENVIRONMENT

Abstract. This paper presents a user tracking algorithm in an IoT-5G Network (or IoT-5GN). Hereby, we aim at studying and evaluating the sensing performances of the IoT-5G Access Point (or IoT-5G AP) primary signal by the IoT-5G user in a cluttered indoor environment using an energy detector (or ED) algorithm and an Alpha-&amp;amp;-Beta Filter (ABF or α-β-F) estimator. The 5G primary signal (or 5G-PS) frequency that we would like to detect is: 60 GHz. As a result, the 5G-PS sensing via the proposed ABF-ED algorithm, enabled us to track the IoT-5G user inside of the IoT-5G AP coverage area. The performances of the proposed ABF-ED algorithm in this paper work is evaluated by the probability of total detection error (or PTDE) measure. Through different scenarios simulations, the performances and robustness of the proffered algorithm are demonstrated.

Comparative Analysis of Deep Neural Networks for the Detection and Decoding of Data Matrix Landmarks in Cluttered Indoor Environments

Data Matrix patterns imprinted as passive visual landmarks have shown to be a valid solution for the self-localization of Automated Guided Vehicles (AGVs) in shop floors. However, existing Data Matrix decoding applications take a long time to detect and segment the markers in the input image. Therefore, this paper proposes a pipeline where the detector is based on a real-time Deep Learning network and the decoder is a conventional method, i.e. the implementation in libdmtx. To do so, several types of Deep Neural Networks (DNNs) for object detection were studied, trained, compared, and assessed. The architectures range from region proposals (Faster R-CNN) to single-shot methods (SSD and YOLO). This study focused on performance and processing time to select the best Deep Learning (DL) model to carry out the detection of the visual markers. Additionally, a specific data set was created to evaluate those networks. This test set includes demanding situations, such as high illumination gradients in the same scene and Data Matrix markers positioned in skewed planes. The proposed approach outperformed the best known and most used Data Matrix decoder available in libraries like libdmtx.

Robust Approach to Supervised Deep Neural Network Training for Real-Time Object Classification in Cluttered Indoor Environment

Unmanned aerial vehicles (UAVs) are being widely utilized for various missions: in both civilian and military sectors. Many of these missions demand UAVs to acquire artificial intelligence about the environments they are navigating in. This perception can be realized by training a computing machine to classify objects in the environment. One of the well known machine training approaches is supervised deep learning, which enables a machine to classify objects. However, supervised deep learning comes with huge sacrifice in terms of time and computational resources. Collecting big input data, pre-training processes, such as labeling training data, and the need for a high performance computer for training are some of the challenges that supervised deep learning poses. To address these setbacks, this study proposes mission specific input data augmentation techniques and the design of light-weight deep neural network architecture that is capable of real-time object classification. Semi-direct visual odometry (SVO) data of augmented images are used to train the network for object classification. Ten classes of 10,000 different images in each class were used as input data where 80% were for training the network and the remaining 20% were used for network validation. For the optimization of the designed deep neural network, a sequential gradient descent algorithm was implemented. This algorithm has the advantage of handling redundancy in the data more efficiently than other algorithms.

Embodied Visual Active Learning for Semantic Segmentation

We study the task of embodied visual active learning, where an agent is set to explore a 3d environment with the goal to acquire visual scene understanding by actively selecting views for which to request annotation. While accurate on some benchmarks, today's deep visual recognition pipelines tend to not generalize well in certain real-world scenarios, or for unusual viewpoints. Robotic perception, in turn, requires the capability to refine the recognition capabilities for the conditions where the mobile system operates, including cluttered indoor environments or poor illumination. This motivates the proposed task, where an agent is placed in a novel environment with the objective of improving its visual recognition capability. To study embodied visual active learning, we develop a battery of agents - both learnt and pre-specified - and with different levels of knowledge of the environment. The agents are equipped with a semantic segmentation network and seek to acquire informative views, move and explore in order to propagate annotations in the neighbourhood of those views, then refine the underlying segmentation network by online retraining. The trainable method uses deep reinforcement learning with a reward function that balances two competing objectives: task performance, represented as visual recognition accuracy, which requires exploring the environment, and the necessary amount of annotated data requested during active exploration. We extensively evaluate the proposed models using the photorealistic Matterport3D simulator and show that a fully learnt method outperforms comparable pre-specified counterparts, even when requesting fewer annotations.

Embodied Visual Navigation With Automatic Curriculum Learning in Real Environments

We present NavACL, a method of automatic curriculum learning tailored to the navigation task. NavACL is simple to train and efficiently selects relevant tasks using geometric features. In our experiments, deep reinforcement learning agents trained using NavACL significantly outperform state-of-the-art agents trained with uniform sampling - the current standard. Furthermore, our agents can navigate through unknown cluttered indoor environments to semantically-specified targets using only RGB images. Obstacle-avoiding policies and frozen feature networks support transfer to unseen real-world environments, without any modification or retraining requirements. We evaluate our policies in simulation, and in the real world on a ground robot and a quadrotor drone. Videos of real-world results are available in the supplementary material.