Real-time attitude commanding to detect coverage gaps and generate high resolution point clouds for RSO shape characterization with a laser rangefinder

This paper discusses the application of a single beam laser rangefinder (LRF) to point cloud generation, shape detection, and shape reconstruction for a space-based space situational awareness (SSA) mission. The LRF is part of the payload of a chaser satellite tasked to image a resident space object (RSO). The one-dimensional (1D) nature of LRF returns significantly increases the complexity of the imaging task. To maximize coverage, a method to autonomously detect and fill gaps in sparse point cloud coverage using a narrow field of view (NFOV) camera in conjunction with the LRF is presented. First, relative orbital motion and scanning attitude motion are combined to generate a baseline 3D point cloud of the RSO. The effectiveness of pregenerated command profiles is analyzed by using a weighted edge reconstruction metric that scores how well a point cloud characterizes RSO shape. The design and characterization of multiple relative motion and attitude maneuver profiles, as well as the time and propellant cost of each profile, are presented with the assumption that the entire metrology chain is error free. Next, a three-part algorithm is used that 1) creates a 3D panoramic map from stitched NFOV camera images, 2) correlates the areas of sparse LRF coverage to the map, and 3) generates attitude commands to close the coverage. This provides a consistent and reliable method to register positions of strike points relative to each other and to the NFOV image of the RSO with a priori knowledge of the RSO attitude. Gaps and sparse areas in LRF coverage are covered with strike points; the result is a point cloud of significantly higher resolution than that obtained with preprogrammed attitude profiles. The analysis bears particular relevance to power-constrained nanosatellite missions for space-based SSA for whom a multibeam LRF payload is not feasible. Maneuvers can now be designed on-line in real time; results presented validate the utility of a single-beam LRF as a tool for 3D imaging of RSO shapes.

This paper focuses on the aerospace application of a single beam laser rangefinder (LRF) for 3D imaging, shape detection, and reconstruction in the context of a space-based space situational awareness (SSA) mission scenario. The primary limitation to 3D imaging from LRF point clouds is the one-dimensional nature of the single beam measurements. A method that combines relative orbital motion and scanning attitude motion to generate point clouds has been developed and the design and characterization of multiple relative motion and attitude maneuver profiles are presented. The target resident space object (RSO) has the shape of a generic telecommunications satellite. The shape and attitude of the RSO are unknown to the chaser satellite however, it is assumed that the RSO is un-cooperative and has fixed inertial pointing. All sensors in the metrology chain are assumed ideal. A previous study by the authors used pure Keplerian motion to perform a similar 3D imaging mission at an asteroid. A new baseline for proximity operations maneuvers for LRF scanning, based on a waypoint adaptation of the Hill-Clohessy-Wiltshire (HCW) equations is examined. Propellant expenditure for each waypoint profile is discussed and combinations of relative motion and attitude maneuvers that minimize the propellant used to achieve a minimum required point cloud density are studied. Both LRF strike-point coverage and point cloud density are maximized; the capability for 3D shape registration and reconstruction from point clouds generated with a single beam LRF without catalog comparison is proven. Next, a method of using edge detection algorithms to process a point cloud into a 3D modeled image containing reconstructed shapes is presented. Weighted accuracy of edge reconstruction with respect to the true model is used to calculate a qualitative “metric” that evaluates effectiveness of coverage. Both edge recognition algorithms and the metric are independent of point cloud density, therefore they are utilized to compare the quality of point clouds generated by various attitude and waypoint command profiles. The RSO model incorporates diverse irregular protruding shapes, such as open sensor covers, instrument pods and solar arrays, to test the limits of the algorithms. This analysis is used to mathematically prove that point clouds generated by a single-beam LRF can achieve sufficient edge recognition accuracy for SSA applications, with meaningful shape information extractable even from sparse point clouds. For all command profiles, reconstruction of RSO shapes from the point clouds generated with the proposed method are compared to the truth model and conclusions are drawn regarding their fidelity.

Biometrics is a method of identifying individuals by their physiological or behavioral characteristics. Among other biometric identifiers, iris recognition has been widely used for various applications that require a high level of security. When a conventional iris recognition camera is used, the size and position of the iris region in a captured image vary according to the X , Y positions of a user’s eye and the Z distance between a user and the camera. Therefore, the searching area of the iris detection algorithm is increased, which can inevitably decrease both the detection speed and accuracy. To solve these problems, we propose a new method of iris localization that uses wide field of view (WFOV) and narrow field of view (NFOV) cameras. Our study is new as compared to previous studies in the following four ways. First, the device used in our research acquires three images, one each of the face and both irises, using one WFOV and two NFOV cameras simultaneously. The relation between the WFOV and NFOV cameras is determined by simple geometric transformation without complex calibration. Second, the Z distance (between a user’s eye and the iris camera) is estimated based on the iris size in the WFOV image and anthropometric data of the size of the human iris. Third, the accuracy of the geometric transformation between the WFOV and NFOV cameras is enhanced by using multiple matrices of the transformation according to the Z distance. Fourth, the searching region for iris localization in the NFOV image is significantly reduced based on the detected iris region in the WFOV image and the matrix of geometric transformation corresponding to the estimated Z distance. Experimental results showed that the performance of the proposed iris localization method is better than that of conventional methods in terms of accuracy and processing time.

Space situational awareness (SSA) is becoming increasingly challenging with the proliferation of resident space objects (RSOs), ranging from CubeSats to mega-constellations. Sensors within the United States Space Surveillance Network are tasked to repeatedly detect, characterize, and track these RSOs to retain custody and estimate their attitude. The majority of these sensors consist of ground-based sensors with a narrow field of view and must be slew at a finite rate from one RSO to another during observations. This results in a complex combinatorial problem that poses a major obstacle to the SSA sensor tasking problem. In this work, we successfully applied deep reinforcement learning (DRL) to overcome the curse of dimensionality and optimally task a ground-based sensor. We trained several DRL agents using proximal policy optimization and population-based training in a simulated SSA environment. The DRL agents outperformed myopic policies in both objective metrics of RSOs’ state uncertainties and the number of unique RSOs observed over a 90-min observation window. The agents’ robustness to changes in RSO orbital regimes, observation window length, observer’s location, and sensor properties are also examined. The robustness of the DRL agents allows them to be applied to any arbitrary locations and scenarios.

An accurate knowledge of the satellites, space debris, and other objects orbiting Earth is critical to mission planning and avoiding on-orbit collisions. Numerous ground and space-based space situational awareness platforms have been developed to aid in the identification and tracking of these objects, collectively known as resident space objects. While space situational awareness satellites are ideal for identifying and tracking space debris, many other satellites have sensors that may be able to contribute to space situational awareness. This paper examines the feasibility of leveraging several existing sensor technologies to aid in identifying and tracking resident space objects. Specifically, we investigate how Synthetic Aperture Radars (SARs), laser rangefinders, Light Detection and Ranging (LIDAR) devices, and laser communication devices can provide range measurements between the observing platform and the resident space object. Based on this survey, we believe that investigating the feasibility of utilizing LIDAR devices for RSO ranging is worth further study.

To maintain a robust catalog of resident space objects (RSOs), space situational awareness (SSA) mission operators depend on ground- and space-based sensors to repeatedly detect, characterize, and track objects in orbit. Although some space sensors are capable of monitoring large swaths of the sky with wide fields of view (FOVs), others—such as maneuverable optical telescopes, narrow-band imaging radars, or satellite laser-ranging systems—are restricted to relatively narrow FOVs and must slew at a finite rate from object to object during observation. Since there are many objects that a narrow FOV sensor could choose to observe within its field of regard (FOR), it must schedule its pointing direction and duration using some algorithm. This combinatorial optimization problem is known as the sensor-tasking problem. In this paper, we developed a deep reinforcement learning agent to task a space-based narrow-FOV sensor in low Earth orbit (LEO) using the proximal policy optimization algorithm. The sensor’s performance—both as a singular sensor acting alone, but also as a complement to a network of taskable, narrow-FOV ground-based sensors—is compared to the greedy scheduler across several figures of merit, including the cumulative number of RSOs observed and the mean trace of the covariance matrix of all of the observable objects in the scenario. The results of several simulations are presented and discussed. Additionally, the results from an LEO SSA sensor in different orbits are evaluated and discussed, as well as various combinations of space-based sensors.

To address the challenges on non-cooperative long-distance human authentication, identification, and verification; we propose an innovative scheme for developing a robust and automatic long-range biometric recognition system by combining face recognition and iris recognition of non-cooperative individuals in 24/7 operations. The system consists of three cameras. One is a wide field of view (WFOV) CCD video camera with InfraRed (IR) filter and powerful IR illuminators for human scan in a wide area and from a long distance. The other two cameras are high resolution video cameras with narrow field of view (NFOV) and IR filter & illuminators, mounted on a pan-tilt-unit (PTU) to capture the frontal view of human face and iris respectively. The WFOV detects the person and the NFOV cameras extract details for person identification. Once the frontal view shots are captured by the NFOV cameras, the face/iris models will be extracted by applying the state-of-the-art face/iris recognizers. In addition, a multimodality fusion approach integrates the face and iris recognition results to improve the overall recognition performance.

The Infrared Eye is a new concept of surveillance system that mimics human eye behavior to improve detection of small or low contrast target. In search and rescue operations (SAR), a wide field of view IR camera (WFOV) of approximately 20 degrees is used for detection of target and switched to a narrow field of view (NFOV) of approximately 5 degrees for a better target identification. In current SAR system, both FOVs cannot be used concurrently on the same display. The system presented in this paper fuses on the same high-resolution display the high- sensitivity WFOV image and the high-resolution NFOV image obtained from two IR cameras. The NFOV image movement within the WFOV image is slaved to the operator's eye movement by an eye-tracking device. The operator's central vision is always looking at the high-resolution IR image of the scene captured by the NFOV camera, while his peripheral vision is filled by the enhanced sensitivity (but low-resolution) image of the WFOV camera. This paper will describe the operation principle and implementation of the display, including its interface with an eye-tracking system and the opto-mechanical system used to steer the NFOV camera.

The United States Space Surveillance Network catalogs around 23,000 Resident Space Objects (RSOs). The completeness of their coverage of the true RSO population decreases gradually with object size and radar reflectivity. While the population of cm level space debris is poorly represented in the catalogs these space bullets can cause severe damage to satellites and spacecrafts in addition to being likely much more numerous than larger pieces. This research project focuses on the ability to peek into this debris population using space-based high sensitivity, fast frame rate, wide field visible imaging from low Earth orbit. The simulator developed focuses on a LEO to LEO (sensor to RSO) scenario and the capacity to constrain their orbit trajectory. In the Matlab simulator, a simple specular/diffuse sphere model is used for the debris in order to generate the object’s apparent magnitude for any sun-debris-observer arrangement. Satellite and debris relative velocities and orbits are also considered in order to determine the length of the streak left by the debris on any given exposure sequence and the number of photons per pixel. The exact timing, position, length and orientation of the streak contains information constraining the object’s orbit. The generation of representative star backgrounds matched to the sensor high sensitivity is also an important part of the simulator since it affects the effective limiting sensitivity to faint transiting source. This simulator allows us to trade various sensor parameters in order to optimize the camera design. The conclusion from this work contribute to the global effort in Space Situational Awareness (SSA) by assessing the impact of including space based optical imagery in the detection mix.

Tracking targets in a panoramic image is in many senses the inverse problem of tracking targets with a narrow field of view camera on a pan-tilt pedestal. In a narrow field of view camera tracking a moving target, the object is constant and the background is changing. A panoramic camera is able to model the entire scene, or background, and those areas it cannot model well are the potential targets and typically subtended far fewer pixels in the panoramic view compared to the narrow field of view. The outputs of an outward staring array of calibrated machine vision cameras are stitched into a single omnidirectional panorama and used to observe False Bay near Simon's Town, South Africa. A ground truth data-set was created by geo-aligning the camera array and placing a differential global position system receiver on a small target boat thus allowing its position in the array's field of view to be determined. Common tracking techniques including level-sets, Kalman filters and particle filters were implemented to run on the central processing unit of the tracking computer. Image enhancement techniques including multi-scale tone mapping, interpolated local histogram equalisation and several sharpening techniques were implemented on the graphics processing unit. An objective measurement of each tracking algorithm's robustness in the presence of sea-glint, low contrast visibility and sea clutter - such as white caps is performed on the raw recorded video data. These results are then compared to those obtained with the enhanced video data.

Taşınabilir ve giyilebilir akıllı mobil cihazların (telefon, tablet, kol saati, gözlük vb.) önemi dijitalleşen mekânsal bilgi endüstrisinde her geçen gün artmaktadır. Akıllı telefonlar gerek kullanım oranı gerekse ekonomik pazar payıyla bu endüstride ön plana çıkmaktadır. Profesyonel donanımlara kıyasla görece düşük maliyetli olan ve birçok sensör özelliğine sahip bu cihazlarda, farklı çözünürlükte kameralar kullanılmaktadır. Son olarak piyasaya sunulan bazı akıllı telefon ve tablet modellerine eklenen lazer tarama (LiDAR) sensör özelliğiyle bu gelişim bir adım daha ileri taşınarak, kamera+LiDAR sensörlerinin mühendislik ölçme uygulamalarında efektif kullanımının altyapısı geliştirilmiştir. 3 boyutlu (3B) modelleme ve artırılmış gerçeklik (Augmented Reality, AR) için bu özellikler maliyet bakımından daha ucuz alternatifler sunmaktadır. Bu çalışmada 3B ölçme ve modelleme ile yüksek doğrulukta mekânsal bilgi üretimi için akıllı cihazlar (telefon+tablet) kullanılarak, iç ve dış mekânlarda farklı boyut ve geometrik şekillerde tanımlanan nesnelerin kamera+LiDAR sensörleriyle elde edilen görüntüleri ve nokta bulutları analiz edilmiş, C2C ve M3C2 sapma analizi yöntemleri kullanılarak karşılaştırılmıştır. Elde edilen bulgular dikkate alındığında, yenilikçi teknolojik sensörlere sahip akıllı mobil cihazlarla elde edilen 3B model uygulama sonuçlarının doğruluğu, bu cihazların mekânsal bilgi endüstrisi açısından birçok farklı sektörde kullanımı için baskın bir alternatif olduğunu ortaya koymuştur.

In this work, we propose a novel technique to generate shapes from point cloud data. A point cloud can be viewed as samples from a distribution of 3D points whose density is concentrated near the surface of the shape. Point cloud generation thus amounts to moving randomly sampled points to high-density areas. We generate point clouds by performing stochastic gradient ascent on an unnormalized probability density, thereby moving sampled points toward the high-likelihood regions. Our model directly predicts the gradient of the log density field and can be trained with a simple objective adapted from score-based generative models. We show that our method can reach state-of-the-art performance for point cloud auto-encoding and generation, while also allowing for extraction of a high-quality implicit surface. Code is available at https://github.com/RuojinCai/ShapeGF .

Optimal range observability maneuvers of a spacecraft formation using angles-only navigation

The inclusion of thermal infrared (TIR) data in point clouds derived from unmanned aircraft system (UAS) imagery can benefit a variety of applications in which surface temperature and 3D geometry are both important discriminants of feature type and condition. Low resolution and narrow fields of view (FOV) of current consumer-grade TIR cameras on UAS, combined with the lack of sharpness and texture in many image regions, may cause failure or poor results from structure from motion (SfM) photogrammetric software, which has gained widespread use for generating point clouds from UAS imagery. This paper proposes a photogrammetric approach for generating 3D multispectral point clouds utilizing coacquired TIR-RGB images. A 3D point cloud is first generated from the RGB imagery using standard SfM procedures. Then the TIR attributes are assigned to points, where the image coordinates of the points in TIR images are estimated using transformation parameters obtained from co-registration procedures. To obtain RGB-to-TIR transformation parameters, this study tests 3D and 2D co-registration approaches. The latter produces better results due to the challenge of calibrating the TIR camera as required for the 3D approach. This proposed approach is advantageous for generating TIR point clouds without loss of photogrammetric precision compared with solely TIR-based SfM, as the 3D accuracy, point density, and reliability are greatly enhanced.

Human tracking is a fundamental technology for mobile robots that work with humans. Various devices are used to observe humans, such as cameras, RGB-D sensors, millimeter-wave radars, and laser range finders (LRF). Typical LRF measurements observe only the surroundings on a particular horizontal plane. Human recognition using an LRF has a low computational load and is suitable for mobile robots. However, it is vulnerable to variations in human height, potentially leading to detection failures for individuals taller or shorter than the standard height. This work aims to develop a method that is robust to height differences among humans using a 3D LiDAR. We observed the environment using a 3D LiDAR and projected the point cloud onto a single horizontal plane to apply a human-tracking method for 2D LRFs. We investigated the optimal height range of the point clouds for projection and found that using 30% of the point clouds from the top of the measured person provided the most stable tracking. The results of the path-following experiments revealed that the proposed method reduced the proportion of outlier points compared to projecting all the points (from 3.63% to 1.75%). As a result, the proposed method was effective in achieving robust human following.