Unknown Objects Research Articles

Semi-supervised Open-World Object Detection

Conventional open-world object detection (OWOD) problem setting first distinguishes known and unknown classes and then later incrementally learns the unknown objects when introduced with labels in the subsequent tasks. However, the current OWOD formulation heavily relies on the external human oracle for knowledge input during the incremental learning stages. Such reliance on run-time makes this formulation less realistic in a real-world deployment. To address this, we introduce a more realistic formulation, named semi-supervised open-world detection (SS-OWOD), that reduces the annotation cost by casting the incremental learning stages of OWOD in a semi-supervised manner. We demonstrate that the performance of the state-of-the-art OWOD detector dramatically deteriorates in the proposed SS-OWOD setting. Therefore, we introduce a novel SS-OWOD detector, named SS-OWFormer, that utilizes a feature-alignment scheme to better align the object query representations between the original and augmented images to leverage the large unlabeled and few labeled data. We further introduce a pseudo-labeling scheme for unknown detection that exploits the inherent capability of decoder object queries to capture object-specific information. On the COCO dataset, our SS-OWFormer using only 50% of the labeled data achieves detection performance that is on par with the state-of-the-art (SOTA) OWOD detector using all the 100% of labeled data. Further, our SS-OWFormer achieves an absolute gain of 4.8% in unknown recall over the SOTA OWOD detector. Lastly, we demonstrate the effectiveness of our SS-OWOD problem setting and approach for remote sensing object detection, proposing carefully curated splits and baseline performance evaluations. Our experiments on 4 datasets including MS COCO, PASCAL, Objects365 and DOTA demonstrate the effectiveness of our approach. Our source code, models and splits are available here https://github.com/sahalshajim/SS-OWFormer

Young children and adults use reasoning by exclusion rather than attraction to novelty to disambiguate novel word meanings.

Upon hearing a novel label, listeners tend to assume that it refers to a novel, rather than a familiar object. While this disambiguation or mutual exclusivity (ME) effect has been robustly shown across development, it is unclear what it involves. Do listeners use their pragmatic and lexical knowledge to exclude the familiar object and thus select the novel one? Or is the effect, at least in early childhood, simply based on an attraction to novelty and a direct mapping of the novel label to a novel object? In a preregistered online study with 2- to 3-year-olds (n = 75) and adults (n = 112), we examined (a) whether relative object novelty alone (without pragmatic or lexical information) could account for participants' disambiguation and (b) whether participants' decision processes involved reasoning by exclusion. Participants encountered either a known and an unknown object (classic ME condition) or two unknown objects, one completely novel and one preexposed (novelty condition) as potential referents of a novel label. Reasoning by exclusion was assessed by children's looking patterns and adults' explanations. In the classic ME condition, children and adults significantly chose the novel object and both used reasoning by exclusion. In contrast, in the novelty condition, children and adults chose randomly. Across conditions, a retention test revealed that adults remembered their prior selections, while children's performance was fragile. These results suggest that referent disambiguation is not based on relative object novelty alone. Instead, to resolve referential ambiguity, both young children and adults seem to make use of pragmatic and/or lexical sources of information and to engage in reasoning by exclusion strategies. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

MiPhDUO: microwave imaging via physics-informed deep unrolled optimization

Microwave imaging (MWI) is a non-invasive technique that can identify unknown scatterer objects’ features while offering advantages such as low cost and portable devices with respect to other imaging methods. However, MWI faces challenges in solving the underlying inverse scattering problem, which involves recovering target properties from its scattered fields. Existing methods include linearized and non-linear optimization approaches, but they have limitations respectively in terms of range of validity and computational complexity (in view of the possible occurrence of ‘false solutions’). In recent years, learning-based approaches have emerged as they can allow real-time imaging but usually lack generalizability and a direct connection to the underlying physics. This paper proposes a physics-informed approach that combines convolutional neural networks with physics-based calculations. It is based on a few cascaded operations, making use of the gradient of the relevant cost function, and successively improving the estimation of the unknown target. The proposed approach is assessed using simulated as well as experimental Fresnel data. The results show that the integration of physics with deep learning can contribute to improve reconstruction accuracy, generalizability, and computational efficiency in MWI.

Hubble Asteroid Hunter

Context. Determining the size distribution of asteroids is key to understanding the collisional history and evolution of the inner Solar System. Aims. We aim to improve our knowledge of the size distribution of small asteroids in the main belt by determining the parallaxes of newly detected asteroids in the Hubble Space Telescope (HST) archive and subsequently their absolute magnitudes and sizes. Methods. Asteroids appear as curved trails in HST images because of the parallax induced by the fast orbital motion of the spacecraft. Taking into account the trajectory of this latter, the parallax effect can be computed to obtain the distance to the asteroids by fitting simulated trajectories to the observed trails. Using distance, we can obtain the absolute magnitude of an object and an estimation of its size assuming an albedo value, along with some boundaries for its orbital parameters. Results. In this work, we analyse a set of 632 serendipitously imaged asteroids found in the ESA HST archive. Images were captured with the ACS/WFC and WFC3/UVIS instruments. A machine learning algorithm (trained with the results of a citizen science project) was used to detect objects in these images as part of a previous study. Our raw data consist of 1031 asteroid trails from unknown objects, not matching any entries in the Minor Planet Center (MPC) database using their coordinates and imaging time. We also found 670 trails from known objects (objects featuring matching entries in the MPC). After an accuracy assessment and filtering process, our analysed HST asteroid set consists of 454 unknown objects and 178 known objects. We obtain a sample dominated by potential main belt objects featuring absolute magnitudes (H) mostly between 15 and 22 mag. The absolute magnitude cumulative distribution log N(H > H0) ∝ α log(H0) confirms the previously reported slope change for 15 < H < 18, from α ≈ 0.56 to α ≈ 0.26, maintained in our case down to absolute magnitudes of around H ≈ 20, and therefore expanding the previous result by approximately two magnitudes. Conclusions. HST archival observations can be used as an asteroid survey because the telescope pointings are statistically randomly oriented in the sky and cover long periods of time. They allow us to expand the current best samples of astronomical objects at no extra cost in regard to telescope time.

Extended scene deep-phase-retrieval Shack-Hartmann wavefront sensors

Context. Strong atmospheric turbulence has been a challenge for high-resolution imaging of solar telescopes. Adaptive optics (AO) systems are capable of improving the quality of imaging by correcting partial aberrations. Thus, the performance of Shack-Hartmann sensors in measuring aberrations generally determines the upper performance bound of AO systems. In solar AO, classic correlation Shack-Hartmann sensors only correct a small number of modal aberrations. Moreover, strong aberrations are difficult to measure stably by correlation Shack-Hartmann. In this context, the improvement in the performance of Shark-Hartmann sensors promises to enable higher-resolution imaging of extended objects for ground-based telescopes or Earth observation. Aims. We propose a new extended scene deep-phase-retrieval Shack-Hartmann wavefront sensing approach to improve the image quality of solar telescopes. It is capable of achieving high-accuracy measurements of high-spatial-resolution wavefronts on extended scene wavefront sensing. Moreover, it has great generalization when observing unknown objects from different fields of view of the telescope. Methods. Our proposed approach can extract features resembling the sub-aperture point spread function (PSF) from a Shack-Hartmann sensor image without any prior information. Then a convolutional neural network is used to establish a nonlinear mapping between the feature image and the wavefront modal coefficients. The extracted feature greatly eliminates the shape information of the extended object while maintaining more information related to aberrations. We verified the performance of the proposed method through simulations and experiments. Results. In the indoor experiment on the ground layer adaptive optics (GLAO) of the 1 m New Vacuum Solar Telescope, compared to the Shack-Hartmann correlation method, the proposed method reduces the correction errors by more than one third. When observing objects from different fields of view in the GLAO that differ from the object in the training data, the relative errors fluctuate within the range of 20% to 26%. The AO system with the proposed wavefront measurement method can obtain higher-resolution focal images of the simulated solar granulation after a round of offline correction. The average latency of the proposed method is about 0.6 ms.

The effect of four different object properties on latency to approach in Goffin's cockatoos (Cacatua goffiniana).

Neophobia and neophilia can be lifesaving as they can facilitate foraging while avoiding predation or intoxication. We investigated the extent to which Goffin's cockatoos (Cacatua goffiniana) exhibit ecollogically relevant and quantifiable neophobic responses toward specific object properties. Twelve cockatoos were presented with 12 novel objects grouped into four distinct categories with unique features: size, color, reflective capacity, and shape. The cockatoos were tested by measuring their latency to approach a high-quality food reward for both novel and control scenarios. Age and sex did not affect the latency to approach food in the presence of a novel object in this species. Additionally, we found no significant differences between the objects of the reflective and color categories. This result is likely due to the plasticity of neophobic behavior related to the benefits and costs of approaching novel stimuli. The cockatoos were significantly slower to approach food in the presence of objects larger than their body size than objects of a similar or smaller size, a phenomenon possibly explained by the increased risk of approaching unknown objects large enough to be a potential predator. They were also significantly more hesitant to approach food in the presence of elongated objects, a phenomenon potentially explained by an ecologically relevant avoidance of snakes. The extent of this neophobia was statistically similar at a group level, indicating that avoidance of elongated and large objects could be an adaptive response aiding survival under natural circumstances and that snakes may impose strong selective pressures on this species. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Method for Unseen Object Six Degrees of Freedom Pose Estimation Based on Segment Anything Model and Hybrid Distance Optimization

Six degrees of freedom pose estimation technology constitutes the cornerstone for precise robotic control and similar tasks. Addressing the limitations of current 6-DoF pose estimation methods in handling object occlusions and unknown objects, we have developed a novel two-stage 6-DoF pose estimation method that integrates RGB-D data with CAD models. Initially, targeting high-quality zero-shot object instance segmentation tasks, we innovated the CAE-SAM model based on the SAM framework. In addressing the SAM model’s boundary blur, mask voids, and over-segmentation issues, this paper introduces innovative strategies such as local spatial-feature-enhancement modules, global context markers, and a bounding box generator. Subsequently, we proposed a registration method optimized through a hybrid distance metric to diminish the dependency of point cloud registration algorithms on sensitive hyperparameters. Experimental results on the HQSeg-44K dataset substantiate the notable improvements in instance segmentation accuracy and robustness rendered by the CAE-SAM model. Moreover, the efficacy of this two-stage method is further corroborated using a 6-DoF pose dataset of workpieces constructed with CloudCompare and RealSense. For unseen targets, the ADD metric achieved 2.973 mm, and the ADD-S metric reached 1.472 mm. This paper significantly enhances pose estimation performance and streamlines the algorithm’s deployment and maintenance procedures.

Resident space objects classification by machine learning techniques

In recent years the number of resident space objects in near-Earth space has been increasing drastically, which can lead to a “cascade collisions” effect and other catastrophic consequences. Therefore, it is crucial to monitor resident space objects, maintain and update catalogues on time, including the identification of unknown space objects to estimate their location and potential orbit. It is necessary to perform resident space object identification quickly and accurately to avoid possible satellite collisions resulting in large clouds of uncontrollable space debris fragments. That is why the task of resident space object preliminary classification by TLEs is stated and solved by applying machine learning approaches to achieve a higher classification quality and speed. This issue is a highly imbalanced classification problem, which narrows and specifies the used models. Applying the proposed resident space object classification model to space surveillance systems can decrease the time required for the object identification significantly.

Sparse multi-apparition linkages in large datasets

We present a new procedure to identify observations of known objects in large data sets of unlinked detections. It begins with a Keplerian integrals method that allows us to link two tracklets, computing preliminary orbits, even when the tracklets are separated in time by a few years. In the second step, we represent the results in a ‘graph’ where the tracklets are the nodes and the preliminary orbits are the edges. Then, acceptable ‘3-cycles’ are identified and a least squares orbit is computed for each of them. Finally, we construct sequences of n≥4 tracklets by searching through the orbits of nearby 3-cycles and attempting to attribute the remaining tracklets. We calculate the technique’s efficiency at identifying unknown objects using real detections that attempt to mimic key parameters of the Minor Planet Center’s (MPC) Isolated Tracklet File (ITF) and then apply the procedure to the ITF. This procedure enables the recovery of several orbits, despite some having few tracklets per apparition. The MPC accepted >95% of our linkages and most of the non-accepted linkages are 2-apparition linkages even when those linkages contained more than half a dozen tracklets.

Space object identification and classification from hyperspectral material analysis

This paper presents a data processing pipeline designed to extract information from the hyperspectral signature of unknown space objects. The methodology proposed in this paper determines the material composition of space objects from single pixel images. Two techniques are used for material identification and classification: one based on machine learning and the other based on a least square match with a library of known spectra. From this information, a supervised machine learning algorithm is used to classify the object into one of several categories based on the detection of materials on the object. The behaviour of the material classification methods is investigated under non-ideal circumstances, to determine the effect of weathered materials, and the behaviour when the training library is missing a material that is present in the object being observed. Finally the paper will present some preliminary results on the identification and classification of space objects.

SRG/ART-XC Galactic Bulge deep survey. II. Catalogue of point sources

ABSTRACT We present a catalogue of point sources detected with the Mikhail Pavlinsky ART-XC telescope onboard the SRG observatory during a wide-field survey of the Galactic Bulge that was conducted as part of the mission’s calibration and performance verification phase in 2019. The survey consisted of nearly 18 d of consecutive scanning observations of the sky region |l| < 6 deg, |b| < 2.5 deg with the median sensitivity of 4 × 10−13 erg s−1 cm−2 in the 4–12 keV energy band, which allows the detection of sources down to a luminosity of 3 × 1033 erg s−1 near the Galactic Centre. Using a maximum-likelihood-based algorithm, 172 sources were detected. Of these, 153 are registered on the average 4–12 keV map and 18 sources are either extremely hard (detected only at energies above 7 keV) or highly variable (detected only in individual scans shorter than a day). For 121 sources, there are plausible counterparts in other X-ray source catalogues, including 43 with known classification. The remaining 51 sources are previously unknown objects, discovered by ART-XC during the Galactic Bulge survey.

Tactile‐Based Negotiation of Unknown Objects during Navigation in Unstructured Environments with Movable Obstacles

Traditional robot navigation passively plans/replans to avoid any contact with obstacles in the scene. This limits the obtained solutions to the collision‐free space and leads to failures if the path to the goal is obstructed. In contrast, humans actively modify their environment by repositioning objects if it assists locomotion. This article aims to bring robots closer to such abilities by providing a framework to detect and clear movable obstacles to continue navigation. The approach leverages a multimodal robot skin that provides both local proximity and tactile feedback regarding physical interactions with the surroundings. This multimodal contact feedback is employed to adapt the robot's behavior when interacting with object surfaces and regulating applied forces. This enables the robot to remove bulky obstacles from its path and solves otherwise infeasible navigation problems. The system's ability is demonstrated in simulation and real‐world scenarios involving movable and nonmovable obstacles.

Compliance motion control of the hydraulic dual-arm manipulator with adaptive mass estimation of unknown object

Compliance motion control of the hydraulic dual-arm manipulator with adaptive mass estimation of unknown object

Analytic and Data-Driven Force Prediction for Vacuum-Based Granular Grippers

As manufacturing and assembly processes continue to require more adaptable systems for automated handling, innovative solutions for universal gripping are emerging. These grasping systems can enable the handling of wide varieties of shapes, with gripping forces varying with grasped geometries. For the efficient usage of handling systems, precise offline and online prediction models for resulting grasping forces for different objects are necessary. In previous research, a flexible vacuum-based granular gripper was developed, for which no option for predicting gripping forces is currently available. Various gripping force prediction methodologies within the current state of the art are examined and evaluated. For an assessment of grasping forces of previously untested objects for the examined gripper with limited data and low computational effort, two methodologies are proposed. An analytical, 2D-geometry-derived gripper-specific metric for geometries is compared to a methodology based on similarities of objects to a small existing dataset. The applicability and prediction quality for different object types is analyzed through validation experiments. Gripping force estimations are possible with both methodologies, with individual weaknesses towards geometric features such as air permeabilities. With further development, robust predictions of gripping forces could be achieved for a wide range of unknown object geometries with limited experimental effort.

Semantic geometric fusion multi-object tracking and lidar odometry in dynamic environment

AbstractSimultaneous localization and mapping systems based on rigid scene assumptions cannot achieve reliable positioning and mapping in a complex environment with many moving objects. To solve this problem, this paper proposes a novel dynamic multi-object lidar odometry (MLO) system based on semantic object recognition technology. The proposed system enables the reliable localization of robots and semantic objects and the generation of long-term static maps in complex dynamic scenes. For ego-motion estimation, the proposed system extracts environmental features that take into account both semantic and geometric consistency constraints. Then, the filtered features can be robust to the semantic movable and unknown dynamic objects. In addition, we propose a new least-squares estimator that uses geometric object points and semantic box planes to realize the multi-object tracking (SGF-MOT) task robustly and precisely. In the mapping module, we implement dynamic semantic object detection using the absolute trajectory tracking list. By using static semantic objects and environmental features, the system eliminates accumulated localization errors and produces a purely static map. Experiments on the public KITTI dataset show that the proposed MLO system provides more accurate and robust object tracking performance and better real-time localization accuracy in complex scenes compared to existing technologies.

SalienDet: A Saliency-Based Feature Enhancement Algorithm for Object Detection for Autonomous Driving

Object detection (OD) is crucial to autonomous driving. On the other hand, unknown objects, which have not been seen in training sample set, are one of the reasons that hinder autonomous vehicles from driving beyond the operational domain. To addresss this issue, we propose a saliency-based OD algorithm (SalienDet) to detect unknown objects. Our SalienDet utilizes a saliency-based algorithm to enhance image features for object proposal generation. Moreover, we design a dataset relabeling approach to differentiate the unknown objects from all objects in training sample set to achieve Open-World Detection. To validate the performance of SalienDet, we evaluate SalienDet on KITTI, nuScenes, and BDD datasets, and the result indicates that it outperforms existing algorithms for unknown object detection. Notably, SalienDet can be easily adapted for incremental learning in open-world detection tasks. The project page is <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/dingmike001/SalienDet-Open-Detection.git</uri> .

Recalling Unknowns without Losing Precision: An Effective Solution to Large Model-Guided Open World Object Detection.

Open World Object Detection (OWOD) aims to adapt object detection to an open-world environment, so as to detect unknown objects and learn knowledge incrementally. Existing OWOD methods typically leverage training sets with a relatively small number of known objects. Due to the absence of generic object knowledge, they fail to comprehensively perceive objects beyond the scope of training sets. Recent advancements in large vision models (LVMs), trained on extensive large-scale data, offer a promising opportunity to harness rich generic knowledge for the fundamental advancement of OWOD. Motivated by Segment Anything Model (SAM), a prominent LVM lauded for its exceptional ability to segment generic objects, we first demonstrate the possibility to employ SAM for OWOD and establish the very first SAM-Guided OWOD baseline solution. Subsequently, we identify and address two fundamental challenges in SAM-Guided OWOD and propose a pioneering SAM-Guided Robust Open-world Detector (SGROD) method, which can significantly improve the recall of unknown objects without losing the precision on known objects. Specifically, the two challenges in SAM-Guided OWOD include: (1) Noisy labels caused by the class-agnostic nature of SAM; (2) Precision degradation on known objects when more unknown objects are recalled. For the first problem, we propose a dynamic label assignment (DLA) method that adaptively selects confident labels from SAM during training, evidently reducing the noise impact. For the second problem, we introduce cross-layer learning (CLL) and SAM-based negative sampling (SNS), which enable SGROD to avoid precision loss by learning robust decision boundaries of objectness and classification. Experiments on public datasets show that SGROD not only improves the recall of unknown objects by a large margin (~ 20%), but also preserves highly-competitive precision on known objects. The program codes are available at https://github.com/harrylin-hyl/SGROD.

Towards utilising Artificial Intelligence for advanced reasoning and adaptability in human-robot collaborative workstations

Significant effort has been allocated recently by the research community for the seamless integration of Artificial Intelligence (AI) in modern human-robot collaborative (HRC) systems. Mainly in the context of human operator support and robotic object manipulation, the adoption of AI could become the main enabler towards the realisation of seamless HRC. This work presents the vision of the European funded project MASTERLY for an AI-based framework to enhance operator assistance and optimize robotic object manipulation. The proposed framework aims to contribute towards the realisation of intelligent, human-centric collaborative workstations focusing on two principal areas. The first area is human operator action recognition, which involves interpreting human actions to provide instructions and notifications to the operator only when it is considered necessary via Augmented Reality (AR) interfaces. The second area is the generation of a strategy for robotic manipulation of unknown objects. The proposed approach utilizes AI methods to identify optimal grasping points on unknown objects for efficient and precise robotic handling. The performance of the proposed framework is tested and validated in a case study from the elevators’ manufacturing industry, and more specifically, the assembly of the elevator's electrical panel.

Towards Generalized Few-Shot Open-Set Object Detection.

Open-set object detection (OSOD) aims to detect the known categories and reject unknown objects in a dynamic world, which has achieved significant attention. However, previous approaches only consider this problem in data-abundant conditions, while neglecting the few-shot scenes. In this paper, we seek a solution for the generalized few-shot open-set object detection (G-FOOD), which aims to avoid detecting unknown classes as known classes with a high confidence score while maintaining the performance of few-shot detection. The main challenge for this task is that few training samples induce the model to overfit on the known classes, resulting in a poor open-set performance. We propose a new G-FOOD algorithm to tackle this issue, named Few-shOt Open-set Detector (FOOD), which contains a novel class weight sparsification classifier (CWSC) and a novel unknown decoupling learner (UDL). To prevent over-fitting, CWSC randomly sparses parts of the normalized weights for the logit prediction of all classes, and then decreases the co-adaptability between the class and its neighbors. Alongside, UDL decouples training the unknown class and enables the model to form a compact unknown decision boundary. Thus, the unknown objects can be identified with a confidence probability without any threshold, prototype, or generation. We compare our method with several state-of-the-art OSOD methods in few-shot scenes and observe that our method improves the F-score of unknown classes by 4.80%-9.08% across all shots in VOC-COCO dataset settings.

A Real-Time Grasping Detection Network Architecture for Various Grasping Scenarios.

In the field of robot grasping detection, due to uncertain factors such as different shapes, distinct colors, diverse materials, and various poses, robot grasping has become very challenging. This article introduces a integrated robotic system designed to address the challenge of grasping numerous unknown objects within a scene from a set of α -channel images. We propose a lightweight and object-independent pixel-level generative adaptive residual depthwise separable convolutional neural network (GARDSCN) with an inference speed of around 28 ms, which can be applied to real-time grasping detection. It can effectively deal with the grasping detection of unknown objects with different shapes and poses in various scenes and overcome the limitations of current robot grasping technology. The proposed network achieves 98.88 % grasp detection accuracy on the Cornell dataset and 95.23 % on the Jacquard dataset. To further verify the validity, the grasping experiment is conducted on a physical robot Kinova Gen2, and the grasp success rate is 96.67 % in the single-object scene and 94.10 % in the multiobject cluttered scene.