Unknown Objects Research Articles

Automated Transient Detection in the Context of the 4m ILMT

In the era of sky surveys like Palomar Transient Factory (PTF), Zwicky Transient Facility (ZTF) and the upcoming Vera Rubin Observatory (VRO) and ILMT, a plethora of image data will be available. ZTF scans the sky with a field of view of 48 deg2 and VRO will have a FoV of 9.6 deg2 but with a much larger aperture. The 4m ILMT covers a 22′ wide strip of the sky. Being a zenith telescope, ILMT has several advantages like low observation air mass, best image quality, minimum light pollution and no pointing time loss. Transient detection requires all these imaging data to be processed through a Difference Imaging Algorithm (DIA) followed by subsequent identification and classification of transients. The ILMT is also expected to discover several known and unknown astrophysical objects including transients. Here, we propose a pipeline with an image subtraction algorithm and a convolutional neural network (CNN) based automated transient discovery and classification system. The pipeline was tested on ILMT data and the transients as well as variable candidates were recovered and classified.

Single View-Based Pose Estimation of Unknown Circular Object with RGB-D Camera

Single View-Based Pose Estimation of Unknown Circular Object with RGB-D Camera

Detecting and Tracking 6-DoF Motion of Unknown Dynamic Objects in Industrial Environments Using Stereo Visual Sensing

Detecting and Tracking 6-DoF Motion of Unknown Dynamic Objects in Industrial Environments Using Stereo Visual Sensing

Mobile robot perception system in ROS 2 on embedded computing platforms

In this paper a perception system of a mobile robot is proposed and implemented combining a LiDAR sensor with a colour camera. Localisation of unknown objects in an a priori known environment and their classification is required in order for the robot to safely and reliably navigate the working area. To this end, a pipeline is proposed to process and fuse the LiDAR 3D point-cloud data with a monocular colour camera image classifications yielding precisely localised 3D detections with class designations. The proposed pipeline is designed to run on limited computational power embedded platforms in the ROS 2 environment, and has been tested on a robotic testbench.

Stochastic Force-Closure Grasp Synthesis for Unknown Objects Using Proximity Perception

Stochastic Force-Closure Grasp Synthesis for Unknown Objects Using Proximity Perception

TIB: Detecting Unknown Objects via Two-Stream Information Bottleneck.

Detecting diverse objects, including ones never-seen-before during training, is critical for the safe application of object detectors. To this end, a task of unsupervised out-of-distribution object detection (OOD-OD) is proposed to detect unknown objects without the reliance on an auxiliary dataset. For this task, it is important to reduce the impact of lacking unknown data for supervision and leverage in-distribution (ID) data to improve the model's discrimination. In this paper, we propose a method of Two-Stream Information Bottleneck (TIB), consisting of a standard IB and a dedicated Reverse Information Bottleneck (RIB). Specifically, after extracting the features of an ID image, we first define a standard IB network to disentangle instance representations that are beneficial for localizing and recognizing objects. Meanwhile, we present RIB to obtain simulative OOD features to alleviate the impact of lacking unknown data. Different from standard IB aiming to extract task-relevant compact representations, RIB is to obtain task-irrelevant representations by reversing the optimization objective of the standard IB. Next, to further enhance the discrimination, a mixture of information bottlenecks is designed to sufficiently capture object-related information. Experimental results on OOD-OD, open-vocabulary object detection, incremental object detection, and open-set object detection show the superiorities of our method.

Construction of statistical recognition algorithms based on two-dimensional threshold functions

The recognition problem often rises in the classification of multidimensional data and is an integral part of artificial intelligence. Currently, there are many well-known methods and algorithms for solving the recognition problem. However, these methods are mainly aimed at solving such problems where objects are described by a set of independent features. A large number of features describe recognition problems in various fields of human activity. In such problems, the probability of violating the assumption of feature independence is quite high and this determines the relevance of this study.The purpose of this study is to develop a model of recognition algorithms based on the method of making statistical decisions, taking into account the high dimensionality of feature space.A statistical approach to solving the problem of recognition under the conditions of the high dimensionality of feature space is proposed. On the basis of this approach, a model of statistical recognition algorithms based on the construction of two-dimensional threshold rule is proposed. The main advantage of the proposed model is the reduction in the volume of computational operations when recognizing objects in the test set. At that, the reduction of the volume of computational operations is ensured using a number of procedures: selection of representative features; building a two-dimensional threshold rule; determination of a set of reference two-dimensional threshold rules; determination of basic two-dimensional threshold rules and calculation of a probabilistic assessment of the object belonging. This characteristic is very important for real-time recognition systems.The operability of the proposed model of recognition algorithms was investigated in the course of experiments in solving a model problem and the problem of recognizing a person's identity from a signature image. The results of the computational experiment confirm the operability of the proposed approach in solving the recognition problem under conditions when the assumption of feature independence is not satisfied. The results of experimental studies show that the proposed statistical algorithms increase the accuracy of recognition and significantly reduce the volume of computational operations by recognizing an unknown object specified in the space of interrelated features. These algorithms can be used in the compilation of various software systems focused on solving applied recognition problems in the conditions of the high dimensionality of feature space.

Robust Data Association Against Detection Deficiency for Semantic SLAM

Robust and accurate object association is essential for precise 3D object landmark inference in semantic Simultaneous Localization and Mapping (SLAM), and yet remains challenging due to the detection deficiency caused by high miss detection rate, false alarm, occlusion and limited field-of-view, etc. The 2D location of an object is a crucial complementary cue to the appearance feature, especially in the case of associating objects across frames under large viewpoint changes. However, motion model or trajectory pattern based methods struggle to infer object motion reliably with a moving camera. In this paper, by exploiting the local projective warping consistency, a local homography based 2D motion inference method is proposed to sequentially estimate the object location along with uncertainty. By integrating the deep appearance feature and semantic information, an object association method, named HOA, which is robust to detection deficiency is proposed. Experimental evaluations suggest that the proposed motion prediction method is capable of maintaining a low cumulative error over a long duration, which enhances the object association performance in both accuracy and robustness. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work aims to consistently associate 2D detection boxes corresponding to the same 3D object across images. In tasks of landmark-based navigation, collision avoidance, grasping and manipulation, objects in the task space are commonly simplified into 3D enveloping surfaces (e.g. cuboid or ellipsoid) by using 2D object detection boxes from multiple image views, and accurate data association is a prerequisite for precise enveloping surface reconstruction. This problem remains challenging considering the imperfect object detections, the appearance similarity of objects and the unpredictable trajectory of the moving camera. This work proposes a long-term reliable 2D location prediction algorithm that is capable of handling the complex motion of the target. Along with the appearance feature extracted by a retrain-free deep learning based model, this work proposes an object association method that can simultaneously deal with multiple objects with unknown object categories under the moving camera scenario.

Artificial Intelligence for the Classification of Plastic Waste Utilizing TinyML on Low-Cost Embedded Systems

BCG's implementation of the economy makes Thailand more environmentally conscious. The consolidation policy encourages consumers to eliminate single-use plastics using the 3Rs. This article introduces a solution to reduce plastic waste drastically using artificial intelligence. Utilizing a low-cost Arducam Pico4ML embedded device and TinyML, a plastic waste classifying system prototype is developed for plastic bottle segregation. The grayscale image datasets of PET, HDPE plastic bottles, and unknown objects are adjusted in the image pre-processing state and utilized to create trained models using MobileNetV2 convolutional-based neural network algorithms. Effective feature extraction and model training are performed on the Edge Impulse platform, and the trained model is exported to an embedded device using the optimized compiler. A further RS485 Modbus communication protocol feature enables integration with a programmable logic controller (PLC). The validation results of the trained model indicate a classification performance of 100% accuracy. Based on the average precision results, it is notable that the trained model can recognize the most common waste with an average accuracy of over 90%. The minimum classification rate of the MobileNetV2 quantized model is 249 milliseconds. It is also implemented in low-cost embedded devices for real-time plastic waste classification using fewer processing resources (185.4K ROM and 88K RAM). The findings exhibit sequential contributions that satisfy the criteria for classifying plastic bottles and the machine's integration capacity. These outcomes are anticipated to foster social shifts in behavior and enhance public awareness about plastic waste management.

Artificial Intelligence for the Classification of Plastic Waste Utilizing TinyML on Low-Cost Embedded Systems

BCG's implementation of the economy makes Thailand more environmentally conscious. The consolidation policy encourages consumers to eliminate single-use plastics using the 3Rs. This article introduces a solution to reduce plastic waste drastically using artificial intelligence. Utilizing a low-cost Arducam Pico4ML embedded device and TinyML, a plastic waste classifying system prototype is developed for plastic bottle segregation. The grayscale image datasets of PET, HDPE plastic bottles, and unknown objects are adjusted in the image pre-processing state and utilized to create trained models using MobileNetV2 convolutional-based neural network algorithms. Effective feature extraction and model training are performed on the Edge Impulse platform, and the trained model is exported to an embedded device using the optimized compiler. A further RS485 Modbus communication protocol feature enables integration with a programmable logic controller (PLC). The validation results of the trained model indicate a classification performance of 100% accuracy. Based on the average precision results, it is notable that the trained model can recognize the most common waste with an average accuracy of over 90%. The minimum classification rate of the MobileNetV2 quantized model is 249 milliseconds. It is also implemented in low-cost embedded devices for real-time plastic waste classification using fewer processing resources (185.4K ROM and 88K RAM). The findings exhibit sequential contributions that satisfy the criteria for classifying plastic bottles and the machine's integration capacity. These outcomes are anticipated to foster social shifts in behavior and enhance public awareness about plastic waste management.

CFSL: A Credible Federated Self-Learning Framework

Federated learning can collaboratively train AI models while protecting data privacy. In practical industry environment, Non-IID characteristics of data affect the effectiveness of federated learning. Personalized federated learning can help resolve this, but it cannot adapt to unknown data. In addition, practical applications also call for trusted training environment and remain stable when there are security threats. In this paper, we propose a Credible Federated Self-Learning (CFSL), based on the idea of hypernetwork supported by blockchain to achieve secured, credible, personalized federated self-learning, especially, for unknown data in Non-IID environment. Extensive experiments on three Non-IID datasets demonstrate the capabilities on adaptive resilience for security attacks and on accuracy of recognizing unknown objects, with good performance at the same time. CFSL outperforms the existing personalized federated learning methods, with an increase in average accuracy by 4.11%.

Enabling High-Performance Pressure and Proximity Dual-Mode Sensing with EGaIn/Ag/ZnO Egg-Shell Ternary Composite Particles.

Eutectic gallium-indium alloy (EGaIn) is a biocompatible liquid metal, promising for wearable electronics. Through functionalization and formation of composites, EGaIn-based materials have shown potential in multifunctional sensing devices. Here, egg-shell EGaIn/Ag/ZnO ternary composite particles were prepared through an ultrasound-assisted displacement reaction combined with room-temperature hydrolysis. The composite was further constructed as a wearable sensor capable of both pressure and proximity detection. For pressure sensing, due to the decrease in the Young's modulus of the egg-shell structure and the presence of the electrical double layers between Ag and ZnO, which enriched surface charges, the sensor showed excellent sensitivity at low pressures (2.17 KPa-1, <0.4 KPa) and thus the ability to sense body movements. For proximity sensing, the composite sensor was able to detect approaching objects that were up to 20 cm away. By combining and fitting the sensing curves for both the touchless and touching modes, the extracted parameters were used to create fingerprints for different objects, demonstrating the great potential of our sensor in the differentiation and identification of unknown objects for future robotics and artificial intelligence.

A Parallel Open-World Object Detection Framework with Uncertainty Mitigation for Campus Monitoring

The recent advancements in artificial intelligence have brought about significant changes in education. In the context of intelligent campus development, target detection technology plays a pivotal role in applications such as campus environment monitoring and the facilitation of classroom behavior surveillance. However, traditional object detection methods face challenges in open and dynamic campus scenarios where unexpected objects and behaviors arise. Open-World Object Detection (OWOD) addresses this issue by enabling detectors to gradually learn and recognize unknown objects. Nevertheless, existing OWOD methods introduce two major uncertainties that limit the detection performance: the unknown discovery uncertainty from the manual generation of pseudo-labels for unknown objects and the known discrimination uncertainty from perturbations that unknown training introduces to the known class features. In this paper, we introduce a Parallel OWOD Framework with Uncertainty Mitigation to alleviate the unknown discovery uncertainty and the known discrimination uncertainty within the OWOD task. To address the unknown discovery uncertainty, we propose an objectness-driven discovery module to focus on capturing the generalized objectness shared among various known classes, driving the framework to discover more potential objects that are distinct from the background, including unknown objects. To mitigate the discrimination uncertainty, we decouple the learning processes for known and unknown classes through a parallel structure to reduce the mutual influence at the feature level and design a collaborative open-world classifier to achieve high-performance collaborative detection of both known and unknown classes. Our framework provides educators with a powerful tool for effective campus monitoring and classroom management. Experimental results on standard benchmarks demonstrate the framework’s superior performance compared to state-of-the-art methods, showcasing its transformative potential in intelligent educational environments.

Finding a Needle in a Haystack: Faint and Small Space Object Detection in 16-Bit Astronomical Images Using a Deep Learning-Based Approach

With the increasing interest in space science exploration, the number of spacecraft in Earth’s orbit has been steadily increasing. To ensure the safety and operational integrity of active satellites, advanced surveillance and early warning of unknown space objects such as space debris are crucial. The traditional threshold-based filter for space object detection heavily relies on manual settings, leading to limitations such as poor flexibility, high false alarm rates, and weak target detection capability in low signal-to-noise ratios. Therefore, detecting faint and small objects against a complex starry background remains a formidable challenge. To address this challenge, we propose a novel, intelligent, and accurate detection method called You Only Look Once for Space Object Detection (SOD-YOLO). Our method includes the following novel modules: Multi-Channel Histogram Truncation (MHT) enhances feature representation, CD-ELAN based on Central Differential Convolution (CDC) facilitates learning contrast information, the Space-to-Depth (SPD) module replaces pooling layer to prevent small object feature loss, a simple and parameter-free attention module (SimAM) expands receptive field for Global Contextual Information, and Alpha-EIoU optimizes the loss function for efficient training. Experiments on our SSOD dataset show SOD-YOLO has the ability to detect objects with a minimum signal-to-noise ratio of 2.08, improves AP by 11.2% compared to YOLOv7, and enhances detection speed by 42.7%. Evaluation on the Spot the Geosynchronous Orbit Satellites (SpotGEO) dataset demonstrates SOD-YOLO’s comparable performance to state-of-the-art methods, affirming its generalization and precision.

MVSSC: Meta-reinforcement learning based visual indoor navigation using multi-view semantic spatial context

In Visual Indoor Navigation (VIN), Deep Reinforcement Learning (DRL) is commonly used by agents to achieve end-to-end mapping from vision to action when navigating toward a target based on observation. However, current DRL-based work suffers from two challenges: partial observability resulting from using solely a single first-person view and poor generalization in the case of unknown scenes and unknown objects. In light of these issues, this paper introduces the integration of multi-view as an expansion of observability and meta-learning as a primary generalization technique into the DRL framework and presents the meta-reinforcement learning method that leverages Multi-View Semantic Spatial Context (MVSSC). Specifically, aiming to explore the informative multi-view context for better efficiency in searching for and navigating to the target, we model the objects’ relationship from two aspects, multi-view semantic context (MVSEC) and multi-view spatial context (MVSPC). MVSEC enables agents to encode prior semantic relationships adaptively via a multi-view modulated graph. Meanwhile, MVSPC enhances the spatial representation of target-related objects’ correlation through similarity grids of multi-view. After adaptively fusing the multi-view and context information under the meta-reinforcement learning framework, our method can encourage efficient target search and robust navigation with stronger generalization performance to unknown scenes and unknown objects. Extensive experimental results on the AI2-THOR simulator demonstrate that our method outperforms the current state-of-the-art approaches.

A lightweight method of pose estimation for indoor object

Due to the multiple types of objects and the uncertainty of their geometric structures and scales in indoor scenes, the position and pose estimation of point clouds of indoor objects by mobile robots has the problems of domain gap, high learning cost, and high computing cost. In this paper, a lightweight 6D pose estimation method is proposed, which decomposes the pose estimation into a viewpoint and the in-plane rotation around the optical axis of the viewpoint, and the improved PointNet+⁣+ network structure and two lightweight modules are used to construct a codebook, and the 6d pose estimation of the point cloud of the indoor objects is completed by building and querying the codebook. The model was trained on the ShapeNetV2 dataset, and reports the ADD-S metric validation on the YCB-Video and LineMOD datasets, reaching 97.0% and 94.6% respectively. The experiment shows that the model can be trained to estimate the 6d position and pose of the unknown object point cloud with lower computation and storage cost, and the model with fewer parameters and better real-time performance is superior to other high-recision methods.

Isolated pure motor apraxia in the setting of depressed skull fracture: A case report and review of the literature

Introduction: An isolated pure apraxia of speech following head trauma is a rare clinical presentation in the absence of any other neurologic deficit. It is more commonly seen in patients with vascular pathology. Confusion can exist in recognizing an apraxia of speech versus a true Broca’s aphasia. We report such a case in a patient undergoing significant head trauma. The pathology and its management are discussed. Case Report: A 41-year-old male presented to our Emergency Department with a 3 cm laceration to the left side of the head following an assault with an unknown object. The patient suffered a depressed skull fracture with underlying epidural hematoma, bilateral traumatic subarachnoid hemorrhages, and a resultant isolated pure motor apraxia of speech. He was treated effectively with an emergency frontoparietal craniotomy, elevation and repair of his skull fracture, and evacuation of his epidural hematoma. His pure motor apraxia was greatly improved post-operatively and the patient was referred to outpatient speech therapy with continued neurosurgical follow-up. Conclusion: This case represents the unusual presentation of a pure motor apraxia of speech following traumatic assault to the left frontoparietal skull. An excellent therapeutic response was obtained following surgical intervention and speech therapy.

A voxel-based machine-learning framework for thermo-fluidic identification of unknown objects

The rapid identification of unknown objects by their thermo-fluid flow field signature is becoming increasingly more important. In this work, a machine-learning framework is developed that efficiently simulates and adapts object geometries in order to match the thermo-flow field signature generated by an unknown object, across a time series of voxel-frames. In order to achieve this, a thermo-fluid model is developed, based on the Navier–Stokes equations and the first law of thermodynamics, using a voxel rendering of the system, which is rapidly solved with a voxel-tailored, temporally-adaptive, iterative solution scheme. This voxel-framework is then combined with a genomic-based machine-learning algorithm to develop a digital-twin (digital-replica) of the system that can run in real-time or faster than the actual physical system. Numerical examples are provided to illustrate the framework.

Super-resolution imaging through scattering media based on improved triple correlation recursion and deterministic iterative estimation.

Iterative phase retrieval algorithms are commonly used in computational techniques and optimization methods to obtain the reconstruction of objects hidden behind opaque scattering media. However, these methods are susceptible to converging to incorrect local minima, and the calculation results tend to be unstable. In this paper, a triple-correlation-based super-resolution imaging (TCSI) framework is proposed to achieve single-shot imaging of unknown objects hidden behind the scattering medium. The amplitude spectrum of the object is obtained by a speckle correlation (SC) method. Iterative relaxation recursion (IRR) sufficiently extracts object information from the triple correlation (TC) of the speckle patterns, serving as the prior initial guess for the iterative estimation algorithm (IE) to obtain a deterministic phase spectrum. Blur correction (BC) is then applied to the diffraction-limited image to achieve super-resolution imaging. Experimental results demonstrate that the flexible framework could effectively overcome the influence of speckle resolution and outperform traditional methods in terms of performance. Our approach provides a basis for non-invasively visualizing various samples behind scattering media.

Robotic Continuous Grasping System by Shape Transformer-Guided Multiobject Category-Level 6-D Pose Estimation

Robotic grasping is one of the key functions for realizing industrial automation and human-machine interaction. However, current robotic grasping methods for unknown objects mainly focus on generating the 6D grasp poses, which cannot obtain rich object pose information and are not robust in challenging scenes. Based on this, we propose a robotic continuous grasping system that achieves end-to-end robotic grasping of intra-class unknown objects in 3D space by accurate category-level 6D object pose estimation. Specifically, to achieve object pose estimation, first, we propose a global shape extraction network (GSENet) based on ResNet1D to extract the global shape of an object category from the 3D models of intra-class known objects. Then, with the global shape as the prior feature, we propose a transformer-guided network to reconstruct the shape of intra-class unknown object. The proposed network can effectively introduce internal and mutual communication between the prior feature, current feature, and their difference feature. The internal communication is performed by self-attention. The mutual communication is performed by cross-attention to strengthen their correlation. To achieve robotic grasping for multiple objects, we propose a low-computation and effective grasping strategy based on the pre-defined vector orientation, and develop a GUI for monitoring and control. Experiments on two benchmark datasets demonstrate that our system achieves state-of-the-art (SOTA) 6D pose estimation accuracy. Moreover, the real-world experiments show that our system also achieves superior robotic grasping performance, with a grasping success rate of 81.6 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> for multiple objects. Code and trained models are released at https://github.com/CNJianLiu/6D-CLGrasp