Unknown Objects Research Articles

Generalized framework for image and video object segmentation using affinity learning and message passing GNNS

Despite significant amount of work reported in the computer vision literature, segmenting images or videos based on multiple cues such as objectness, texture and motion, is still a challenge. This is particularly true when the number of objects to be segmented is not known or there are objects that are not classified in the training data (unknown objects). A possible remedy to this problem is to utilize graph-based clustering techniques such as Correlation Clustering. It is known that using long range affinities (Lifted multicut), makes correlation clustering more accurate than using only adjacent affinities (Multicut). However, the former is computationally expensive and hard to use. In this paper, we introduce a new framework to perform image/motion segmentation using an affinity learning module and a Message Passing Graph Neural Network (MPGNN). The affinity learning module uses a permutation invariant affinity representation to overcome the multi-object problem. The paper shows, both theoretically and empirically, that the proposed MPGNN aggregates higher order information and thereby converts the Lifted Multicut Problem (LMP) to a Multicut Problem (MP), which is easier and faster to solve. Importantly, the proposed method can be generalized to deal with different clustering problems with the same MPGNN architecture. For instance, our method produces competitive results for single image segmentation (on BSDS dataset) as well as unsupervised video object segmentation (on DAVIS17 dataset), by only changing the feature extraction part. In addition, using an ablation study on the proposed MPGNN architecture, we show that the way we update the parameterized affinities directly contributes to the accuracy of the results.

On the use of low-cost 3D stereo depth camera to drive robot trajectories in contact-based applications

In production systems characterized by small batches and high customization levels, operations are required to be flexible to adapt to different products within the shortest possible time and with the minimum effort for system setup. Contact-based operations such as surface finishing, polishing, deburring, and material deposition are mandatory in the fabrication of numerous products. To maintain consistent performance over time, many of these operations require a high level of accuracy, both in end-effector positioning and contact force level. This paper proposes a robotic solution to generate the robot working trajectory for contact-based operations over the external surface of unknown objects of which a digital model is not available or different from the actual state of the workpiece. The paper introduces the integration process of RGBD images to construct a 3D model and its elaboration to extract the workpiece. Different searching subroutines have been developed to select different areas of the workpiece based on the operation to be carried on and generate the related trajectory. The evaluation metric of the proposed robotic solution is given by the stability of the contact force exerted by the robotic tool and the error between generated and the actually followed trajectory due to the depth estimation of a low-cost camera. A few millimeters of the inaccuracy of the trajectory are obtained; these inaccuracies are compensated using force control. Different tests with different nominal values of the force control loop are carried out. Statistical analysis shows that the mean values of the contact force obtained coincide with the nominal value of the single tests.

A Variable Photo-Model Method for Object Pose and Size Estimation with Stereo Vision in a Complex Home Scene.

Model-based stereo vision methods can estimate the 6D poses of rigid objects. They can help robots to achieve a target grip in complex home environments. This study presents a novel approach, called the variable photo-model method, to estimate the pose and size of an unknown object using a single photo of the same category. By employing a pre-trained You Only Look Once (YOLO) v4 weight for object detection and 2D model generation in the photo, the method converts the segmented 2D photo-model into 3D flat photo-models assuming different sizes and poses. Through perspective projection and model matching, the method finds the best match between the model and the actual object in the captured stereo images. The matching fitness function is optimized using a genetic algorithm (GA). Unlike data-driven approaches, this approach does not require multiple photos or pre-training time for single object pose recognition, making it more versatile. Indoor experiments demonstrate the effectiveness of the variable photo-model method in estimating the pose and size of the target objects within the same class. The findings of this study have practical implications for object detection prior to robotic grasping, particularly due to its ease of application and the limited data required.

Robot Active Neural Sensing and Planning in Unknown Cluttered Environments

Active sensing and planning in unknown, cluttered environments is an open challenge for robots intending to provide home service, search and rescue, narrow-passage inspection, and medical assistance. Although many active sensing methods exist, they often consider open spaces, assume known settings, or mostly do not generalize to real-world scenarios. In this article, we present the active neural sensing approach that generates the kinematically feasible viewpoint sequences for the robot manipulator with an in-hand camera to gather the minimum number of observations needed to reconstruct the underlying environment. Our framework actively collects the visual RGBD observations, aggregates them into scene representation, and performs object shape inference to avoid unnecessary robot interactions with the environment. We train our approach on synthetic data with domain randomization and demonstrate its successful execution via sim-to-real transfer in reconstructing narrow, covered, real-world cabinet environments cluttered with unknown objects. The natural cabinet scenarios impose significant challenges for robot motion and scene reconstruction due to surrounding obstacles and low ambient lighting conditions. However, despite unfavorable settings, our method exhibits high performance compared to its baselines in terms of various environment reconstruction metrics, including planning speed, the number of viewpoints, and overall scene coverage.

Application of improved you only look once model in road traffic monitoring system

<span lang="EN-US">The present research focuses on developing an intelligent traffic management solution for tracking the vehicles on roads. Our proposed work focuses on a much better you only look once (YOLOv4) traffic monitoring system that uses the CSPDarknet53 architecture as its foundation. Deep-sort learning methodology for vehicle multi-target detection from traffic video is also part of our research study. We have included features like the Kalman filter, which estimates unknown objects and can track moving targets. Hungarian techniques identify the correct frame for the object. We are using enhanced object detection network design and new data augmentation techniques with YOLOv4, which ultimately aids in traffic monitoring. Until recently, object identification models could either perform quickly or draw conclusions quickly. This was a big improvement, as YOLOv4 has an astoundingly good performance for a very high frames per second (FPS). The current study is focused on developing an intelligent video surveillance-based vehicle tracking system that tracks the vehicles using a neural network, image-based tracking, and YOLOv4. Real video sequences of road traffic are used to test the effectiveness of the method that has been suggested in the research. Through simulations, it is demonstrated that the suggested technique significantly increases graphics processing unit (GPU) speed and FSP as compared to baseline algorithms.</span>

The evolvement of the architect I.N. Kudryavtsev’s creative method in the 1930s on the example of the Houses of Communication in Leningrad, Murmansk and Petrozavodsk

The article reviews the architecture of the Houses of Communication designed by the architect I. N. Kudryavtsev, a graduate of the Leningrad VKhUTEIN, in the context of the architectural styles of the 1930s. Most of these projects were implemented in the North-Western region: Leningrad, Murmansk and Petrozavodsk. Based on archival sources, the authorship of I. N. Kudryavtsev was first established for a number of previously unknown objects. The study revealed specific features of the architect's creative method, which evolved under the influence of artistic, stylistic, social and political factors. For the first time, a whole typological group of specific buildings, which are Houses of Communication, have been identified as pertaining to the style of constructivism and post-constructivism. The role and place of the architect I. N. Kudryavtsev in Leningrad’s professional community before his expatriation to Finland in 1938 is ascertained.

Model-Based Adaptive Collaboration of Multi-Terminal Internal Force Tracking

This paper proposes a multi-terminal adaptive collaborative operation method for solving the problem of unstable internal force tracking in the clamping and handling of unknown objects by multi-terminal robots. In the proposed method, the internal command force changes the complex internal force control problem into an internal force tracking problem from multi-slave to master. Moreover, we develop an algorithm for multi-slave setups to estimate the object stiffness and motion uncertainty in the direction of the internal command force according to Lyapunov theory. Finally, the impedance control generates a reference trajectory for the multi-slave to maintain the desired internal force and track the master’s motion. Several experiments were conducted on a self-made robot. The experimental results show that the oscillation amplitude of each slave end is less than 1 mm and the directional oscillation amplitude is less than 1 degree during the tracking of the desired commanded internal force. For objects with a low stiffness, the error of the commanded internal force is less than 1 N (6%) per slave. The error in tracking the commanded internal force for objects with a high stiffness is less than 2 N (8%). The results prove the feasibility and effectiveness of the proposed method.

Komparasi Algoritma Data Mining Dalam Ketuntasan Belajar Daring Siswa Pada Masa Pandemi Covid 19

This research was conducted at SMA Negri 3 Selong and became the focus of students in class XI IPA and Social Studies. The sampling technique used purposive sampling method. This study aims to describe the extent to which the level of completeness of students during post-covid-19 pandemic learning with online media. This study uses a classification algorithm that functions to find a model that distinguishes data classes or data concepts, with the specific objective of determining the class of unknown object labels. The method used is the PSO-based Naïve Bayes and Naïve Bayes Comparison Algorithms. The results of this study indicate that the use of online media during online learning using the naïve Bayes algorithm is 83.91%, and the PSO-based naïve Bayes algorithm is 91.98%, from the experimental results and testing of the two algorithms, the results of the confusion matrix and AUC testing can be obtained which can be determined the best accuracy value is the PSO-based Naïve Bayes algorithm. As for the comparison of the results in the form of an accuracy value obtained by the Naïve Bayes Algorithm of 83.91% and the PSO-Based Naïve Bayes Algorithm of 91.98% and the difference in the level of accuracy of 8.07%, so it can be concluded that the algorithm that is suitable for classifying student learning completeness during the covid 19 pandemic is Naive Bayes based on particle swarm optimization.

Environment-aware motion planning for EAST Optical Diagnostic Beam Orientation Robot

EAST Optical Diagnostic Beam Orientation Robot (EODBOR) is designed for the calibration of the visible optical diagnostic system before the plasma experiment, which can orientate light beams with high precision. At present, the motion of EODBOR is purely controlled by the operators during the whole calibration process since planning algorithms based on prior knowledge of the environment cannot be adopted due to the intrinsically unstructured property of EAST’s vacuum vessel. To automate the calibration, we upgraded our previous system and propose a method for environment-aware motion planning based on Robot Operating System (ROS). In this method, the probabilistic octree is used to represent unknown objects in the environment, which is updated with point clouds from an RGB-D camera mounted at the end of the manipulator. We verify the feasibility and stability of the proposed method through simulations and experiments.

Research on Perception and Control Technology for Dexterous Robot Operation

Robotic grasping in cluttered environments is a fundamental and challenging task in robotics research. The ability to autonomously grasp objects in cluttered scenes is crucial for robots to perform complex tasks in real-world scenarios. Conventional grasping is based on the known object model in a structured environment, but the adaptability of unknown objects and complicated situations is constrained. In this paper, we present a robotic grasp architecture of attention-based deep reinforcement learning. To prevent the loss of local information, the prominent characteristics of input images are automatically extracted using a full convolutional network. In contrast to previous model-based and data-driven methods, the reward is remodeled in an effort to address the sparse rewards. The experimental results show that our method can double the learning speed in grasping a series of randomly placed objects. In real-word experiments, the grasping success rate of the robot platform reaches 90.4%, which outperforms several baselines.

Unsupervised image segmentation with robust virtual class contrast

Unsupervised Image Segmentation (UIS) is a challenging problem in computer vision that aims to classify pixels in an image into different semantic classes without using any labels. Among the various UIS methods, MaskContrast performs well when dealing with complex objects containing lots of detail. It utilizes prior knowledge from a pretrained saliency detection model to identify foreground pixels and leverages contrastive learning to learn the representations of those foreground pixels. However, this method is not designed to handle situations where foreground pixels belong to multiple semantic classes. To mitigate this problem, we propose a novel method for UIS called Robust Virtual Class Contrast (RVCC), which characterizes foreground pixels based on their “virtual classes” rather than just saliency masks. A virtual class can be viewed as the representation of some unknown object in the training data. Our method employs K-means clustering to find the pseudo virtual class label for every foreground pixel at the start of each training epoch. These pseudo-labels are then used to guide the learning of pixel representations. Additionally, to avoid the overconfidence of pseudo-label prediction, RVCC incorporates an additional regularization term that encourages consistency between the predictions under weak and strong augmentations. Our experiments demonstrate that RVCC outperforms existing baselines on the Pascal VOC2012 and MSRCv2 datasets, showcasing its capability for the UIS problem.

Haptic Exploration of Unknown Objects for Robust In-Hand Manipulation.

Human-like robot hands provide the flexibility to manipulate a variety of objects that are found in unstructured environments. Knowledge of object properties and motion trajectory is required, but often not available in real-world manipulation tasks. Although it is possible to grasp and manipulate unknown objects, an uninformed grasp leads to inferior stability, accuracy, and repeatability of the manipulation. Therefore, a central challenge of in-hand manipulation in unstructured environments is to acquire this information safely and efficiently. We propose an in-hand manipulation framework that does not assume any prior information about the object and the motion, but instead extracts the object properties through a novel haptic exploration procedure and learns the motion from demonstration using dynamical movement primitives. We evaluate our approach by unknown object manipulation experiments using a human-like robot hand. The results show that haptic exploration improves the manipulation robustness and accuracy significantly, compared to the virtual spring framework baseline method that is widely used for grasping unknown objects.

DSQNet: A Deformable Model-Based Supervised Learning Algorithm for Grasping Unknown Occluded Objects

DSQNet: A Deformable Model-Based Supervised Learning Algorithm for Grasping Unknown Occluded Objects

Fast Force-Closure Grasp Synthesis With Learning-Based Sampling

Anthropomorphic robotic hands have been widely investigated to dexterously manipulate objects because of their anatomical similarity to the human hand. However, the large dimension of configuration space challenges the real-time performance of existing grasp planning methods and drastically limits the application of anthropomorphic hands. In this letter, we propose a fast force-closure grasp synthesis (FFCGS) method for the anthropomorphic hand to efficiently grasp unknown objects. The FFCGS is implemented by using a signed distance field (SDF) as input. Firstly, a network that samples feasible 6D wrist poses is trained in an end-to-end fashion to reduce the dimension of search space. Furthermore, a fast optimization algorithm is presented to find finger configurations for force-closure precision grasp based on the differentiable Q-distance metric. We validate our method in both a simulated and a real-world environment. Experiment results show that the proposed FFCGS achieves a significantly improved performance in terms of time efficiency (5 times faster), grasp quality metrics, and success rate (5%-10% improvement) over benchmark methods. The outcomes of this study have great significance in promoting the motion planning of robot hand-arm systems and upper-limb prostheses.

Zero Moment Two Edge Pushing of Novel Objects With Center of Mass Estimation

Pushing is one of the fundamental nonprehensile manipulation skills to impart to an object changes in position and orientation. To exploit this skill to manipulate novel objects, explicit knowledge of their physical properties should be given a priori. In this work, we estimate the center of mass (CoM) of an object by narrowing down its probable location with a deep learning model and Mason’s voting theorem. In addition, we propose the Zero Moment Two Edge Pushing (ZMTEP) method to translate a novel object without rotation to a goal pose. The proposed method enables a pusher to select the most suitable two-edge-contact configuration for a given object using the estimated CoM and the geometrical shape of the object. Notably, neither the friction between the object and its support plane nor the friction between the object and the pusher are assumed to be known. We evaluate the proposed CoM estimation and ZMTEP methods through a series of experiments in both simulation and real robotic pusher settings. The result shows that the CoM estimation method has good mean squared error properties and small standard deviation, and the ZMTEP method significantly outperforms competitive baseline methods. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article aims to endow robotic arms with the capability of moving or aligning objects by pushing, which is much more simple and secure than pick-and-place or in-hand manipulations. Most in-demand manipulation skills require sophisticated hand design and control, which might not be affordable for industrial applications staying cost-competitive. In contrast, robot pushing can be implemented with different types of simple pushers and straightforwardly applied to pre-grasp manipulation. This article makes the estimation of an object’s CoM location practical. Building upon the estimation method, a robust and noise-tolerant two-edge-contact pushing configuration selection method is presented to translate an arbitrarily shaped unknown object to its goal pose.

Satellite Collision and Fragmentation Probabilities Using Radar-Based Size and Mass Estimates

Most conjunctions between Earth-orbiting satellites involve unknown objects, typically debris created by explosions or collisions. This study formulates methods to estimate probabilities of collision and fragmentation for such conjunctions, which depend on the estimated sizes and masses of the unknown objects. Analysis of radar cross-section (RCS) measurements provides estimated sizes, found to be accurate at the 90% confidence level to within a factor of 0.59 for potential underestimations and to within a factor of 3.1 for potential overestimations. For satellites that experience measurable atmospheric drag orbital perturbations, combining RCS data with orbit determination ballistic coefficients provides mass estimates accurate to within factors of 0.47 to 10.9 at the 90% confidence level. For satellites with perigee altitudes above 450 km, combining RCS data with solar radiation pressure coefficients provides mass estimates accurate to within factors of 0.44 to 5.6 at 90% confidence. The collision and fragmentation risk assessment formulation accounts for these size and mass estimation uncertainties. Specifically, conjunction collision probabilities formulated as statistically expected values account for the RCS-based size estimation uncertainties. Fragmentation probabilities, which measure the likelihood of producing more than a threshold number of collision fragments, account for both size and mass estimation uncertainties.

One-Shot Replay: Boosting Incremental Object Detection via Retrospecting One Object

Modern object detectors are ill-equipped to incrementally learn new emerging object classes over time due to the well-known phenomenon of catastrophic forgetting. Due to data privacy or limited storage, few or no images of the old data can be stored for replay. In this paper, we design a novel One-Shot Replay (OSR) method for incremental object detection, which is an augmentation-based method. Rather than storing original images, only one object-level sample for each old class is stored to reduce memory usage significantly, and we find that copy-paste is a harmonious way to replay for incremental object detection. In the incremental learning procedure, diverse augmented samples with co-occurrence of old and new objects to existing training data are generated. To introduce more variants for objects of old classes, we propose two augmentation modules. The object augmentation module aims to enhance the ability of the detector to perceive potential unknown objects. The feature augmentation module explores the relations between old and new classes and augments the feature space via analogy. Extensive experimental results on VOC2007 and COCO demonstrate that OSR can outperform the state-of-the-art incremental object detection methods without using extra wild data.

DMA-Net: Decoupled Multi-Scale Attention for Few-Shot Object Detection

As one of the most important fields in computer vision, object detection has undergone marked development in recent years. Generally, object detection requires many labeled samples for training, but it is not easy to collect and label samples in many specialized fields. In the case of few samples, general detectors typically exhibit overfitting and poor generalizability when recognizing unknown objects, and many FSOD methods also cannot make good use of support information or manage the potential problem of information relationships between the support branch and the query branch. To address this issue, we propose in this paper a novel framework called Decoupled Multi-scale Attention (DMA-Net), the core of which is the Decoupled Multi-scale Attention Module (DMAM), which consists of three primary parts: a multi-scale feature extractor, a multi-scale attention module, and a decoupled gradient module (DGM). DMAM performs multi-scale feature extraction and layer-to-layer information fusion, which can use support information more efficiently, and DGM can reduce the impact of potential optimization information exchange between two branches. DMA-Net can implement incremental FSOD, which is suitable for practical applications. Extensive experimental results demonstrate that DMA-Net has comparable results on generic FSOD benchmarks, particularly in the incremental FSOD setting, where it achieves a state-of-the-art performance.

Photon-noise: is a single-pixel camera better than point scanning? A signal-to-noise ratio analysis for Hadamard and Cosine positive modulation

In a single-pixel camera, an unknown object is sequentially illuminated by intensity patterns. The total reflected or transmitted intensity is summed in a single-pixel detector from which the object is computationally reconstructed. In the situation where the measurements are limited by photon-noise, it is questionable whether a single-pixel camera performs better or worse than simply scanning the object with a focused intensity spot—a modality known as point raster scanning and employed in many laser scanning systems. Here, we solve this general question and report that positive intensity modulation based on Hadamard or Cosine patterns does not necessarily improve the single-to-noise ratio (SNR) of single-pixel cameras, as compared to point raster scanning (RS). Instead, we show that the SNR is only improved on object pixels at least k times brighter than the object mean signal , where k is a constant that depends on the modulation scheme (modulation matrix, number of detectors, etc). The constant k is derived for several widespread cases and has important consequences on the choice of the optical deign. This fundamental property is demonstrated theoretically, numerically, and is experimentally confirmed in the spatial domain (widefield fluorescence imaging) and in the spectral domain (spontaneous Raman spectral measurements). Finally, we provide user-oriented guidelines that help decide when and how multiplexing under photon-noise should be used instead of point RS.

Edge detection ghost imaging using frequency-domain filtering

Ghost imaging is a novel imaging method, which enables imaging under harsh environments or quickly extracting image information. The advantage of Ghost imaging in edge detection is that it can directly obtain the edge information of an unknown target. We propose an edge detection method based on two dimensional (2D) discrete cosine transform (DCT) - frequency domain filtering ghost imaging (FDFGI). This method combines the frequency-domain filtering and anisotropic gradient operator to extract the edge information of the unknown object under the under-sampling rate. Compared with traditional methods, this method has a higher signal-to-noise ratio of image edge information extracted under Gaussian noise. It can even achieve edge information extraction under an extremely low sampling rate of about 1.12%.