Detection Of Transparent Objects Research Articles

MSCAF-Net: A General Framework for Camouflaged Object Detection via Learning Multi-Scale Context-Aware Features

The aim of camouflaged object detection (COD) is to find objects that are hidden in their surrounding environment. Due to the factors like low illumination, occlusion, small size and high similarity to the background, COD is recognized to be a very challenging task. In this paper, we propose a general COD framework, termed as MSCAF-Net, focusing on learning multi-scale context-aware features. To achieve this target, we first adopt the improved Pyramid Vision Transformer (PVTv2) model as the backbone to extract global contextual information at multiple scales. An enhanced receptive field (ERF) module is then designed to refine the features at each scale. Further, a cross-scale feature fusion (CSFF) module is introduced to achieve sufficient interaction of multi-scale information, aiming to enrich the scale diversity of extracted features. In addition, inspired the mechanism of the human visual system, a dense interactive decoder (DID) module is devised to output a rough localization map, which is used to modulate the fused features obtained in the CSFF module for more accurate detection. The effectiveness of our MSCAF-Net is validated on four benchmark datasets. The results show that the proposed method significantly outperforms state-of-the-art (SOTA) COD models by a large margin. Besides, we also investigate the potential of our MSCAF-Net on some other vision tasks that are highly related to COD, such as polyp segmentation, COVID-19 lung infection segmentation, transparent object detection and defect detection. Experimental results demonstrate the high versatility of the proposed MSCAF-Net. The source code and results of our method are available at https://github.com/yuliu316316/MSCAF-COD.

TGSNet: Multi-Field Feature Fusion for Glass Region Segmentation Using Transformers

Glass is a common object in living environments, but detecting it can be difficult because of the reflection and refraction of various colors of light in different environments; even humans are sometimes unable to detect glass. Currently, many methods are used to detect glass, but most rely on other sensors, which are costly and have difficulty collecting data. This study aims to solve the problem of detecting glass regions in a single RGB image by concatenating contextual features from multiple receptive fields and proposing a new enhanced feature fusion algorithm. To do this, we first construct a contextual attention module to extract backbone features through a self-attention approach. We then propose a VIT-based deep semantic segmentation architecture called MFT, which associates multilevel receptive field features and retains the feature information captured by each level of features. It is shown experimentally that our proposed method performs better on existing glass detection datasets than several state-of-the-art glass detection and transparent object detection methods, which fully demonstrates the better performance of our TGSNet.

LidSonic for Visually Impaired: Green Machine Learning-Based Assistive Smart Glasses with Smart App and Arduino

Smart wearable technologies such as fitness trackers are creating many new opportunities to improve the quality of life for everyone. It is usually impossible for visually impaired people to orientate themselves in large spaces and navigate an unfamiliar area without external assistance. The design space for assistive technologies for the visually impaired is complex, involving many design parameters including reliability, transparent object detection, handsfree operations, high-speed real-time operations, low battery usage, low computation and memory requirements, ensuring that it is lightweight, and price affordability. State-of-the-art visually impaired devices lack maturity, and they do not fully meet user satisfaction, thus more effort is required to bring innovation to this field. In this work, we develop a pair of smart glasses called LidSonic that uses machine learning, LiDAR, and ultrasonic sensors to identify obstacles. The LidSonic system comprises an Arduino Uno device located in the smart glasses and a smartphone app that communicates data using Bluetooth. Arduino collects data, manages the sensors on smart glasses, detects objects using simple data processing, and provides buzzer warnings to visually impaired users. The smartphone app receives data from Arduino, detects and identifies objects in the spatial environment, and provides verbal feedback about the object to the user. Compared to image processing-based glasses, LidSonic requires much less processing time and energy to classify objects using simple LiDAR data containing 45-integer readings. We provide a detailed description of the system hardware and software design, and its evaluation using nine machine learning algorithms. The data for the training and validation of machine learning models are collected from real spatial environments. We developed the complete LidSonic system using off-the-shelf inexpensive sensors and a microcontroller board costing less than USD 80. The intention is to provide a design of an inexpensive, miniature, green device that can be built into, or mounted on, any pair of glasses or even a wheelchair to help the visually impaired. This work is expected to open new directions for smart glasses design using open software tools and off-the-shelf hardware.

Non-line-of-sight transparent object detection and reconstruction based on passive single-pixel imaging

Single-pixel imaging (SPI) uses point detectors instead of expensive array sensors to capture images under low light conditions. It therefore provides a solution in detecting objects with weak signals. In this research, we propose a non-line-of-sight (NLOS) 2D transparent object inspection system based on SPI technique. The 2D shape of the transparent object is reconstructed using Total Variation Augmented Lagrangian and Alternating Direction Algorithm (TVAL3) approach by collecting indirect light reflected from the object surface. Theoretical analysis shows that 300 illumination patterns are required to reconstruct a pixel image, and those patterns are streamed to the digital micromirror device at high speed. Compared to TVAL3 with other methods, better image quality can be obtained with only one-third of illumination patterns. Also, experimental results confirm that this system can obtain transparent object information from reflected light even at a sampling rate of 24%. Furthermore, adjusting the laser wavelength and laser power can improve image quality. The proposed SPI detection system for NLOS transparent objects is an effective method that requires fewer illumination patterns and provides new insights for the detection and reconstruction of NLOS transparent object.

3D Transparent Object Detection and Reconstruction Based on Passive Mode Single-Pixel Imaging.

Transparent object detection and reconstruction are significant, due to their practical applications. The appearance and characteristics of light in these objects make reconstruction methods tailored for Lambertian surfaces fail disgracefully. In this paper, we introduce a fixed multi-viewpoint approach to ascertain the shape of transparent objects, thereby avoiding the rotation or movement of the object during imaging. In addition, a simple and cost-effective experimental setup is presented, which employs two single-pixel detectors and a digital micromirror device, for imaging transparent objects by projecting binary patterns. In the system setup, a dark framework is implemented around the object, to create shades at the boundaries of the object. By triangulating the light path from the object, the surface shape is recovered, neither considering the reflections nor the number of refractions. It can, therefore, handle transparent objects with a relatively complex shape with the unknown refractive index. The implementation of compressive sensing in this technique further simplifies the acquisition process, by reducing the number of measurements. The experimental results show that 2D images obtained from the single-pixel detectors are better in quality with a resolution of . Additionally, the obtained disparity and error map indicate the feasibility and accuracy of the proposed method. This work provides a new insight into 3D transparent object detection and reconstruction, based on single-pixel imaging at an affordable cost, with the implementation of a few numbers of detectors.

Structured-light modulation analysis technique for contamination and defect detection of specular surfaces and transparent objects.

With the rapid development of electronic industry, higher and higher requirements are placed on the surface quality of optical components in electronic devices. Structured-Light Modulation Analysis Technique (SMAT) was recently proposed to detect the contamination and defects on specular surfaces. In this paper, the proposed mechanisms and mathematical models of SMAT are analyzed and established based on the theory of photometry and the optical characteristics of contamination and defects for the first time. What's more, a novel transmission system adopting SMAT is especially designed for the defect detection of transparent objects. For both reflection and transmission system, simulations and experiments were conducted, and comparative studies with uniform planar illumination were also carried out. Simulations on the influence of incident light source region showed that SMAT can eliminate the interference of ambient light while uniform planar illumination technique cannot. Experiments on samples with specular surface and transparent material demonstrated that the modulation values at the contamination and defects are much less than that at clean and intact place, and defects and contaminations were clearly distinguished based on SMAT, while they were almost indiscernible with uniform planar illumination. Therefore, SMAT can be applied to the whole-field inspection of optical components in industrial environments.

Transparent object detection and location based on RGB-D camera

In order to improve the accuracy and efficiency of robot grasping, we propose a new method for transparent object detection and location that utilize depth image, RGB image and IR image. In detection process, an active depth sensor (RealSense) is firstly employed to retrieve the transparent candidates from the depth image and the corresponding candidates in the RGB image and IR image are then extracted separately. A transparent candidate classification algorithm is subsequently presented that uses SIFT features to recognize the transparent ones from the candidates. In location process, we obtain a new group of RGB images and IR images by adjusting camera orientation to make its optical axis perpendicular to the normal direction of the plane on which the object is placed. The object contours in RGB image and IR image are then extracted, respectively. The three-dimensional object is finally reconstructed by means of stereo matching of the two contours, and the current pose information of the object is calculated in the end. In order to verify the feasibility of the method, we built a hand-eye test system with a movable industrial robot to detect and capture transparent objects at different locations. The final test results demonstrate that the method is more general and effective than the traditional one.

Depth Resolution Enhancement in Time-of-Flight Cameras Using Polarization State of the Reflected Light

Time-of-Flight (ToF) cameras are nowadays commonly used for depth measurements and are useful in applications, such as robotic navigation, gaming, and others. The depth measurements by the ToF camera are influenced by the systematic errors of the camera and the nonsystematic errors of the environment. This paper presents a novel method for reduction of depth measurement error using polarization state of the reflected light. Using polarization state of the reflected light, the nonsystematic errors introduced due to multipath interference, material properties, and color of the target object reduces, which reduces the depth errors. The idea is tested using SR-4000 3-D sensor and a linear polarizer in front of it. The transmission angle of the linear polarizer is varied for a fixed object distance from the camera. The depth error has a sinusoidal variation with clear maxima and minima. It is observed that the depth error is minimum when the polarization angle is equal to the ToF phase shift. The experimental results show a maximum of 87% reduction in depth error at 3000 mm distance using polarization angle equal to ToF phase shift. The reduction in the nonsystematic error in the ToF-based distance measurements using polarization of the reflected light will enable new applications, such as diffuser imaging, detection of transparent objects, and among others.

6-5 テクスチャ特徴の屈折による変化を用いた透明物体の検出(第6部門 メディア工学(画像認識))

6-5 テクスチャ特徴の屈折による変化を用いた透明物体の検出(第6部門 メディア工学(画像認識))

Low coherence interferometric technique for detection of transparent objects in turbid media

This paper shows the ability to detect transparent objects in scattering media by an interferometric method. The technique used is an optical coherent one and allows direct detection of objects in turbid medium. The advantages and drawbacks of this method will be discussed.