Low-cost Visual Sensor Research Articles

Efficient Fine Tuning for Fashion Object Detection.

Pre-trained models have achieved success in object detection. However, challenges remain due to dataset noise and lack of domain-specific data, resulting in weaker zero-shot capabilities in specialized fields such as fashion imaging. We addressed this by constructing a novel clothing object detection benchmark, Garment40K, which includes more than 140,000 human images with bounding boxes and over 40,000 clothing images. Each clothing item within this dataset is accompanied by its corresponding category and textual description. The dataset covers 2 major categories, pants and tops, which are further divided into 15 fine-grained subclasses, providing a rich and high-quality clothing resource. Leveraging this dataset, we propose an efficient fine-tuning method based on the Grounding DINO framework to tackle the issue of missed and false detections of clothing targets. This method incorporates additional similarity loss constraints and adapter modules, leading to a significantly enhanced model named Improved Grounding DINO. By fine-tuning only a small number of additional adapter module parameters, we considerably reduced computational costs while achieving performance comparable to full parameter fine tuning. This allows our model to be conveniently deployed on a variety of low-cost visual sensors. Our Improved Grounding DINO demonstrates considerable performance improvements in computer vision applications in the clothing domain.

Automated Guided Vehicle (AGV) Driving System Using Vision Sensor and Color Code

Recently, the use of Automated Guided Vehicles (AGVs) at production sites has been increasing due to industrial development, such as the introduction of smart factories. AGVs utilizing the wired induction method, which is cheaper and faster than the wireless induction method, are mainly used at production sites. However, the wired guidance AGV operation system has the disadvantage of being limited to small-batch or collaboration-based production sites, since it is difficult to change the driving route. In this paper, we propose an AGV line-scan algorithm that can perform route recognition, driving commands, and operation through color-code recognition using an Arduino controller and a low-cost vision sensor, instead of the optical sensor conventionally used for these functions. When the proposed algorithm is applied to the AGV car, the CMUcam 5 Pixy2 camera identifies the driving path to follow by tracking a black line using the Otsu method. In addition, it can be confirmed that the driving command is executed using the proposed color code by applying the color recognition function of the CMUcam 5 Pixy2.

Roadside HD Map Object Reconstruction Using Monocular Camera

The traditional HD map production based on mapping vehicles equipped with expensive sensors (e.g. LiDAR) struggles to keep the frequently changing HD maps up-to-date, thus calling for great research efforts in the HD map reconstruction using low-cost vision sensors. However, most existing works in this area limit their focus on reconstructing the landmarks on the road surface. This letter widens this limited scope by focusing on the roadside object reconstruction with a monocular camera. We propose a novel object-level reconstruction framework that boasts two advantageous features: 1) a novel 3D model estimation method to directly reconstruct the specific roadside objects in a vectorized format using their prior geometric knowledge, and 2) a novel data association method to solve the complex tracking problem of roadside objects that often yield thin and coherent observations. The proposed framework is evaluated on both typical highway and urban scenarios in public KAIST Urban dataset. The results demonstrate that our algorithm can reconstruct the roadside objects with high accuracy and recall, and outperforms classic Structure from Motion (SfM) and deep-learning-based methods by a large margin.

Precise Inspection of Geometric Parameters for Polyvinyl Chloride Pipe Section Based on Computer Vision

The precise inspection of geometric parameters is crucial for quality control in the context of Industry 4.0. The current technique of precise inspection depends on the operation of professional personnel, and the measuring accuracy is restricted by the proficiency of operators. To solve the defects, this paper proposes a precise inspection framework for the geometric parameters of polyvinyl chloride (PVC) pipe section (G-PVC), using low-cost visual sensors and high-precision computer vision algorithms. Firstly, a robust imaging system was built to acquire images of a PVC pipe section under irregular illumination changes. Next, an engineering semantic model was established to calculate G-PVC like inner diameter, outer diameter, wall thickness, and roundness. After that, a region-of-interest (ROI) extraction algorithm was combined with an improved edge operator to obtain the coordinates of measured points on PVC end-face image in a stable and precise manner. Finally, our framework was proved highly precise and robust through experiments.

Evaluating Feature Extraction Methods with Synthetic Noise Patterns for Image-Based Modelling of Texture-Less Objects

Image-based three-dimensional (3D) reconstruction is a process of extracting 3D information from an object or entire scene while using low-cost vision sensors. A structure-from-motion coupled with multi-view stereo (SFM-MVS) pipeline is a widely used technique that allows 3D reconstruction from a collection of unordered images. The SFM-MVS pipeline typically comprises different processing steps, including feature extraction and feature matching, which provide the basis for automatic 3D reconstruction. However, surfaces with poor visual texture (repetitive, monotone, etc.) challenge the feature extraction and matching stage and affect the quality of reconstruction. The projection of image patterns while using a video projector during the image acquisition process is a well-known technique that has been shown to be successful for such surfaces. In this study, we evaluate the performance of different feature extraction methods on texture-less surfaces with the application of synthetically generated noise patterns (images). Seven state-of-the-art feature extraction methods (HARRIS, Shi-Tomasi, MSER, SIFT, SURF, KAZE, and BRISK) are evaluated on problematic surfaces in two experimental phases. In the first phase, the 3D reconstruction of real and virtual planar surfaces evaluates image patterns while using all feature extraction methods, where the patterns with uniform histograms have the most suitable morphological features. The best performing pattern from Phase One is used in Phase Two experiments in order to recreate a polygonal model of a 3D printed object using all of the feature extraction methods. The KAZE algorithm achieved the lowest standard deviation and mean distance values of 0.0635 mm and −0.00921 mm, respectively.

Low Memory Architectures of Fractional Wavelet Filter for Low-Cost Visual Sensors and Wearable Devices

This paper proposes a low memory architecture of fractional wavelet filter for computing two-dimensional discrete wavelet transform of gray-scale images. The proposed architecture uses a data scanning order which mainly contributes to memory reduction. A multiplierless version of the proposed architecture is also designed to reduce the critical path delay equal to the delay of an adder. The proposed architectures and an existing low memory discrete wavelet transform architecture are implemented on the same field programmable gate array board and their performance are compared for different image resolutions. The results show that the proposed architectures (with and without multipliers) uses lesser basic blocks, such as look-up tables (about 85-94% less), flip-flops (about 85-90% less), consumes lesser dynamic power (about 58-75% less), and results in critical path delay reduction of (about 23-77%) compared to one of the best existing low memory discrete wavelet transform architecture for images of $1024\times {1024}$ pixels. These advantages are achieved in addition to reduction in memory requirement over the existing state-of-the-art architectures. Thus, the proposed architectures are viable options for calculating discrete wavelet transform coefficients of gray-scale images on low-cost memory-constrained platforms like visual sensors/wearable devices and Internet of things.

Angle-Constrained Formation Control for Circular Mobile Robots

In this letter, we investigate the formation control problem of mobile robots moving in the plane where, instead of assuming robots to be simple points, each robot is assumed to have the form of a disk with equal radius. Based on interior angle measurements of the neighboring robots' disk, which can be obtained from low-cost vision sensors, we propose a gradient-based distributed control law and show the exponential convergence property of the associated error system. By construction, the proposed control law has the appealing property of ensuring collision avoidance between neighboring robots. We also present simulation results for {a team} of four circular mobile robots forming a rectangular shape.

Automatic surface inspection for S-PVC using a composite vision-based method.

Appearance defect inspection is crucial for quality control in the context of Industry 4.0. This research introduces a joint surface defect inspection and classification framework for polyvinyl chloride (PVC) pipe based on the low-cost visual sensors and high-efficiency computer vision algorithms. First, we build a robust imaging system to acquire the surface of PVC (S-PVC) by considering its characteristics and the illumination condition into the modeling process. Second, we adopt the region of interest method to eliminate the background interference captured in the S-PVC imaging and design an efficient S-PVC defect inspection and classification method. Third, we build an automatic machine prototype to evaluate the efficiency of the proposed method. Experimental results demonstrate that our framework has the advantages of low latency, high precision, and robustness.

SMFrWF: Segmented Modified Fractional Wavelet Filter: Fast Low-Memory Discrete Wavelet Transform (DWT)

This paper proposes a novel algorithm to compute the 2-D discrete wavelet transform (DWT) of high-resolution (HR) images on low-cost visual sensor and Internet of Things (IoT) nodes. The main advantages of the proposed segmented modified fractional wavelet filter (SMFrWF) are reduced computation (time) complexity and energy consumption compared to the state-of-the-art low-memory 2-D DWT computation methods. In particular, the conventional convolution-based DWT is very fast but requires large random access memory (RAM), as the entire image needs to be in the system memory. The fractional wavelet filter (FrWF) requires only a small RAM but has high complexity due to multiple readings of image lines. The proposed SMFrWF avoids the multiple readings of image lines, thus reducing the memory read access time and, thereby, the complexity. We evaluated the proposed SMFrWF through MATLAB simulations with 70 popular gray-scale test images of dimensions ranging from $256 \times 256$ up to $8192 \times 8192$ pixels. The results show that for images of size $2048\times 2048$ pixels, the proposed SMFrWF (with four segments per line) has 16.8% and 53.6% lower time complexities than the conventional DWT and FrWF, respectively. The proposed SMFrWF has also been modeled in a hardware description language (HDL) and implemented on an Artix-7 field-programmable gate array (FPGA) platform to evaluate the hardware performance. We observed that the proposed SMFrWF has 65% lower energy consumption than the FrWF (both implemented on the same board). Thus, the proposed SMFrWF appears suitable for computing the wavelet transform coefficients of HR images on low-cost visual sensors and IoT platforms.

Design of Low-Cost Personal Identification System That Uses Combined Palm Vein and Palmprint Biometric Features

This paper proposes a low-cost personal identification system that uses the combined palm vein and palmprint biometric features. The system consists of near-infrared and visible light-emitting diode (LED) arrays, a low-cost visual sensor, a Xilinx chip, and other components. A real-time image quality assessment (IQA) method for the combined palm vein and palmprint biometric features is also proposed. Two types of the LED with central frequency spectra of 890 and 680 nm are used to capture the palm vein and palmprint, respectively. The adaptive feedback control of the diode brightness is in accordance with the image quality assessed by the combined 2D entropy and local 2D entropy. The palm vein and palmprint images are acquired nearly simultaneously, and each acquired image undergoes a few preprocessing steps for extraction of the vein and print patterns. We use an image-level wavelet-based fusion strategy to reduce image storage requirement for the embedded platform and implement a complex wavelet-based fusion strategy for the PC platform. A deep scattering convolutional network is applied for extracting the features of the fused images, and a multi-class support vector machine is used for training and recognition. Characteristics of some vision-based personal identification systems are discussed. The proposed real-time IQA method with fusion strategy and feature extraction algorithm in our prototype system has substantially less operational requirements than that of the previous fusion strategies. It also demands less memory and yields lower equal error rate than the classical feature extraction algorithms.

A Distributed Strategy for Target Tracking and Rendezvous Using UAVs Relying on Visual Information Only

This paper proposes a distributed target tracking solution using a team of Unmanned Aerial Vehicles (UAVs) equipped with low-cost visual sensors capable of measuring targets bearing information only. The team of UAVs moves along circular orbits and uses consensus–based distributed Kalman Filtering to identify the position of the target. We show that the centre of the orbit eventually converges to the target position using theoretical arguments and extensive simulation data. By using the same approach, we can solve the rendezvous problem: the team first scans an area in search of a target; as soon as one of the UAVs spots, the other components converge on the target position.

ZM-SPECK: A Fast and Memoryless Image Coder for Multimedia Sensor Networks

The set partitioned embedded block (SPECK) algorithm is an efficient block-based image coder to encode wavelet transformed images. SPECK uses linked lists to track significant/insignificant coefficients and block sets, thereby having a large memory requirement that increases with the encoding rate. Furthermore, multiple memory read/write operations and list management slow down the algorithm. In addition, the implementation of the traditional discrete wavelet transform (DWT) is memory intensive and time-consuming. Therefore, it is difficult to implement image coding using the traditional DWT and SPECK algorithm on low-cost visual sensor nodes. Most of the existing studies on low-memory implementations of the SPECK algorithm attempt to replace the dynamic memory of linked lists by a static memory in the form of fixed-length state tables/markers. In this paper, a fast and memoryless image coder is proposed, which uses the fractional wavelet filter to calculate the DWT coefficients of the image and a zero-memory listless SPECK algorithm for quantization and coding of the DWT coefficients. The proposed algorithm, referred as zero-memory SPECK (ZM-SPECK), completely eliminates the linked lists and only uses a few registers to perform some low-level arithmetic/logical operations. The elimination of linked lists also reduces the memory access time, thereby making ZM-SPECK faster than the original SPECK algorithm. Simulation results show that the proposed ZM-SPECK coder outperforms the contemporary state-of-the-art wavelet image coders in terms of memory requirement and computational complexity, while retaining their coding efficiency. The proposed ZM-SPECK image coder is thus very well suited for image communication in visual sensor networks.

Vision Based Localization for Multiple Mobile Robots Using Low-cost Vision Sensor

This paper presents an efficient approach for a vision based localization of multiple mobile robots in an indoor environment by using a low cost vision sensor. The proposed vision sensor system that uses a single camera mounted over the mobile robots field takes advantages of small size, low energy consumption, and high flexibility to play an important role in the field of robotics. The nRF24L01 RF transceiver is connected to the vision system to enable wireless communication with multiple devices through 6 different data pipes. The downward-facing camera provides excellent performance that has the ability to identify a number of objects based on color codes, which form colored landmarks that provide mobile robots with useful image information for localization in the image view, which is then transformed to real world coordinates. Experimental results are given to show that the proposed method can obtain good localization performance in multi-mobile robots setting.

지능형 운행체를 위한 비전 센서 기반 자이로 드리프트 감소

Accurate heading information is crucial for the navigation of intelligent vehicles. In outdoor environments, GPS is usually used for the navigation of vehicles. However, in GPS-denied environments such as dense building areas, tunnels, underground areas and indoor environments, non-GPS solutions are required. Yaw-rates from a single gyro sensor could be one of the solutions. In dealing with gyro sensors, the drift problem should be resolved. HDR (Heuristic Drift Reduction) can reduce the average heading error in straight line movement. However, it shows rather large errors in some moving environments, especially along curved lines. This paper presents a method called VDR (Vision-based Drift Reduction), a system which uses a low-cost vision sensor as compensation for HDR errors.

A low-cost visual sensor network for elderly care

A low-resolution visual sensor network enables monitoring of elderly people's health and safety at home, postponing institutionalized healthcare.

Time-monitoring sensor based on oxygen diffusion in an indicator/polymer matrix

A time-monitoring sensor based on the oxidation of leuco methylene blue (LMB) to methylene blue (MB) was developed. The sensor changes its color from yellow to green in the presence of oxygen and was integrated into a poly(vinyl alcohol) matrix. The diffusion of the oxygen in the polymer matrix as well as the oxygen uptake due to the oxidation reaction determines the time monitoring of the sensor. A physical model has been developed that accounts for both the diffusion as well as the oxidation reaction. For this purpose, the reaction kinetics was determined experimentally. Moreover, the diffusion coefficient of oxygen was determined and concentration profiles in the polymer matrix modeled. Based on these modeling, the time sensor could be calibrated very precisely. This widely applicable, low-cost visual sensor is compatible with current technologies for the processing of plastics and can be integrated into different types of packaging, e.g. for application in freshness monitoring of consumer goods.

Floating Car Observer – An innovative vehicle-sensor for urban and highway scenarios

Stationary detectors together with Floating Car Data (FCD) are used today to generate information about the current situation on the roads. Without the need of equipping vehicle fleets with FCD sensors or installing lots of stationary detectors, the novel technique of Floating Car Observer (FCO) offers an alternative for capaciously measuring the oncoming traffic. Thereby, the speeds of oncoming vehicles are measured from a moving vehicle using a special sensor module. According to this principle, a low-cost visual sensor module has been developed. This sensor uses the reflection capability of passing vehicles' number plates in order to achieve data about driving direction and speed of these vehicles. The detection capabilities of the developed FCO module has been tested in several field tests. In urban scenarios the detection has been adapted to public transport requirements where the highway scenarios have been specialised concerning the detection of traffic situations in roadwork construction sites. This paper describes the hardware and software components of the sensor module as well as the field test results with detailed explanations.

Monitoring time and temperature by methylene blue containing polyacrylate film

The redox characteristic of methylene blue in polyacrylate film has been investigated for applications in monitoring time and temperature. This monitoring capability is based on a color change from colorless to blue under exposure to air. The oxidation process is controlled by the air diffusion which is affected by the chemical structure of the polyacrylate. The activation energy of diffusion, which was calculated by Arrhenius equation, goes up with increasing the rigid structure of the polyacrylate. The results correlate well with the glass transition temperature of different polyacrylates. The unique feature of the methylene blue to change color under oxidation, can be used for the fabrication of low cost visual sensors for monitoring time and temperature.

Low-cost vision sensors

The introduction of low-cost machine vision sensors is bringing machine vision to the mass automation market. The authors explain what to look for to ensure a successful application.

The Use of Multiple Low Cost Vision Sensors in Fabric Pick and Place Tasks

Abstract The use of several low cost dynamic RAM cameras to facilitate the accurate assembly of fabric panels is described. Two cases are considered, the assembly of shirt collars and a stage in the production of mens' underwear. The cameras are mounted on a robot gripper and used to locate features on the panels so that the gripper can be positionedaccurately. Details of the hardware and software aspects of the cameras are given.