Machine Vision Research Articles

Integration and performance analysis of artificial intelligence and computer vision based on deep learning algorithms

This paper focuses on the analysis of the application effectiveness of the integration of deep learning and computer vision technologies. Deep learning achieves a historic breakthrough by constructing hierarchical neural networks, enabling end-to-end feature learning and semantic understanding of images. The successful experiences in the field of computer vision provide strong support for training deep learning algorithms. The tight integration of these two fields has given rise to a new generation of advanced computer vision systems, significantly surpassing traditional methods in tasks such as machine vision image classification and object detection. In this paper, typical image classification cases are combined to analyze the superior performance of deep neural network models while also pointing out their limitations in generalization and interpretability, proposing directions for future improvements. Overall, the efficient integration and development trend of deep learning with massive visual data will continue to drive technological breakthroughs and application expansion in the field of computer vision, making it possible to build truly intelligent machine vision systems. This deepening fusion paradigm will powerfully promote unprecedented tasks and functions in computer vision, providing stronger development momentum for related disciplines and industries.

Fringe-based depth segmentation via minimum-fringe-period-based singular points extraction

In the field of machine vision, depth segmentation plays a crucial role in dividing targets into different regions based on abrupt changes in depth. Phase-shifting depth segmentation is a technique that extracts singular points to form segmentation lines by leveraging the phase-shifting invariance of singular points in different wrapped phase maps. This makes it immune to color, texture, and camera exposure. However, current phase-shifting depth segmentation techniques face challenges in the precision of segmentation. To overcome this issue, this paper proposes a singular points extraction technique by constructing a more comprehensive threshold with the help of the minimum period of the phase map. Taking full advantage of the proposed technique, mean-value points and order singular points are accurately filtered out, and the integrity of segmentation lines in high-curvature regions can be guaranteed. During optimization processing, the precision of segmentation is improved by employing a low-cost morphology-based optimization model. Simulation results demonstrate the segmentation accuracy reaches up to 98.58% even in a noisy condition. Experimental results on different objects indicate that the proposed method exhibits good generalization and robustness.

A Reconfigurable Bipolar Image Sensor for High-Efficiency Dynamic Vision Recognition.

Dynamic vision perception and processing (DVPP) is in high demand by booming edge artificial intelligence. However, existing imaging systems suffer from low efficiency or low compatibility with advanced machine vision techniques. Here, we propose a reconfigurable bipolar image sensor (RBIS) for in-sensor DVPP based on a two-dimensional WSe2/GeSe heterostructure device. Owing to the gate-tunable and reversible built-in electric field, its photoresponse shows bipolarity as being positive or negative. High-efficiency DVPP incorporating front-end RBIS and back-end CNN is then demonstrated. It shows a high recognition accuracy of over 94.9% on the derived DVS128 data set and requires much fewer neural network parameters than that without RBIS. Moreover, we demonstrate an optimized device with a vertically stacked structure and a stable nonvolatile bipolarity, which enables more efficient DVPP hardware. Our work demonstrates the potential of fabricating DVPP devices with a simple structure, high efficiency, and outputs compatible with advanced algorithms.

A comprehensive research on measurement and evaluation of intelligent meter reading systems

Based on the machine vision and artificial intelligence technologies, the reading and acquisition of meter values have become the mainstream technical solutions for real-time monitoring of production data in the current industrial field. At the same time, the number of intelligent instruments and meters is increasing, and the quality varies. There is a lack of scientifically measurement methods and quality trust for intelligent meter reading algorithms. To promote their reliable and widespread application in various fields, it is necessary to conduct in-depth research on the assessment system of artificial intelligence in intelligent meter reading systems. This paper comprehensively studies various aspects involved in the emerging measurement field of artificial intelligence evaluation at the current stage, providing scientific measurement and evaluation references for the quality trust of artificial intelligence in the field of meters.

An Element Perception and State Recognition Method Oriented to Complex Assembled Product

Abstract Quality inspection during assembly process has an important impact on the performance and service life of the final product. Automated inspection solutions based on machine vision and artificial intelligence greatly improve the efficiency of inspection. However, the current mainstream large-scene inspection solutions, such as mobile vision platforms, are difficult to apply widely due to cost issues. Therefore, portable vision inspection solutions have gradually gained attention. However, the bottleneck technologies that limit this solution are full-element perception and occlusion judgment under free viewing angle, robust object matching method. To address the above problems, This paper proposes a free-viewpoint-based element perception and state recognition scheme. An improved structure estimation model is adopted to establish a bridge between virtual and actual mapping under the support of small-sample data, and all elements are located and occlusion analyzed by a 3D model-constrained view evolution algorithm. In addition, a hierarchy matching and optimization method is established to complete visual analysis. Experimental results show that this scheme performs well in structure estimation, element localization, occlusion judgment, state recognition, and work efficiency, demonstrating high application value.

Hybrid synaptic structure for spiking neural network realization

Neural networks and neuromorphic computing represent fundamental paradigms as alternative approaches to Von-Neumann-based implementations, advancing in the applications of deep learning and machine vision. Nonetheless, conventional semiconductor circuits encounter challenges in achieving ultra-fast processing speed and low power consumption due to their dissipative properties. Conversely, single flux quantum circuits exhibit inherent spiking behavior, showcasing their characteristics as a promising candidate for spiking neural networks (SNNs). In this work, we present a compact hybrid synapse circuit to mimic the biological interconnect functionality, enabling the weighting operations for excitatory and inhibitory impulses. Additionally, the proposed structure facilitates input accumulation, which is performed before the activation function. In the experiments, our synaptic structure interfaces with a soma circuit fabricated using a commercial Nb process, underscoring its compatibility and supporting its potential for integration into efficient neural network architectures. The weight value on the synapse is configurable by utilizing cryo-CMOS circuits, providing adaptability to the inference networks. We’ve successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system, enabling high-speed inference implementation for SNNs.

Vehicle Detection Algorithms for Autonomous Driving: A Review

Autonomous driving, as a pivotal technology in modern transportation, is progressively transforming the modalities of human mobility. In this domain, vehicle detection is a significant research direction that involves the intersection of multiple disciplines, including sensor technology and computer vision. In recent years, many excellent vehicle detection methods have been reported, but few studies have focused on summarizing and analyzing these algorithms. This work provides a comprehensive review of existing vehicle detection algorithms and discusses their practical applications in the field of autonomous driving. First, we provide a brief description of the tasks, evaluation metrics, and datasets for vehicle detection. Second, more than 200 classical and latest vehicle detection algorithms are summarized in detail, including those based on machine vision, LiDAR, millimeter-wave radar, and sensor fusion. Finally, this article discusses the strengths and limitations of different algorithms and sensors, and proposes future trends.

Tool wear monitoring based on scSE-ResNet-50-TSCNN model integrating machine vision and force signals

In the realm of mechanical machining, tool wear is an unavoidable phenomenon. Monitoring the condition of tool wear is crucial for enhancing machining quality and advancing automation in the manufacturing process. This paper investigates an innovative approach to tool wear monitoring that integrates machine vision with force signal analysis. It relies on a deep residual two-stream convolutional model optimized with the scSE (concurrent spatial and channel squeeze and excitation) attention mechanism (scSE-ResNet-50-TSCNN). The force signals are converted into the corresponding wavelet scale images following wavelet threshold denoising and continuous wavelet transform. Concurrently, the images undergo processing using contrast limited adaptive histogram equalization and the structural similarity index method, allowing for the selection of the most suitable image inputs. The processed data are subsequently input into the developed scSE-ResNet-50-TSCNN model for precise identification of the tool wear state. To validate the model, the paper employed X850 carbon fibre reinforced polymer and Ti–6Al–4V titanium alloy as laminated experimental materials, conducting a series of tool wear tests while collecting pertinent machining data. The experimental results underscore the model’s effectiveness, achieving an impressive recognition accuracy of 93.86%. When compared with alternative models, the proposed approach surpasses them in performance on the identical dataset, showcasing its efficient monitoring capabilities in contrast to single-stream networks or unoptimized networks. Consequently, it excels in monitoring tool wear status and promots crucial technical support for enhancing machining quality control and advancing the field of intelligent manufacturing.

A Novel Pretrained General-purpose Vision Language Model for the Vietnamese Language

Lying in the cross-section of computer vision and natural language processing, vision language models are capable of processing images and text at once. These models are helpful in various tasks: text generation from image and vice versa, image-text retrieval, or visual navigation. Besides building a model trained on a dataset for a task, people also study general-purpose models to utilize many datasets for multitasks. Their two primary applications are image captioning and visual question answering. For English, large datasets and foundation models are already abundant. However, for Vietnamese, they are still limited. To expand the language range, this work proposes a pretrained general-purpose image-text model named VisualRoBERTa. A dataset of 600k images with captions (translated MS COCO 2017 from English to Vietnamese) is introduced to pretrain VisualRoBERTa. The model’s architecture is built using Convolutional Neural Network and Transformer blocks. Fine-tuning VisualRoBERTa shows promising results on the ViVQA dataset with 34.49% accuracy, 0.4173 BLEU 4, and 0.4390 RougeL (in visual question answering task), and best outcomes on the sViIC dataset with 0.6685 BLEU 4, 0.6320 RougeL (in image captioning task).

Research on load monitoring technology of mine hoist based on machine vision

A hoist load monitoring method based on machine vision technology is proposed in this paper to address the frequent overloading accidents of mine hoists and the low safety and reliability of existing contact load monitoring technologies. The depth image of the skip undergoes time domain and spatial bilateral filtering algorithms for noise reduction, followed by conditional filtering and downsampling algorithms to remove redundant point cloud data. Point cloud recognition, extraction, segmentation, and alignment algorithms are then applied to quickly generate a skip point cloud model. A surface reconstruction optimization process combining greedy projection triangulation algorithm and void repair algorithm is proposed to obtain a smooth and complete sealing of the skip. The closed surface model volume is calculated using VTK volume function. Based on single-rope winding hoist, a load visual monitoring system is constructed for relevant experimental research. Results show that this method can accurately measure the loaded coal volume with a relative error range of 0.05%–4.13%, meeting practical application requirements while providing an effective way for non-contact accurate measurement of hoist loads in mines.

Nighttime Harvesting of OrBot (Orchard RoBot)

The Robotics Vision Lab of Northwest Nazarene University has developed the Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 s. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted in images with the lowest noise. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94% compared to 88% during the daytime operations.

Study on positioning and detection of crayfish body parts based on machine vision

Study on positioning and detection of crayfish body parts based on machine vision

Displacement sensing based on microscopic vision with high resolution and large measuring range

AbstractMicroimage strain sensing (MISS) is a novel piston‐type sensor based on microscopic vision. In this study, optical disc slice is used as information carriers to improve MISS. There are multiple pits on the surface of an optical disc. By using machine vision algorithms, the pits can be converted into digital information, making them scales for recording displacements. By this means, we proposed a sensing method that combines high resolution, wide range, and strong robustness. The study measured displacement under different conditions. To address inevitable factors such as pixel drift, and manufacturing errors, corresponding compensation methods were provided. The results show that the measurements closely match those of the linear variable differential transformer, with a resolution of up to 20 nm and a range approaching the sensor size. Despite the sensor's dependence on machine vision, it demonstrates strong resistance to environmental factors such as brightness and angle. Combining compensation methods for pixel drift, and manufacturing errors, this sensor can be well‐applied in various working conditions.

Detection method for contact stress distribution of tapered roller bearings

The axle box of high-speed train adopts double row tapered roller bearings as transmission parts, and the reliability of bearings directly affects the safety of train operation. Tapered roller bearings can withstand axial and radial loads, their service life being closely related to the distribution of contact stress. The test object is the axle box bearing of high-speed train. According to bearing’s structural characteristics, based on the digital speckle correlation method (DSCM) and machine vision, the contact stress distribution detection devices for rollers and complete sets of bearings are developed respectively and an effective detection method is proposed. The contact stress data of the bearing which has reached the service life and the new bearing under the condition of no lubrication and grease lubrication are collected and normalized. Through the geometric relationship, based on the measured data of the selected detection points, the normal and tangential contact stress distribution of the roller and raceway under different contact conditions is obtained by data fitting. The test can be used as an evaluation basis for the effectiveness of bearing modification and provide reference for bearing design.

Pointer generation and main scale detection for occluded meter reading based on generative adversarial network

The meter reading with machine vision greatly improves the efficiency of industrial monitoring. However, the pointer and scales of the meter can be occluded by rain or dirt, which greatly reduces the accuracy of the meter reading recognition. To solve this problem, we propose a generative adversarial network (PMS-GAN) with pointer generation and main scale detection for occluded meter reading. Specifically, dilated convolution block is designed to correlate separated pointer features. Then multi-scale feature fusion mechanism is proposed to guarantee the precision of pointer generation and main scale detection with guidance of semantic information. Moreover, feature enhancement mechanism is proposed to construct the long-range relationship for generating pointer under high occlusion. Finally, the reading is accomplished by calculating local angle with generated pointer and detected main scales. Experiments show that PMS-GAN can generate more intact pointer and detect main scales to guarantee the success and accuracy of occluded meter reading.

Correction: Real-time Machine Vision System for the Visually Impaired

Correction: Real-time Machine Vision System for the Visually Impaired

Camera calibration optimization algorithm that uses a step function

Camera calibration is very important when planning machine vision tasks. Calibration may involve 3D reconstruction, size measurement, or careful target positioning. Calibration accuracy directly affects the accuracy of machine vision. The parameters in many image distortion models are usually applied to all image pixels. However, this may be associated with rather high pixel reprojection errors at image edges, compromising camera calibration accuracy. In this paper, we present a new camera calibration optimization algorithm that features a step function that splits images into center and edge regions. First, based on the increasing pixel reprojection errors according to the pixel distance away from the image center, we gave a flexible method to divide an image into two regions, center and boundary. Then, the algorithm automatically determines the step position, and the calibration model is rebuilt. The new model can calibrate the distortions at the center and boundary regions separately. Optimized by the method, the number of distortion parameters in the old model is doubled, and different parameters represent different distortions within two regions. In this way, our method can optimize traditional calibration models, which define a global model to describe the distortion of the whole image and get a higher calibration accuracy. Experimentally, the method significantly improved pixel reprojection accuracy, particularly at image edges. Simulations revealed that our method was more flexible than traditional methods.

Multi-model feature aggregation for classification of laser welding images with vision transformer

The laser welding technique is quite common in various manufacturing lines, including those for lithium-ion power batteries, due to its remarkable productivity, effectiveness, and flexibility. However, it has been observed that the consistency of the welding quality is not always optimal. This study investigates the adoption of laser welding in the manufacturing of lithium-ion batteries, such as their anode, cathode and safety vent, where consistent weld quality is crucial for battery performance and safety. Therefore, evaluating the laser-welding product is indispensable for industrial production and the public domain. Several techniques have been utilized for laser welding evaluation, both destructive and non-destructive. However, these methods are ineffective and too cumbersome to be adopted in mass manufacturing. In contrast, a machine vision strategy has recently been adopted to distinguish between successful and unsuccessful laser welding items. This opened new perspectives for evaluating the quality of the laser welding product using digital image techniques. However, these methods cannot deliver outstanding performance across multiple laser-welding products and achieve optimal classification accuracy. Deep learning has evolved remarkably in recent years and gained popularity in detecting welding defects. This paper presents the observation of a Hybrid Vision Transformer (HViT) to classify the laser welding images corresponding to their feature patch images based on multi-model feature aggregation. The VGG-16 and MobileNet, which were pre-trained on ImageNet, were utilized as the core models to extract the rich features of the laser welding image. To integrate these features, the squeeze excitation module (SE) was utilized, and a multi-layer perception (MLP) approach with a label smoothing optimizer was used for classification. To determine the effectiveness of the proposed strategy, a comparative analysis was conducted with a multitude of alternative machine learning and deep learning approaches. The results indicate that the proposed strategy surpasses all other methods in terms of classification accuracy and evaluation metrics on the test dataset.

Movement and binding control strategy based on a new type of rebar-binding robot

Purpose The paper focuses on the issue of manual rebar-binding tasks in the construction industry, which are marked by high labor intensity, high costs and inefficient operations. The rebar-binding robots that are currently available are not fully mature. Most of them can only bind one or two nodes in one position, which leads to significant time wastage in movement. Based on a new type of rebar-binding robot, this paper aims to propose a new movement and binding control that reduces manpower and enhances efficiency. Design/methodology/approach The robot is combined with photoelectric sensors, travel switches and other sensors. It is supposed to move accurately and run in a limited area on the rebar mesh through logical judgment, speed control and position control. Machine vision is used by the robot to locate the rebar nodes and then adjusts the binding-gun position to ensure that multiple rebar nodes are bound sequentially. Findings By moving on the rebar mesh with accuracy, the robot meets the positioning accuracy requirements of the binding module, with experimental testing accuracy within 5 mm. Furthermore, its ability to bind four rebar nodes in one place results in a high efficiency and a binding effect that meets building standards. Originality/value The innovative design of the robot can adapt itself to the rebar mesh, move accurately to the target position and bind four nodes at that position, which reduces the number of movements on the mesh. Repetitive and heavy rebar-binding tasks can be efficiently completed by the robot, which saves human resources, reduces worker labor intensity and reduces construction overhead. It provides a more feasible and practical solution for using robots to bind rebar nodes.

Automation inspection in metal fabrication: enhancing sheet metal forming processes with automation, machine vision, and image correlation for quality assurance

Automation inspection in metal fabrication: enhancing sheet metal forming processes with automation, machine vision, and image correlation for quality assurance