Traditional Computer Vision Techniques Research Articles

A review of deep learning-based stereo vision techniques for phenotype feature and behavioral analysis of fish in aquaculture

The industrialization, high-density, and greener aquaculture requires a more precise and intelligent aquaculture management. Phenotypic and behavioral information of fish, which can reflect fish growth and welfare status, play a crucial role in aquaculture management. Stereo vision technology, which simulates parallax perception of the human eye, can obtain the three-dimensional phenotypic characteristics and movement trajectories of fish through different types of sensors. It can overcome the limitations in dealing with fish deformation, frequent occlusions and understanding three-dimension scenes compared to the traditional two-dimensional computer vision techniques. With the deep learning development and application in aquaculture, stereo vision has become a super computer vision technology that can provide more precise and interpretable information for intelligent aquaculture management, such as size estimation, counting and behavioral analysis of fish. Hence, it is very beneficial for researchers, managers, and entrepreneurs to possess a thorough comprehension about the fast-developing stereo vision technology for modern aquaculture. This study provides a critical review of relevant topics, including the four-layer application structure of stereo vision technology in aquaculture, various deep learning-based technologies used, and specific application scenarios. The review contributes to research development by identifying the current challenges and provide valuable suggestions for future research directions. This review can serve as a useful resource for developing future studies and applications of stereo vision technology in smart aquaculture, focusing on phenotype feature extraction and behavioral analysis of fish.

Research on the detection and recognition system of target vehicles based on fusion algorithm

<p>In modern society, the monitoring of vehicles is paramount for upholding traffic safety and order. This paper delves into a Graphical User Interface (GUI)-driven system for vehicle target detection and integration. This system seamlessly integrates three pivotal functionalities: vehicle type recognition, license plate recognition, and driver face recognition. It automates the vehicle inspection assessment process through an intuitive user interface. Specifically, for license plate recognition, the system leverages traditional computer vision techniques, including color and grayscale processing, radon transform, morphological operations, edge detection, character segmentation, and character recognition. The driver face recognition module incorporates methods such as image gray scaling, Gaussian filtering for denoising, face detection, and feature extraction. Meanwhile, vehicle type recognition is achieved by extracting Histogram of Oriented Gradients (HOG) features and utilizing a Support Vector Machine (SVM) classifier. Experimental findings highlight the system’s remarkable recognition accuracy, offering a highly efficient and dependable solution for the automated inspections of vehicles.</p>

Fractional Calculus Meets Neural Networks for Computer Vision: A Survey

Traditional computer vision techniques aim to extract meaningful information from images but often depend on manual feature engineering, making it difficult to handle complex real-world scenarios. Fractional calculus (FC), which extends derivatives to non-integer orders, provides a flexible way to model systems with memory effects and long-term dependencies, making it a powerful tool for capturing fractional rates of variation. Recently, neural networks (NNs) have demonstrated remarkable capabilities in learning complex patterns directly from raw data, automating computer vision tasks and enhancing performance. Therefore, the use of fractional calculus in neural network-based computer vision is a powerful method to address existing challenges by effectively capturing complex spatial and temporal relationships in images and videos. This paper presents a survey of fractional calculus neural network-based (FC NN-based) computer vision techniques for denoising, enhancement, object detection, segmentation, restoration, and NN compression. This survey compiles existing FFC NN-based approaches, elucidates underlying concepts, and identifies open questions and research directions. By leveraging FC’s properties, FC NN-based approaches offer a novel way to improve the robustness and efficiency of computer vision systems.

Identification of TRISO pebbles at arbitrary orientation using pairs of X-ray radiographs

TRISO (tri-structural isotropic) fuel is anticipated to be one of the fuel types for future small modular reactors. A TRISO-fuelled pebble in a pebble bed reactor (PBR) consists of thousands of TRISO fuel particles embedded inside a graphite sphere. The diameter of each of the fuel particles is approximately 1 mm. A PBR can contain hundreds of thousands of pebbles, each with a diameter of approximately 6 cm. Unlike in traditional reactor designs, which have a relatively small number of discrete fuel assemblies, the large number of pebbles inside a PBR makes the tracking of individual fuel pebbles difficult. The distribution of TRISO particles within the pebble provides a unique fingerprint that identifies a pebble and can be used to track it. This fingerprint can be determined using a standard X-ray radiograph or a more time-consuming X-ray computed tomography scan. When an individual radiograph is used, the pebble image is compared to a library of reference images of the pebbles. To obtain a good match, the orientation of the pebble must be close to the orientation used for the reference image. As a pebble exits the reactor in a random orientation, the pebble must be reorientated before its identification can take place. This paper discusses the addition of a reference particle in the non-fuelled region of the pebble, as well as the simultaneous acquisition of pairs of radiographs at 90° with respect to each other, to aid pebble reorientation and the development of an identification framework. The framework follows a hybrid approach in which the pebble is first reorientated using machine learning techniques before being identified using traditional computer vision techniques. Other effects such as the impact of transmutation of the elements inside the fuel and the radioactivity of irradiated pebbles are also discussed.

Using Deep Learning to Increase Eye-Tracking Robustness, Accuracy, and Precision in Virtual Reality.

Algorithms for the estimation of gaze direction from mobile and video-based eye trackers typically involve tracking a feature of the eye that moves through the eye camera image in a way that covaries with the shifting gaze direction, such as the center or boundaries of the pupil. Tracking these features using traditional computer vision techniques can be difficult due to partial occlusion and environmental reflections. Although recent efforts to use machine learning (ML) for pupil tracking have demonstrated superior results when evaluated using standard measures of segmentation performance, little is known of how these networks may affect the quality of the final gaze estimate. This work provides an objective assessment of the impact of several contemporary ML-based methods for eye feature tracking when the subsequent gaze estimate is produced using either feature-based or model-based methods. Metrics include the accuracy and precision of the gaze estimate, as well as drop-out rate.

Detecting irradiation defects in materials: A machine learning approach to analyze helium bubble images

Additive manufacturing technology has received significant attention in the field of nuclear materials due to its potential to improve the radiation resistance of materials and components. In our research, we propose to use 304 L stainless steel fabricated by Selective Laser Melting (SLM) as a replacement for traditional casting due to its superior performance and optimized manufacturing technique. During testing of its radiation resistance, we found that analyzing helium bubbles captured by transmission electron microscopy (TEM) is crucial for understanding and predicting material behavior under irradiation. However, this task typically involves manual counting, which is both time-consuming and prone to errors due to the fatigue of human annotators. To address this challenge, we present a novel machine learning model for automatic helium bubbles detection and counting through images. Our model leverages a fusion of traditional computer vision techniques with cutting-edge machine learning methods to achieve superior detection of helium bubbles in nuclear materials. This approach is based on two core designs: a novel generative model to generate training data and a gradient convergence layer to limit the range of parameters. Experiments show that our model outperforms previous state-of-the-art unsupervised detectors, and achieves highly accurate helium bubble segmentation in a few-shot scenario, with an F1 score typically exceeding 90 % and an average size error less than 0.1 nm. Our model achieves over 100 times more efficiency than manual counting, which takes approximately 20 s per image. Our work demonstrates a successful collaboration between the fields of material characterization and artificial intelligence (AI). By leveraging the respective strengths of material science and machine learning, we have achieved surprising results that could have a significant impact on the development of new materials and components for nuclear applications.

Deep-Learning-Based Action and Trajectory Analysis for Museum Security Videos

Recent advancements in deep learning and video analysis, combined with the efficiency of contemporary computational resources, have catalyzed the development of advanced real-time computational systems, significantly impacting various fields. This paper introduces a cutting-edge video analysis framework that was specifically designed to bolster security in museum environments. We elaborate on the proposed framework, which was evaluated and integrated into a real-time video analysis pipeline. Our research primarily focused on two innovative approaches: action recognition for identifying potential threats at the individual level and trajectory extraction for monitoring museum visitor movements, serving the dual purposes of security and visitor flow analysis. These approaches leverage a synergistic blend of deep learning models, particularly CNNs, and traditional computer vision techniques. Our experimental findings affirmed the high efficacy of our action recognition model in accurately distinguishing between normal and suspicious behaviors within video feeds. Moreover, our trajectory extraction method demonstrated commendable precision in tracking and analyzing visitor movements. The integration of deep learning techniques not only enhances the capability for automatic detection of malevolent actions but also establishes the trajectory extraction process as a robust and adaptable tool for various analytical endeavors beyond mere security applications.

Data-driven approach for AI-based crack detection: techniques, challenges, and future scope

This article provides a systematic literature review on the application of artificial intelligence (AI) technology for detecting cracks in civil infrastructure, which is a critical issue affecting the performance and longevity of these structures. Traditional crack detection methods involve manual inspection, which is laborious and time-consuming, especially in urban areas. Therefore, automatic crack detection with AI technology has gained popularity due to its ability to identify degradation of roads in real-time, leading to increased safety and reliability. This review emphasizes two key approaches for crack detection: deep learning and traditional computer vision, with a focus on data-driven aspects that rely primarily on data from training datasets to detect and quantify the severity level of the crack. The article highlights the advantages and drawbacks of each approach and provides an overview of various crack detection models, feature extraction techniques, datasets, potential issues, and future directions. The research concludes that deep learning-based methods used for crack classification, localization and segmentation have shown better performance than traditional computer vision techniques, especially in terms of accuracy. However, deep learning methods require large amounts of training data and computational power, which can be a significant limitation. Additionally, the article identifies a lack of 3D datasets, unsupervised learning algorithms are rarely used to train crack detection model, and datasets having road images with variety of road textures such as asphalt and cement etc. as challenges for future research in this field. A need for 3D and combined texture datasets as challenges for future research in this field.

Defect detection and classification on semiconductor wafers using two-stage geometric transformation-based data augmentation and SqueezeNet lightweight convolutional neural network

The manufacturing industry is evolving in line with the principles of Industry 4.0, with the aim of achieving higher levels of automation and digitization. In particular, deep learning algorithms such as convolutional neural networks (CNNs) are key enabling technologies to achieve this goal. The semiconductor industry is a particular case where CNNs are used to assist inspection systems and human operators in defect classification of wafers. However, deep CNN models are time consuming and resource intensive. It is therefore necessary to look for alternatives. One of these alternatives is the use of lightweight models, which provide competitive classification performance with low time and resource consumption. Therefore, the motivation of this work is to apply these lightweight models to semiconductor defect classification and compare their performance with that obtained by deep CNN models in similar work. In this line, this paper introduces an efficient two-step approach combining traditional computer vision techniques and a lightweight SqueezeNet CNN for defect detection and classification. The lightweight SqueezeNet model is tuned using a grid search algorithm. After obtaining the optimal model, its metrics are presented and compared with results from related work. Using a semiconductor surface defect dataset from a multinational semiconductor company, our lightweight model can achieve really competitive classification results (99.356% versus 99.443% obtained by ResNet50) while consuming significantly less time than other heavyweight models (80.146% less time than ResNet50).

Artificial Intelligence framework with traditional computer vision and deep learning approaches for optimal automatic segmentation of left ventricle with scar

Automatic segmentation of the cardiac left ventricle with scars remains a challenging and clinically significant task, as it is essential for patient diagnosis and treatment pathways. This study aimed to develop a novel framework and cost function to achieve optimal automatic segmentation of the left ventricle with scars using LGE-MRI images. To ensure the generalization of the framework, an unbiased validation protocol was established using out-of-distribution (OOD) internal and external validation cohorts, and intra-observation and inter-observer variability ground truths. The framework employs a combination of traditional computer vision techniques and deep learning, to achieve optimal segmentation results. The traditional approach uses multi-atlas techniques, active contours, and k-means methods, while the deep learning approach utilizes various deep learning techniques and networks. The study found that the traditional computer vision technique delivered more accurate results than deep learning, except in cases where there was breath misalignment error. The optimal solution of the framework achieved robust and generalized results with Dice scores of 82.8 ± 6.4% and 72.1 ± 4.6% in the internal and external OOD cohorts, respectively. The developed framework offers a high-performance solution for automatic segmentation of the left ventricle with scars using LGE-MRI. Unlike existing state-of-the-art approaches, it achieves unbiased results across different hospitals and vendors without the need for training or tuning in hospital cohorts. This framework offers a valuable tool for experts to accomplish the task of fully automatic segmentation of the left ventricle with scars based on a single-modality cardiac scan.

Deep Learning for Visual SLAM: The State-of-the-Art and Future Trends

Visual Simultaneous Localization and Mapping (VSLAM) has been a hot topic of research since the 1990s, first based on traditional computer vision and recognition techniques and later on deep learning models. Although the implementation of VSLAM methods is far from perfect and complete, recent research in deep learning has yielded promising results for applications such as autonomous driving and navigation, service robots, virtual and augmented reality, and pose estimation. The pipeline of traditional VSLAM methods based on classical image processing algorithms consists of six main steps, including initialization (data acquisition), feature extraction, feature matching, pose estimation, map construction, and loop closure. Since 2017, deep learning has changed this approach from individual steps to implementation as a whole. Currently, three ways are developing with varying degrees of integration of deep learning into traditional VSLAM systems: (1) adding auxiliary modules based on deep learning, (2) replacing the original modules of traditional VSLAM with deep learning modules, and (3) replacing the traditional VSLAM system with end-to-end deep neural networks. The first way is the most elaborate and includes multiple algorithms. The other two are in the early stages of development due to complex requirements and criteria. The available datasets with multi-modal data are also of interest. The discussed challenges, advantages, and disadvantages underlie future VSLAM trends, guiding subsequent directions of research.

Automatic measuring of finger joint space width on hand radiograph using deep learning and conventional computer vision methods

Hand osteoarthritis (OA) severity can be assessed visually through radiographs using semi-quantitative grading systems. However, these grading systems are subjective and cannot distinguish minor differences. Joint space width (JSW) compensates for these disadvantages, as it quantifies the severity of OA by accurately measuring the distances between joint bones. Current methods used to assess JSW require users’ interaction to identify the joints and delineate the initial joint boundary, which is time-consuming. To automate this process and offer a robust measurement for JSW, we proposed two novel methods to measure JSW: (1) The segmentation-based (SEG) method, which uses traditional computer vision techniques to calculate JSW; (2) The regression-based (REG) method, which is a deep learning approach employing a modified VGG-19 network to predict JSW. On a dataset with 3591 hand radiographs, 10,845 DIP joints were cut as ROI and served as input to the SEG and REG methods. The bone masks of the ROI images generated by a U-Net model were sent as input in addition to the ROIs. The ground truth of JSW was labeled by a trained research assistant using a semi-automatic tool. Compared with the ground truth, the REG method achieved a correlation coefficient (r) of 0.88 and a mean square error (MSE) of 0.02 mm on the testing set; the SEG method achieved a correlation coefficient of 0.42 and an MSE of 0.15 mm. Results show the REG method has promising performance in JSW measurement and, in general, Deep Learning approaches can facilitate the automatic quantification of distance features in medical images.

A depth information aided real-time instance segmentation method for space task scenarios under CPU platform

Visual instance segmentation ability is one of the effective means to promote the autonomy and intelligence of space agents. However, due to the limited airborne computing capacity, current methods are difficult to deploy on space agents, because these methods are developed based on GPU. To this end, this paper proposes a novel framework of instance segmentation by introducing depth information and combining traditional computer vision techniques with an object detection method. This framework provides a new idea for the implementation of instance segmentation. The experiment results show that the proposed method achieves a real-time performance under a common laptop CPU platform. In addition, thanks to the introduction of depth information, the proposed method can obtain better segmentation results compared to Mask R–CNN and SOLOv2 in complex scenes (poor illumination and occlusion). Finally, because the semantic information is obtained by the object detection method in this paper, the model training adopts a weakly supervised manner from bounding-box annotations, which can reduce various costs of data labeling to a certain extent.

Nature inspired firefighter assistant by unmanned aerial vehicle (UAV) data

One of the most hazardous phenomena in forests is wildfire or bush fire and early detection of massive damage prevention is vital. Employing Unmanned Aerial Vehicles (UAV) as a visual and extinguisher tool in order to prevent this tragedy which brings fatal effects on humans and wildlife has high importance. Additionally, using aerial imagery could assist firefighters to recognize fire intensity and localize and route the fire in the forest which shrinks down casualties of firefighters. All these benefits and more is just possible by employing cheap UAVs. The proposed research uses nature-inspired image processing techniques in order to segment and classify fire in color and thermal images. Multiple nature-inspired and traditional computer vision techniques, including Chicken Swarm Algorithm (CSA) intensity adjustment (contrast enhancement), Denoising Convolutional Neural Network (DnCNN), Local Phase Quantization (LPQ) feature extraction, Bees Image Segmentation, Biogeography-Based Optimization (BBO) feature selection, Firefly Algorithm (FA) classification and more are employed to achieve high classification and segmentation accuracy. The system evaluates nine performance metrics including, F-Score, Accuracy, and Jaccard for the segmentation stage and four performance metrics for the classification stage. All experiments are conducted on the two most recent UAV fire datasets of FLAME (2021) and DeepFire (2022). Additionally, fire intensity, fire direction, and fire geometrical calculation are calculated which assists firefighters even more. As smoke shows the location of the fire, a smoke detection workflow is proposed, too. Proposed system Compared with traditional and novel methods for segmentation and classification leading to satisfactory and promising results for almost all metrics. The trained model of this system could be used in most of the current rescue UAVs in real-time applications. For the FLAME dataset (color data), segmentation precision is 95.57 % and classification accuracy is 91.33 %. Also, For the DeepFire dataset segmentation precision is 91.74 % and classification accuracy is 96.88 %.

The EMory BrEast imaging Dataset (EMBED): A Racially Diverse, Granular Dataset of 3.4 Million Screening and Diagnostic Mammographic Images.

Supplemental material is available for this article. Keywords: Mammography, Breast, Machine Learning © RSNA, 2023.

Small object detection using retinanet with hybrid anchor box hyper tuning using interface of Bayesian mathematics

In recent years object detection system has been improved by many folds due to many novel deep learning models. Deep learning has outperformed the existing traditional computer vision techniques. In recent many deep learning models uses the concept of anchor box, the model proposes various anchor boxes on the images. The models generally use a classification model and a regression models, the regression model is used to predict the position of next possible anchor box and the classification is used to validate the anchor box. The hyper tuning of these models are generally based on the anchor box specifications, many researchers have used an optimized anchor box dimensions which is obtained for a specific dataset, due to which the accuracy increases drastically but the model are not scalable on any other data set. We propose a new hybrid anchor box optimization technique by using a variant of Bayesian optimization and sub sampling for small object detection using retina net model with resnet backbone. Our hybrid model is scalable over various datasets, the model is used on visdrone dataset and the result shows a 3.7% improvement in MAP result.

A deep residual neural network for semiconductor defect classification in imbalanced scanning electron microscope datasets

The detection of defects using inspection systems is common in a wide range of corporations such as semiconductor industries. The use of techniques based on deep learning (DL) and, in particular, convolutional neural networks (CNNs), emerges as a powerful tool to classify aesthetic defects and other unwanted anomalies in industrial automation applications marked by their natural complexity and high degree of variability. In this paper, an effective hybrid deep residual neural network approach is presented which merges traditional computer vision techniques to perform an appropriate defect segmentation and a deep residual neural network-grid search-based hyperparameter optimization for defect classification. The proposed model has been compared with other baseline algorithms and with other hybrid methods-based CNNs using different performance metrics such as F1-score, Cohen’s kappa coefficient, confusion matrix and computing time, on imbalanced datasets obtained from scanning electron microscope (SEM) images. The results obtained illustrate that the designed hybrid method provides the best defect classification of defects in semiconductor wafers in terms of F1-score (99.443%) while consuming the least computational time.

Obstacle detection method of unmanned electric locomotive in coal mine based on YOLOv3-4L

It is one of the most critical technologies for unmanned electric locomotives to detect the obstacles in front of their operation quickly and accurately, which is of great significance for the safe operation of electric locomotives. Aiming at the problems of current computer vision detection methods, such as error warning, low detection accuracy, and slow detection speed, an obstacle intelligent detection method for unmanned electric locomotives based on an improved YOLOv3 (YOLOv3-4L) algorithm is proposed. The obstacle image data set of the electric locomotive running area is constructed to provide a testing environment for various obstacle detection algorithms. In the network structure, the darknet-53 feature extraction network is simplified, and the four-scale detection structure is formed by adding the shallow layer detection scale to the detection layer, which can improve the detection speed and accuracy of the algorithm for obstacles in front of the locomotive. Distance intersection over union loss function and Focal loss function are adopted to redesign the loss function of the target detector to further improve the detection accuracy of the algorithm. Traditional computer vision techniques such as perspective transformation, sliding window, and least square cubic polynomial are used to detect the track lines. By finding the area where the track was located and extending a certain distance to the outside of the track, the dangerous area of electric locomotive running is obtained. The improved YOLOv3 algorithm is utilized to detect obstacles, and only the types and positions of obstacles coincident with dangerous areas are output. The experimental results show that the traditional computer vision techniques such as perspective transformation, sliding window, and least square cubic polynomial can detect not only straight track but also curved track, which makes up for the shortcomings of the Hough transforms in detecting curved tracks. Compared with the original YOLOv3 algorithm, the YOLOv3-4L algorithm improves the mean average precision by 5.1%, and the detection speed increases by 7 fps. YOLOv3-4L detection model has high detection accuracy and speed, which can meet the actual working conditions and provide technical reference for unmanned driving of electric locomotives in underground coal mines.

Deep Learning-Based Abdominal Muscle Segmentation on CT Images of Surgical Patient Populations.

Computed tomography (CT) is commonly used for the characterization and tracking of abdominal muscle mass in surgical patients for both pre-surgical outcome predictions and post-surgical monitoring of response to therapy. In order to accurately track changes of abdominal muscle mass, radiologists must manually segment CT slices of patients, a time-consuming task with potential for variability. In this work, we combined a fully convolutional neural network (CNN) with high levels of preprocessing to improve segmentation quality. We utilized a CNN based approach to remove patients' arms and fat from each slice and then applied a series of registrations with a diverse set of abdominal muscle segmentations to identify a best fit mask. Using this best fit mask, we were able to remove many parts of the abdominal cavity, such as the liver, kidneys, and intestines. This preprocessing was able to achieve a mean Dice similarity coefficient (DSC) of 0.53 on our validation set and 0.50 on our test set by only using traditional computer vision techniques and no artificial intelligence. The preprocessed images were then fed into a similar CNN previously presented in a hybrid computer vision-artificial intelligence approach and was able to achieve a mean DSC of 0.94 on testing data. The preprocessing and deep learning-based method is able to accurately segment and quantify abdominal muscle mass on CT images.

A Novel Road Maintenance Prioritisation System Based on Computer Vision and Crowdsourced Reporting

The maintenance of critical infrastructure is a costly necessity that developing countries often struggle to deliver timely repairs. The transport system acts as the arteries of any economy in development, and the formation of potholes on roads can lead to injuries and the loss of lives. Recently, several countries have enabled pothole reporting platforms for their citizens, so that repair work data can be centralised and visible for everyone. Nevertheless, many of these platforms have been interrupted because of the rapid growth of requests made by users. Not only have these platforms failed to filter duplicate or fake reports, but they have also failed to classify their severity, albeit that this information would be key in prioritising repair work and improving the safety of roads. In this work, we aimed to develop a prioritisation system that combines deep learning models and traditional computer vision techniques to automate the analysis of road irregularities reported by citizens. The system consists of three main components. First, we propose a processing pipeline that segments road sections of repair requests with a UNet-based model that integrates a pretrained Resnet34 as the encoder. Second, we assessed the performance of two object detection architectures—EfficientDet and YOLOv5—in the task of road damage localisation and classification. Two public datasets, the Indian Driving Dataset (IDD) and the Road Damage Detection Dataset (RDD2020), were preprocessed and augmented to train and evaluate our segmentation and damage detection models. Third, we applied feature extraction and feature matching to find possible duplicated reports. The combination of these three approaches allowed us to cluster reports according to their location and severity using clustering techniques. The results showed that this approach is a promising direction for authorities to leverage limited road maintenance resources in an impactful and effective way.