Computer Vision Applications Research Articles

Deep Learning-Based Dzongkha Handwritten Digit Classification

In computer vision applications, pattern recognition is one of the important fields in artificial intelligence. With the advancement in deep learning technology, many machine learning algorithms were developed to tackle the problem of pattern recognition. The purpose of conducting the research is to create the first-ever Dzongkha handwritten digit dataset and develop a model to classify the digit. In the study, the 3 layer set of CONV → ReLU → POOL, followed by a fully connected layer, dropout layer, and softmax function were used to train the digit. In the dataset, each class (0-9) contains 1500 images which are split into train, validation, and test sets: 70:20:10. The model was trained on three different image dimensions: 28 by 28, 32 by 32, and 64 by 64. Compared to image dimensions 28 by 28 and 32 by 32, 64 by 64 gave the highest train, validation, and test accuracy of 98.66%, 98.9%, and 99.13% respectively. In the future, the sample of digits needs to be increased and use the transfer learning concept to train the model.

Low-Cost Classification of Close and Open Shell Antep Pistachio Nuts based on Image Analysis and Machine Learning

The effectiveness of post-harvest industrial processes is critical to maintaining the economic worth of pistachio nuts, which play an essential role in the agricultural economy. To achieve this level of efficiency, updated applications and technology for pistachio separation and categorization are required. Different pistachio species target different markets, highlighting the need for pistachio species classification. This work aims to develop a classification model that is distinct from existing separation approaches, based on image processing and machine learning, and can provide the required categorization. A computer vision application was done to identify between three types of pistachios. A high-resolution camera was used to capture 385 images of these pistachios. The photos of the pistachio samples were processed using image processing techniques like segmentation and feature extraction. On the given dataset, an advanced classifier based on Decision Tree and Random Forest predictions was constructed, as well as a simple and successful classifier. In the research, an application with feature extraction based on the dimension and pixel measurement is proposed. The proposed approach attained a classification success rate of 100% at 70% train and 30% test, and also, 80% train and 20% test data rate with Random Forest prediction, according to the experimental data. The provided high-performance classification model fills an important demand for the separation of pistachio types while increasing the economic worth of the species.

MMDCP: An Image Enhancement Algorithm Incorporating Multi-Channel Phase Activation and Multi-Constrained Dark Channel Prior

The quality of visual media is critically impacted by low illumination and the presence of airborne particulates, leading to challenges in brightness balance, color saturation, and texture clarity which are detrimental to various applications in image processing and computer vision. Addressing these challenges, this study introduces a novel image enhancement algorithm that significantly improves the quality of degraded images. Our proposed method, the multi-channel phase activation and multi-constraint dark channel prior (MMDCP), leverages an innovative approach by integrating the phase-adjusted Gaussian kernel function for brightness channel optimization in the Fourier transform frequency domain. This optimization is enhanced through the application of a saturated dark channel prior, achieving simultaneous brightness enhancement and color fidelity. Furthermore, we refine the dark channel prior deblurring algorithm by incorporating intensity, brightness, and color constraints to correct overexposure issues and color offsets in the reconstructed images. The efficacy of the MMDCP algorithm is demonstrated through extensive experimentation, comparing it against six contemporary image enhancement algorithms using two types of objective indicators and subjective assessments across four public datasets. The MMDCP algorithm consistently outperforms the existing methods, with a notable average improvement of 20% in PSNR and 19.6% in SSIM metrics, substantiating its superiority in enhancing brightness, detail, and color accuracy. This study’s results underline the MMDCP algorithm’s robustness and versatility in improving image quality across various conditions, including daytime, nighttime, indoor, and outdoor settings.

An Intelligent Detection and Classification Model Based on Computer Vision for Pavement Cracks in Complicated Scenarios

With the extension of road service life, cracks are the most significant type of pavement distress. To monitor road conditions and avoid excessive damage, pavement crack detection is absolutely necessary and an indispensable part of road periodic maintenance and performance assessment. The development and application of computer vision have provided modern methods for crack detection, which are low in cost, less labor-intensive, continuous, and timely. In this paper, an intelligent model based on a target detection algorithm in computer vision was proposed to accurately detect and classify four classes of cracks. Firstly, by vehicle-mounted camera capture, a dataset of pavement cracks with complicated backgrounds that are the most similar to actual scenarios was built, containing 4007 images and 7882 crack samples. Secondly, the YOLOv5 framework was improved from the four aspects of the detection layer, anchor box, neck structure, and cross-layer connection, and thereby the network’s feature extraction capability and small-sized-target detection performance were enhanced. Finally, the experimental results indicated that the proposed model attained an AP of the four classes of 81.75%, 83.81%, 98.20%, and 92.83%, respectively, and a mAP of 89.15%. In addition, the proposed model achieved a 2.20% missed detection rate, representing a 6.75% decrease over the original YOLOv5. These results demonstrated the effectiveness and practicality of our proposed model in addressing the issues of low accuracy and missed detection for small targets in the original network. Overall, the implementation of computer vision-based models in crack detection can promote the intellectualization of road maintenance.

Applied Artificial Intelligence in Healthcare: A Review of Computer Vision Technology Application in Hospital Settings.

Computer vision (CV), a type of artificial intelligence (AI) that uses digital videos or a sequence of images to recognize content, has been used extensively across industries in recent years. However, in the healthcare industry, its applications are limited by factors like privacy, safety, and ethical concerns. Despite this, CV has the potential to improve patient monitoring, and system efficiencies, while reducing workload. In contrast to previous reviews, we focus on the end-user applications of CV. First, we briefly review and categorize CV applications in other industries (job enhancement, surveillance and monitoring, automation, and augmented reality). We then review the developments of CV in the hospital setting, outpatient, and community settings. The recent advances in monitoring delirium, pain and sedation, patient deterioration, mechanical ventilation, mobility, patient safety, surgical applications, quantification of workload in the hospital, and monitoring for patient events outside the hospital are highlighted. To identify opportunities for future applications, we also completed journey mapping at different system levels. Lastly, we discuss the privacy, safety, and ethical considerations associated with CV and outline processes in algorithm development and testing that limit CV expansion in healthcare. This comprehensive review highlights CV applications and ideas for its expanded use in healthcare.

Tensorial Evolutionary Optimization for Natural Image Matting

Natural image matting has garnered increasing attention in various computer vision applications. The matting problem aims to find the optimal foreground/background (F/B) color pair for each unknown pixel and thus obtain an alpha matte indicating the opacity of the foreground object. This problem is typically modeled as a large-scale pixel pair combinatorial optimization (PPCO) problem. Heuristic optimization is widely employed to tackle the PPCO problem owing to its gradient-free property and promising search ability. However, traditional heuristic methods often encode F/B solutions to a one-dimensional (1D) representation and then evolve the solutions in a 1D manner. This 1D representation destroys the intrinsic two-dimensional (2D) structure of images, where the significant spatial correlations among pixels are ignored. Moreover, the 1D representation also brings operation inefficiency. To address the above issues, this article develops a spatial-aware tensorial evolutionary image matting (TEIM) method. Specifically, the matting problem is modeled as a 2D Spatial-PPCO (S-PPCO) problem, and a global tensorial evolutionary optimizer is proposed to tackle the S-PPCO problem. The entire population is represented as a whole by a third-order tensor, in which individuals are classified into two types: F and B individuals for denoting the 2D F/B solutions, respectively. The evolution process, consisting of three tensorial evolutionary operators, is implemented based on pure tensor computation for efficiently seeking F/B solutions. The local spatial smoothness of images is also integrated into the evaluation process for obtaining a high-quality alpha matte. Experimental results compared with state-of-the-art methods validate the effectiveness of TEIM.

Unified Framework for Diffusion Generative Models in SO(3): Applications in Computer Vision and Astrophysics

Diffusion-based generative models represent the current state-of-the-art for image generation. However, standard diffusion models are based on Euclidean geometry and do not translate directly to manifold-valued data. In this work, we develop extensions of both score-based generative models (SGMs) and Denoising Diffusion Probabilistic Models (DDPMs) to the Lie group of 3D rotations, SO(3). SO(3) is of particular interest in many disciplines such as robotics, biochemistry and astronomy/cosmology science. Contrary to more general Riemannian manifolds, SO(3) admits a tractable solution to heat diffusion, and allows us to implement efficient training of diffusion models. We apply both SO(3) DDPMs and SGMs to synthetic densities on SO(3) and demonstrate state-of-the-art results. Additionally, we demonstrate the practicality of our model on pose estimation tasks and in predicting correlated galaxy orientations for astrophysics/cosmology.

Towards Robust Image Stitching: An Adaptive Resistance Learning against Compatible Attacks

Image stitching seamlessly integrates images captured from varying perspectives into a single wide field-of-view image. Such integration not only broadens the captured scene but also augments holistic perception in computer vision applications. Given a pair of captured images, subtle perturbations and distortions which go unnoticed by the human visual system tend to attack the correspondence matching, impairing the performance of image stitching algorithms. In light of this challenge, this paper presents the first attempt to improve the robustness of image stitching against adversarial attacks. Specifically, we introduce a stitching-oriented attack (SoA), tailored to amplify the alignment loss within overlapping regions, thereby targeting the feature matching procedure. To establish an attack resistant model, we delve into the robustness of stitching architecture and develop an adaptive adversarial training (AAT) to balance attack resistance with stitching precision. In this way, we relieve the gap between the routine adversarial training and benign models, ensuring resilience without quality compromise. Comprehensive evaluation across real-world and synthetic datasets validate the deterioration of SoA on stitching performance. Furthermore, AAT emerges as a more robust solution against adversarial perturbations, delivering superior stitching results. Code is available at: https://github.com/Jzy2017/TRIS.

Make Lossy Compression Meaningful for Low-Light Images

Low-light images frequently occur due to unavoidable environmental influences or technical limitations, such as insufficient lighting or limited exposure time. To achieve better visibility for visual perception, low-light image enhancement is usually adopted. Besides, lossy image compression is vital for meeting the requirements of storage and transmission in computer vision applications. To touch the above two practical demands, current solutions can be categorized into two sequential manners: ``Compress before Enhance (CbE)'' or ``Enhance before Compress (EbC)''. However, both of them are not suitable since: (1) Error accumulation in the individual models plagues sequential solutions. Especially, once low-light images are compressed by existing general lossy image compression approaches, useful information (e.g., texture details) would be lost resulting in a dramatic performance decrease in low-light image enhancement. (2) Due to the intermediate process, the sequential solution introduces an additional burden resulting in low efficiency. We propose a novel joint solution to simultaneously achieve a high compression rate and good enhancement performance for low-light images with much lower computational cost and fewer model parameters. We design an end-to-end trainable architecture, which includes the main enhancement branch and the signal-to-noise ratio (SNR) aware branch. Experimental results show that our proposed joint solution achieves a significant improvement over different combinations of existing state-of-the-art sequential ``Compress before Enhance'' or ``Enhance before Compress'' solutions for low-light images, which would make lossy low-light image compression more meaningful. The project is publicly available at: https://github.com/CaiShilv/Joint-IC-LL.

GLDL: Graph Label Distribution Learning

Label Distribution Learning (LDL), as a more general learning setting than generic single-label and multi-label learning, has been commonly used in computer vision and many other applications. To date, existing LDL approaches are designed and applied to data without considering the interdependence between instances. In this paper, we propose a Graph Label Distribution Learning (GLDL) framework, which explicitly models three types of relationships: instance-instance, label-label, and instance-label, to learn the label distribution for networked data. A label-label network is learned to capture label-to-label correlation, through which GLDL can accurately learn label distributions for nodes. Dual graph convolution network (GCN) Co-training with heterogeneous message passing ensures two GCNs, one focusing on instance-instance relationship and the other one targeting label-label correlation, are jointly trained such that instance-instance relationship can help induce label-label correlation and vice versa. Our theoretical study derives the error bound of GLDL. For verification, four benchmark datasets with label distributions for nodes are created using common graph benchmarks. The experiments show that considering dependency helps learn better label distributions for networked data, compared to state-of-the-art LDL baseline. In addition, GLDL not only outperforms simple GCN and graph attention networks (GAT) using distribution loss but is also superior to its variant considering label-label relationship as a static network. GLDL and its benchmarks are the first research endeavors to address LDL for graphs. Code and benchmark data are released for public access.

BadSAM: Exploring Security Vulnerabilities of SAM via Backdoor Attacks (Student Abstract)

Image segmentation is foundational to computer vision applications, and the Segment Anything Model (SAM) has become a leading base model for these tasks. However, SAM falters in specialized downstream challenges, leading to various customized SAM models. We introduce BadSAM, a backdoor attack tailored for SAM, revealing that customized models can harbor malicious behaviors. Using the CAMO dataset, we confirm BadSAM's efficacy and identify SAM vulnerabilities. This study paves the way for the development of more secure and customizable vision foundation models.

Finding Visual Saliency in Continuous Spike Stream

As a bio-inspired vision sensor, the spike camera emulates the operational principles of the fovea, a compact retinal region, by employing spike discharges to encode the accumulation of per-pixel luminance intensity. Leveraging its high temporal resolution and bio-inspired neuromorphic design, the spike camera holds significant promise for advancing computer vision applications. Saliency detection mimic the behavior of human beings and capture the most salient region from the scenes. In this paper, we investigate the visual saliency in the continuous spike stream for the first time. To effectively process the binary spike stream, we propose a Recurrent Spiking Transformer (RST) framework, which is based on a full spiking neural network. Our framework enables the extraction of spatio-temporal features from the continuous spatio-temporal spike stream while maintaining low power consumption. To facilitate the training and validation of our proposed model, we build a comprehensive real-world spike-based visual saliency dataset, enriched with numerous light conditions. Extensive experiments demonstrate the superior performance of our Recurrent Spiking Transformer framework in comparison to other spike neural network-based methods. Our framework exhibits a substantial margin of improvement in capturing and highlighting visual saliency in the spike stream, which not only provides a new perspective for spike-based saliency segmentation but also shows a new paradigm for full SNN-based transformer models. The code and dataset are available at https://github.com/BIT-Vision/SVS.

Sparse Resultant-Based Minimal Solvers in Computer Vision and Their Connection with the Action Matrix

Many computer vision applications require robust and efficient estimation of camera geometry from a minimal number of input data measurements. Minimal problems are usually formulated as complex systems of sparse polynomial equations. The systems usually are overdetermined and consist of polynomials with algebraically constrained coefficients. Most state-of-the-art efficient polynomial solvers are based on the action matrix method that has been automated and highly optimized in recent years. On the other hand, the alternative theory of sparse resultants based on the Newton polytopes has not been used so often for generating efficient solvers, primarily because the polytopes do not respect the constraints amongst the coefficients. In an attempt to tackle this challenge, here we propose a simple iterative scheme to test various subsets of the Newton polytopes and search for the most efficient solver. Moreover, we propose to use an extra polynomial with a special form to further improve the solver efficiency via Schur complement computation. We show that for some camera geometry problems our resultant-based method leads to smaller and more stable solvers than the state-of-the-art Gröbner basis-based solvers, while being significantly smaller than the state-of-the-art resultant-based methods. The proposed method can be fully automated and incorporated into existing tools for the automatic generation of efficient polynomial solvers. It provides a competitive alternative to popular Gröbner basis-based methods for minimal problems in computer vision. Additionally, we study the conditions under which the minimal solvers generated by the state-of-the-art action matrix-based methods and the proposed extra polynomial resultant-based method, are equivalent. Specifically, we consider a step-by-step comparison between the approaches based on the action matrix and the sparse resultant, followed by a set of substitutions, which would lead to equivalent minimal solvers.

A two-stream conditional generative adversarial network for improving semantic predictions in urban driving scenes

Semantic segmentation is a well-studied topic and one of the most challenging tasks in computer vision applications, such as autonomous driving. Deep learning approaches based on convolutional neural networks (CNN) have demonstrated exceptional success on this task in recent years. Despite this success, existing approaches are plagued by higher-order inconsistencies between the ground truth images and the ones predicted by the segmentation model. This paper proposes a novel post-processing scheme based on adversarial learning to counter these inconsistencies. Such a scheme can be combined with a variety of existing CNN-based semantic segmentation networks to improve their segmentation performances. The proposed scheme is a Two-Stream Conditional Generative Adversarial Network (TScGAN), with one stream having initial semantic segmentation masks predicted by an existing CNN, while the other stream utilizes scene images to retain high-level information under a supervised residual network structure. In addition, TScGAN incorporates a novel dynamic weighting mechanism, which leads to significant and consistent gains in segmentation performance. Several comparative tests on public benchmark driving databases, including Cityscapes, Mapillary, and Berkeley DeepDrive100K, demonstrate the effectiveness of the proposed method when used with state-of-the-art CNN-based semantic segmentation models. Furthermore, the ablation experiment proved the structural rationality of our two-stream structure. The code for TScGAN could be found at https://github.com/epan-utbm/TScGAN-for-Improving-Semantic-Predictions

Irregular boundaries stereo images dataset creating using depth estimation model

AbstractThis paper introduces a stereoscopic image and depth dataset created using a deep learning model. It addresses the challenge of obtaining accurate and annotated stereo image pairs with irregular boundaries for deep learning model training. Stereoscopic image and depth dataset provides a unique resource for training deep learning models to handle irregular boundary stereoscopic images, which are valuable for real-world scenarios with complex shapes or occlusions. The dataset is created using monocular depth estimation, a state-of-the-art depth estimation model, and it can be used in applications like rectifying images, estimating depth, detecting objects, and autonomous driving. Overall, this paper presents a novel dataset that demonstrates its effectiveness and potential for advancing stereo vision and developing deep learning models for computer vision applications.

Modelling flame-to-fuel heat transfer by deep learning and fire images

In numerical fire simulations, the calculation of thermal feedback from the flame to the solid and liquid fuel surface plays a critical role as it connects the fundamental gas-phase flame burning and condensed-phase fuel gasification. However, it is a computationally intensive task in CFD fire modelling methods because of the requirement of a high-resolution grid for calculating the interface heat transfer. This paper proposed a real-time prediction of the flame-to-fuel heat transfer by using simulated flame images and a computer-vision deep learning method. Different methanol pool fires were selected to produce the image database for training the model. As the pool diameters increase from 20 to 40 cm, the dominant flame-to-fuel heat transfer shifts from convection to radiation. Results show that the proposed AI algorithm trained by flame images can predict both the convective and radiative heat flux distributions on the condensed fuel surface with a relative error below 20%, based on the input of real-time flame morphology that can be captured by a larger grid size. Regardless of growing or decaying fires or puffing flames induced by buoyancy, this method can further predict the non-uniform distribution of heat transfer coefficient on the interface rather than using empirical correlations. This work demonstrates the use of AI and computer vision in accelerating numerical fire simulation, which helps simulate complex fire behaviours with simpler models and smaller computational costs. Highlights A total framework between AI model and fire simulation software is designed to further enhance the reliability of AI-based fire simulations. A standard pool fire simulation database is built using numerical model recommend by the IAFSS Computation Group. A deep learning model is developed to predict both the convective and radiative heat flux distributions on the condensed fuel surface using numerical images database. The demonstration showcases the application of AI and computer vision to accelerate numerical fire simulation.

Edge Boosted Global Awared Low-light Image Enhancement Network

Low-light images are captured in situations where the lighting is poor or the camera hardware is not capable of producing good quality images. These types of images tend to have low contrast, blurry details, noise, and color distortion. In computer vision applications, image brightness plays a crucial role, and therefore, low-light image enhancement is used as a preprocessing step. In this study, we have improved the Low-Light Enhancement Network with Global Awareness (GLADNet) method by adding a UNet-based edge information extraction unit. The channel attention mechanism was also incorporated into the edge information extraction unit to achieve color preservation. Our experiments show that our proposed method has achieved higher PSNR, SSIM, and FSIM metrics compared to reference images. Additionally, it has produced lower NIQE and BRISQUE values for non-reference performance evaluation. Moreover, our proposed method removes noise better and produces visual results that are closer to the target images.

Visual tracking in camera-switching outdoor sport videos: Benchmark and baselines for skiing

Skiing is a globally popular winter sport discipline with a rich history of competitive events. This domain offers ample opportunities for the application of computer vision to enhance the understanding of athletes’ performances. However, this potential has remained relatively untapped in comparison to other sports, primarily due to the limited availability of dedicated research studies and datasets. The present paper takes a significant stride towards bridging these gaps. It conducts a comprehensive examination of skier appearance tracking in videos capturing their entire performance—an essential step for more advanced performance analyses. To implement this investigation, we introduce SkiTB, the largest and most annotated dataset tailored for computer vision applications in skiing. We subject a range of visual object tracking algorithms to rigorous testing, including both well-established methodologies and a novel skier-specific baseline algorithm. The results yield valuable insights into the suitability of various tracking techniques for vision-based skiing analysis and into the generalization of state-of-the-art algorithms to complex target behaviors and conditions set by winter outdoor environments. To foster further development, we make SkiTB, the associated code, and the obtained results accessible through https://machinelearning.uniud.it/datasets/skitb.

Visual resource extraction and artistic communication model design based on improved CycleGAN algorithm.

Through the application of computer vision and deep learning methodologies, real-time style transfer of images becomes achievable. This process involves the fusion of diverse artistic elements into a single image, resulting in the creation of innovative pieces of art. This article centers its focus on image style transfer within the realm of art education and introduces an ATT-CycleGAN model enriched with an attention mechanism to enhance the quality and precision of style conversion. The framework enhances the generators within CycleGAN. At first, images undergo encoder downsampling before entering the intermediate transformation model. In this intermediate transformation model, feature maps are acquired through four encoding residual blocks, which are subsequently input into an attention module. Channel attention is incorporated through multi-weight optimization achieved via global max-pooling and global average-pooling techniques. During the model's training process, transfer learning techniques are employed to improve model parameter initialization, enhancing training efficiency. Experimental results demonstrate the superior performance of the proposed model in image style transfer across various categories. In comparison to the traditional CycleGAN model, it exhibits a notable increase in structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) metrics. Specifically, on the Places365 and selfi2anime datasets, compared with the traditional CycleGAN model, SSIM is increased by 3.19% and 1.31% respectively, and PSNR is increased by 10.16% and 5.02% respectively. These findings provide valuable algorithmic support and crucial references for future research in the fields of art education, image segmentation, and style transfer.

FPGA and computer-vision-based atom tracking technology for scanning probe microscopy

Atom tracking technology enhanced with innovative algorithms has been implemented in this study, utilizing a comprehensive suite of controllers and software independently developed domestically. Leveraging an on-board field-programmable gate array (FPGA) with a core frequency of 100 MHz, our system facilitates reading and writing operations across 16 channels, performing discrete incremental proportional-integral-derivative (PID) calculations within 3.4 microseconds. Building upon this foundation, gradient and extremum algorithms are further integrated, incorporating circular and spiral scanning modes with a horizontal movement accuracy of 0.38 pm. This integration enhances the real-time performance and significantly increases the accuracy of atom tracking. Atom tracking achieves an equivalent precision of at least 142 pm on a highly oriented pyrolytic graphite (HOPG) surface under room temperature atmospheric conditions. Through applying computer vision and image processing algorithms, atom tracking can be used when scanning a large area. The techniques primarily consist of two algorithms: the region of interest (ROI)-based feature matching algorithm, which achieves 97.92% accuracy, and the feature description-based matching algorithm, with an impressive 99.99% accuracy. Both implementation approaches have been tested for scanner drift measurements, and these technologies are scalable and applicable in various domains of scanning probe microscopy with broad application prospects in the field of nanoengineering.