Computer Vision Research Research Articles

Bioinspired Garra Rufa Optimization-Assisted Deep Learning Model for Object Classification on Pedestrian Walkways.

Object detection in pedestrian walkways is a crucial area of research that is widely used to improve the safety of pedestrians. It is not only challenging but also a tedious process to manually examine the labeling of abnormal actions, owing to its broad applications in video surveillance systems and the larger number of videos captured. Thus, an automatic surveillance system that identifies the anomalies has become indispensable for computer vision (CV) researcher workers. The recent advancements in deep learning (DL) algorithms have attracted wide attention for CV processes such as object detection and object classification based on supervised learning that requires labels. The current research study designs the bioinspired Garra rufa optimization-assisted deep learning model for object classification (BGRODL-OC) technique on pedestrian walkways. The objective of the BGRODL-OC technique is to recognize the presence of pedestrians and objects in the surveillance video. To achieve this goal, the BGRODL-OC technique primarily applies the GhostNet feature extractors to produce a set of feature vectors. In addition to this, the BGRODL-OC technique makes use of the GRO algorithm for hyperparameter tuning process. Finally, the object classification is performed via the attention-based long short-term memory (ALSTM) network. A wide range of experimental analysis was conducted to validate the superior performance of the BGRODL-OC technique. The experimental values established the superior performance of the BGRODL-OC algorithm over other existing approaches.

DALib: A Curated Repository of Libraries for Data Augmentation in Computer Vision.

Data augmentation is a fundamental technique in machine learning that plays a crucial role in expanding the size of training datasets. By applying various transformations or modifications to existing data, data augmentation enhances the generalization and robustness of machine learning models. In recent years, the development of several libraries has simplified the utilization of diverse data augmentation strategies across different tasks. This paper focuses on the exploration of the most widely adopted libraries specifically designed for data augmentation in computer vision tasks. Here, we aim to provide a comprehensive survey of publicly available data augmentation libraries, facilitating practitioners to navigate these resources effectively. Through a curated taxonomy, we present an organized classification of the different approaches employed by these libraries, along with accompanying application examples. By examining the techniques of each library, practitioners can make informed decisions in selecting the most suitable augmentation techniques for their computer vision projects. To ensure the accessibility of this valuable information, a dedicated public website named DALib has been created. This website serves as a centralized repository where the taxonomy, methods, and examples associated with the surveyed data augmentation libraries can be explored. By offering this comprehensive resource, we aim to empower practitioners and contribute to the advancement of computer vision research and applications through effective utilization of data augmentation techniques.

EVALUATING MONOCULAR DEPTH ESTIMATION METHODS

Abstract. Depth estimation from monocular images has become a prominent focus in photogrammetry and computer vision research. Monocular Depth Estimation (MDE), which involves determining depth from a single RGB image, offers numerous advantages, including applications in simultaneous localization and mapping (SLAM), scene comprehension, 3D modeling, robotics, and autonomous driving. Depth information retrieval becomes especially crucial in situations where other sources like stereo images, optical flow, or point clouds are not available. In contrast to traditional stereo or multi-view methods, MDE techniques require fewer computational resources and smaller datasets. This research work presents a comprehensive analysis and evaluation of some state-of-the-art MDE methods, considering their ability to infer depth information in terrestrial images. The evaluation includes quantitative assessments using ground truth data, including 3D analyses and inference time.

Recursive neural programs: A differentiable framework for learning compositional part-whole hierarchies and image grammars.

Human vision, thought, and planning involve parsing and representing objects and scenes using structured representations based on part-whole hierarchies. Computer vision and machine learning researchers have recently sought to emulate this capability using neural networks, but a generative model formulation has been lacking. Generative models that leverage compositionality, recursion, and part-whole hierarchies are thought to underlie human concept learning and the ability to construct and represent flexible mental concepts. We introduce Recursive Neural Programs (RNPs), a neural generative model that addresses the part-whole hierarchy learning problem by modeling images as hierarchical trees of probabilistic sensory-motor programs. These programs recursively reuse learned sensory-motor primitives to model an image within different spatial reference frames, enabling hierarchical composition of objects from parts and implementing a grammar for images. We show that RNPs can learn part-whole hierarchies for a variety of image datasets, allowing rich compositionality and intuitive parts-based explanations of objects. Our model also suggests a cognitive framework for understanding how human brains can potentially learn and represent concepts in terms of recursively defined primitives and their relations with each other.

Improving human–object interaction with auxiliary semantic information and enhanced instance representation

Human–Object Interaction (HOI) detection has garnered considerable attention among computer vision researchers as it involves identifying and describing actions between humans and objects. Numerous approaches, such as sequential and end-to-end methods, have been proposed to tackle this problem, with a recent focus on exploring end-to-end systems. This study presents an enhanced end-to-end transformer-based human–object detector based on HOTR, which introduces three improvements. The proposed model improves instance representation through a simple yet effective mechanism, utilizes semantic information to provide contextual understanding and additional knowledge, and incorporates a cross-attention mechanism for fusing multi-level high-level feature maps within the Transformer architecture. Experimental results demonstrate significant performance gains over the baseline HOTR model, making it competitive with other state-of-the-art models on two widely-used datasets: V-COCO and HICO-DET.

CherryChèvre: A fine-grained dataset for goat detection in natural environments

We introduce a new dataset for goat detection that contains 6160 annotated images captured under varying environmental conditions. The dataset is intended for developing machine learning algorithms for goat detection, with applications in precision agriculture, animal welfare, behaviour analysis, and animal husbandry. The annotations were performed by expert in computer vision, ensuring high accuracy and consistency. The dataset is publicly available and can be used as a benchmark for evaluating existing algorithms. This dataset advances research in computer vision for agriculture.

Badminton video action recognition based on time network

With the continuous development of artificial intelligence research, computer vision research has shifted from traditional “feature engineering”-based methods to deep learning-based “network engineering” methods, which automatically extracts and classifies features by using deep neural networks. Traditional methods based on artificial design features are computationally expensive and are usually used to solve simple research problems, which is not conducive for large-scale data feature extraction. Deep learning-based methods greatly reduce the difficulty of artificial features by learning features from large-scale data and are successfully applied in many visual recognition tasks. Video action recognition methods also shift from traditional methods based on artificial design features to deep learning-based methods, which is oriented to building more effective deep neural network models. Through collecting and sorting related research results found that academic for timing segment network of football and basketball video action research is relatively rich, but lack of badminton research given the above research results, this study based on timing segment network of badminton video action identification can enrich the research results, provide reference for follow-up research. This paper introduces the lightweight attention mechanism into the temporal segmentation network, forming the attention mechanism-timing segmentation network, and trains the neural network to get the classifier of badminton stroke action, which can be predicted as four common types: forehand stroke, backhand stroke, overhead stroke and pick ball. The experimental results show that the recognition recall and accuracy of various stroke movements reach more than 86%, and the average size of recall and accuracy is 91.2% and 91.6% respectively, indicating that the method based on timing segmentation network can be close to the human judgment level and can effectively conduct the identification task of badminton video strokes.

Applications of AI in Computer Vision and NLP

Computer vision and language AI (or speech AI) are two rapidly growing fields that have the potential to revolutionize the way when interacted with technology. When it comes to artificial intelligence, there are two main branches: computer vision, which focuses on visual data, and language AI (or voice AI), which focuses on both written and spoken language. The purpose of this research is to offer a snapshot of where things stand in this area right now, highlighting recent research and key findings in both computer vision and language AI (or speech AI). The literature review will focus on the most recent advancements and trends in these fields, as well as identify gaps in the literature that need further research.

Enhancing gland segmentation in colon histology images using an instance-aware diffusion model

In pathological image analysis, determination of gland morphology in histology images of the colon is essential to determine the grade of colon cancer. However, manual segmentation of glands is extremely challenging and there is a need to develop automatic methods for segmenting gland instances. Recently, due to the powerful noise-to-image denoising pipeline, the diffusion model has become one of the hot spots in computer vision research and has been explored in the field of image segmentation. In this paper, we propose an instance segmentation method based on the diffusion model that can perform automatic gland instance segmentation. Firstly, we model the instance segmentation process for colon histology images as a denoising process based on a diffusion model. Secondly, to recover details lost during denoising, we use Instance Aware Filters and multi-scale Mask Branch to construct global mask instead of predicting only local masks. Thirdly, to improve the distinction between the object and the background, we apply Conditional Encoding to enhance the intermediate features with the original image encoding. To objectively validate the proposed method, we compared several state-of-the-art deep learning models on the 2015 MICCAI Gland Segmentation challenge (GlaS) dataset (165 images), the Colorectal Adenocarcinoma Glands (CRAG) dataset (213 images) and the RINGS dataset (1500 images). Our proposed method obtains significantly improved results for CRAG (Object F1 0.853 ± 0.054, Object Dice 0.906 ± 0.043), GlaS Test A (Object F1 0.941 ± 0.039, Object Dice 0.939 ± 0.060), GlaS Test B (Object F1 0.893 ± 0.073, Object Dice 0.889 ± 0.069), and RINGS dataset (Precision 0.893 ± 0.096, Dice 0.904 ± 0.091). The experimental results show that our method significantly improves the segmentation accuracy, and the experiment results demonstrate the efficacy of the method.

Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective

Transferring knowledge from pre-trained deep models for downstream tasks, particularly with limited labeled samples, is a fundamental problem in computer vision research. Recent advances in large-scale, task-agnostic vision-language pre-trained models, which are learned with billions of samples, have shed new light on this problem. In this study, we investigate how to efficiently transfer aligned visual and textual knowledge for downstream visual recognition tasks. We first revisit the role of the linear classifier in the vanilla transfer learning framework, and then propose a new paradigm where the parameters of the classifier are initialized with semantic targets from the textual encoder and remain fixed during optimization. To provide a comparison, we also initialize the classifier with knowledge from various resources. In the empirical study, we demonstrate that our paradigm improves the performance and training speed of transfer learning tasks. With only minor modifications, our approach proves effective across 17 visual datasets that span three different data domains: image, video, and 3D point cloud.

Hierarchical painter: Chinese landscape painting restoration with fine-grained styles

Image inpainting is a critical area of research in computer vision with a broad range of applications, including image restoration and editing. However, current inpainting models often struggle to learn the specific painting styles and fine-grained brushstrokes of individual artists when restoring Chinese landscape paintings. To address this challenge, this paper proposes a novel inpainting model specifically designed for Chinese landscape paintings, featuring a hierarchical structure that can be applied to restore the famous Dwelling in the Fuchun Mountains with remarkable fidelity. The proposed method leverages an image processing algorithm to extract the structural information of Chinese landscape paintings. This approach enables the model to decompose the inpainting process into two separate steps, generating less informative backgrounds and more detailed foregrounds. By seamlessly merging the generated results with the remaining portions of the original work, the proposed method can faithfully restore Chinese landscape paintings while preserving their rich details and fine-grained styles. Overall, the results of this study demonstrate that the proposed method represents a significant step forward in the field of image inpainting, particularly for the restoration of Chinese landscape paintings. The hierarchical structure and image processing algorithm used in this model is able to faithfully restore delicate and intricate details of these paintings, making it a promising tool for art restoration professionals and researchers.

Computer Vision Driven Precision Dairy Farming for Efficient Cattle Management

Precision Dairy Farming (PDF)” or “The Per Animal Approach” can be enhanced through the implementation of three-dimensional computer vision, which offers improved cattle identification, disease monitoring, and growth assessment. The integration of 3D vision systems is particularly vital for advancing dairy farming practices in the next generation. These systems facilitate the automation of various animal husbandry tasks, including monitoring, herding, feeding, milking, and bedding of animals. The applications of 3D computer vision in PLF encompass diverse platforms, such as 3D camera installations for monitoring cow walking postures, and intelligent systems that interact safely with animals, capable of identifying dairy cattle and detecting health indicators like animal identification, recognition, body condition score, and lameness. To be effective, systems must be adaptable to unconstrained environments, varying herd characteristics, weather conditions, farmyard layouts, and animal-machine interaction scenarios. Considering these requirements, this paper proposes the application of emerging computer vision and artificial intelligence techniques in dairy farming. This review encourages future research in three-dimensional computer vision for cattle growth management and its potential extension to other livestock and wild animals

A Novel Probabilistic Network Model for Estimating Cognitive-Gait Connection Using Multimodal Interface

Research in human gait analysis has captivated several computer vision researchers to solve human identification problems. The proposed work provides a novel approach for cognitive state estimation via multi-modal analysis. The advantage of multi-modal system is to provide adequate motion signatures with ensemble of multimodal gait data to ensure data reliability for classification. The relationship between human cognitive states and gait is predicted using both temporal and non-temporal probabilistic models. We estimate prior probability tables for non-temporal probabilistic model known as Simple Bayesian Model after analysing the data acquired from Inertial Measurement Unit (IMU), Electroencephalography (EEG) and multiple Kinect V2.0 sensors. A novel Dynamic Bayesian Network (DBN) is used as probabilistic temporal model for estimating most probable sequence of transitions among human cognitive states. We apply Gaussian Mixture Modelling with Expectation maximisation (GMM-EM) to tune transition and emission probabilities for maximizing the probability of the observed sequence. A promising estimation accuracy of 88.6% is obtained. Also, Principal Component Analysis (PCA) and k-nearest neighbour (kNN) algorithms are applied separately to calculate the input probabilities for DBN model. It is observed that both GMM-EM and k-NN based approaches outperform all the state-of-the-art techniques. Moreover, standard statistical tests are performed on acquired dataset to validate the experimental results.

Fake Face Detection Based on Colour Textual Analysis Using Deep Convolutional Neural Network

Detecting fake faces has become a crucial endeavour within the realm of computer vision. The widespread availability of digital media has facilitated the creation and dissemination of deceptive and misleading content. A prominent strategy for identifying counterfeit faces employs advanced deep-learning methodologies that scrutinise both colour and textural attributes. This investigation is geared towards devising a method for discerning fake faces by leveraging the capabilities of convolutional neural networks (CNNs). These networks are trained to discriminate between authentic and forged images by discerning nuances in their colour characteristics. To achieve this, the MSU MFSD dataset will be harnessed, allowing for exploring colour textures and extracting facial traits across diverse colour channels, including RGB, HSV, and YCbCr.The proposed framework marks a notable stride in the realm of computer vision research, particularly given the prevalent employment of digital media, which has eased the generation and distribution of misleading or deceitful content. Developing dependable systems for identifying counterfeit faces holds immense potential in curtailing the proliferation of false information and upholding the integrity of digital media platforms.

A Deep Learning Approach for Human Facial Expression Recognition using Residual Network – 101

Emotion recognition is a dynamic process that focuses on a person's emotional state, which implies that the emotions associated with each individual's activities are unique. Human emotion analysis and recognition have been popular study areas among computer vision researchers. High dimensionality, execution time, and cost are the main difficulties in human emotion detection. To deal with these issues, the proposed model aims to design a human emotion recognition model using Residual Networks-101 (ResNet-101). A Convolutional Neural Network (CNN) design called ResNet-101 solves the vanishing gradient issue and makes it possible to build networks with thousands of convolutional layers that outperform networks with fewer layers. An image dataset was used for this emotion recognition. Then, this image dataset was subjected to preprocessing to resize the image and eliminate the noise contents present in the images. After preprocessing, the image was given to the classifier to recognize the emotions effectively. Here, ResNet-101 was used for the classification of six classes. The experimental results demonstrate that ResNet-101 models outperform the most recent techniques for emotion recognition. The proposed model was executed in MATLAB software and carried out several performance metrics. The proposed architecture attained better performance in terms of accuracy 92% and error with 0.08 and other performances like 92% of precision, 85% of specificity and 98% of sensitivity so on, and this shows the effectiveness of the proposed model to existing approaches such as LeNet, AlexNet and VGG. In comparison to current techniques, the suggested model provides improved recognition accuracy for low intensity or mild emotional expressions.

A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition

In recent years, facial expression recognition (FER) has garnered significant attention within the realm of computer vision research. This paper presents an innovative network called the Dual-Direction Attention Mixed Feature Network (DDAMFN) specifically designed for FER, boasting both robustness and lightweight characteristics. The network architecture comprises two primary components: the Mixed Feature Network (MFN) serving as the backbone, and the Dual-Direction Attention Network (DDAN) functioning as the head. To enhance the network’s capability in the MFN, resilient features are extracted by utilizing mixed-size kernels. Additionally, a new Dual-Direction Attention (DDA) head that generates attention maps in two orientations is proposed, enabling the model to capture long-range dependencies effectively. To further improve the accuracy, a novel attention loss mechanism for the DDAN is introduced with different heads focusing on distinct areas of the input. Experimental evaluations on several widely used public datasets, including AffectNet, RAF-DB, and FERPlus, demonstrate the superiority of the DDAMFN compared to other existing models, which establishes that the DDAMFN as the state-of-the-art model in the field of FER.

A Benchmark for Maritime Object Detection with Centernet on an Improved Dataset, ABOships-PLUS

Object detection from waterborne imagery is an essential aspect in maritime traffic management, navigation safety and coastal security. Building efficient autonomous systems, which can take decisions in critical situations, requires an integration of complex object detectors in real time. Object detectors trained on generic datasets often give unsatisfactory results in complex scenarios like the maritime environment, since only a fraction of their images contain maritime vessels. Publicly available domain-specific datasets are scarce, and they are limited in the number of images and annotations. Compared to object detection in applications such as autonomous vehicles, maritime vessel detection is considerably reduced in computer vision research. This creates a deficit in exhaustive benchmarking studies for maritime detection datasets. To bridge this gap, we relabel the ABOships dataset and benchmark a state-of-the-art center-based detector, Centernet, on the newly relabeled dataset, ABOships-PLUS. We explore its performance under different feature extractors, and investigate the effect of object size and inter-class variation on detection accuracy. The reported benchmarking illustrates that the ABOships-PLUS dataset is adequate to use in supervised domain adaptation. The experimental results show that Centernet with DLA (Deep Layer Aggregation) as a feature extractor achieved the highest accuracy in detecting maritime objects overall (with mean average precision 74.4%).

A survey of text detection and recognition algorithms based on deep learning technology

Optical Character Recognition (OCR) poses a crucial challenge within the realm of computer vision research, as it plays a pivotal role in converting vast amounts of unstructured text data into structured formats to support diverse artificial intelligence applications. The OCR process encompasses two core components: text detection and text recognition. Text detection involves identifying and extracting text regions, achieved through either object detection or segmentation techniques, while text recognition focuses on accurately deciphering the content within these identified regions. In recent years, remarkable strides have been made in the domain of text recognition, primarily driven by deep learning-based models. These models eliminate the need for manual feature processing and excel in recognizing text even within complex scenes, surpassing the performance of traditional text recognition methods and subsequently emerging as the dominant approach. The objective of this paper is to present a comprehensive survey of both text detection and text recognition models. Firstly, we systematically categorize and provide an overview of existing off-the-shelf text detection methods. Subsequently, we conduct an in-depth investigation of six distinct text recognition models, taking into account their unique implementations. Additionally, we explore and analyze the principal datasets that currently prevail in the field of text detection and recognition. Furthermore, this research entails a meticulous performance comparison of various text detection algorithms on the CTW1500, TotalText, and ICDAR2015 datasets. Additionally, we evaluate and scrutinize the efficacy of mainstream text recognition algorithms on the IIIT-5K, SVT, ICDAR2013, SVT-P, CUTE80, and ICDAR2015 datasets. Finally, we conclude with a discussion on the future development and research trends concerning text detection and recognition, providing insights that can further drive progress in this crucial area.

Research on an Underwater Object Detection Network Based on Dual-Branch Feature Extraction

Underwater object detection is challenging in computer vision research due to the complex underwater environment, poor image quality, and varying target scales, making it difficult for existing object detection networks to achieve high accuracy in underwater tasks. To address the issues of limited data and multi-scale targets in underwater detection, we propose a Dual-Branch Underwater Object Detection Network (DB-UODN) based on dual-branch feature extraction. In the feature extraction stage, we design a dual-branch structure by combining the You Only Look Once (YOLO) v7 backbone with the Enhanced Channel and Dilated Block (ECDB). It allows for the extraction and complementation of multi-scale features, which enable the model to learn both global and local information and enhance its perception of multi-scale features in underwater targets. Furthermore, we employ the DSPACSPC structure to replace the SPPCSPC structure in YOLOv7. The DSPACSPC structure utilizes atrous convolutions with different dilation rates to capture contextual information at various scales, compensating for potential information loss caused by pooling operations. Additionally, we utilize a dense connection structure to facilitate feature reuse and enhance the network’s representation and generalization capabilities. Experimental results demonstrate that the proposed DB-UODN outperforms the most commonly used object detection networks in underwater scenarios. On the URPC2020 dataset, the network achieves an average detection accuracy of 87.36%.

MSPIF: Multi-stage progressive visible and infrared image fusion with structures preservation

Infrared and visible image fusion is one of the hottest research in computer vision to improve image quality. Traditional fusion methods suffer from detail loss, low resolution, and application scenarios limitation. To address these problems, this paper proposes a multi-stage progressive visible and infrared image fusion strategy (MSPIF), including the following key stages: Firstly, the visible image is enhanced using a weighted fusion visible image enhancement algorithm. Secondly, the infrared and enhanced visible images are both decomposed by a pre-trained network, namely Retinex_Net, to obtain their respective illumination and reflectance components. Thirdly, the reflectance components are decomposed by discrete wavelet transform (DWT). The low-frequency components are fused by a weighted information entropy fusion strategy while the high-frequency components are fused by a local energy fusion strategy. Such two fused parts contribute to the final fused reflectance component with inverse DWT (IDWT). Meanwhile, the illumination components are fused using a multi-scale fusion strategy based on the Laplace pyramid. Finally, the improved result is generated by the fused reflectance and illumination components based on Retinex theory. Quantitative and qualitative results of experiments on the TNO, KAIST, and ROAD datasets show that the proposed MSPIF is effective and achieves good results with structures preservation, even is superior to existing methods.