Computer Vision Research Articles

Efficient Visual Computing With Camera RAW Snapshots.

Conventional cameras capture image irradiance (RAW) on a sensor and convert it to RGB images using an image signal processor (ISP). The images can then be used for photography or visual computing tasks in a variety of applications, such as public safety surveillance and autonomous driving. One can argue that since RAW images contain all the captured information, the conversion of RAW to RGB using an ISP is not necessary for visual computing. In this paper, we propose a novel ρ-Vision framework to perform high-level semantic understanding and low-level compression using RAW images without the ISP subsystem used for decades. Considering the scarcity of available RAW image datasets, we first develop an unpaired CycleR2R network based on unsupervised CycleGAN to train modular unrolled ISP and inverse ISP (invISP) models using unpaired RAW and RGB images. We can then flexibly generate simulated RAW images (simRAW) using any existing RGB image dataset and finetune different models originally trained in the RGB domain to process real-world camera RAW images. We demonstrate object detection and image compression capabilities in RAW-domain using RAW-domain YOLOv3 and RAW image compressor (RIC) on camera snapshots. Quantitative results reveal that RAW-domain task inference provides better detection accuracy and compression efficiency compared to that in the RGB domain. Furthermore, the proposed ρ-Vision generalizes across various camera sensors and different task-specific models. An added benefit of employing the ρ-Vision is the elimination of the need for ISP, leading to potential reductions in computations and processing times.

Understanding the Learnability of a Novel Mobile App for Personalised Food Guidance Leveraging Persuasive Technology and Computer Vision: A Mixed-Methods Study

Understanding the Learnability of a Novel Mobile App for Personalised Food Guidance Leveraging Persuasive Technology and Computer Vision: A Mixed-Methods Study

Face Recognition Using LBPH and CNN

Objective:: The purpose of this paper was to use Machine Learning (ML) techniques to extract facial features from images. Accurate face detection and recognition has long been a problem in computer vision. According to a recent study, Local Binary Pattern (LBP) is a superior facial descriptor for face recognition. A person's face may make their identity, feelings, and ideas more obvious. In the modern world, everyone wants to feel secure from unauthorized authentication. Face detection and recognition help increase security; however, the most difficult challenge is to accurately recognise faces without creating any false identities. Methods:: The proposed method uses a Local Binary Pattern Histogram (LBPH) and Convolution Neural Network (CNN) to preprocess face images with equalized histograms. Results:: LBPH in the proposed technique is used to extract and join the histogram values into a single vector. The technique has been found to result in a reduction in training loss and an increase in validation accuracy of over 96.5%. Prior algorithms have been reported with lower accuracy when compared to LBPH using CNN. Conclusion:: This study demonstrates how studying characteristics produces more precise results, as the number of epochs increases. By comparing facial similarities, the vector has generated the best result.

Enhancing Image Captioning Using Deep Convolutional Generative Adversarial Networks

Introduction:: Introduction: Image caption generation has long been a fundamental challenge in the area of computer vision (CV) and natural language processing (NLP). In this research, we present an innovative approach that harnesses the power of Deep Convolutional Generative Adversarial Networks (DCGAN) and adversarial training to revolutionize the generation of natural and contextually relevant image captions. Method:: Our method significantly improves the fluency, coherence, and contextual relevance of generated captions and showcases the effectiveness of RL reward-based fine-tuning. Through a comprehensive evaluation of COCO datasets, our model demonstrates superior performance over baseline and state-of-the-art methods. On the COCO dataset, our model outperforms current state-of-the-art (SOTA) models across all metrics, achieving BLEU-4 (0.327), METEOR (0.249), Rough (0.525) and CIDEr (1.155) scores. Result:: The integration of DCGAN and adversarial training opens new possibilities in image captioning, with applications spanning from automated content generation to enhanced accessibility solutions. Conclusion:: This research paves the way for more intelligent and context-aware image understanding systems, promising exciting future exploration and innovation prospects.

PlaceFormer: Transformer-Based Visual Place Recognition Using Multi-Scale Patch Selection and Fusion

PlaceFormer: Transformer-Based Visual Place Recognition Using Multi-Scale Patch Selection and Fusion

Efficient Saliency Encoding for Visual Place Recognition: Introducing the Lightweight Pooling-Centric Saliency-Aware VPR Method

Efficient Saliency Encoding for Visual Place Recognition: Introducing the Lightweight Pooling-Centric Saliency-Aware VPR Method

Learnable Graph Matching: A Practical Paradigm for Data Association.

Data association is at the core of many computer vision tasks, e.g., multiple object tracking, image matching, and point cloud registration. however, current data association solutions have some defects: they mostly ignore the intra-view context information; besides, they either train deep association models in an end-to-end way and hardly utilize the advantage of optimization-based assignment methods, or only use an off-the-shelf neural network to extract features. In this paper, we propose a general learnable graph matching method to address these issues. Especially, we model the intra-view relationships as an undirected graph. Then data association turns into a general graph matching problem between graphs. Furthermore, to make optimization end-to-end differentiable, we relax the original graph matching problem into continuous quadratic programming and then incorporate training into a deep graph neural network with KKT conditions and implicit function theorem. In MOT task, our method achieves state-of-the-art performance on several MOT datasets. For image matching, our method outperforms state-of-the-art methods on a popular indoor dataset, ScanNet. For point cloud registration, we also achieve competitive results.

Learning With Constraint Learning: New Perspective, Solution Strategy and Various Applications.

The complexity of learning problems, such as Generative Adversarial Network (GAN) and its variants, multi-task and meta-learning, hyper-parameter learning, and a variety of real-world vision applications, demands a deeper understanding of their underlying coupling mechanisms. Existing approaches often address these problems in isolation, lacking a unified perspective that can reveal commonalities and enable effective solutions. Therefore, in this work, we proposed a new framework, named Learning with Constraint Learning (LwCL), that can holistically examine challenges and provide a unified methodology to tackle all the above-mentioned complex learning and vision problems. Specifically, LwCL is designed as a general hierarchical optimization model that captures the essence of these diverse learning and vision problems. Furthermore, we develop a gradient-response based fast solution strategy to overcome optimization challenges of the LwCL framework. Our proposed framework efficiently addresses a wide range of applications in learning and vision, encompassing three categories and nine different problem types. Extensive experiments on synthetic tasks and real-world applications verify the effectiveness of our approach. The LwCL framework offers a comprehensive solution for tackling complex machine learning and computer vision problems, bridging the gap between theory and practice.

Automated cooling tower detection through deep learning for Legionnaires’ disease outbreak investigations: a model development and validation study

Cooling towers containing Legionella spp are a high-risk source of Legionnaires' disease outbreaks. Manually locating cooling towers from aerial imagery during outbreak investigations requires expertise, is labour intensive, and can be prone to errors. We aimed to train a deep learning computer vision model to automatically detect cooling towers that are aerially visible. Between Jan 1 and 31, 2021, we extracted satellite view images of Philadelphia (PN, USA) and New York state (NY, USA) from Google Maps and annotated cooling towers to create training datasets. We augmented training data with synthetic data and model-assisted labelling of additional cities. Using 2051 images containing 7292 cooling towers, we trained a two-stage model using YOLOv5, a model that detects objects in images, and EfficientNet-b5, a model that classifies images. We assessed the primary outcomes of sensitivity and positive predictive value (PPV) of the model against manual labelling on test datasets of 548 images, including from two cities not seen in training (Boston [MA, USA] and Athens [GA, USA]). We compared the search speed of the model with that of manual searching by four epidemiologists. The model identified visible cooling towers with 95·1% sensitivity (95% CI 94·0-96·1) and a PPV of 90·1% (95% CI 90·0-90·2) in New York City and Philadelphia. In Boston, sensitivity was 91·6% (89·2-93·7) and PPV was 80·8% (80·5-81·2). In Athens, sensitivity was 86·9% (75·8-94·2) and PPV was 85·5% (84·2-86·7). For an area of New York City encompassing 45 blocks (0·26 square miles), the model searched more than 600 times faster (7·6 s; 351 potential cooling towers identified) than did human investigators (mean 83·75 min [SD 29·5]; mean 310·8 cooling towers [42·2]). The model could be used to accelerate investigation and source control during outbreaks of Legionnaires' disease through the identification of cooling towers from aerial imagery, potentially preventing additional disease spread. The model has already been used by public health teams for outbreak investigations and to initialise cooling tower registries, which are considered best practice for preventing and responding to outbreaks of Legionnaires' disease. None.

A Multi-Level Interpretable Sleep Stage Scoring System by Infusing Experts' Knowledge Into a Deep Network Architecture.

In recent years, deep learning has shown potential and efficiency in a wide area including computer vision, image and signal processing. Yet, translational challenges remain for user applications due to a lack of interpretability of algorithmic decisions and results. This black box problem is particularly problematic for high-risk applications such as medical-related decision-making. The current study goal was to design an interpretable deep learning system for time series classification of electroencephalogram (EEG) for sleep stage scoring as a step toward designing a transparent system. We have developed an interpretable deep neural network that includes a kernel-based layer guided by a set of principles used for sleep scoring by human experts in the visual analysis of polysomnographic records. A kernel-based convolutional layer was defined and used as the first layer of the system and made available for user interpretation. The trained system and its results were interpreted in four levels from microstructure of EEG signals, such as trained kernels and effect of each kernel on the detected stages, to macrostructures, such as transitions between stages. The proposed system demonstrated greater performance than prior studies and the system learned information consistent with expert knowledge.

Every Problem, Every Step, All in Focus: Learning to Solve Vision-Language Problems With Integrated Attention.

Integrating information from vision and language modalities has sparked interesting applications in the fields of computer vision and natural language processing. Existing methods, though promising in tasks like image captioning and visual question answering, face challenges in understanding real-life issues and offering step-by-step solutions. In particular, they typically limit their scope to solutions with a sequential structure, thus ignoring complex inter-step dependencies. To bridge this gap, we propose a graph-based approach to vision-language problem solving. It leverages a novel integrated attention mechanism that jointly considers the importance of features within each step as well as across multiple steps. Together with a graph neural network method, this attention mechanism can be progressively learned to predict sequential and non-sequential solution graphs depending on the characterization of the problem-solving process. To tightly couple attention with the problem-solving procedure, we further design new learning objectives with attention metrics that quantify this integrated attention, which better aligns visual and language information within steps, and more accurately captures information flow between steps. Experimental results on VisualHow, a comprehensive dataset of varying solution structures, show significant improvements in predicting steps and dependencies, demonstrating the effectiveness of our approach in tackling various vision-language problems.

CL-TransFER: Collaborative Learning based Transformer for Facial Expression Recognition with Masked Reconstruction

Facial expression recognition (FER) has attracted intensive attention due to its critical role in various computer vision tasks. However, existing FER approaches suffer from either noisy annotations or expression ambiguity (high inter-class and low intra-class similarity), limiting the FER performance. To this end, we propose a robust end-to-end collaborative learning based transformer for FER (CL-TransFER) in this paper. Specifically, CL-TransFER co-trains a CNN feature extractor and a transformer feature extractor jointly to extract both rich local semantic features as well as global structural information from facial images. By enforcing the consensus between the predictions of two extractors, the CL-TransFER could suppress the influence of noisy annotations. To further tackle the expression ambiguity problem, we design a simple yet efficient self-supervised masked reconstruction (SSMR) task to pre-train the transformer feature extractor of CL-TransFER. This enhances the model's capability of learning fine-grained discriminative representations. Extensive experiments on three popular benchmarks have demonstrated the effectiveness and superiority of our method.

Enhanced Medical Image Segmentation using Transfer Learning with Res101_UNet: Experimental Insights and Comparative Performance Analysis

Throughout the past few decades, artificial intelligence and machine learning have seen a lot of active research in areas such as computer vision, natural language processing, and speech processing. As a result, deep learning models became state-of-the-art for computer vision tasks such as object detection, classification, segmentation, and other allied tasks. Of course, the fruits of this research are extended to the design of robust and reliable digital health systems as well as other applications in the healthcare sector. Many clinical applications require the automatic segmentation of medical images. Recent deep learning-based approaches have demonstrated state-of-the-art performance in medical image segmentation tasks. In addition to their ability to automatically extract features and generalize over large amounts of data, transfer learning based deep learning models have proven to be handy for data scared areas like medical domains. In this research, we investigate and demonstrate the efficacy of a DCNN-based transfer learning model -Res101_Unet, which has been trained and/or fine-tuned to execute tumor tissue segmentation tasks in MRI, CT, PET, and X-RAY pictures of medical organ scans with little data. For our experimental study, we employed two image datasets: 'Liver Tumor' and 'Gland Colon Cancer', both obtained from the Kaggle portal. This experimental setup includes an Open-Source segmentation model API. Our findings indicate that domain similarity-based transfer learning can be used to data-scarce sectors. We achieved 98.47% accuracy and a IoU score of 0.9891 on Liver Tumor data and 0.6956 accuracy and a IoU score of 0.7043 on gland colon dataset.

Black and White Image Colorization with Deep Learning

Abstract: The project "Black and White Image Colorization with Deep Learning" develops a system to automatically infuse colors into grayscale images through cutting-edge deep learning approaches. Utilizing convolutional neural networks (CNNs) and generative adversarial networks (GANs), the system aims to deliver both realistic and visually pleasing colorization outcomes. This process includes training the deep learning models on diverse image datasets, optimizing the model parameters, and using advanced loss functions to ensure precise color representation.This project addresses the complex challenges associated with adding colors to monochrome images and also seeks to improve user experience by introducing customizable options such as style modification and color tweaks. By merging deep learning with computer vision techniques, this initiative is poised to contribute significantly to various image processing fields, especially in the restoration of historical images and the enhancement of visual content production. With thorough experimentation and ongoing refinements, the project aspires to establish a reliable and easy-to-use system that advances the state of image colorization technology in the modern digital age..

Lane Line Detection System for Vehicle Using Python

Abstract: This research presents a real-time collision detection and lane assistance system implemented using Python and OpenCV. The system leverages computer vision techniques to detect vehicles and lane lines, calculate distances to detected objects, and issue timely alerts to prevent collisions. The implementation aims to enhance driver safety by providing real-time feedback about the vehicle's surroundings. This paper details the system's design, methodology, and performance evaluation.

Weapon Detection Using Faster R-CNN

Abstract: As criminal activities continue to rise, ensuring security has become a top priority across various sectors. Computer vision technology is being extensively employed to address this problem by detecting and monitoring abnormalities. Video surveillance systems capable of identifying and analyzing scenes and detecting anomalous events have become increasingly essential to safeguard personal belongings, ensure safety, and enhance security. Such systems play a crucial role in intelligence monitoring. In this study, we implemented automatic weapon (or gun) detection using Faster RCNN techniques. Two datasets were utilized: one consisting of pre-labeled photos, and the other a collection of manually labeled images. Upon analyzing the data, both algorithms yielded highly precise outcomes. However, the practicality of these systems in real-world scenarios will ultimately depend on the trade-off between time and accuracy

Unveiling Anomalies in Surveillance Videos

Abstract: The implementation of Convolutional Neural Network (CNN) algorithm for the detection of anomalies like fire and potholes in images or videos. Leveraging deep learning technique, the model aims to accurately identify fire, potholes, etc. within various contexts. The CNN architecture is trained on a dataset comprising diverse images to enhance its ability to generalize. This proposed project not only addresses safety concerns by promoting helmet usage and proper maintenance of roads but also contributes to the broader field of computer vision and object detection. This implementation involves training the CNN on annotated datasets, fine-tuning the model for optimal performance, and integrating it into a practical application for efficient anomaly identification. With a focus on real-time detection, the system utilizes image processing techniques and a trained CNN model to analyze visual data from cameras or video feeds. The system's versatility allows integration into surveillance systems, industrial sites, or any environment where fire detection is crucial for safety. By automating the detection process, the project contributes to minimizing human error and ensuring consistent monitoring. The proposed solution holds the potential to significantly impact safety protocols, particularly in industries where protective headgear is paramount. Potholes pose a significant threat to both drivers and pedestrians, leading to accidents and infrastructure damage. The proposed solution leverages CNN's ability to effectively process visual data, making it well-suited for image recognition tasks. Our approach involves training the CNN on a diverse dataset of road images to enable it to accurately identify potholes. The model will learn distinctive features and patterns associated with potholes, allowing for robust detection under various environmental conditions. Real-time implementation on embedded systems or cameras along roadways will enable instantaneous identification and alerting.

MEASURING THICKNESS OF FLAX FIBRES IN A CLUSTER USING COMPUTER VISION

Authors analyse a method of indirect measurement of flax fibre linear density using its thickness, based on computer vision approach and “distance transform” algorithm in particular. A number of flaws of this method are identified. The main reason for the discrepancy is the effect of crossed fibres within the analysed sample. The secondary reason is the specific shape of fibre’s topological skeletonnear its tips. Based on these observations, a software model of the problem has been created. The software produces a simplified image of crossed fibres with specified properties, and measures this image using the above method. The effect of various factors on the discrepancy in the distribution of thickness values appears to match well with theoretical analysis. Additionally, a number of ways to improve the software model are outlined, related to how typicallyfibres are oriented within the analysed sample.

Development of the Anthropomorphic Arm for Collaborative and Home Service Robot CHARMIE

Service robots are rapidly transitioning from concept to reality, making significant strides in development. Similarly, the field of prosthetics is evolving at an impressive pace, with both areas now being highly relevant in the industry. Advancements in these fields are continually pushing the boundaries of what is possible, leading to the increasing creation of individual arm and hand prosthetics, either as standalone units or combined packages. This trend is driven by the rise of advanced collaborative robots that seamlessly integrate with human counterparts in real-world applications. This paper presents an open-source, 3D-printed robotic arm that has been assembled and programmed using two distinct approaches. The first approach involves controlling the hand via teleoperation, utilizing a camera and machine learning-based hand pose estimation. This method details the programming techniques and processes required to capture data from the camera and convert it into hardware signals. The second approach employs kinematic control using the Denavit-Hartenbergmethod to define motion and determine the position of the end effector in 3D space. Additionally, this work discusses the assembly and modifications made to the arm and hand to create a cost-effective and practical solution. Typically, implementing teleoperation requires numerous sensors and cameras to ensure smooth and successful operation. This paper explores methods enabled by artificial intelligence (AI) that reduce the need for extensive sensor arrays and equipment. It investigates how AI-generated data can be translated into tangible hardware applications across various fields. The advancements in computer vision, combined with AI capable of accurately predicting poses, have the potential to revolutionize the way we control and interact with the world around us.

Advancing autism prediction through visual-based AI approaches: integrating advanced eye movement analysis and shape recognition with Kalman filtering

Advancing autism prediction through visual-based AI approaches: integrating advanced eye movement analysis and shape recognition with Kalman filtering