Computer Vision Research Articles

Traffic Signal Optimization using Computer Vision

Traffic Signal Optimization using Computer Vision

Top-down feedback normalizes distortion in early visual word recognition: Insights from masked priming.

The identification of visually presented words tolerates distortions in the input format, as Hannagan et al. Plos One, 7, e32121, (2012) demonstrated in a masked priming lexical decision task, showing sizable identity-priming effects with CAPTCHA-like primes. This tolerance to distortion has two potential explanations: bottom-up normalization in the encoding stage (Dehaene et al., Trends in Cognitive Sciences, 9, 335-341, 2005) or top-down lexical feedback (McClelland & Rumelhart, Psychological Review, 88, 375-407, 1981). To disentangle the predictions of these accounts, we conducted two masked identity-priming experiments with printed and CAPTCHA-like distorted primes on high- and low-frequency words. The rationale was that, in the distorted format, high-frequency words would benefit more from top-down feedback than low-frequency words. Results in the lexical decision experiment showed that, for high-frequency words, identity-priming effects were only slightly greater for printed than for CAPTCHA-like primes, whereas this difference was larger for low-frequency words. In contrast, when employing the same-different matching task, which does not require lexical access, the identity-priming effect was greater for printed primes and was unaffected by word frequency. Thus, during lexical access, top-down feedback may help normalize the visual input in the early stages of word recognition, challenging bottom-up models of visual word recognition.

Geometric neural network based on phase space for BCI-EEG decoding.

\textbf{Objective:} 
The integration of Deep Learning (DL) algorithms on brain signal analysis is still in its nascent stages compared to their success in fields like Computer Vision. This is particularly true for BCI, where the brain activity is decoded to control external devices without requiring muscle control.
Electroencephalography (EEG) is a widely adopted choice for designing BCI systems due to its non-invasive and cost-effective nature and excellent temporal resolution. Still, it comes at the expense of limited training data, poor signal-to-noise, and a large variability across and within-subject recordings. 
Finally, setting up a BCI system with many electrodes takes a long time, hindering the widespread adoption of reliable DL architectures in BCIs outside research laboratories. To improve adoption, we need to improve user comfort using, for instance, reliable algorithms that operate with few electrodes. 
\textbf{Approach:} Our research aims to develop a DL algorithm that delivers effective results with a limited number of electrodes. Taking advantage of the Augmented Covariance Method and the framework of SPDNet, we propose the \method{} architecture and analyze its performance and the interpretability of the results. The evaluation is conducted on 5-fold cross-validation, using only three electrodes positioned above the Motor Cortex. The methodology was tested on nearly 100 subjects from several open-source datasets using the Mother Of All BCI Benchmark (MOABB) framework. 
\textbf{Main results:} The results of our \method{} demonstrate that the augmented approach combined with the SPDNet significantly outperforms all the current state-of-the-art DL architecture in MI decoding. 
\textbf{Significance:} This new architecture is explainable and with a low number of trainable parameters.

Enabling Energy-Efficient Deployment of Large Language Models on Memristor Crossbar: A Synergy of Large and Small.

Large language models (LLMs) have garnered substantial attention due to their promising applications in diverse domains. Nevertheless, the increasing size of LLMs comes with a significant surge in the computational requirements for training and deployment. Memristor crossbars have emerged as a promising solution, which demonstrated a small footprint and remarkably high energy efficiency in computer vision (CV) models. Memristors possess higher density compared to conventional memory technologies, making them highly suitable for effectively managing the extreme model size associated with LLMs. However, deploying LLMs on memristor crossbars faces three major challenges. Firstly, the size of LLMs increases rapidly, already surpassing the capabilities of state-of-the-art memristor chips. Secondly, LLMs often incorporate multi-head attention blocks, which involve non-weight stationary multiplications that traditional memristor crossbars cannot support. Third, while memristor crossbars excel at performing linear operations, they are not capable of executing complex nonlinear operations in LLM such as softmax and layer normalization. To address these challenges, we present a novel architecture for the memristor crossbar that enables the deployment of state-of-the-art LLM on a single chip or package, eliminating the energy and time inefficiencies associated with off-chip communication. Our testing on BERT Large showed negligible accuracy loss. Compared to traditional memristor crossbars, our architecture achieves enhancements of up to 39× in area overhead and 18× in energy consumption. Compared to modern TPU/GPU systems, our architecture demonstrates at least a 68× reduction in the area-delay product and a significant 69% energy consumption reduction.

Remote sensing and computer vision for marine aquaculture.

Aquaculture, the cultivation of aquatic plants and animals, has grown rapidly since the 1990s, but sparse, self-reported, and aggregated production data limit the effective understanding and monitoring of the industry's trends and potential risks. Building on a manual survey of aquaculture production from remote sensing imagery, we train a computer vision model to identify marine aquaculture cages from aerial and satellite imagery and generate a spatially explicit dataset of finfish production locations in the French Mediterranean from 2000 to 2021 including 4010 cages (average cage area, 69 square meters). We demonstrate the value of our method as an easily adaptable, cost-effective approach that can improve the speed and reliability of aquaculture surveys and enables downstream analyses relevant to researchers and regulators. We illustrate its use to compute independent estimates of production and develop a flexible framework to quantify uncertainty in these estimates. Overall, our study presents an efficient, scalable, and adaptable method for monitoring aquaculture production from remote sensing imagery.

Deep learning-based surgical step recognition for laparoscopic right-sided colectomy.

Understanding the complex anatomy and surgical steps involved in laparoscopic right-sided colectomy (LAP-RC) is essential for standardizing the surgical procedure. Deep-learning (DL)-based computer vision can achieve this. This study aimed to develop a step recognition model for LAP-RC using a dataset of surgical videos with annotated step information and evaluate its recognition performance. This single-center retrospective study utilized a video dataset of laparoscopic ileocecal resection (LAP-ICR) and laparoscopic right-sided hemicolectomy (LAP-RHC) for right-sided colon cancer performed between January 2018 and March 2022. The videos were split into still images, which were divided into training, validation, and test sets using 66%, 17%, and 17% of the data, respectively. Videos were manually classified into eight main steps: 1) medial mobilization, 2) central vascular ligation, 3) dissection of the superior mesenteric vein, 4) retroperitoneal mobilization, 5) lateral mobilization, 6) cranial mobilization, 7) mesocolon resection, and 8) intracorporeal anastomosis. In a simpler version, consecutive surgical steps were combined, resulting in five steps. Precision, recall, F1 scores, and overall accuracy were assessed to evaluate the model's performance in the surgical step classification task. Seventy-eight patients were included; LAP-ICR and LAP-RHC were performed in 35 (44%) and 44 (56%) patients, respectively. The overall accuracy was 72.1% and 82.9% for the eight-step and combined five-step classification tasks, respectively. The automatic surgical step-recognition model for LAP-RCs, developed using a DL algorithm, exhibited a fairly high classification performance. A model that understands the complex steps of LAP-RC will aid the standardization of the surgical procedure.

High-fidelity Image Restoration of Large 3D Electron Microscopy Volume.

Volume electron microscopy (VEM) is an essential tool for studying biological structures. Due to the challenges of sample preparation and continuous volumetric imaging, image artifacts are almost inevitable. Such image artifacts complicate further processing both for automated computer vision methods and human experts. Unfortunately, the widely used contrast limited adaptive histogram equalization (CLAHE) can alter the essential relative contrast information about some biological structures. We developed an image-processing pipeline to remove the artifacts and enhance the images without CLAHE. We apply our method to VEM datasets of a Microwasp head. We demonstrate that our method restores the images with high fidelity while preserving the original relative contrast. This pipeline is adaptable to other VEM datasets.

Deep learning in disease vector image identification.

Vector-borne diseases (VBDs) represent a critical global public health concern, with approximately 80% of the world's population at risk of one or more VBD. Manual disease vector identification is time-consuming and expert-dependent, hindering disease control efforts. Deep learning (DL), widely used in image, text, and audio tasks, offers automation potential for disease vector identification. This paper explores the substantial potential of combining DL with disease vector identification. Our aim is to comprehensively summarize the current status of DL in disease vector identification, covering data collection, data preprocessing, model construction, evaluation methods, and applications in identification spanning from species classification to object detection and breeding site identification. We also discuss the challenges and possible prospects for DL in disease vector identification for further research. © 2024 Society of Chemical Industry.

The spatiotemporal dynamics of bottom-up and top-down processing during at-a-glance reading.

Like all domains of cognition, language processing is affected by top-down knowledge. Classic evidence for this is missing blatant errors in the signal. In sentence comprehension, one instance is failing to notice word order errors, such as transposed words in the middle of a sentence: you that read wrong (Mirault et al., 2018). Our brains seem to fix such errors, since they are incompatible with our grammatical knowledge. But how do our brains do this? Following behavioral work on inner transpositions, we flashed four-word sentences for 300ms using rapid parallel visual presentation (RPVP, Snell and Grainger, 2017). We compared magnetoencephalography responses to fully grammatical and reversed sentences (24 human participants: 21 female, 4 male). Left lateral language cortex robustly distinguished grammatical and reversed sentences starting at 213ms. Thus, the influence of grammatical knowledge begun rapidly after visual word form recognition (Tarkiainen et al., 1999). At the earliest stage of this neural "sentence superiority effect," inner transpositions patterned between grammatical and reversed sentences, showing evidence that the brain initially "noticed" the error. However, 100 milliseconds later, inner transpositions became indistinguishable from grammatical sentences, suggesting at this point, the brain had "fixed" the error. These results show that after a single glance at a sentence, syntax impacts our neural activity almost as quickly as higher-level object recognition is assumed to take place (Cichy et al., 2014). The earliest stage involves detailed comparisons between the bottom-up input and grammatical knowledge, while shortly afterwards, top-down knowledge can override an error in the stimulus.Significance statement Language processing, like all cognitive domains, is profoundly influenced by top-down knowledge. One type of evidence of this is oversight of errors in the signal. For example, in sentence comprehension, individuals often fail to detect word order errors, such as transposed words mid-sentence. Utilizing rapid parallel visual presentation (RPVP), we investigated this phenomenon by exposing participants to four-word sentences for 300ms. Magnetoencephalography revealed robust differentiation between grammatical and reversed sentences in left lateral language cortex starting at 213ms post-presentation. Intriguingly, initial neural responses to inner transpositions treated them as deviant, but 100ms later, neural signals grouped them with grammatical sentence, indicating rapid error correction. These findings reveal the brain's remarkable capacity to reconcile bottom-up input with linguistic knowledge almost instantaneously.

A Survey of Computer Vision Detection, Visual SLAM Algorithms, and Their Applications in Energy-Efficient Autonomous Systems

A Survey of Computer Vision Detection, Visual SLAM Algorithms, and Their Applications in Energy-Efficient Autonomous Systems

Axes-aligned non-linear optimized PnP algorithm

Pose estimation is an important component of many real-world computer vision systems. Most existing pose estimation algorithms need a large number of point correspondences to accurately determine the pose of an object. Since the number of point correspondences depends on the object’s appearance, lighting and other external conditions, detecting many points may not be feasible. In many real-world applications, the movement of objects is limited, e.g. due to gravity. Hence, detecting objects with only three degrees of freedom is usually sufficient. This allows us to improve the accuracy of pose estimation by changing the underlying equations of the perspective-n-point problem to three variables instead of six. By using the simplified equations, our algorithm is more robust against detection errors with limited point correspondences. In this article, we study three scenarios where such constraints apply. The first one is about parking a vehicle on a specific spot. Here, a stationary camera is detecting the vehicle to assist the driver. The second scenario describes the perspective of a moving camera detecting objects in its environment. This scenario is common for driver assistance systems, autonomous cars or mobile robots. Third, we describe a camera observing objects from a birds-eye view, which occurs in industrial applications. In all three scenarios, observed objects can only move in the ground plane and rotate around the vertical axis. Hence, three degrees of freedom are sufficient to estimate the pose. Experiments with synthetic data and real-world photographs have shown that our algorithm outperforms state-of-the-art pose estimation algorithms. Depending on the scenario, our algorithm is able to achieve 50% better accuracy, while being equally fast.

Geometry-Aware Enhanced Mutual-Supervised Point Elimination with Overlapping Mask Contrastive Learning for Partitial Point Cloud Registration

Point cloud registration is one of the fundamental tasks in computer vision, but faces challenges under low overlap conditions. Recent approaches use transformers and overlapping masks to improve perception, but mask learning only considers Euclidean distances between features, ignores mismatches caused by fuzzy geometric structures, and is often computationally inefficient. To address these issues, we introduce a novel matching framework. Firstly, we fuse adaptive graph convolution with PPF features to obtain rich feature perception. Subsequently, we construct a PGT framework that uses GeoTransformer and combines it with location information encoding to enhance the geometry perception between source and target clouds. In addition, we improve the visibility of overlapping regions through information exchange and the AIS module, aiming at subsequent keypoint extraction, preserving points with distinct geometrical structures while suppressing the influence of non-overlapping regions to improve computational efficiency. Finally, the mask is refined through contrast learning to preserve geometric and distance similarity, which helps to compute the transformation parameters more accurately. We have conducted comprehensive experiments on synthetic and real-world scene datasets, demonstrating superior registration performance compared to recent deep learning methods. Our approach shows remarkable improvements of 68.21% in RRMSE and 76.31% in tRMSE on synthetic data, while also excelling in real-world scenarios with enhancements of 76.46% in RRMSE and 45.16% in tRMSE.

Editorial: Artificial Intelligence and Cybersecurity in Pervasive Computing

Pervasive computing, or ubiquitous computing, is rapidly increasing in capacity and capabilities. With the Internet of Things (IoT) becoming an integral part of daily life and the growing availability of edge computing resources, automation guided by data is advancing applications in healthcare, manufacturing, automotive, and other areas. It's natural that pervasive computing will intersect with artificial intelligence (AI) and cybersecurity. AI can improve detection, prediction, and anticipative responses to human needs, while cybersecurity addresses topics like misuse prevention, ethics, policies, and governance. This issue features seven articles on these intersections, including four AI articles exploring natural language processing and computer vision, and three cybersecurity articles covering cryptography, medical devices, and maritime security.

Fostering ayurvedic plant wellness: Innovative leaf disease detection using computer vision and machine learning

Ayurveda is a conventional medicinal approach that has its roots in India and has been used for hundreds of years. It is still popular today since it is entirely natural and free of side effects. Although ayurvedic medicines are made from natural botanical substances, their safety depends on the way they are administered, taking into account the needs of the individual and the specific disease states they are treating. Diseases that affect the plants' leaves are regarded to be one of the main reasons for a decline in the output of ayurvedic plants in India's agricultural, economic, cosmetic, and pharmaceutical sectors. A plant exhibits signs of plant diseases in diverse sections of the plant, however, leaves are the most frequently observed component for spotting an infection. The objectives of this work are to modernize and innovate an autonomous ayurvedic plant leaf disease detection system by examining the scientific basis of computer vision. Additionally, a full analysis of computer vision techniques such as image pre-processing, segmentation, and feature extraction is covered to detect defects in plant leaves. This work investigates cutting-edge machine learning methods for identifying plant diseases that employ a variety of computer vision techniques. Based on the review, the article addresses the challenges and offers recommendations for changes that could be made in the future.

Cylinder Object 6-DOF Pose Estimation via Single Perspective Circle on Cylindrical Surface

Abstract Pose estimation is an important research topic in computer vision. In this paper, a novel method for six-degree-of-freedom (6-DOF) pose estimation of cylindrical objects via a single perspective circle on cylindrical surface is proposed. Unlike traditional methods that rely on planar circular features, a circular feature target which is affixed to the side of cylindrical objects is utilized. The geometric properties of the spatial curve formed by its contour are leveraged for pose estimation. Specifically, the imaging model of the spatial curve is established to derive the general equation of its projection curve. The initial pose is determined based on the perspective principle, and the final optimized pose is obtained with the least squares method. To avoid local optima, a verification and correction method is proposed to enhance estimation accuracy. Synthetic and physical experiments demonstrate that the proposed method can achieve real-time, high-precision pose estimation for cylindrical objects, exhibiting robustness against occlusion and noise interference.

Fostering ayurvedic plant wellness: Innovative leaf disease detection using computer vision and machine learning

Ayurveda is a conventional medicinal approach that has its roots in India and has been used for hundreds of years. It is still popular today since it is entirely natural and free of side effects. Although ayurvedic medicines are made from natural botanical substances, their safety depends on the way they are administered, taking into account the needs of the individual and the specific disease states they are treating. Diseases that affect the plants' leaves are regarded to be one of the main reasons for a decline in the output of ayurvedic plants in India's agricultural, economic, cosmetic, and pharmaceutical sectors. A plant exhibits signs of plant diseases in diverse sections of the plant, however, leaves are the most frequently observed component for spotting an infection. The objectives of this work are to modernize and innovate an autonomous ayurvedic plant leaf disease detection system by examining the scientific basis of computer vision. Additionally, a full analysis of computer vision techniques such as image pre-processing, segmentation, and feature extraction is covered to detect defects in plant leaves. This work investigates cutting-edge machine learning methods for identifying plant diseases that employ a variety of computer vision techniques. Based on the review, the article addresses the challenges and offers recommendations for changes that could be made in the future.

Image segmentation of tunnel water leakage defects in complex environments using an improved Unet model

Computer vision technology provides an intelligent means for detecting tunnel water leakage areas. However, the accuracy of defect feature extraction and segmentation is limited by factors such as insufficient lighting and environmental interference inside tunnels. To address the problem, this paper proposes a tunnel water leakage area segmentation network model called Customized Side Guided-Unet (CSG-Unet), using Unet as the baseline model. The main contributions are: (1) To improve the accuracy of water leakage area extraction, a customized side guided term is introduced to direct the net’s attention to the changes in light and shade within the image. A parallel attention network module is designed to extract internal information from the guided term. Subsequently, a strengthened channel attention module aggregates the guided term and the original information to achieve accurate segmentation of water leakage areas; (2) To address the scarcity of tunnel water leakage area datasets, a basic dataset is constructed by collecting data from open-source datasets and manually gathered data in tunnels. On this basis, perspective transformation is used to change the camera viewpoint, gaussian noise is randomly added to the images in the dataset to simulate images taken in dimly lit scenes, thereby expanding the dataset and enhancing the network’s generalization. The CSG-Unet network was trained using the constructed training set, achieving a mean Intersection over Union (mi IoU) of 85.54%, a mean Dice coefficient (mi Dice) of 85.26%, and a mean Pixel Accuracy (mi PA) of 90.85%. Compared to its baseline network, U-Net (tiny), these metrics show an improvement of over 3.2% in each indicator. Finally, a visual comparison between the improved network and the baseline network further confirms that the proposed model can effectively adapt to the segmentation of water leakage areas in complex environments.

Designing an artificial intelligence-powered video assistant referee system for team sports using computer vision

This paper investigates the efficacy of an AI-powered Video Assistant Referee (VAR) system in enhancing officiating accuracy, efficiency, and consistency in team sports. Employing a combination of artificial intelligence and computer vision, the system was tested in a local championship in Almaty, involving eight football teams. Through the analysis of decision-making accuracy, time efficiency, and consistency across officiating scenarios, the study employed chi-squared tests, paired t-tests, and Cohen's Kappa statistics to quantitatively assess improvements over traditional VAR systems. Results indicated that the AI-powered VAR system significantly increased the accuracy of decisions and reduced the decision-making time, thereby maintaining the fluidity of gameplay. Although the system also demonstrated enhanced consistency in officiating decisions, it highlighted areas needing further refinement to handle complex game situations effectively. The findings suggest that AI integration into sports officiating can substantially benefit the fairness and dynamics of team sports, provided that ongoing technological advancements continue to address current limitations. This study contributes to the growing body of knowledge on the intersection of technology and sports, offering a framework for future enhancements in digital officiating systems.

Precision and speed: The AI revolution in oral squamous cell carcinoma detection

Precision and speed: The AI revolution in oral squamous cell carcinoma detection

AI-Driven Quality Control in PCB Manufacturing: Enhancing Production Efficiency and Precision

This paper investigates the application of Artificial Intelligence (AI) in enhancing quality control within Printed Circuit Board (PCB) manufacturing processes, focusing on how AI-driven technologies improve production efficiency and precision. The research addresses traditional quality control methods, such as manual inspection and Automated Optical Inspection (AOI), highlighting their limitations in keeping up with the complexities and demand for high-precision PCBs in modern electronics. The study delves into how AI technologies, particularly machine learning, computer vision, and predictive analytics, are being leveraged to overcome these limitations. By automating defect detection, improving accuracy, and enabling real-time analysis, AI systems not only streamline the quality control process but also significantly reduce human error and production costs. AI-driven quality control systems are shown to increase defect detection rates, reduce inspection times, and enhance overall production throughput. The paper includes a comparative analysis between traditional and AI-based quality control methods, revealing a notable improvement in both detection accuracy and production speed when AI systems are employed. Additionally, cost efficiency is explored, demonstrating how AI systems reduce waste, minimize rework, and lower operational costs. The potential challenges of AI implementation, such as the high initial costs, data requirements, and integration with legacy systems, are also discussed. The paper concludes that despite these challenges, AI offers a transformative solution to the growing demands of the PCB manufacturing industry, with the ability to scale effectively and adapt to future technological advancements. Ultimately, this research underscores the significant role of AI in revolutionizing PCB quality control by providing a more efficient, precise, and cost-effective approach, paving the way for further innovation in the electronics manufacturing sector.