Computer Vision Research Articles

Illumination and Shadows in Head Rotation: Experiments with Denoising Diffusion Models

Accurately modeling the effects of illumination and shadows during head rotation is critical in computer vision for enhancing image realism and reducing artifacts. This study delves into the latent space of denoising diffusion models to identify compelling trajectories that can express continuous head rotation under varying lighting conditions. A key contribution of our work is the generation of additional labels from the CelebA dataset, categorizing images into three groups based on prevalent illumination direction: left, center, and right. These labels play a crucial role in our approach, enabling more precise manipulations and improved handling of lighting variations. Leveraging a recent embedding technique for Denoising Diffusion Implicit Models (DDIM), our method achieves noteworthy manipulations, encompassing a wide rotation angle of ±30°. while preserving individual distinct characteristics even under challenging illumination conditions. Our methodology involves computing trajectories that approximate clouds of latent representations of dataset samples with different yaw rotations through linear regression. Specific trajectories are obtained by analyzing subsets of data that share significant attributes with the source image, including light direction. Notably, our approach does not require any specific training of the generative model for the task of rotation; we merely compute and follow specific trajectories in the latent space of a pre-trained face generation model. This article showcases the potential of our approach and its current limitations through a qualitative discussion of notable examples. This study contributes to the ongoing advancements in representation learning and the semantic investigation of the latent space of generative models.

Menumbuhkan Literasi Teknologi Melalui Pengenalan Aplikasi Computer Vision Di Kalangan Pelajar

The purpose of this community service project is to provide education to enhance technological literacy by introducing the basics of Computer Vision applications among students. The activities included a seminar attended by 170 students. An analysis using the Wilcoxon statistical test on pre-post test results showed a significant improvement in participants' understanding of Computer Vision applications. The test results indicated a significant difference before and after the activity with a value of 0.011. Through this community service, participants have successfully grasped the material presented to enhance literacy as change agents in the digital era, positively impacting societal progress through improved understanding of technology in the field of Computer Vision.

Assessing CNN Architectures for Estimating Correct Posture in Cruise Machinists

Cruise machinists operate in dynamic and physically demanding environments where improper posture can lead to musculoskeletal disorders, adversely affecting their health and work efficiency. Current ergonomic assessments in such settings are often generic and not tailored to the unique challenges of maritime operations. This paper presents a novel application of artificial intelligence tools for real-time posture estimation specifically designed for cruise machinists. The primary aim is to enhance occupational health and safety by providing precise, real-time feedback on ergonomic practices. We developed a dataset by capturing video recordings of cruise machinists at work, which were processed to extract skeletal outlines using advanced computer vision techniques. This dataset was used to train deep neural networks, optimizing them for accuracy in diverse and constrained shipboard environments. The networks were tested across various computational platforms to ensure robustness and adaptability. The AI model demonstrated high efficacy in recognizing both correct and incorrect postures under real-world conditions aboard ships. The system significantly outperformed traditional ergonomic assessment tools in terms of speed, accuracy, and the ability to provide instant feedback. The findings suggest that AI-enhanced ergonomic assessments could be a transformative approach for occupational health across various industries.

Enhancing User Perception of Reliability in Computer Vision: Uncertainty Visualization for Probability Distributions

Non-expert users often find it challenging to perceive the reliability of computer vision systems accurately. In human–computer decision-making applications, users’ perceptions of system reliability may deviate from the probabilistic characteristics. Intuitive visualization of system recognition results within probability distributions can serve to enhance interpretability and support cognitive processes. Different visualization formats may impact users’ reliability perceptions and cognitive abilities. This study first compared the mapping relationship between users’ perceived values of system recognition results and the actual probabilistic characteristics of the distribution when using density strips, violin plots, and error bars to visualize normal distributions. The findings indicate that when density strips are used for visualization, users’ perceptions align most closely with the probabilistic integrals, exhibiting the shortest response times and highest cognitive arousal. However, users’ perceptions often exceed the actual probability density, with an average coefficient of 2.53 times, unaffected by the form of uncertainty visualization. Conversely, this perceptual bias did not appear in triangular distributions and remained consistent across symmetric and asymmetric distributions. The results of this study contribute to a better understanding of user reliability perception for interaction designers, helping to improve uncertainty visualization and thereby mitigate perceptual biases and potential trust risks.

Toward Automated Structural Design for Controlled Vibration Characteristics Using Topology Optimization and Computer Vision in Space Missions

This study explores the integration of computer vision with topology optimization for additive manufacturing, with a focus on maximizing eigenfrequency in a design domain. Utilizing custom-developed photogrammetry software, high-resolution images are processed to generate detailed 3D models, which are subsequently converted to STL files with precision. Adaptive meshing in COMSOL 5.3 Multiphysics, controlled through a MATLAB 2023 API, ensures optimal mesh resolution. Prioritizing resource conservation in extraterrestrial environments, the original volume is reduced by 50% while preserving structural integrity. The design domain undergoes rigorous topology optimization in MATLAB, supported by COMSOL’s advanced FEM simulation. The optimized design exhibits a 57% performance improvement and a 50% weight reduction, maintaining the desired vibration characteristics, validating the efficacy of the modifications. Moreover, the case with an eccentric mass shows a significant 64% increase in eigenfrequency.

Examining the emergence of the ‘AI eye’ and its effect on the ‘creative treatment of actuality’ in computational non-fiction

ABSTRACT Documentaries have frequently captured journeys to places audiences might not have seen before. The first documentary film Nanook of the North (1922), often referred to as a travel film, took viewers to the Canadian Artic for the first time. In 1929, The Man with a Movie Camera revealed the rhythm of the Soviet cities through the Kino-Eye. Now the documentary film genre, the City Film is taking viewers on a virtual tour into new digital synthetic domains as they contain artificially generated images produced by GenAI. In response to this emergence of the ‘AI eye’, a computational vision of the world, the following attributes of AI as an asset, tool and collaborator shape the discussion of novel creative documentary processes that are emerging when Gen AI is integrated into computational non-fiction. This article explores a computational vision of the world in relation to the City Film documentary genre.

Multi-task prompt tuning with soft context sharing for vision–language models

Vision–language models have recently shown great potential on many tasks in computer vision. Meanwhile, prior work demonstrates prompt tuning designed for vision–language models could acquire superior performance on few-shot image recognition compared to linear probe, a strong baseline. In practice, many few-shot tasks are inherently correlated, particularly within specialized domains. However, such information is overlooked previously. Inspired by the fact that modeling task relationship by multi-task learning can usually boost performance, we propose a novel method SoftCPT (Soft Context Sharing for Prompt Tuning) to tune pre-trained vision–language models on multiple target few-shot tasks jointly. Specifically, we design a task-shared meta network to generate prompt context for each task using task name together with a learnable task context as input. The parameters of this meta network as well as the task context are tuned on the joint training set of all tasks. As such, the prompt context of all tasks will be shared in a soft manner. Extensive experiments across four multi-task few-shot datasets covering 44 tasks and 1593 categories demonstrate that SoftCPT significantly outperforms single-task prompt tuning methods, highlighting the effectiveness of multi-task learning for vision–language prompt tuning.

The Emotional Power of Partisan Media: A Computer Vision Analysis of the 2020 Democratic Party Presidential Primaries

ABSTRACT Building on theories of emotional contagion and affective priming, the study assesses the emotional impact of the U.S. partisan media’s coverage of six leading political candidates’ images during the 2020 New Hampshire Democratic Party presidential primary. Results show that the partisan media’s representations of the candidates’ facial expressions of emotions are associated with the voters’ feelings toward the politicians in their desired directions. Additionally, the partisan-media-primed emotions among the voters consistently align with their voting decisions. Methodologically, the study combines state-of-the-art computer vision techniques and survey research conducted with a representative sample of New Hampshire voters.

A Children's Psychological and Mental Health Detection Model by Drawing Analysis based on Computer Vision and Deep Learning

A Children's Psychological and Mental Health Detection Model by Drawing Analysis based on Computer Vision and Deep Learning

Assessing opportunities for enhanced lighting energy conservation via occupancy and daylight monitoring

Efficient energy utilization in buildings is crucial for sustainability. This work proposes an intelligent system that leverages computer vision techniques and CCTV images to assess indoor lighting energy usage based on occupancy, artificial lighting, and daylight conditions. Object detection models - You Only Look Once (YOLO) version 3 (v3) and v8 are employed to detect people, lights, and windows, achieving promising accuracies of 94.9 ​%, 73.3 ​%, and 98.7 ​%, respectively. The system categorizes scenarios as energy-efficient, wasteful, or neutral by integrating these outputs, highlighting opportunities for improving efficiency by harmonizing lighting infrastructure with occupancy and daylight exposure. Performance analyses, including training and validation metrics, are presented. This study demonstrates the potential of computer vision and AI for optimizing energy utilization, enabling sustainable building operation and promoting energy-positive occupant behaviors through sensor-driven methodologies.

Enhancing the Quality of Ambulance Crew Work by detecting Ambulance Equipment using Computer Vision and Deep Learning

Enhancing the Quality of Ambulance Crew Work by detecting Ambulance Equipment using Computer Vision and Deep Learning

An automated lightweight approach for detecting dead fish in a recirculating aquaculture system

Accurate methods for detecting dead fish are of great significance for fisheries because of their potential to improve production and reduce water pollution. Small-scale fisheries typically apply manual observation, but this approach is labor-intensive and time-consuming, which limits its application on large-scale farms. Advances in computer vision tools have provided the foundation for revolutionary methods of automated identification of individual-animal behavior. However, computer vision systems often suffer from unstable accuracy, low efficiency, and limited detection capability. Thus, the aim of this study was to design a deep learning system, designated “Deadfish-YOLO”, to detect dead fish based solely on standard underwater camera images with no additional hardware. First, eight selected rearing tanks were monitored by a self-developed computer version system. From these tank images, a dataset containing 18,114 frames was built by manual labeling. Next, a lightweight backbone network was generated with YOLOv4 to ensure fast computations, and an attention mechanism was introduced into the model to suppress unimportant features. Finally, the ReLU-memristor-like activation function was adopted to improve neural-network performance. The accuracy and processing speed of Deadfish-YOLO were superior to those of other state-of-the-art single- and two-stage detection models; Deadfish-YOLO ran at 85 frames per second with a mean average precision of 0.946 and an average ratio of 0.924 between intersection and union. These results demonstrate that Deadfish-YOLO may be used to automatically monitor dead fish in real circulating aquaculture systems. In addition, the results of this study should facilitate the wider application of artificial-intelligence-based animal monitoring in aquaculture.

Multi-fidelity wavelet neural operator surrogate model for time-independent and time-dependent reliability analysis

Operator learning frameworks have recently emerged as an effective scientific machine learning tool for learning complex nonlinear operators of differential equations. Since neural operators learn an infinite-dimensional functional mapping, it is useful in applications requiring rapid prediction of solutions for a wide range of input functions. A task of a similar nature arises in many applications of uncertainty quantification, including reliability estimation and design under uncertainty, each of which demands thousands of samples subjected to a wide range of possible input conditions, an aspect to which neural operators are specialized. Although the neural operators are capable of learning complex nonlinear solution operators, they require an extensive amount of data for successful training. Unlike the applications in computer vision, the computational complexity of the numerical simulations and the cost of physical experiments contributing to the synthetic and real training data compromise the performance of the trained neural operator model, thereby directly impacting the accuracy of uncertainty quantification results. We aim to alleviate the data bottleneck by using multi-fidelity learning in neural operators, where a neural operator is trained by using a large amount of inexpensive low-fidelity data along with a small amount of expensive high-fidelity data. We propose the multi-fidelity wavelet neural operator, capable of learning solution operators from a multi-fidelity dataset, for efficient and effective data-driven reliability analysis of dynamical systems. We illustrate the performance of the proposed framework on bi-fidelity data simulated on coarse and refined grids for spatial and spatiotemporal systems.

A Scale-space Approach for Surface Normal Vector Estimation from Depth Maps

Surface normal vectors provide cues about the local geometric features of the scene which are utilized in many computer vision and computer graphics applications. Thus, the estimation of surface normals by utilizing structured range sensor data is an important research field. Thereupon, we propose a learning-free algorithm to estimate the surface normal vectors from depth maps. Our simple yet effective method relies on computations carried out in scale-space. Our main idea is to estimate the surface normals which cannot be properly computed in the finest scale from the coarser scales. Our method can estimate the surface normals even for images included in datasets that have challenging characteristics such as noisy real-world data or significantly large planar regions that either have a small or no gradient change. We analyze our algorithm’s performance by utilizing five benchmarks, namely, the MIT-Berkeley Intrinsic Images dataset, the New Tsukuba Dataset, the SceneNet RGB-D dataset, the IID-NORD dataset, and the NYU Depth Dataset V2, and by using two different evaluation strategies. According to the experimental results, our method can estimate surface normals efficiently without requiring neither complex computations nor huge amounts of data.

ETLP: event-based three-factor local plasticity for online learning with neuromorphic hardware

Neuromorphic perception with event-based sensors, asynchronous hardware, and spiking neurons shows promise for real-time, energy-efficient inference in embedded systems. Brain-inspired computing aims to enable adaptation to changes at the edge with online learning. However, the parallel and distributed architectures of neuromorphic hardware based on co-localized compute and memory imposes locality constraints to the on-chip learning rules. We propose the event-based three-factor local plasticity (ETLP) rule that uses the pre-synaptic spike trace, the post-synaptic membrane voltage and a third factor in the form of projected labels with no error calculation, that also serve as update triggers. ETLP is applied to visual and auditory event-based pattern recognition using feedforward and recurrent spiking neural networks. Compared to back-propagation through time, eProp and DECOLLE, ETLP achieves competitive accuracy with lower computational complexity. We also show that when using local plasticity, threshold adaptation in spiking neurons and a recurrent topology are necessary to learn spatio-temporal patterns with a rich temporal structure. Finally, we provide a proof of concept hardware implementation of ETLP on FPGA to highlight the simplicity of its computational primitives and how they can be mapped into neuromorphic hardware for online learning with real-time interaction and low energy consumption.

Towards Automatic Mango Grading: A Computer Vision System Design for Measuring a Mango's Weight and Skin Defect Areas

Towards Automatic Mango Grading: A Computer Vision System Design for Measuring a Mango's Weight and Skin Defect Areas

Vision-Language Models for Vision Tasks: A Survey.

Most visual recognition studies rely heavily on crowd-labelled data in deep neural networks (DNNs) training, and they usually train a DNN for each single visual recognition task, leading to a laborious and time-consuming visual recognition paradigm. To address the two challenges, Vision-Language Models (VLMs) have been intensively investigated recently, which learns rich vision-language correlation from web-scale image-text pairs that are almost infinitely available on the Internet and enables zero-shot predictions on various visual recognition tasks with a single VLM. This paper provides a systematic review of visual language models for various visual recognition tasks, including: (1) the background that introduces the development of visual recognition paradigms; (2) the foundations of VLM that summarize the widely-adopted network architectures, pre-training objectives, and downstream tasks; (3) the widely-adopted datasets in VLM pre-training and evaluations; (4) the review and categorization of existing VLM pre-training methods, VLM transfer learning methods, and VLM knowledge distillation methods; (5) the benchmarking, analysis and discussion of the reviewed methods; (6) several research challenges and potential research directions that could be pursued in the future VLM studies for visual recognition.

An overview of hand gesture recognition based on computer vision

Hand gesture recognition emerges as one of the foremost sectors which has gone through several developments within pattern recognition. Numerous studies and research endeavors have explored methodologies grounded in computer vision within this domain. Despite extensive research endeavors, there is still a need for a more thorough evaluation of the efficiency of various methods in different environments along with the challenges encountered during the application of these methods. The focal point of this paper is the comparison of different research in the domain of vision-based hand gesture recognition. The objective is to find out the most prominent methods by reviewing efficiency. Concurrently, the paper delves into presenting potential solutions for challenges faced in different research. A comparative analysis particularly centered around traditional methods and convolutional neural networks like random forest, long short-term memory (LSTM), heatmap, and you only look once (YOLO). considering their efficacy. Where convolutional neural network-based algorithms performed best for recognizing the gestures and gave effective solutions for the challenges faced by the researchers. In essence, the findings of this review paper aim to contribute to future implementations and the discovery of more efficient approaches in the gesture recognition sector.

Does Negative Sampling Matter? a Review With Insights Into its Theory and Applications.

Negative sampling has swiftly risen to prominence as a focal point of research, with wide-ranging applications spanning machine learning, computer vision, natural language processing, data mining, and recommender systems. This surge in interest prompts us to question the fundamental impact of negative sampling: Does negative sampling really matter? Is there a general framework that can incorporate all negative sampling methods? In what fields is it applied? Addressing these questions, we propose a general framework that using negative sampling. Delving into the history of negative sampling, we chart its evolution across five distinct trajectories. We dissect and categorize the strategies used to select negative sample candidates, detailing global, local, mini-batch, hop, and memory-based approaches. Our comprehensive review extends to an analysis of current negative sampling methodologies, systematically grouping them into five classifications: static, hard, GAN-based, Auxiliary-based, and In-batch. Beyond detailed categorization, we explore the practical application of negative sampling across various fields. Finally, we briefly discuss open problems and future directions for negative sampling.

Edge-platforms based decision-support approach for solar panels inspection with YOLOv8 deep neural-network

<div class="page" title="Page 1"><div class="layoutArea"><div class="column"><p>This paper presents an innovative AI-based method for autonomous inspection, designed to enhance energy production efficiency by optimizing cleaning strategies for soiled photovoltaic panels, using advanced artificial intelligence algorithms to analyze panel conditions and environmental factors in real-time, allowing for targeted cleaning interventions. Based on the advanced YOLOv8 deep learning algorithm and computer vision approach, the proposed method offers distinct advantages in real-time detection and classification of various types of soiling and dust accumulation compared on solar panels to traditional methods, and underwent satisfactory testing across diverse scenarios. The NVIDIA Jetson Nano, the Raspberry Pi4 embedded devices, and the Raspberry Pi4 combined with NCS2 accelerator are used for implementing our approach. A comparison aims to provide a detailed exploration of the most suitable embedded platform for deploying our advanced system was discussed. This comparison considers processing speed and accuracy, energy consumption, and overall performance in executing the computationally intensive tasks. The results demonstrate that our model achieves high accuracy in detecting soiling and enhancing the model's detection speed. With an average precision of 99.5%, this approach ensures accurate fault identification, underscoring the effectiveness of computer vision using deep learning algorithms for detection tasks across a wide range of scenarios.</p></div></div></div>