Vision Applications Research Articles

A computer vision approach to estimate the localized sea state

This research presents a novel application of computer vision (CV) and deep learning methods for real-time sea state recognition, aiming to contribute to improving the operational safety and energy efficiency of seagoing vessels, key factors in meeting the legislative carbon reduction targets. Our work focuses on utilizing sea images in operational envelopes captured by a single stationary camera mounted on the ship bridge. The collected images are used to train a deep learning model to automatically recognize the state of the sea based on the Beaufort scale. To recognize the sea state, we used 4 state-of-the-art deep neural networks with different characteristics that proved useful in various computer vision tasks: Resnet-101, NASNet, MobileNet_v2, and Transformer ViT -b32. Furthermore, we have defined a unique large-scale dataset, collected over a broad range of sea conditions from an ocean-going vessel prepared for machine learning. We used the transfer learning approach to fine-tune the models on our dataset. The obtained results demonstrate the potential for this approach to complement traditional methods, particularly where in-situ measurements are unfeasible or interpolated weather buoy data is insufficiently accurate. This study sets the groundwork for further development of sea state classification models to address recognized gaps in maritime research and enable safer and more efficient maritime operations.

Development of a real-time cattle lameness detection system using a single side-view camera

Recent advancements in machine learning and deep learning have revolutionized various computer vision applications, including object detection, tracking, and classification. This research investigates the application of deep learning for cattle lameness detection in dairy farming. Our study employs image processing techniques and deep learning methods for cattle detection, tracking, and lameness classification. We utilize two powerful object detection algorithms: Mask-RCNN from Detectron2 and the popular YOLOv8. Their performance is compared to identify the most effective approach for this application. Bounding boxes are drawn around detected cattle to assign unique local IDs, enabling individual tracking and isolation throughout the video sequence. Additionally, mask regions generated by the chosen detection algorithm provide valuable data for feature extraction, which is crucial for subsequent lameness classification. The extracted cattle mask region values serve as the basis for feature extraction, capturing relevant information indicative of lameness. These features, combined with the local IDs assigned during tracking, are used to compute a lameness score for each cattle. We explore the efficacy of various established machine learning algorithms, such as Support Vector Machines (SVM), AdaBoost and so on, in analyzing the extracted lameness features. Evaluation of the proposed system was conducted across three key domains: detection, tracking, and lameness classification. Notably, the detection module employing Detectron2 achieved an impressive accuracy of 98.98%. Similarly, the tracking module attained a high accuracy of 99.50%. In lameness classification, AdaBoost emerged as the most effective algorithm, yielding the highest overall average accuracy (77.9%). Other established machine learning algorithms, including Decision Trees (DT), Support Vector Machines (SVM), and Random Forests, also demonstrated promising performance (DT: 75.32%, SVM: 75.20%, Random Forest: 74.9%). The presented approach demonstrates the successful implementation for cattle lameness detection. The proposed system has the potential to revolutionize dairy farm management by enabling early lameness detection and facilitating effective monitoring of cattle health. Our findings contribute valuable insights into the application of advanced computer vision methods for livestock health management.

Multiclass Classification using Enhanced MobileNet V2 Architecture

In computer vision applications including object detection, face recognition, image classification, etc., the Convolutional Neural Network (CNN) predominates. MobileNet v2 is one such CNN architecture which can significantly cut down the parameters and cost of calculation while giving comparatively higher accuracy. In this paper, we propose the enhanced MobileNet architecture which shows an improvement in the accuracy. We have applied transfer learning techniques to fine-tune the MobileNet V2 model on the food dataset. The proposed model’s generalization capacity, it’s robustness and recognition accuracy shows a considerable amount of improvement with respect to the base architecture. The findings demonstrate that the enhanced MobileNet architecture’s accuracy for food images recognition and classification has increased to 97.16%. Key Words: Convolutional Neural Network (CNN), MobileNet v2, transfer learning, image recognition and classification

Novel Deep Learning Domain Adaptation Approach for Object Detection Using Semi-Self Building Dataset and Modified YOLOv4

Moving object detection is a vital research area that plays an essential role in intelligent transportation systems (ITSs) and various applications in computer vision. Recently, researchers have utilized convolutional neural networks (CNNs) to develop new techniques in object detection and recognition. However, with the increasing number of machine learning strategies used for object detection, there has been a growing need for large datasets with accurate ground truth used for the training, usually demanding their manual labeling. Moreover, most of these deep strategies are supervised and only applicable for specific scenes with large computational resources needed. Alternatively, other object detection techniques such as classical background subtraction need low computational resources and can be used with general scenes. In this paper, we propose a new a reliable semi-automatic method that combines a modified version of the detection-based CNN You Only Look Once V4 (YOLOv4) technique and background subtraction technique to perform an unsupervised object detection for surveillance videos. In this proposed strategy, background subtraction-based low-rank decomposition is applied firstly to extract the moving objects. Then, a clustering method is adopted to refine the background subtraction (BS) result. Finally, the refined results are used to fine-tune the modified YOLO v4 before using it in the detection and classification of objects. The main contribution of this work is a new detection framework that overcomes manual labeling and creates an automatic labeler that can replace manual labeling using motion information to supply labeled training data (background and foreground) directly from the detection video. Extensive experiments using real-world object monitoring benchmarks indicate that the suggested framework obtains a considerable increase in mAP compared to state-of-the-art results on both the CDnet 2014 and UA-DETRAC datasets.

Learning manufacturing computer vision systems using tiny YOLOv4.

Implementing and deploying advanced technologies are principal in improving manufacturing processes, signifying a transformative stride in the industrial sector. Computer vision plays a crucial innovation role during this technological advancement, demonstrating broad applicability and profound impact across various industrial operations. This pivotal technology is not merely an additive enhancement but a revolutionary approach that redefines quality control, automation, and operational efficiency parameters in manufacturing landscapes. By integrating computer vision, industries are positioned to optimize their current processes significantly and spearhead innovations that could set new standards for future industrial endeavors. However, the integration of computer vision in these contexts necessitates comprehensive training programs for operators, given this advanced system's complexity and abstract nature. Historically, training modalities have grappled with the complexities of understanding concepts as advanced as computer vision. Despite these challenges, computer vision has recently surged to the forefront across various disciplines, attributed to its versatility and superior performance, often matching or exceeding the capabilities of other established technologies. Nonetheless, there is a noticeable knowledge gap among students, particularly in comprehending the application of Artificial Intelligence (AI) within Computer Vision. This disconnect underscores the need for an educational paradigm transcending traditional theoretical instruction. Cultivating a more practical understanding of the symbiotic relationship between AI and computer vision is essential. To address this, the current work proposes a project-based instructional approach to bridge the educational divide. This methodology will enable students to engage directly with the practical aspects of computer vision applications within AI. By guiding students through a hands-on project, they will learn how to effectively utilize a dataset, train an object detection model, and implement it within a microcomputer infrastructure. This immersive experience is intended to bolster theoretical knowledge and provide a practical understanding of deploying AI techniques within computer vision. The main goal is to equip students with a robust skill set that translates into practical acumen, preparing a competent workforce to navigate and innovate in the complex landscape of Industry 4.0. This approach emphasizes the criticality of adapting educational strategies to meet the evolving demands of advanced technological infrastructures. It ensures that emerging professionals are adept at harnessing the potential of transformative tools like computer vision in industrial settings.

A review on deep learning-based object tracking methods

A deep learning algorithm tracks an object’s movement during object tracking and the main challenge in the tracking of objects is to estimate or forecast the locations and other pertinent details of moving objects in a video. Typically, object tracking entails the process of object detection. In computer vision applications the detection, classification, and tracking of objects play a vital role, and gaining information about the various techniques available also provides significance. In this research, a systematic literature review of the object detection techniques is performed by analyzing, summarizing, and examining the existing works available. Various state of art works are collected from standard journals and the methods available, cons, and pros along with challenges are determined based on this the research questions are also formulated. Overall, around 50 research articles are collected, and the evaluation based on various metrics shows that most of the literary works used Deep convolutional neural networks (Deep CNN), and while tracking the objects object detection helps in enhancing the performance of these networks. The important issues that need to be resolved are also discussed in this research, which helps in leveling up the object-tracking techniques.

AVA: Towards Autonomous Visualization Agents through Visual Perception‐Driven Decision‐Making

AbstractWith recent advances in multi‐modal foundation models, the previously text‐only large language models (LLM) have evolved to incorporate visual input, opening up unprecedented opportunities for various applications in visualization. Compared to existing work on LLM‐based visualization works that generate and control visualization with textual input and output only, the proposed approach explores the utilization of the visual processing ability of multi‐modal LLMs to develop Autonomous Visualization Agents (AVAs) that can evaluate the generated visualization and iterate on the result to accomplish user‐defined objectives defined through natural language. We propose the first framework for the design of AVAs and present several usage scenarios intended to demonstrate the general applicability of the proposed paradigm. Our preliminary exploration and proof‐of‐concept agents suggest that this approach can be widely applicable whenever the choices of appropriate visualization parameters require the interpretation of previous visual output. Our study indicates that AVAs represent a general paradigm for designing intelligent visualization systems that can achieve high‐level visualization goals, which pave the way for developing expert‐level visualization agents in the future.

Should I make it round? Suitability of circular and linear layouts for comparative tasks with matrix and connective data

AbstractVisual representations based on circular shapes are frequently used in visualization applications. One example are circos plots within bioinformatics, which bend graphs into a wheel of information with connective lines running through the center like spokes. The results are aesthetically appealing and impressive visualizations that fit long data sequences into a small quadratic space. However, the authors' experiences are that when asked, a visualization researcher would generally advise against making visualizations with radial layouts. Upon reviewing the literature we found that there is evidence that circular layouts are preferable in some cases, but we found no clear evidence for what layout is preferable for matrices and connective data in particular, which both are common data types in circos plots. In this work, we thus performed a user study to compare circular and linear layouts. The tasks are inspired by genomics data, but our results generalize to many other application areas, involving comparison and connective data. To build the prototype we utilized Gosling, a grammar for visualizing genomics data. We contribute empirical evidence on the suitedness of linear versus circular layouts, adding to the specific and general knowledge concerning perception of circular graphs. In addition, we contribute a case study evaluation of the grammar Gosling as a rapid prototyping language, confirming its utility and providing guidance on suitable areas for future development.

Application of Photogrammetric Computer Vision and Deep Learning in High-Resolution Underwater Mapping: A Case Study of Shallow-Water Coral Reefs

Abstract. Underwater mapping is vital for engineering applications and scientific research in ocean environments, with coral reefs being a primary focus. Unlike more uniform and predictable terrestrial environments, coral reefs present a unique challenge for 3D reconstruction due to their intricate and irregular structures. Traditional 3D reconstruction methods struggle to accurately capture the nuances of coral reefs. This is primarily because coral reefs exhibit a high degree of spatial heterogeneity, featuring diverse shapes, sizes, and textures. Additionally, the dynamic nature of underwater conditions, such as varying light, water clarity, and movement, further complicates the accurate geometrical estimation of these ecosystems. With the rapid advancement of photogrammetric computer vision and deep learning technologies, there are emerging methods that have potential to enhance the quality of its 3D reconstruction. In this context, this study formulates a coral reef reconstruction workflow that incorporates these cutting-edge technologies. This workflow is divided into two core stages: sparse reconstruction and dense reconstruction. We conduct individual summaries of the relevant research efforts in these stages and outline the available methods. To assess the specific capabilities of these methods, we apply them to real-world coral reef images and conduct a comprehensive evaluation. Additionally, we analyze the strengths and weaknesses of different methods and identify areas for improvement. We believe this study offers valuable references for future research in underwater mapping.

Multifunctional Phosphor with High-Efficient Near-Infrared Emission Based on Antimony-Zinc Halides.

Metal halide-based broadband near-infrared (NIR) luminescent materials face problems such as complicated preparation, high cost, low photoluminescence quantum yield, and high excitation energy. Here, incorporating Sb3+ and Br- into (C20H20P)2ZnCl4 crystals allowed for the achievement of efficient broadband near-infrared emission under 400 nm excitation while maintaining satisfactory environmental and thermal stability. The compounds exhibit a broad range of emission bands from 550 to 1050 nm, with a photoluminescence quantum yield of 93.57%. This is a groundbreaking achievement for organic-inorganic hybrid metal halide NIR luminescent materials. The near-infrared emission is suggested to originate from [SbX5]2-, as supported by the femtosecond transient absorption spectra and density-functional theory calculations. This phosphor-based NIR LEDs successfully demonstrate potential applications in night vision, medical imaging, information encryption, and anticounterfeiting.

From Pixels to Principles: A Decade of Progress and Landscape in Trustworthy Computer Vision

The rapid development of computer vision technologies and applications has brought forth a range of social and ethical challenges. Due to the unique characteristics of visual technology in terms of data modalities and application scenarios, computer vision poses specific ethical issues. However, the majority of existing literature either addresses artificial intelligence as a whole or pays particular attention to natural language processing, leaving a gap in specialized research on ethical issues and systematic solutions in the field of computer vision. This paper utilizes bibliometrics and text-mining techniques to quantitatively analyze papers from prominent academic conferences in computer vision over the past decade. It first reveals the developing trends and specific distribution of attention regarding trustworthy aspects in the computer vision field, as well as the inherent connections between ethical dimensions and different stages of visual model development. A life-cycle framework regarding trustworthy computer vision is then presented by making the relevant trustworthy issues, the operation pipeline of AI models, and viable technical solutions interconnected, providing researchers and policymakers with references and guidance for achieving trustworthy CV. Finally, it discusses particular motivations for conducting trustworthy practices and underscores the consistency and ambivalence among various trustworthy principles and technical attributes.

Uncertainty Quantification with Deep Ensembles for 6D Object Pose Estimation

Abstract. The estimation of 6D object poses is a fundamental task in many computer vision applications. Particularly, in high risk scenarios such as human-robot interaction, industrial inspection, and automation, reliable pose estimates are crucial. In the last years, increasingly accurate and robust deep-learning-based approaches for 6D object pose estimation have been proposed. Many top-performing methods are not end-to-end trainable but consist of multiple stages. In the context of deep uncertainty quantification, deep ensembles are considered as state of the art since they have been proven to produce well-calibrated and robust uncertainty estimates. However, deep ensembles can only be applied to methods that can be trained end-to-end. In this work, we propose a method to quantify the uncertainty of multi-stage 6D object pose estimation approaches with deep ensembles. For the implementation, we choose SurfEmb as representative, since it is one of the top-performing 6D object pose estimation approaches in the BOP Challenge 2022. We apply established metrics and concepts for deep uncertainty quantification to evaluate the results. Furthermore, we propose a novel uncertainty calibration score for regression tasks to quantify the quality of the estimated uncertainty.

A workflow to systematically design uncertainty-aware visual analytics applications

A workflow to systematically design uncertainty-aware visual analytics applications

A review: Data visualization for COVID-19

As COVID-19 raged across the globe, it seriously disrupted social health security, daily work, and the rest of the people, which aroused great panic among all classes. These facts extensively affected the regional circulation and economic growth of the epidemic area. Data visualization, an advanced technology, can help humans understand epidemic transmission trends, identify high-risk areas of outbreak, and evaluate prevention and control policies. Previous essays immediately established databases to organize and build datasets. Other researchers built new visualization models to analyze the causes of virus transmission and pointed out the low efficiency of the existing COVID-19 system response. Although various research and data visualization methods have been used on COVID-19, there has not been a comprehensive, systematic essay that covers the entire span of COVID-19 and its data visualization usage. Therefore, this paper will provide a new systematic and comprehensive perspective to explore the analysis of data visualization to solve COVID-19 data challenges and provide research methods and applications for COVID-19. To be specific, this paper first introduces the characteristics and challenges of COVID-19 epidemic data people may face during the data visualization process, which includes data complexity problems, data timeliness problems, data regional problems, and data quality problems. Then, this paper discusses data visualization methods used by predecessors in previous research. After that, this paper describes the urgent data visualization analysis requirements of COVID-19, including regional differences in virus spread, the influence of temperature on the virus, transmission trends, and death rates for different genders and ages, etc. Finally, this paper reviews applications of data visualization in COVID-19 analysis, followed by earlier-stage prevention, mid-term containment, and later-term reduction of impact. This paper provides an important reference for further research and application of data visualization technology in COVID-19 prevention and control, which brings new ideas for strengthening future epidemic surveillance and data visualization for global epidemic prevention and control.

Enhancing soccer pass receiver prediction in broadcast images through advanced deep learning techniques: A comprehensive study on model optimization and performance evaluation

In this study, we present a graph neural network (GNN) model specifically designed for football pass receiver prediction in Broadcast Images is presented in this study. Important node properties, including ball possession indicators, hot-encoded team values, and normalized ground placements, are incorporated into the model along with a careful weighting of edges to account for player distances. With weighted BCE loss used to overcome class imbalance, its architecture consists of a linear layer, numerous GNN Message Passing layers, a SoftMax activation, and binary cross-entropy (BCE) loss for training. Of 206 examples, 101 valid predictions were made, indicating a predictive accuracy of 0.50 according to the evaluation data. Comparative analyzes show that GAT-V2 (0.85) and GAT (0.63) perform better in terms of optimization and accuracy, respectively. The effectiveness in recognizing football pass receivers is demonstrated in this paper, highlighting developments in computer vision applications for sports analytics.

Bridging the gap: Active learning for efficient domain adaptation in object detection

In practical object detection computer vision applications, training commonly incorporates multiple data sources. Domain adaptation enhances the models’ capacity to generalize across different source and target domain data. Active learning, which serves as a bridge between unsupervised and supervised domain adaptation, is an area that has been underexplored for object detection. Our methods are specifically designed to tackle the challenges of selecting target domain images for annotation, complementing the existing source domain images. First, this paper formally describes active learning in domain adaptation for object detection. We propose three new selection strategies for active learning in domain adaptation for object detection. We utilize multiple machine learning models when selecting target domain data for annotation. In particular, our best-performing selection method utilizes a domain discriminator to predict the target domain likelihood of each sample. Additionally, we introduce an evaluation framework that evaluates per-box and per-image labeling efforts in the unique object detection application. Through extensive experiments we could show that our best-performing method could improve the active learning object detection performance over random selection by about 2.4% while requiring about 13% less annotated bounding boxes on the target domain.

Fabric4show: real-time vision system for fabric defect detection and post-processing

The exploration of computer vision applications for fabric defect detection has immense potential value. However, current relevant research in this area has primarily focused on detection models that aim for high detection accuracy and algorithmic efficiency, while neglecting the practical industrial production requirements. Therefore, we propose a fabric defect detection and post-processing system that integrates an optimized region with convolutional neural network (CNN) features (i.e., Faster R-CNN) for defect detection, defect localization and detection model evaluation. In addition, the proposed intelligent system incorporates novel approaches, such as a rearranged fabric dataset, anomaly detection, recommended clipping region division, and a replenishment device. This study illustrates an example of artificial intelligence (AI)-driven automated technology in fabric manufacturing. The accuracy and detection speed of different detection models under identical hardware conditions are evaluated and compared with related work. Experimental results demonstrate that the proposed approach achieves comparable performance to other models, while significantly reducing computational resource requirements. The potential efficiency of using two-stage networks on hardware systems for fabric defect detection tasks is highlighted, which is likely to have relevant implications for the textile industry.

Underwater Image Enhancement Algorithm Based on Adversarial Training

Ocean observation is the first step in the development of the ocean, whose abundant resources and strategic significance are attracting increasing attention. Observation methods based on visual sensor networks have received great attention from researchers due to their visualization capability and high information capacity. However, below the sea surface, objective factors such as blurriness, turbulence, and underwater color casting can cause image distortion and affect the acquisition of images. In this paper, the enhancement of underwater images is tackled using an adversarial learning-based approach. First, pre-processing is applied to address the significant color casting in the dataset, thus enhancing feature learning for subsequent style transfer. Then, corresponding improvements are made to a generative adversarial network’s structure and loss functions to better restore the features of the network output. Finally, evaluations and comparisons are performed using underwater image quality assessment metrics and several public datasets. Through multidimensional experiments, the proposed algorithm is shown to exhibit excellent performance in both subjective and objective evaluation metrics compared to state-of-the-art algorithms, as well as in practical visual applications.

Opportunities for improving the use of computer vision in municipal solid waste sorting

This article explores the possibilities of improving the application of optical sorting of municipal solid waste on conveyor belts using computer vision and machine learning technologies in order to replace manual sorting lines. Also the relevance of improving waste sorting is emphasized. This work expands the understanding of the application of computer vision in the field of waste management and offers practical recommendations for its implementation in industrial processes. The paper reviews current approaches to sorting municipal solid waste, analyzes their effectiveness, and explores the potential for improving waste management processes and increasing recycling efficiency based on recent research in this area. The most popular equipment used in sorting and its features are considered. A generalization of the morphological composition of waste is carried out, and the minimum and maximum possible shares of each class of waste in the recycling stream are identified. Based on this data, sufficient proportions of images in each category are selected to train a convolutional neural network for waste classification. Applied research in terms of detection and classification of waste objects on conveyor belts using well-known neural network architectures is considered. The analysis showed that most solutions are based on 2018 architectures, while the reference classification accuracy of more modern architectures, such as OmniVec, is 17 % higher, and detection efficiency is similar orders of magnitude higher. The resulting classification accuracy is significantly affected by image augmentation in the set of images for training the neural network — its presence can increase accuracy by 10 to 15 %. The minimum training data set has been determined, which should be about 2 000 images due to the fact that after a further increase in the number, the accuracy does not change significantly.

45‐1: Invited Paper: Design Strategies for NIR Emitting Materials

The development of OLED materials by molecular design is important for high performance organic light emitting diode because tailoring the molecular structures can dramatically tune the properties such as efficiency, color, stability of device. Near‐infrared organic light‐emitting diodes (NIR OLEDs) with heavy metals or TADF materials are regularly reported due to the advantages of their various applications in healthcare services, veil authentication, and night vision displays. In this paper, we are summarized the collection of innovative organic near‐infrared (NIR) emitters which is crucial factor to design specific NIR emitter for NIR OLEDs