Computer Vision Applications Research Articles

The role of driver head pose dynamics and instantaneous driving in safety critical events: Application of computer vision in naturalistic driving

This paper investigates the role of driver behavior especially head pose dynamics in safety–critical events (SCEs). Using a large dataset collected in a naturalistic driving study, this paper analyzes the head pose dynamics and driving behavior in moments leading up to crashes or near-crashes. The study uses advanced computer vision and mixed logit modeling techniques to identify patterns and relationships between drivers’ head pose dynamics and crash involvement. The results suggest that driver-head pose dynamics, especially poses that indicate distraction and movement volatility, are important factors that can contribute to undesirable safety outcomes. Marginal effects show that angular deviation for head pose dynamics indicated by yaw, pitch and roll increase the likelihood of crash intensity by 4.56%, 4.92% and 8.26% respectively. Furthermore, traffic flow and lane changing also contribute to increase in likelihood of crash intensity. These findings provide new insights into pre-crash factors, especially human factors and safety–critical events. The study highlights the importance of considering human factors in designing driver assistance systems and developing safer vehicles. This research contributes by examining naturalistic driving data at the microscopic level with early detection of behaviors that lead to SCEs and provides a basis for future research on automation.

Mathematical modeling and problem solving: from fundamentals to applications

The rapidly advancing fields of machine learning and mathematical modeling, greatly enhanced by the recent growth in artificial intelligence, are the focus of this special issue. This issue compiles extensively revised and improved versions of the top papers from the workshop on Mathematical Modeling and Problem Solving at PDPTA'23, the 29th International Conference on Parallel and Distributed Processing Techniques and Applications. Covering fundamental research in matrix operations and heuristic searches to real-world applications in computer vision and drug discovery, the issue underscores the crucial role of supercomputing and parallel and distributed computing infrastructure in research. Featuring nine key studies, this issue pushes forward computational technologies in mathematical modeling, refines techniques for analyzing images and time-series data, and introduces new methods in pharmaceutical and materials science, making significant contributions to these areas.

Image and video analysis using graph neural network for Internet of Medical Things and computer vision applications

AbstractGraph neural networks (GNNs) have revolutionised the processing of information by facilitating the transmission of messages between graph nodes. Graph neural networks operate on graph‐structured data, which makes them suitable for a wide variety of computer vision problems, such as link prediction, node classification, and graph classification. The authors explore in depth the applications of GNNs in computer vision, including their design considerations, architectural challenges, applications, and implementation concerns. While conventional convolutional neural networks (CNNs) excel at object recognition in images and videos, GNN architectures offer a novel method for addressing various image and video comprehension challenges. A novel deep neural network‐based model for image and video analysis is proposed, which combines a neural network with fully connected layers on a graph. The proposed architecture extracts highly discriminative information from images and videos by leveraging the graph structure. Also, the investigation focuses on the enhancement of underlying connection network estimation using cutting‐edge graph learning algorithms. Experimental results on real‐world datasets demonstrate that the proposed GNN model is preferable to existing state‐of‐the‐art methods. It obtains a remarkable 96.63% accuracy on the ImageNet dataset, outperforming heuristic approaches, artificial neural networks, and conventional CNN techniques. From the results, we can see that GNNs are a potent instrument for graph data analysis and pave the way for machines to achieve human‐level visual intuition.

FishSegSSL: A Semi-Supervised Semantic Segmentation Framework for Fish-Eye Images.

The application of large field-of-view (FoV) cameras equipped with fish-eye lenses brings notable advantages to various real-world computer vision applications, including autonomous driving. While deep learning has proven successful in conventional computer vision applications using regular perspective images, its potential in fish-eye camera contexts remains largely unexplored due to limited datasets for fully supervised learning. Semi-supervised learning comes as a potential solution to manage this challenge. In this study, we explore and benchmark two popular semi-supervised methods from the perspective image domain for fish-eye image segmentation. We further introduce FishSegSSL, a novel fish-eye image segmentation framework featuring three semi-supervised components: pseudo-label filtering, dynamic confidence thresholding, and robust strong augmentation. Evaluation on the WoodScape dataset, collected from vehicle-mounted fish-eye cameras, demonstrates that our proposed method enhances the model's performance by up to 10.49% over fully supervised methods using the same amount of labeled data. Our method also improves the existing image segmentation methods by 2.34%. To the best of our knowledge, this is the first work on semi-supervised semantic segmentation on fish-eye images. Additionally, we conduct a comprehensive ablation study and sensitivity analysis to showcase the efficacy of each proposed method in this research.

Proposed Approach for Object Detection and Recognition by Deep Learning Models Using Data Augmentation

Object detection and recognition play a crucial role in computer vision applications, ranging from security systems to autonomous vehicles. Deep learning algorithms have shown remarkable performance in these tasks, but they often require large, annotated datasets for training. However, collecting such datasets can be time-consuming and costly. Data augmentation techniques provide a solution to this problem by artificially expanding the training dataset. In this study, we propose a deep learning approach for object detection and recognition that leverages data augmentation techniques. We use deep convolutional neural networks (CNNs) as the underlying architecture, specifically focusing on popular models such as You Only Look Once version 3 (YOLOv3). By augmenting the training data with various transformations, such as rotation, scaling, and flipping, we can effectively increase the diversity and size of the dataset. Our approach not only improves the robustness and generalization of the models but also reduces the risk of overfitting. By training on augmented data, the models can learn to recognize objects from different viewpoints, scales, and orientations, leading to improved accuracy and performance. We conduct extensive experiments on benchmark datasets and evaluate the performance of our approach using standard metrics such as precision, recall, and mean average precision (mAP). The experimental results demonstrate that our data augmentation-based deep learning approach achieves superior object detection and recognition accuracy compared to traditional training methods without data augmentation. We compare the average accuracy of the YOLOv3-SPP model with two other variants of the YOLOv3 algorithm: one with a feature extraction network consisting of 53 convolutional layers and the other with 13 convolutional layers. The average accuracy of the proposed model (YOLOv3-SPP) is reported as accuracy of 97%, F1-score of 96%, precision of 94%, and average Intersection over Union (IoU) of 78.04%.

Detection of an in-housed pig using modified YOLOv5 model

Object detection has made significant strides in recent years, but it remains a challenging task to accurately and quickly identify and detect objects. While humans can easily recognize objects in images or videos regardless of their appearance, computers face difficulties in this task. Object detection plays a crucial role in computer vision and finds applications in various domains such as healthcare, security, agriculture, home automation and more. To address the challenges of object detection, several techniques have been developed including RCNN, Faster RCNN, YOLO and Single Shot Detector (SSD). In this paper, we propose a modified YOLOv5s architecture that aims to improve detection performance. Our modified architecture incorporates the C3Ghost module along with the SPP and SPPF modules in the YOLOv5s backbone network. We also utilize the Adam and Stochastic Gradient Descent (SGD) optimizers. The paper also provides an overview of three major versions of the YOLO object detection model: YOLOv3, YOLOv4 and YOLOv5. We discussed their respective performance analyses. For our evaluation, we collected a database of pig images from the ICAR-National Research Centre on Pig farm. We assessed the performance using four metrics such as Precision (P), Recall (R), F1-score and mAP @ 0.50. The computational results demonstrate that our method YOLOv5s architecture achieves a 0.0414 higher mAP while utilizing less memory space compared to the original YOLOv5s architecture. This research contributes to the advancement of object detection techniques and showcases the potential of our modified YOLOv5s architecture for improved performance in real world applications.

Safety Measure Detection Using Deep Learning

This implementation is for a computer vision application that detects individuals and verifies their compliance with safety gear regulations, such as safety jackets and hard-hats. The system counts the number of individuals violating safety standards and keeps track of the total number of individuals detected. The system uses advanced image processing techniques, including object detection and classification, to accurately identify the presence or absence of safety gear. The user interface provides real-time analysis of the data, with the option to alert the user of any violations. This implementation is a valuable tool for organizations looking to ensure the safety of their employees and customers, providing a comprehensive solution for monitoring compliance with safety regulations. It can also be used to analyze trends and identify areas for improvement, making it an essential tool for safety professionals and facilities managers.

OSPC: Online Sequential Photometric Calibration

Photometric calibration is essential to many computer vision applications. One of its key benefits is enhancing the performance of Visual SLAM, especially when it depends on a direct method for tracking, such as the standard KLT algorithm. Additionally, it proves valuable in extracting sensor irradiance values from measured intensities, serving as a pre-processing step for a number of vision algorithms such as shape-from-shading. Current photometric calibration systems rely on a joint optimization problem and encounter an ambiguity in the estimates, which can only be resolved using ground truth information. We propose a novel method that solves for photometric parameters using a sequential estimation approach. To enhance the decoupling of CRF and Vignette estimation, we strategically utilize keyframes with high exposure ratios and small displacements for the former, and keyframes with relatively large displacements for the latter. Our proposed method achieves high accuracy in estimating all parameters; furthermore, the formulations are linear and convex, which makes the solution fast and suitable for online applications. Experiments on a Visual Odometry system validate the proposed method and demonstrate its advantages.

XVDPU: A High-Performance CNN Accelerator on the Versal Platform Powered by the AI Engine

Today, convolutional neural networks (CNNs) are widely used in computer vision applications. However, the trends of higher accuracy and higher resolution generate larger networks. The requirements of computation or I/O are the key bottlenecks. In this article, we propose XVDPU: the AI Engine (AIE)-based CNN accelerator on Versal chips to meet heavy computation requirements. To resolve the IO bottleneck, we adopt several techniques to improve data reuse and reduce I/O requirements. An arithmetic logic unit is further proposed that can better balance resource utilization, new feature support, and efficiency of the whole system. We have successfully deployed more than 100 CNN models with our accelerator. Our experimental results show that the 96-AIE-core implementation can achieve 1,653 frames per second (FPS) for ResNet50 on VCK190, which is 9.8× faster than the design on ZCU102 running at 168.5 FPS. The 256-AIE-core implementation can further achieve 4,050 FPS. We propose a tilling strategy to achieve feature-map-stationary for high-definition CNN with the accelerator, achieving 3.8× FPS improvement on the residual channel attention network and 3.1× on super-efficient super-resolution. This accelerator can also solve the 3D convolution task in disparity estimation, achieving end-to-end performance of 10.1 FPS with all the optimizations.

Applications of Machine Learning and Computer Vision in Industry 4.0

Among the most important economic activities of humankind is industry [...]

Evolution and Efficiency in Neural Architecture Search: Bridging the Gap between Expert Design and Automated Optimization

The paper provides a comprehensive overview of Neural Architecture Search (NAS), emphasizing its evolution from manual design to automated, computationally driven approaches. It covers the inception and growth of NAS, highlighting its application across various domains, including medical imaging and natural language processing. The document details the shift from expert-driven design to algorithm-driven processes, exploring initial methodologies like reinforcement learning and evolutionary algorithms. It also discusses the challenges of computational demands and the emergence of efficient NAS methodologies, such as Differentiable Architecture Search and hardware-aware NAS. The paper further elaborates on NAS's application in computer vision, NLP, and beyond, demonstrating its versatility and potential for optimizing neural network architectures across different tasks. Future directions and challenges, including computational efficiency and the integration with emerging AI domains, are addressed, showcasing NAS's dynamic nature and its continued evolution towards more sophisticated and efficient architecture search methods.

AI Powered Echocardiography

The purpose of this paper is to discuss the current technological developments in diagnostic cardiovascular care. Echocardiography, a widely known imaging tool, extracts insights about a patients’ cardiac anatomy and perform necessary treatments or procedures based on their diagnoses. AI models respond to vast amounts of raw cardiac data and use Deep Learning algorithms to identify images with remarkable speed and accuracy. AI applications in computer vision offer key benefits in the healthcare industry. Companies such as Siemens are the key players – the commercialization of new AI technology has enabled healthcare organizations to streamline workflows, reduce errors, and lower costs.Potentially, there will be no reproducibility issues thereby redirecting clinical efforts towards patient treatment planning and research to prevent uptrends of heart disease.

Deep CNN Model for Condition Monitoring of Road Traffic: An Application Of Computer Vision

The traffic surveillance system accumulates an enormous amount of data regarding road traffic each second. Monitoring these data with the human eye is a tedious process and it also requires manpower for monitoring. Deep learning Convolutional Neural Network (DLCNN) can be utilized for traffic monitoring and control. The traffic surveillance data are preprocessed to construct the training dataset. The Traffic net is constructed by transferring the network to traffic applications and retraining it with a self-established data set. This Traffic-net can be used for regional detection in large scale applications. Further, it can be implemented across-the-board. Further, DLCNN is used for prediction of traffic status i.e., dense traffic, low traffic, accident, and fire occurred from test sample. Finally, the simulations revealed that the proposed DLCNN resulted in superior performance as compared to existing model.

Black-Box Boundary Attack Based on Gradient Optimization

Deep neural networks have gained extensive applications in computer vision, demonstrating significant success in fundamental research tasks such as image classification. However, the robustness of these networks faces severe challenges in the presence of adversarial attacks. In real-world scenarios, addressing hard-label attacks often requires the execution of tens of thousands of queries. To combat these challenges, the Black-Box Boundary Attack leveraging Gradient Optimization (GOBA) has been introduced. This method employs a binary search strategy to acquire an initial adversarial example with significant perturbation. The Monte Carlo algorithm is utilized to estimate the gradient of the sample, facilitating iterative movement along the estimated gradient and the direction of the malicious label. Moreover, query vectors positively correlated with the gradient are extracted to construct a sampling space with an optimal scale, thereby enhancing the efficiency of the Monte Carlo algorithm. Experimental evaluations were conducted using the HSJA, QEBA, and NLBA attack methodologies on the ImageNet, CelebA, and MNIST datasets, respectively. The results indicate that, under the constraint of 3 k query times, the GOBA, compared to other methods, can, on average, reduce perturbation (L2 distance) by 55.74% and simultaneously increase the attack success rate by an average of 13.78%.

GAIT analysis based on GENDER detection using pre-trained models and tune parameters

In past several decades, gait biometrics has emerged as a viable alternative to traditional identification methods, offering advancements in surveillance, monitoring, and analysis techniques. However, determining gender based on gait remains a challenge, particularly in computer vision applications. This study proposes a robust and adaptable approach to address this issue by leveraging gait analysis. There is a growing need for datasets tailored to gait analysis and recognition to facilitate the extraction of relevant data. While most existing research relies on image-based gait datasets, this study utilizes the OULP-Age dataset from OU-ISIR, representing gait through gait energy images (GEIs). The methodology involves feature extraction from GEIs using pre-trained models, followed by classification with the XGBoost classifier. Gender prediction is enhanced through parameter fine-tuning of the XGBoost classifier. Comparative analysis of 11 pre-trained models for feature extraction reveals that DenseNet models, combined with optimized XGBoost parameters, demonstrate promising results for gender prediction. This study contributes to advancing gender prediction based on gait analysis and underscores the efficacy of integrating deep learning models with traditional classifiers for improved accuracy and reliability.

Advancements in Computer Vision Applications for Traffic Surveillance Systems

With the increasing need for efficient traffic surveillance systems, computer vision emerges as a promising technology to address various challenges in this domain. This paper aims to provide an overview of recent advancements in the application of computer vision in traffic surveillance. It commences by discussing fundamental concepts of computer vision and its role in traffic monitoring, followed by an exploration of various techniques and algorithms used in computer vision systems, including object detection, tracking, and classification. Furthermore, it investigates the integration of computer vision with other technologies such as machine learning and deep learning, emphasizing their impact on enhancing the accuracy and efficiency of traffic surveillance systems. The review explores specific applications of computer vision in traffic surveillance, such as traffic flow monitoring, anomaly detection, and license plate recognition. Additionally, it addresses challenges and limitations associated with the implementation of computer vision in real-world traffic scenarios. Synthesizing existing literature, case studies, and research findings, this review offers a comprehensive insight into the current state-of-the-art in the application of computer vision for traffic surveillance systems. The paper concludes with a discussion of future directions and potential research opportunities in this rapidly evolving field.

Exploring Novel Self-Supervised Learning Techniques for Image Reconstruction Tasks

Image reconstruction tasks, such as super-resolution, inpainting, and denoising, play a crucial role in various computer vision applications. Traditional methods often rely heavily on large labeled datasets for training, which can be costly and time-consuming to acquire. Self-supervised learning has emerged as a promising alternative, aiming to reduce this dependency by leveraging the inherent structures within the data itself. In this paper, we explore novel self-supervised learning techniques tailored specifically for image reconstruction tasks. We propose approaches that exploit the inherent relationships between low and high-resolution images, utilize context-aware information for inpainting, and incorporate generative adversarial networks for denoising. Through extensive experimentation, we demonstrate the efficacy of our methods in achieving competitive performance compared to supervised approaches while significantly reducing the need for labeled data. Our findings pave the way for more efficient and scalable solutions in image reconstruction, offering practical benefits across a wide range of applications

A Compact Depth Separable Convolutional Image Filter for Clinical Color Perception Test

<p>Deep convolutional neural networks have achieved good performance in the application of computer vision, but there are also problems, such as a large amount of computation, time consuming, and high memory demand. In this paper, a depthwise separable convolution filter pruning method based on PCA is proposed. First, this paper uses depthwise separable convolution to replace the conventional convolution in ResNet to reduce the number of parameters and the amount of computation in the network. The specific operation process is to first use depthwise convolution to separate the spatial dimension to increase the network width and expand the range of feature extraction, and then use pointwise convolution to reduce the computational complexity of conventional convolution operation. Second, PCA is used to distinguish stacked similar filters and perform dimensionality reduction, which not only alleviates the dimensional disaster, but also achieves compression of data and minimizes information loss. Experimental results show that this method can significantly improve the calculation speed and accuracy of the deep convolutional neural network model, and further compress the model size. On the clinical Color Perception Test Chart, this method reduced the amount of model parameters and MACs on ResNet by about 91% while maintaining the test accuracy at about 95%. With almost no loss of accuracy, this method greatly shortened the running time of the model.</p> <p> </p>

A practical guide to the development and deployment of deep learning models for the orthopaedic surgeon: Part III, focus on registry creation, diagnosis,and data privacy.

Deep learning is a subset of artificial intelligence (AI) with enormous potential to transform orthopaedic surgery. As has already become evident with the deployment of Large Language Models (LLMs) like ChatGPT (OpenAI Inc.), deep learning can rapidly enter clinical and surgical practices. As such, it is imperative that orthopaedic surgeons acquire a deeper understanding of the technical terminology, capabilitiesand limitations associated with deep learning models. The focus of this series thus far has been providing surgeons with an overview of the steps needed to implement a deep learning-based pipeline, emphasizing some of the important technical details for surgeons to understand as they encounter, evaluateor lead deep learning projects. However, this series would be remiss without providing practical examples of how deep learning models have begun to be deployed and highlighting the areas where the authors feel deep learning may have the most profound potential. While computer vision applications of deep learning were the focus of Parts I and II, due to the enormous impact that natural language processing (NLP) has had in recent months, NLP-based deep learning models are also discussed in this final part of the series. In this review, three applications that the authors believe can be impacted the most by deep learning but with which many surgeons may not be familiar are discussed: (1) registry construction, (2) diagnostic AIand (3) data privacy. Deep learning-based registry construction will be essential for the development of more impactful clinical applications, with diagnostic AI being one of those applications likely to augment clinical decision-making in the near future. As the applications of deep learning continue to grow, the protection of patient information will become increasingly essential; as such, applications of deep learning to enhance data privacy are likely to become more important than ever before. Level of Evidence: Level IV.

Detecting Shadows in Computer Vision: A MATLAB-Based Approach

The abstract outlines a novel MATLAB-based technique for enhancing shadow detection accuracy in computer vision applications. This MATLAB-based method uses thresholding and color analysis to improve the accuracy of shadow identification. Luminance (L*) is distinguished from chromaticity (a* and b*) in digital photographs by converting them into the Lab color space. This makes it easier to identify possible shadowed areas. The luminance channel's thresholding makes it possible to distinguish between regions that are well-lit and those that are shadowed. The computation of chromaticity distance also helps in detecting modest color changes that are suggestive of shadows. The culmination of the suggested technique is the creation of a binary shadow mask, which isolates the image's shaded regions. The research addresses limitations of existing approaches by maintaining robustness in diverse lighting conditions and complex scenarios, thereby enhancing image comprehension and the precision of shadow detection. The proposed technique holds promise for improving object recognition and scene analysis in computer vision applications, offering a robust and reliable solution for identifying and isolating shadows in digital images.