Advanced Computer Vision Research Articles

Accelerating biomedical image segmentation using equilibrium optimization with a deep learning approach

<abstract> <p>Biomedical image segmentation is a vital task in the analysis of medical imaging, including the detection and delineation of pathological regions or anatomical structures within medical images. It has played a pivotal role in a variety of medical applications, involving diagnoses, monitoring of diseases, and treatment planning. Conventionally, clinicians or expert radiologists have manually conducted biomedical image segmentation, which is prone to human error, subjective, and time-consuming. With the advancement in computer vision and deep learning (DL) algorithms, automated and semi-automated segmentation techniques have attracted much research interest. DL approaches, particularly convolutional neural networks (CNN), have revolutionized biomedical image segmentation. With this motivation, we developed a novel equilibrium optimization algorithm with a deep learning-based biomedical image segmentation (EOADL-BIS) technique. The purpose of the EOADL-BIS technique is to integrate EOA with the Faster RCNN model for an accurate and efficient biomedical image segmentation process. To accomplish this, the EOADL-BIS technique involves Faster R-CNN architecture with ResNeXt as a backbone network for image segmentation. The region proposal network (RPN) proficiently creates a collection of a set of region proposals, which are then fed into the ResNeXt for classification and precise localization. During the training process of the Faster RCNN algorithm, the EOA was utilized to optimize the hyperparameter of the ResNeXt model which increased the segmentation results and reduced the loss function. The experimental outcome of the EOADL-BIS algorithm was tested on distinct benchmark medical image databases. The experimental results stated the greater efficiency of the EOADL-BIS algorithm compared to other DL-based segmentation approaches.</p> </abstract>

Automatic Recognition of Bio-Colonization Processes on Historic Facades: Application on Case Studies

Many factors (physical, chemical, natural, and human activities) contribute to the degradation of historic buildings. Preventive conservation is a cost-effective approach international preservation bodies recommend to mitigate risks to built cultural heritage. A substantial challenge is bio-colonization, especially by microalgae, which affects brick-facing masonry surfaces due to environmental factors (i.e., temperature and moisture), leading to progressively increasing deterioration. Early detection systems could be useful to reduce damage from these organisms. Advances in computer vision and machine learning, such as convolutional neural networks, offer promising solutions for automating the identification of building pathologies using image collection. This research focuses on developing predictive models using convolutional neural networks to monitor bio-deterioration on historic facades, specifically targeting early-stage microalgae colonization. After a training phase using laboratory-induced bio-colonization on brick samples, the method was applied to real case studies of architecturally significant buildings affected by bio-colonization. In fact, a substantial number of digital images of these buildings, even if taken for other purposes, are available. The work shows that analyzing these images with the trained network facilitates the early detection of bio-colonization, providing a contribution to the field of built cultural heritage conservation.

Revolutionising the Sustainability of Steel Manufacturing Using Computer Vision

The pressure of the global sustainability problem is growing faster than ever after the social and economic havoc wreaked by the pandemic, as well as the present time nearing to the 2050 net zero carbon goal. In 2021 the steel industry had a turnover of around €125 billion and was directly responsible for providing at least 300000 jobs. In 2022, it was responsible for roughly 5% of CO2 emissions in the EU and 7% globally, and was responsible for at least 20278 injuries and 90 deaths. Environmental solutions for mitigating negative consequences of the steel industry exist such as hydrogen-based steelmaking, carbon capture, utilisation and storage and electrolysis, whilst safety regulations and personal protective equipment have been used to improve safety. However, little has been done to combat the problem using the most powerful emerging technology of the fourth industrial revolution, artificial intelligence. In particular, computer vision has already shown great competence in a range of applications related to steelmaking, but without seriously considering sustainability, resulting in limited awareness of the potential benefits computer vision can bring to industry. A lack of this awareness leads to missed opportunities for sustainable development. This paper aims to address the gap in research that discusses computer vision capabilities for enhancing the sustainability of steel production by providing a literature review covering recent advances in computer vision, as well as an industry 4.0 approach for integrating computer vision systems with steelworks. Research presented here successfully exposes the untapped potential of computer vision in the steel industry and paves the way for future developments by exhibiting a blueprint for simultaneously elevating sustainability and technological advancement.

Swin-GA-RF: genetic algorithm-based Swin Transformer and random forest for enhancing cervical cancer classification.

Cervical cancer is a prevalent and concerning disease affecting women, with increasing incidence and mortality rates. Early detection plays a crucial role in improving outcomes. Recent advancements in computer vision, particularly the Swin transformer, have shown promising performance in image classification tasks, rivaling or surpassing traditional convolutional neural networks (CNNs). The Swin transformer adopts a hierarchical and efficient approach using shifted windows, enabling the capture of both local and global contextual information in images. In this paper, we propose a novel approach called Swin-GA-RF to enhance the classification performance of cervical cells in Pap smear images. Swin-GA-RF combines the strengths of the Swin transformer, genetic algorithm (GA) feature selection, and the replacement of the softmax layer with a random forest classifier. Our methodology involves extracting feature representations from the Swin transformer, utilizing GA to identify the optimal feature set, and employing random forest as the classification model. Additionally, data augmentation techniques are applied to augment the diversity and quantity of the SIPaKMeD1 cervical cancer image dataset. We compare the performance of the Swin-GA-RF Transformer with pre-trained CNN models using two classes and five classes of cervical cancer classification, employing both Adam and SGD optimizers. The experimental results demonstrate that Swin-GA-RF outperforms other Swin transformers and pre-trained CNN models. When utilizing the Adam optimizer, Swin-GA-RF achieves the highest performance in both binary and five-class classification tasks. Specifically, for binary classification, it achieves an accuracy, precision, recall, and F1-score of 99.012, 99.015, 99.012, and 99.011, respectively. In the five-class classification, it achieves an accuracy, precision, recall, and F1-score of 98.808, 98.812, 98.808, and 98.808, respectively. These results underscore the effectiveness of the Swin-GA-RF approach in cervical cancer classification, demonstrating its potential as a valuable tool for early diagnosis and screening programs.

Battery Energy Storage System Planning Based on Super-Resolution Source-Load Uncertainty Reconstruction

Low-time resolution electricity data have been used to drive battery energy storage system (BESS) planning due to data barriers. However, the coarse-resolution time series cannot reflect real power variation, and the planning results may be inappropriate due to the unrealistic representation of source-load uncertainties. To this end, this paper proposes a BESS planning method based on super-resolution (SR) source-load uncertainty reconstruction. The Gramian angular field (GAF) is employed to transform the 1D time series into 2D images to enrich the time dependency of signals. With advanced computer vision (CV), the entropy decrease generative adversarial network (EDGAN) based on information-growth attention is designed to recover the source-load profile from low resolution to high resolution. To depict source-load uncertainties in BESS sizing, a two-stage multi-distributionally robust optimization (2S-MDRO) is established under SR ambiguity sets. Case studies validated the accuracy of the proposed SR method and proved the effectiveness of the proposed sizing model. The recommended time resolution for BESS planning is presented in the comparative results.

Optimized real-time sudoku puzzle solving and solution overlay on images and videos using advanced computer vision and TensorFlow

This paper presents a novel approach to solving Sudoku puzzles using computer vision techniques. Sudoku, a popular logic-based number placement game, poses an interesting challenge for automated solutions. The proposed Sudoku solver employs image processing and computer vision algorithms to extract the puzzle grid from a given image. The system then employs optical character recognition (OCR) to recognize and digitize the numbers within the grid. The image preprocessing involves techniques such as edge detection, contour identification, and perspective transformation to enhance the accuracy of grid extraction. Subsequently, OCR is applied to recognize the digits within each cell of the Sudoku grid. The solver then utilizes a backtracking algorithm to systematically fill in the missing numbers while adhering to the rules of Sudoku. The computer vision-based Sudoku solver offers advantages in terms of versatility, as it can handle puzzles of varying difficulty levels and appearances. The system's performance is evaluated on a diverse set of Sudoku puzzles, demonstrating its effectiveness in accurately solving puzzles with different grid sizes and layouts. The results indicate that the proposed approach achieves competitive solving times while maintaining a high level of accuracy. The presented Sudoku solver serves as a testament to the potential of computer vision in solving complex puzzles, opening avenues for further research in the integration of artificial intelligence and image processing for recreational problem-solving applications.

A computational framework for quantifying electrical conductance in metallic nanomesh using image processing and computer vision technologies.

This study introduces a computational framework for precisely quantifying electrical conductance in metallic nanomesh structures, leveraging advanced image processing and computer vision algorithms. Nanomesh images obtained via scanning electron microscopy are subjected to preprocessing operations, including threshold-based binary conversion and convolution techniques, to mitigate defects and delineate conductive pathways. A rigorous equivalent electrical path model is subsequently established through keypoint identification via mean-shift segmentation. Kirchhoff's current law is applied to the model to deduce the conductance of the nanomesh. The computationally estimated conductance results are stringently validated against experimental measurements, confirming the model's accuracy. Further, the framework is applied to ascertain the resistivity of nanoscale Ag films deposited on glass substrates, revealing a resistivity higher than that of bulk materials-a finding corroborated by both computational predictions and experimental data. The methodology can be a robust, automated analytical tool for assessing conductance in comparable nanostructured materials.

Alleviate light pollution by recognizing urban night-time light control area based on computer vision techniques and remote sensing imagery

Light pollution caused by unreasonable and excessive use of outdoor artificial light at night (ALAN) has become one of the fastest growing and most urgent global environmental problems. Therefore, the International Dark-Sky Association advocated the more controlled use of ALAN in urban area to protect night-time environment and enable sustainable development. However, relevant research and lighting planning are still in the enlightenment stage. This study focused on a typical city experiencing significant ALAN growth, combining night-time remote sensing imagery, urban big data and other muti-source heterogeneous data to systematically research urban night-time light status. We first analyzed the spatio-temporal evolution pattern of ALAN. Subsequently, advanced computer vision algorithms were proposed to recognize the night-time light control area (NLCA) based on the perspective of supply and demand balance. Results suggest both the ALAN intensity and area have grown rapidly over the past decade in the study area, with the intensity increasing by 82 % and the area expanding by 42 %. And a 108.5-square-kilometre area of the NLCA was intelligently delineated, for where the ALAN intensity was more than the demand of population, indicating that the management of the night-time light environment an urgent matter. Therefore, this study proposed practical management solutions according to the different functional zones in city within the NLCA, for further promoting the green transformation and smart upgrading of urban outdoor lighting. To sum up, our research offers a novel approach to resolving the issue of urban light pollution by recognizing the NLCA, as well as facilitating energy conservation and emission reduction.

Optimized zone-based vehicle speed estimation and classification

Abstract. This article pioneers the fusion of advanced computer vision, and environmental science in order to be a starting point in ecological tasks and environmental benefits. Utilizing state-of-the-art tools like YOLOv7 and innovative algorithms, the study achieves unmatched accuracy in vehicle identification, classification, tracking, and speed analysis. By optimizing YOLOv7-e6e-1280 architecture using TensorRT and reduced precision, real-time analysis becomes possible without compromising accuracy. The integration of the Vanishing Point Principle for road zoning and zone-based speed calculation provides nuanced insights into driving behaviors. Detailed vehicle classification and robust tracking offer valuable data for urban planning and ecological studies. This approach increase our potential in vehicular analysis, setting new standards for research in urban development, transportation, and environmental science.

Object Recognition System for the Spinbotics Robotic Arm

This study focuses on the development of a visual system designed to facilitate object detection for the Spinbotics robotic arm in spatial environments. The primary objective is to enable accurate detection and classification of diverse objects, enhancing the arm's capability to grasp and manipulate items effectively. The system employs the YOLOv7 deep neural network, fine-tuned using transfer learning on a local computing infrastructure. Compared to traditional methods like R-CNN and SSD, YOLOv7 offers superior real-time processing capabilities and efficiency, making it well-suited for dynamic environments. Through extensive training and testing, the system demonstrates robust performance in detecting objects across varied scenes and identifying optimal grasp points. This research underscores the effectiveness of integrating advanced computer vision techniques to enhance the operational efficiency and versatility of robotic manipulators in real-world applications.

Exploring the Application of Computer Visual Arts in the Context of Digital Media

Abstract This paper provides an in-depth discussion on the critical technologies of computer visual arts in digital media, mainly focusing on virtual character modeling and human body tracking technologies. The study merges the models into the same coordinate system by combining point cloud data from different angles and calculating the transformation matrix. Accurate 3D human body models were reconstructed using point cloud alignment and fusion techniques. A particle filtering method combined color, motion and shape information for effective tracking. The experimental results show that the success rate of hand tracking reaches 95.35% in the case of no background interference, while the success rate is 90.97% in the environment with background interference. In addition, the study shows that low resolution images (584×388) outperform high resolution (968×676) in terms of tracking accuracy and speed. The study confirms that the combination of advanced computer vision techniques can effectively construct and follow virtual character models and realize high-precision human motion capture under different environmental conditions.

Rapid-Motion-Track: Markerless tracking of fast human motion with deep learning

Human movement patterns reflect central nervous system function. Small deficits in repetitive fast movements, such as slightly slowed finger-tapping or mildly irregular rhythm of stepping, are often an early sign of a neurological disorder. Accessible tools that precisely measure the individual components of fast movements would thus enhance disease detection, monitoring and research. Deep learning-based computer vision methods applied to digital video-recordings hold promise but current state-of-the-art tools, including DeepLabCut (DLC) and other advanced models, fail to accurately track the fastest range of human movements, primarily due to image blur. To solve this, we developed a new end-to-end, Rapid-Motion-Track (RMT) computer vision tool. This study aimed to evaluate the accuracy of RMT compared to DLC and other advanced computer vision tools. 220 finger-tapping tests were performed at frequencies between 0.5Hz and 6Hz and recorded simultaneously with a standard 30 frames/sec 2D laptop camera and a high-speed 250 frames/sec 3D motion tracking system (ground-truth). Bland-Altman plots and paired Welch's t-test were used to quantify the validity of movement features extracted by computer vision methods with the ground-truth. The movement features extracted by RMT (e.g. frequency, speed, variance) exhibited high concurrent validity across all tapping-frequencies. RMT outperformed other computer vision methods for very fast movements >4Hz. RMT also robustly tracked other fast motions including sit-to-stand, head-turning, foot-tapping, and leg agility. This new tool provides an accurate method to precisely and automatically measure even the fastest and finest human movements. It holds potential of wide reach as digital cameras are ubiquitous in homes, clinics and research centres.

Use of laser technology for the postural classification of bedridden people

This work presents an innovative method for the automated classification of postures of bedridden people using laser technology. With the aim of improving the quality of medical care and facilitating the continuous monitoring of patients in clinical and home environments, a system is proposed that uses laser sensors to capture the three-dimensional geometry of body postures. Through advanced computer vision processing techniques, a robust classification algorithm is developed capable of identifying and categorizing lateral decubitus postures, also known as lateral safety position, which are commonly used to lay down patients who are permanently bedridden. Experimental results, performed so far, show sufficient accuracy in posture classification, suggesting the potential of this technology to improve the monitoring and care of bedridden patients, while providing periodic warnings to medical staff when a patient has exceeded the recommended time in the same posture, to avoid the appearance of skin ulcers.

Image to Audio Conversion for Blind People Using Neural Network

Abstract: The development of an image-to-audio conversion system represents a significant stride towards enhancing accessibility and autonomy for visually impaired individuals. This innovative technology leverages computer vision and audio synthesis techniques to convert visual information from images into auditory cues, enabling blind users to interpret and comprehend their surroundings more effectively. The core of this system relies on advanced computer vision algorithms that process input images, recognizing objects, text, and scene elements. These algorithms employ deep learning models to extract meaningful visual features and convert them into a structured representation of the image content. Simultaneously, natural language processing techniques are employed to extract and interpret textual information within the image, such as signs, labels, or written instructions. Once the image content is comprehended, an audio synthesis engine generates a corresponding auditory output. This auditory output is designed to convey the information in a clear and intuitive manner. Additionally, the system can adapt its output based on user preferences and environmental context, providing a customizable and dynamic auditory experience. It empowers blind individuals to independently access visual information from a variety of sources, including printed materials, digital displays, and real-world scenes. Moreover, it promotes inclusion by reducing the reliance on sighted assistance and fostering greater self-reliance and confidence among visually impaired individuals. By harnessing computer vision and audio synthesis, it provides a means for blind individuals to access and interpret visual information independently, thereby enhancing their autonomy, inclusion, and overall quality of life. This innovative solution underscores the potential of technology to bridge accessibility gaps and empower individuals with disabilities.

Pneumonia Detection using Chest X-Ray Images and Machine Learning Techniques

Abstract— Chest X-rays have been a crucial diagnostic tool in detecting pneumonia. The disease manifests as inflammation and filling of the air spaces within the lungs, leading to changes in the appearance of the lungs on X-rays. In recent times, advances in computer vision and machine learning have allowed for the development of automated systems for pneumonia detection using chest X-rays. These systems leverage deep learning techniques to analyze X-ray images and identify patterns associated with pneumonia. The use of these systems can aid in early diagnosis and prompt treatment, potentially leading to improved patient outcomes. Despite their promise, there are still challenges to be addressed, including the need for large and diverse datasets for training and the need for further validation of these systems in real-world settings. Nevertheless, the development of automated pneumonia detection systems using chest X-rays has the potential to revolutionize the field of radiology and improve patient care. Index Terms—Pneumonia; artificial intelligence; machine learning; image processing; convolutional neural network; chest x-rays.

SOCA-PRNet: Spatially Oriented Attention-Infused Structured-Feature-Enabled PoseResNet for 2D Human Pose Estimation.

In the recent era, 2D human pose estimation (HPE) has become an integral part of advanced computer vision (CV) applications, particularly in understanding human behaviors. Despite challenges such as occlusion, unfavorable lighting, and motion blur, advancements in deep learning have significantly enhanced the performance of 2D HPE by enabling automatic feature learning from data and improving model generalization. Given the crucial role of 2D HPE in accurately identifying and classifying human body joints, optimization is imperative. In response, we introduce the Spatially Oriented Attention-Infused Structured-Feature-enabled PoseResNet (SOCA-PRNet) for enhanced 2D HPE. This model incorporates a novel element, Spatially Oriented Attention (SOCA), designed to enhance accuracy without significantly increasing the parameter count. Leveraging the strength of ResNet34 and integrating Global Context Blocks (GCBs), SOCA-PRNet precisely captures detailed human poses. Empirical evaluations demonstrate that our model outperforms existing state-of-the-art approaches, achieving a Percentage of Correct Keypoints at 0.5 (PCKh@0.5) of 90.877 at a 50% threshold and a Mean Precision (Mean@0.1) score of 41.137. These results underscore the potential of SOCA-PRNet in real-world applications such as robotics, gaming, and human-computer interaction, where precise and efficient 2D HPE is paramount.

Alleviating distribution shift and mining hidden temporal variations for ultra-short-term wind power forecasting

Randomness and non-stationarity are common challenges in wind power forecasting (WPF). Many studies focus on randomness but usually ignore the non-stationarity which leads to distribution shift and affects prediction accuracy. To address the distribution shift problem, an alleviating distribution shift using the recent difference characterization (Dish-RDC) method is proposed as a general neural paradigm for WPF. Dish-RDC categorizes the distribution shift into intra-space and inter-space shifts. By employing the RDC, the method facilitates the mapping of input sequences to learnable distribution coefficients that better estimate the distribution. Furthermore, real-world time series often exhibit multi-periodicity, yet existing models face limitations in capturing this temporal variation. To address this issue, our research introduces the Temporal 2D-Variation Model (TimesNet) in WPF. This innovative model extends time variation analysis into a 2D space based on multi-periodicity. By using 2D kernels to model these variations, TimesNet can effectively incorporate advanced computer vision techniques into WPF. Combining these approaches, we developed Dish-RDC-TimesNet, a hybrid model. Experiments show it reduced mean absolute error (MAE) by 47.10 % and 20.63 % on two datasets compared to benchmark models. Moreover, integrating Dish-RDC with benchmark models decreased MAE by 39.79 % and 17.85 % on these datasets.

RescueNet: A High Resolution UAV Semantic Segmentation Dataset for Natural Disaster Damage Assessment

Recent advancements in computer vision and deep learning techniques have facilitated notable progress in scene understanding, thereby assisting rescue teams in achieving precise damage assessment. In this paper, we present RescueNet, a meticulously curated high-resolution post-disaster dataset that includes detailed classification and semantic segmentation annotations. This dataset aims to facilitate comprehensive scene understanding in the aftermath of natural disasters. RescueNet comprises post-disaster images collected after Hurricane Michael, obtained using Unmanned Aerial Vehicles (UAVs) from multiple impacted regions. The uniqueness of RescueNet lies in its provision of high-resolution post-disaster imagery, accompanied by comprehensive annotations for each image. Unlike existing datasets that offer annotations limited to specific scene elements such as buildings, RescueNet provides pixel-level annotations for all classes, including buildings, roads, pools, trees, and more. Furthermore, we evaluate the utility of the dataset by implementing state-of-the-art segmentation models on RescueNet, demonstrating its value in enhancing existing methodologies for natural disaster damage assessment.

A computer vision-based concept model to recommend domestic overseas-like travel experiences: A design science study

Travel location recommendation systems have long been used by travellers for their ability to suggest destinations and potential travel experiences that match travellers' desires. Recently, a new type of domestic travel has emerged, namely domestic overseas-like travel experiences. These experiences are attractive to travellers who have a preference for exotic locations but no desire to travel internationally. Existing travel recommendation systems were not designed for such applications, nor do they have the relevant ability to recommend domestic overseas-like travel experiences to support travel decision making. To address this challenge, this paper focuses on the development of a recommendation model based on the visual content of photos for domestic overseas-like travel experiences and a prototype application. The application uses the latest advancement in computer vision — the concept model — to learn high-level concepts in an overseas travel destination photo collection to identify similar domestic travel experiences. We demonstrate the usability of the prototype application with a large-scale data set of approximately 479,000 travel photos taken in several countries and evaluate its utility and efficacy through four focus groups with target users.

Advancing Tuberculosis Detection in Chest X-rays: A YOLOv7-Based Approach

In this work, we propose a CAD (computer-aided diagnosis) system using advanced deep-learning models and computer vision techniques that can improve diagnostic accuracy and reduce transmission risks using the YOLOv7 (You Only Look Once, version 7) object detection architecture. The proposed system is capable of accurate object detection, which provides a bounding box denoting the area in the X-rays that shows some possibility of TB (tuberculosis). The system makes use of CNNs (Convolutional Neural Networks) and YOLO models for the detection of the consolidation of cavitary patterns of the lesions and their detection, respectively. For this study, we experimented on the TBX11K dataset, which is a publicly available dataset. In our experiment, we employed class weights and data augmentation techniques to address the data imbalance present in the dataset. This technique shows a promising improvement in the model’s performance and thus better generalization. In addition, it also shows that the developed model achieved promising results with a mAP (mean average precision) of 0.587, addressing class imbalance and yielding a robust performance for both obsolete pulmonary TB and active TB detection. Thus, our CAD system, rooted in state-of-the-art deep-learning and computer vision methodologies, not only advances diagnostic accuracy but also contributes to the mitigation of TB transmission risks. The substantial improvement in the model’s performance and the ability to handle class imbalance underscore the potential of our approach for real-world TB detection applications.