Vision Applications Research Articles

Innovative Application of Computer Vision and Motion Tracking Technology in Sports Training

The use of cutting-edge technology has resulted in a significant enhancement in athletic training. Computer vision and motion tracking are very important for enhancing performance, reducing the risk of accidents, and training in general. Some computer vision algorithms investigate how a sportsperson moves when competing or practising. It is possible that coaches who continuously evaluate their players’ posture, muscle activation, and joint angles would have a better understanding of biomechanical efficiency. It is possible to generate performance measurements from the real-time surveillance of athletes while competing in sports. Through the use of computer vision, it is possible to identify acts that might be hazardous. Notifications are given to coaches if there is a deviation in the form of an athlete, which enables them to address the situation as soon as possible. The three variables that these sensors monitor are the direction, speed, and acceleration. Athletes can encounter realistic environments thanks to the integration of motion tracking with virtual reality. One may use the feedback loop to increase their spatial awareness and decision-making ability. Augmented reality allows for enhancing an athlete’s eyesight by providing them with real-time data while practising. Last but not least, the use of computer vision and motion tracking is bringing about a significant improvement in the sporting training process. Through collaborative efforts, researchers, athletes, and coaches can accelerate humans' performance to levels that have never been seen before.

Multimodal Convolutional Neural Networks for Sperm Motility and Concentration Predictions

Semen analysis is an important analysis for male infertility primary investigation and manual semen analysis is a conventional method to assess it. Manual semen analysis has been revealed with accuracy and precision limitations due to noncompliance to guidelines and procedures. Sperm motility and concentration are the main indicators for pregnancy and conception rate hence they were selected for parameters prediction. Convolutional neural network (CNN) has benefited computer vision application industry in recent years and has been widely applied in computer vision research tasks. In this paper, three-dimensional CNN (3DCNN) was designed to extract motion and temporal features, which are vital for sperm motility prediction. For sperm concentration, since two-dimensional CNN (2DCNN) is efficient in recognizing and extracting spatial features, well-established Residual Network (ResNet) architecture was adopted and customized for sperm concentration prediction. Multimodal learning approach is a technique to aggregate learnt features from different deep learning architecture that adopted other forms of modalities, which could provide deep learning model with better insights on their tasks. Hence, a multimodal learning deep learning architecture was designed to receive both image-based (frames extracted from video samples) and video-based (stacked frames pre-processed from video samples) input that could provide well-extracted spatial and temporal features for sperm parameters prediction. The results obtained using the proposed methodology have surpassed other similar research works who used deep learning approach. For sperm motility, its best achieved average mean absolute error (MAE) was 8.048, and sperm concentration obtained a competent Pearson’s correlation coefficient (RP) value of 0.853.

Unveiling the Future of Human and Machine Coding: A Survey of End-to-End Learned Image Compression.

End-to-end learned image compression codecs have notably emerged in recent years. These codecs have demonstrated superiority over conventional methods, showcasing remarkable flexibility and adaptability across diverse data domains while supporting new distortion losses. Despite challenges such as computational complexity, learned image compression methods inherently align with learning-based data processing and analytic pipelines due to their well-suited internal representations. The concept of Video Coding for Machines has garnered significant attention from both academic researchers and industry practitioners. This concept reflects the growing need to integrate data compression with computer vision applications. In light of these developments, we present a comprehensive survey and review of lossy image compression methods. Additionally, we provide a concise overview of two prominent international standards, MPEG Video Coding for Machines and JPEG AI. These standards are designed to bridge the gap between data compression and computer vision, catering to practical industry use cases.

Analysis of Music Brand Similarity Levels Using a Visual Computing Approach

Currently, many trademarks have emerged, especially for music equipment products circulating in the community. Especially in Indonesia, musical instrument brands with various models have emerged. The problem occurs when the brand is considered to resemble the original brand, which makes it uncomfortable for musicians. Even though the quality of the tone produced is not assessed from a brand perspective, it is felt by brand owners to be quite a violation of the code of ethics. This has an impact on marketing products that are considered genuine. In this paper, a concept will be proposed in determining whether a trademark is considered authentic in terms of the logo. Use of Visual Computing with SIFT and SURF algorithms. SIFT (Scale-Invariant Feature Transform) and SURF (Speeded-Up Robust Features) are two popular algorithms for feature extraction and matching in image processing. Both are frequently used in computer vision applications such as object detection, image matching, and object recognition. The results of this analysis will be used for musicians who want to buy musical equipment to be able to detect the logo to compare the success of a musical instrument product. The appropriateness of the quality itself will be discussed in a different discussion. The benefits that can be generated will make musicians, especially in Indonesia, more conscientious. The possibility of sorting so that you can like your own product will be very possible.

КОМПЛЕКСНІ ОНТОЛОГІЧНІ ТА НЕЙРОМЕРЕЖЕВІ МОДЕЛІ ФОТОГРАФІЧНИХ ОБРАЗІВ

This article explores an innovative approach to developing computer vision systems, based on the deep integration of complex ontological and neural network models for effective analysis of photographic images. This work proposes a novel hybrid method in the context of contemporary challenges associated with image recognition, such as a high diversity of classes and problems with the occlusion of objects in low-quality photos. This method provides computer vision models with a deeper understanding of the context and semantics of images, utilizing ontological structures to represent visual content. The analytical process includes the identification of key objects in the image and the determination of their ontological context, allowing neural networks to effectively perform a gradient transformation of the input data for more accurate recognition and classification. The proposed model demonstrates the potential for gaining an advantage over traditional methods in computer vision tasks, opening new possibilities for expanding computer vision applications in scientific, industrial, and domestic spheres.

SwinSOD: Salient object detection using swin-transformer

The Transformer structure has achieved excellent performance in a wide range of applications in computer vision, and Swin-Transformer also shows strong feature representation capabilities. On this basis, we proposed a fusion model SwinSOD for RGB salient object detection. This model used a Swin-Transformer as the encoder to extract hierarchical features, was driven by a multi-head attention mechanism to bridge the gap between hierarchical features, progressively fused adjacent layer feature information under the guidance of global information, and refined the boundaries of saliency objects through the feedback information. Specifically, the Swin-Transformer encoder extracted multi-level features and then recalibrated the channels to optimize intra-layer channel features. The feature fusion module realized feature fusion between each layer under the guidance of global information. In order to clarify the fuzzy boundaries, the second stage feature fusion achieved edge refinement under the guidance of feedback information. The proposed model outperforms state-of-the-art models on five popular SOD datasets, demonstrating the advanced performance of this network. Code released: https://github.com/user-wu/SwinSOD.

Real-time multi-camera 3D human pose estimation at the edge for industrial applications

There is an increasing interest in exploiting human pose estimation (HPE) software in human–machine interaction systems. Nevertheless, adopting such a computer vision application in real industrial scenarios is challenging. To overcome occlusion limitations, it requires multiple cameras, which in turn require multiple, distributed, and synchronized HPE software nodes running on resource-constrained edge devices. We address this challenge by presenting a real-time distributed 3D HPE platform, which consists of a set of 3D HPE software nodes on edge devices (i.e., one per camera) to redundantly extrapolate the human pose from different points of view. A centralized aggregator collects the pose information through a shared communication network and merges them, in real time, through a pipeline of filtering, clustering and association algorithms. It addresses network communication issues (e.g., delay and bandwidth variability) through a two-levels synchronization, and supports both single and multi-person pose estimation. We present the evaluation results with a real case of study (i.e., HPE for human–machine interaction in an intelligent manufacturing line), in which the platform accuracy and scalability are compared with state-of-the-art approaches and with a marker-based infra-red motion capture system.

Image Morpher

The "Image Morphing with the Beier-Neely Method" research paper extensively explores image transformation techniques, with a special focus on the Beier- Neely approach. Image morphing, the art of seamlessly transforming one image into another, has garnered significant attention for its applications in animation, filmmaking, and scientific visualization. The research paper delves into the historical context and importance of image morphing, particularly emphasizing the role of the Beier-Neely method in addressing the limitations of earlier techniques. Classical image morphing methods, characterized by pixel-level interpolation, suffer from visual artifacts, manual correspondence definition, and sluggishness. These shortcomings set the stage for the introduction of the feature-based Beier-Neely method, which revolutionized image morphing by allowing control points on features, facilitating smooth transitions and natural transformations. This research papers objectives are clearly articulated, with the primary aim of implementing the Beier- Neely method to produce high-quality image morphs. The research paper proposes a system architecture that underscores the use of feature-based control points, Delaunay triangulation, affine transformations, and the significance of visual feedback. Additionally, it provides a detailed analysis of the mathematical foundations and algorithmic intricacies of the Beier-Neely method, elucidating the theoretical framework driving its effectiveness. Overall, the research paper provides a comprehensive exploration of image morphing techniques, highlighting the pivotal role played by the Beier-Neely method in advancing the field.. Key Words: Image Morphing, Beier-Neely Method, Seamless Transitions, Feature-based Control Points, System Architecture, Mathematical Foundations

End-to-end tire defect detection model based on transfer learning techniques

Visual inspection of defective tires post-production is vital for human safety, as faulty tires can lead to explosions, accidents, and loss of life. With the advancement of technology, transfer learning (TL) plays an influential role in many computer vision applications, including the tire defect detection problem. However, automatic tire defect detection is difficult for two reasons. The first is the presence of complex anisotropic multi-textured rubber layers. Second, there is no standard tire X-ray image dataset to use for defect detection. In this study, a TL-based tire defect detection model is proposed using a new dataset from a global tire company. First, we collected and labeled the dataset consisting of 3366 X-ray images of faulty tires and 20,000 images of qualified tires. Although the dataset covers 15 types of defects arising from different design patterns, our primary focus is on binary classification to detect the presence or absence of defects. This challenging dataset was split into 70, 15, and 15% for training, validation, and testing, respectively. Then, nine common pre-trained models were fine-tuned, trained, and tested on the proposed dataset. These models are Xception, InceptionV3, VGG16, VGG19, ResNet50, ResNet152V2, DenseNet121, InceptionResNetV2, and MobileNetV2. The results show that the fine-tuned VGG19, DenseNet21 and InceptionNet models achieve compatible results with the literature. Moreover, the Xception model outperformed the compared TL models and literature methods in terms of recall, precision, accuracy, and F1 score. Moreover, it achieved on the testing dataset 73.7, 88, 80.2, and 94.75% of recall, precision, F1 score, and accuracy, respectively, and on the validation dataset 73.3, 90.24, 80.9, and 95% of recall, precision, F1 score, and accuracy, respectively.

Application of Wireless Network Remote Sensing Image and Video Processing Technology in Smart City Design and planning

With the rapid development of society, regional urban construction has become the main guarantee of economic development, and smart city construction has become the main trend of regional development. Smart cities consist of information images and urban development, which can meet the needs of the modern generation for material life, knowledge and scientific life. Therefore, wireless remote sensing (RS) and video image processing have become key tools for the development of smart cities, which can solve the problems encountered by many professionals before, and become an integral part of urban construction. On this basis, this paper summarized the video processing image and wireless network RS image, and analyzed their development and application in urban design and layout. Based on this, this paper expounded the application of wireless network RS image in smart city design and layout, and analyzed the application prospects of RS technology in building smart city space-time information framework, RS+urban management, RS+smart environmental protection, RS+smart city agriculture and video processing technology in smart city design and planning. It analyzed the application in intelligent security, intelligent transportation and management visualization, and then used the two-dimensional discrete Fourier transform (2D-DFT) definition to strengthen the design and layout of smart cities. According to experiments and surveys, introducing 2D-DFT into the design and layout of smart cities can build a more intelligent city and improve residents’ satisfaction by 30%.

Computer Vision-Based Path Planning with Indoor Low-Cost Autonomous Drones: An Educational Surrogate Project for Autonomous Wind Farm Navigation

The application of computer vision in conjunction with GPS is essential for autonomous wind turbine inspection, particularly when the drone navigates through a wind farm to detect the turbine of interest. Although drones for such inspections use GPS, our study only focuses on the computer vision aspect of navigation that can be combined with GPS information for better navigation in a wind farm. Here, we employ an affordable, non-GPS-equipped drone within an indoor setting to serve educational needs, enhancing its accessibility. To address navigation without GPS, our solution leverages visual data captured by the drone’s front-facing and bottom-facing cameras. We utilize Hough transform, object detection, and QR codes to control drone positioning and calibration. This approach facilitates accurate navigation in a traveling salesman experiment, where the drone visits each wind turbine and returns to a designated launching point without relying on GPS. To perform experiments and investigate the performance of the proposed computer vision technique, the DJI Tello EDU drone and pedestal fans are used to represent commercial drones and wind turbines, respectively. Our detailed and timely experiments demonstrate the effectiveness of computer vision-based path planning in guiding the drone through a small-scale surrogate wind farm, ensuring energy-efficient paths, collision avoidance, and real-time adaptability. Although our efforts do not replicate the actual scenario of wind turbine inspection using drone technology, they provide valuable educational contributions for those willing to work in this area and educational institutions who are seeking to integrate projects like this into their courses, such as autonomous systems.

A Basketball Big Data Platform for Box Score and Play-by-Play Data.

This is the second part of a research diptych devoted to improving basketball data management in Spain. The Spanish ACB (Association of Basketball Clubs, acronym in Spanish) is the top European national competition. It attracts most of the best foreign players outside the NBA (National Basketball Association, in North America) and also accelerates the development of Spanish players who ultimately contribute to the success of the Spanish national team. However, this sporting excellence is not reciprocated by an advanced treatment of the data generated by teams and players, the so-called statistics. On the contrary, their use is still very rudimentary. An earlier article published in this journal in 2020 introduced the first open web application for interactive visualization of the box score data from three European competitions, including the ACB. Box score data refer to the data provided once the game is finished. Following the same inspiration, this new research aims to present the work carried out with more advanced data, namely, play-by-play data, which are provided as the game runs. This type of data allow us to gain greater insight into basketball performance, providing information that cannot be revealed with box score data. A new dashboard is developed to analyze play-by-play data from a number of different and novel perspectives. Furthermore, a comprehensive data platform encompassing the visualization of the ACB box score and play-by-play data is presented.

Lit me up: A reference free adaptive low light image enhancement for in-the-wild conditions

Images captured with different devices in uneven conditions (e.g., invariable lighting, low lighting, weather changes, exposure time, etc.) often lead to low image visibility and poor color and contrast, affecting the performance of computer vision and pattern recognition applications. The pre-trained convolutional neural networks (CNNs) solely rely on the training data and lack adaptation due to the uncertainty in the lighting conditions. Moreover, capturing large-scale datasets to train CNNs also raises the computational complexity and overall cost. This work integrates the knowledge and data and proposes a two-stage Uneven-to-Enliven network (U2E-Net), which rapidly learns to see in uneven conditions. A multiple-layered Uneven network learns to distinguish the reflection and illumination in the input images, and an encoder–decoder-based Enliven-Net contextualizes the illumination information. A key component in such ill-posed problems is to obtain information from priors and pairs; however, we present the compelling idea of information trade-off followed by decomposition consistency, thereby progressively improving the visual quality with the subsequent enhancement operations. To this end, we proposed a two-faceted framework that can work without depending on the data type. A novel color and contrast preservation strategy (CPS) is proposed following the decomposition of input data. CPS is integrated within the network to extract contrast in the darkest background regions.

Exploring simple triplet representation learning

Fully supervised learning methods necessitate a substantial volume of labelled training instances, a process that is typically both labour-intensive and costly. In the realm of medical image analysis, this issue is further amplified, as annotated medical images are considerably more scarce than their unlabelled counterparts. Consequently, leveraging unlabelled images to extract meaningful underlying knowledge presents a formidable challenge in medical image analysis. This paper introduces a simple triple-view unsupervised representation learning model (SimTrip) combined with a triple-view architecture and loss function, aiming to learn meaningful inherent knowledge efficiently from unlabelled data with small batch size. With the meaningful representation extracted from unlabelled data, our model demonstrates exemplary performance across two medical image datasets. It achieves this using only partial labels and outperforms other state-of-the-art methods. The method we present herein offers a novel paradigm for unsupervised representation learning, establishing a baseline that is poised to inspire the development of more intricate SimTrip-based methods across a spectrum of computer vision applications. Code and user guide are released at https://github.com/JerryRollingUp/SimTripSystem, the system also runs at http://43.131.9.159:5000/.

Improved stochastic configuration networks with vision patch fusion method for industrial image classification

This paper contributes to the advancement of stochastic configuration neural networks (SCN) in the field of visual applications. The proposed image classification randomized algorithm is an extension of deep stochastic configuration networks (DeepSCN), known as an improved SCN with vision patch fusion (Vi-SCN). Compared to existing two dimensional stochastic configuration networks (2DSCN) and DeepSCN, we have developed an incremental modeling method that employs a stochastic configuration patch fusion method. This method extracts randomly fused image features from three-channel high-resolution image data, improving the network's ability to extract features. Moreover, we have introduced a strategy for dynamically determining the depth structure of the network, enabling flexible adjustments to the network structure and the number of nodes in each layer based on the complexity of image recognition tasks. Through a series of comparisons on four image classification benchmark datasets, we assess the superior learning performance of our design compared to 2DSCN and DeepSCN. Furthermore, to evaluate the performance of Vi-SCN in practical visual application scenarios, we collect three industrial datasets with distinct characteristics. Vi-SCN demonstrate outstanding performance in all three tasks, showing its significant potential in image recognition.

Crawler Data Visualization in the Tourism Industry Based on Python

Data visualization is being widely used in various fields, such as medicine, biology, economics, and other disciplines. In the tourism industry, whether the display of tourism resources, the planning of tourism routes, the feedback of tourists' experience, and so on, all of them become clearer and more intuitive through data visualization. In particular, Python-based data visualization has many advantages in the tourism industry. This paper mainly analyzes the current situation and advantages of Python-based data visualization in the tourism industry, and puts forward corresponding improvement measures and suggestions, aiming to help tourism practitioners and tourists use data visualization more conveniently and efficiently. It is found that data visualization plays an important role in displaying tourist attractions, planning tourist routes, and enhancing the experience of tourism services. However, the problems of data visualization in tourism, such as insufficient interactivity and single visualization style, require improvement measures. For example, methods such as optimizing algorithms and introducing interactive visualization tools can solve these problems. The purpose of this study is to promote a wider and more effective application of data visualization in the tourism industry so that tourism-related industries can be more efficient and intuitive when using data visualization tools.

A Comprehensive Review and New Taxonomy on Superpixel Segmentation

Superpixel segmentation consists of partitioning images into regions composed of similar and connected pixels. Its methods have been widely used in many computer vision applications, since it allows for reducing the workload, removing redundant information, and preserving regions with meaningful features. Due to the rapid progress in this area, the literature fails to catch up on more recent works among the compared ones and to categorize the methods according to all existing strategies. This work fills this gap by presenting a comprehensive review with a new taxonomy for superpixel segmentation, in which methods are classified according to their processing steps and processing levels of image features. We revisit the recent and popular literature according to our taxonomy and evaluate 23 strategies and a grid segmentation baseline based on nine criteria: connectivity, compactness, delineation, control over the number of superpixels, color homogeneity, robustness, running time, stability, and visual quality. Our experiments show the trends of each approach in superpixel segmentation and discuss individual trade-offs. Finally, we provide a new benchmark for superpixel assessment, available at https://github.com/IMScience-PPGINF-PucMinas/superpixel-benchmark .

Strain-insensitive viscoelastic perovskite film for intrinsically stretchable neuromorphic vision-adaptive transistors

Stretchable neuromorphic optoelectronics present tantalizing opportunities for intelligent vision applications that necessitate high spatial resolution and multimodal interaction. Existing neuromorphic devices are either stretchable but not reconcilable with multifunctionality, or discrete but with low-end neurological function and limited flexibility. Herein, we propose a defect-tunable viscoelastic perovskite film that is assembled into strain-insensitive quasi-continuous microsphere morphologies for intrinsically stretchable neuromorphic vision-adaptive transistors. The resulting device achieves trichromatic photoadaptation and a rapid adaptive speed (<150 s) beyond human eyes (3 ~ 30 min) even under 100% mechanical strain. When acted as an artificial synapse, the device can operate at an ultra-low energy consumption (15 aJ) (far below the human brain of 1 ~ 10 fJ) with a high paired-pulse facilitation index of 270% (one of the best figures of merit in stretchable synaptic phototransistors). Furthermore, adaptive optical imaging is achieved by the strain-insensitive perovskite films, accelerating the implementation of next-generation neuromorphic vision systems.

Feature Engineering Versus Deep Learning for Scene Recognition: A Brief Survey

Scene recognition is an important computer vision task that has evolved from the study of the biological visual system. Its applications range from video surveillance, autopilot systems, to robotics. The early works were based on feature engineering that involved the computation and aggregation of global and local image descriptors. Several popular image features such as SIFT, SURF, HOG, ORB, LBP, KAZE, etc. have been proposed and applied to the task with successful results. Features can be either computed from the entire image on a global scale, or extracted from local sub-regions and aggregated across the image. Suitable classifier models are deployed that learn to classify these features. This review paper analyzes several of these handcrafted features that have been applied to the scene recognition task over the past decades, and tracks the transition from the traditional feature engineering to deep learning which forms the current state of the art in computer vision. Deep learning is now deemed to have overtaken feature engineering in several computer vision applications. Deep convolutional neural networks and vision transformers are the current state of the art for object recognition. However, scenes from urban landscapes are bound to contain similar objects posing a challenge to deep learning solutions for scene recognition. In our study, a critical analysis of feature engineering and deep learning methodologies for scene recognition is provided, and results on benchmark scene datasets are presented, concluding with a discussion on challenges and possible solutions that may facilitate more accurate scene recognition.

Optimized Image Captioning: Hybrid Transformers Vision Transformers and Convolutional Neural Networks: Enhanced with Beam Search

Deep learning has improved image captioning. Transformer, a neural network architecture built for natural language processing, excels at image captioning and other computer vision applications. This paper reviews Transformer-based image captioning methods in detail. Convolutional neural networks (CNNs) extracted image features and RNNs or LSTM networks generated captions in traditional image captioning. This method often has information bottlenecks and trouble capturing long-range dependencies. Transformer architecture revolutionized natural language processing with its attention strategy and parallel processing. Researchers used Transformers' language success to solve image captioning problems. Transformer-based image captioning systems outperform previous methods in accuracy and efficiency by integrating visual and textual information into a single model. This paper discusses how the Transformer architecture's self-attention mechanisms and positional encodings are adapted for image captioning. Vision Transformers (ViTs) and CNN-Transformer hybrid models are discussed. We also discuss pre-training, fine-tuning, and reinforcement learning to improve caption quality. Transformer-based image captioning difficulties, trends, and future approaches are also examined. Multimodal fusion, visual-text alignment, and caption interpretability are challenges. We expect research to address these issues and apply Transformer-based image captioning to medical imaging and distant sensing. This paper covers how Transformer-based approaches have changed image captioning and their potential to revolutionize multimodal interpretation and generation, advancing artificial intelligence and human-computer interactions.