Video Images Research Articles

Benchmarking the Functional, Technical, and Business Characteristics of Automated Passenger Counting Products

Urban transport planning and the integration of various mobility options have become increasingly complex, necessitating a thorough understanding of user mobility patterns and their diverse needs. This paper focuses on benchmarking different Automatic Passenger Counting (APC) technologies, which play a key role in Mobility as a Service (MaaS) systems. APC systems provide valuable data for analysing mobility patterns and informing decisions about resource allocation. Our study presents a comprehensive data collection and benchmark analysis of APC solutions. The literature review emphasises the significance of passenger counting for transport companies and discusses various existing APC technologies, such as pressure sensors, wireless sensors, optical infrared sensors (IR), and video image technology. Real-world applications of APC systems are examined, highlighting experimental results and their potential for improving accuracy. The methodology outlines the data collection process, which involved identifying APC companies, conducting interviews with companies and customers, and administering an ad hoc survey to gather specific information about APC systems. The collected data were used to establish criteria and key performance indicators (KPIs) for the benchmarking analysis. The benchmarking analysis compares APC devices and companies based on ten criteria: technology, accuracy, environment, coverage, interface, interference, robustness (for devices), price, pricing model, and system integration (for companies). KPIs were developed to measure performance and make comparison easier. The results of the benchmarking analysis offer insights into the costs and accuracy of different APC systems, enabling informed decision making regarding system selection and implementation. The findings fill a research gap and provide valuable information for transport companies and policy makers, and we offer a comprehensive analysis of APC systems, highlighting their strengths, weaknesses, and business strategies. The paper concludes by discussing limitations and suggesting future research directions for APC technologies.

Experimental Study on Measuring and Tracking Structural Displacement Based on Surveillance Video Image Analysis.

The digital image method of monitoring structural displacement is receiving more attention today, especially in non-contact structure health monitoring. Some obvious advantages of this method, such as economy and convenience, were shown while it was used to monitor the deformation of the bridge structure during the service period. The image processing technology was used to extract structural deformation feature information from surveillance video images containing structural displacement in order to realize a new non-contact online monitoring method in this paper. The influence of different imaging distances and angles on the conversion coefficient (η) that converts the pixel coordinates to the actual displacement was first studied experimentally. Then, the measuring and tracking of bridge structural displacement based on surveillance video images was investigated by laboratory-scale experiments under idealized conditions. The results showed that the video imaging accuracy can be affected by changes in the relative position of the imaging device and measured structure, which is embodied in the change in η (actual size of individual pixel) on the structured image. The increase in distance between the measured structure and the monitoring equipment will have a significant effect on the change in the η value. The value of η varies linearly with the change in shooting distance. The value of η will be affected by the changes in shooting angle. The millimeter-level online monitoring of the structure displacement can be realized using images based on surveillance video images. The feasibility of measuring and tracking structural displacement based on surveillance video images was confirmed by a laboratory-scale experiment.

A System for Monitoring Animals Based on Behavioral Information and Internal State Information.

Managing the risk of injury or illness is an important consideration when keeping pets. This risk can be minimized if pets are monitored on a regular basis, but this can be difficult and time-consuming. However, because only the external behavior of the animal can be observed and the internal condition cannot be assessed, the animal's state can easily be misjudged. Additionally, although some systems use heartbeat measurement to determine a state of tension, or use rest to assess the internal state, because an increase in heart rate can also occur as a result of exercise, it is desirable to use this measurement in combination with behavioral information. In the current study, we proposed a monitoring system for animals using video image analysis. The proposed system first extracts features related to behavioral information and the animal's internal state via mask R-CNN using video images taken from the top of the cage. These features are used to detect typical daily activities and anomalous activities. This method produces an alert when the hamster behaves in an unusual way. In our experiment, the daily behavior of a hamster was measured and analyzed using the proposed system. The results showed that the features of the hamster's behavior were successfully detected. When loud sounds were presented from outside the cage, the system was able to discriminate between the behavioral and internal changes of the hamster. In future research, we plan to improve the accuracy of the measurement of small movements and develop a more accurate system.

Analysis of the organizational culture of the company Genesem Inc. from the perspective of visual anthropology

Organizational culture in companies is of great importance, since it allows you to know how the company is inside, this organizational culture shows you how communication is managed, who manages it, how is the work climate and culture, as well as the use of technology by employees, especially when it is used as a communication tool. Within the organizational culture it is also possible to see how the relationship of employees is, both within the company, as well as their relationship with customers and suppliers. Through this qualitative analysis from the perspective of visual anthropology, it is possible to have more detail on all these issues, especially through the analysis of video images where you can perceive the nonverbal communication and micro expressions of the people who participated in the data collection techniques.

Abnormal Temperature Detection of Blower Components Based on Infrared Video Images Analysis

Abnormal Temperature Detection of Blower Components Based on Infrared Video Images Analysis

Physics-informed deep learning for scattered full wavefield reconstruction from a sparse set of sensor data for impact diagnosis in structural health monitoring

This paper presents a physics-informed deep learning framework for the reconstruction of full scattered spatiotemporal Lamb wavefields (video images) in plate-like structures from a sparse set of time-series sensor data. The reconstructed scattered wavefield contains a wealth of information about the wave propagation phenomenon including any interactions of the propagating wave with damage in the structure. This information is paramount for damage diagnosis as is demonstrated in this paper via impact diagnosis—a key structural health monitoring application. A physics-informed neural network (PINN) that encodes the underlying elastodynamic field equations into the learning/training process in the neural network is proposed for this purpose. This prior wavefield physics knowledge embedded in the loss function acts as a regularization agent for the minimization problem in the neural network training, thereby enabling the extrapolation of a sparse set of one-dimensional time-series signals into two-dimensional scattered wavefield. The wavefield reconstruction capabilities of the proposed supervised forward PINN framework are first verified both numerically and experimentally for a stiffened aluminum panel under a couple of narrowband ultrasonic-frequency excitations, and the results confirm its robustness to low spatial resolution and substantial noise in the measured sensor data. The PINN requires far fewer sensors for scattered wavefield reconstruction, thereby permitting for a higher sensor spacing or lower spatial sampling. To this end, it is shown that a sensor spacing of 5λ generates good wavefield reconstruction accuracy, which is a 10-fold increase over the Nyquist–Shannon sampling limit (λ/2). Two sets of experiments are then conducted on a long-stiffened aluminum panel to validate the proposed framework via low-velocity impact diagnosis in the near-ultrasonic frequency range. The first set of experiments, with the known excitation force incorporated into the PINN, allows the wavefields to be accurately reconstructed with the sensor spacing up to 5λ as expected. The second set of experiments assumes unknown impact force history—a classical case of impact diagnosis where the impact force history is not known a priori. It is shown that the wavefield reconstruction through PINN still provides good accuracy albeit with a less generous sensor spacing of 2λ. A convolutional neural network long short-term memory (CNN-LSTM) model then solves the mathematically inverse problem of inferring the impact location and impact force history by analyzing the reconstructed impact generated wavefield. The impact location is predicted well with 93% accuracy, and the impact force history is reconstructed with 90% accuracy, further validating the proposed framework.

Yolov5s-MSD: a multi-scale ship detector for visible video image

Yolov5s-MSD: a multi-scale ship detector for visible video image

Masked face age and gender identification using CAFFE-modified MobileNetV2 on photo and real-time video images by transfer learning and deep learning techniques

One of the most challenging factors related to masked face age and gender identification (MFAGI) is developing a technique to quickly carry out identification and maintain accuracy without needing people to remove their masks. The eyes, forehead and important frontal features including the ears were the main focus of this investigation. In this study, we evaluated and compared ten (10) different MFAGI techniques, which include traditional methods such as MobileNet, MobileNetV2, ResNet101, DenseNet169, DenseNet201, EfficientNetB0, ResNet152V2, InceptionV3, Xception and our one proposed approach. These 10 models were used to classify masked faces of people among sixteen (16) classes. Combining the convolutional architecture of fast feature embedding (CAFFE) with the modified MobileNetV2 (MNV2) model developed a novel method known as the “CMNV2” model. To achieve the goal of model performance and classification accuracy for MFAGI of people’s faces, this work introduces a unique CMNV2 model by developing a pre-trained model with eight (8) different layers. Using deep learning (DL) and a transfer learning approach, we developed a unique novel approach to image feature extraction that outperforms the nine (9) traditional methods for masked face classification (MFC), masked face age identification (MFAI) and masked face gender identification (MFGI). This approach developed a Python model that was trained and evaluated on 12,160 masked face images. This study demonstrates that an efficient CMNV2 method can be achieved by combining transfer learning with deep neural network (DNN) development. This proposed CMNV2 model achieved 96.54% accuracy and 3.46% error rate with less parameters and computational time exceeding the other 9 different traditional masked identification methods. It also shows that our proposed CMNV2 strategy is more successful than the other 9 models and can identify masked people’s faces age and gender in photo and real-time video images. Overall, the results of this research indicate that DL methods have the capability to improve the accuracy and speed with which masked face classification and identification is performed. The artificially masked faces of images dataset created by us utilized in this work is publicly made available on the Kaggle website https://www.kaggle.com/datasets/banilkumar20phd7071/masked-face-age-and-gender-identify-artificial-masking.

Study of stability and object tracking of traffic video image for smart cities

Study of stability and object tracking of traffic video image for smart cities

An Approach Based on Video Image Intelligent Recognition for Water Turbidity Monitoring in River

An Approach Based on Video Image Intelligent Recognition for Water Turbidity Monitoring in River

A kick monitoring method for deepwater open-circuit drilling based on convolutional neural network

In the case of shallow drilling in deepwater without installation of riser and subsea BOP, open-circuit drilling is usually used. In the process of open-circuit drilling, drilling fluid is directly returned to the seabed. Therefore, observing the return of seabed drilling fluid through Remotely Operated Vehicle (ROV) has become one of the main methods for monitoring the kick in open-circuit drilling. However, the current ROV monitoring method only relies on the professional technical personnel to observe the video screen, which is difficult to identify the kick quickly and accurately in the early stage. In view of this problem, an intelligent kick monitoring method based on deep learning image recognition technology is proposed in this paper. By collecting the actual video image data shot by ROV and simulating the drilling fluid return morphology under different gas influx using Ansys Fluent software, a typical sample database is constructed. Then, the pretrained GoogLeNet neural network model is used for transfer learning, and the video images shot by ROV on site are used to test the trained model. The results show that this method can identify the kick more quickly and accurately, and the model has good robustness. Using this method can greatly reduce the economic losses in the process of deepwater open-circuit drilling, and provides technical support for safe and efficient drilling of deepwater open-circuit drilling.

Monitoring respiratory state from near-infrared face video images

AbstractIn this paper, we propose an algorithm for estimating respiratory state using near-infrared facial video images. Estimation of respiratory state is an important indicator for early detection of respiratory diseases. In particular, there is a demand for monitoring respiratory state during the night. One method of monitoring respiratory state is to use contact-type sensors. However, this method requires the installation of many sensors and a visit to a hospital, which place a burden on patients. Therefore, we propose to acquire respiratory-induced features from near-infrared face video images and investigate their similarity to measurements obtained with a respirometer for non-contact monitoring of respiratory state in the dark. Respiratory-induced features were obtained from pulse wave signals extracted from the face video images. The results showed correlations in several respiratory states. This study opens some perspectives in non-contact monitoring of respiratory states.

Inferring benthic megafaunal sediment reworking activity in relation to bottom water oxygen in Barkley Canyon, NE Pacific from video and acoustic imaging analysis

Sediment reworking activity influences benthic functioning expressed as nutrient fluxes and carbon cycling. Multiple studies have addressed sediment reworking based on observations from individual instrument types (e.g., video camera, side-scan sonar, multibeam), but none have considered reworking based on two or more complementary instruments. We therefore analyzed deep-sea megafaunal and reworked sediment traces by combining and comparing observations from non-invasive instruments, an underwater video camera and a rotary sonar, as part of the node of Ocean Networks Canada's NEPTUNE cabled observatory, located at 396 m depth in Barkley Canyon Upper Slope, NE Pacific Ocean. Specifically, we examined sediment traces and benthic megafaunal communities during two different sampling periods (May and September 2013) and at two different spatial scales of analysis. The camera images (∼0.5 m2) documented significantly lower megafaunal density during the period of reduced oxygen concentrations (May). Although we did not observe significant differences in sediment trace diversity and density between the two study periods, we were interested in how the relief traces generated by unidentified gastropods (likely the family Solariellidae) influenced sediment mixing. Relief traces showed higher density in low oxygen (May, 1.87 count m−2) than in high oxygen (September, 1.17 count m−2) conditions. Sonar images (∼1268 m2), which lacked sufficient resolution to allow identification of benthic organisms, documented distributions of biological pits (regions of low backscatter), a significantly greater proportion of bioturbated seafloor area, and increased in pit size with increased oxygen levels. Pits dominated sonar images at this location and persisted through the two study periods. Most sonar field of view subsections showed a significant increase in circularity of pit shapes, likely explained by increased reworked sediment with increasing oxygen concentration. We conclude that, except for relief burrows, higher reworked sediment area coincided with the highest oxygen concentration, which aligns with previously established reduced metabolic activity by benthos adapted to oxygen minimum zones. Furthermore, we emphasize the complementarity of the two imaging techniques (video and sonar) in understanding deep-sea benthic ecosystem dynamics, as well as the importance of considering multiple spatial scales when investigating proxies of bioturbation activity.

Registration of preoperative temporal bone CT-scan to otoendoscopic video for augmented-reality based on convolutional neural networks.

Patient-to-image registration is a preliminary step required in surgical navigation based on preoperative images. Human intervention and fiducial markers hamper this task as they are time-consuming and introduce potential errors. We aimed to develop a fully automatic 2D registration system for augmented reality in ear surgery. CT-scans and corresponding oto-endoscopic videos were collected from 41 patients (58 ears) undergoing ear examination (vestibular schwannoma before surgery, profound hearing loss requiring cochlear implant, suspicion of perilymphatic fistula, contralateral ears in cases of unilateral chronic otitis media). Two to four images were selected from each case. For the training phase, data from patients (75% of the dataset) and 11 cadaveric specimens were used. Tympanic membranes and malleus handles were contoured on both video images and CT-scans by expert surgeons. The algorithm used a U-Net network for detecting the contours of the tympanic membrane and the malleus on both preoperative CT-scans and endoscopic video frames. Then, contours were processed and registered through an iterative closest point algorithm. Validation was performed on 4 cases and testing on 6 cases. Registration error was measured by overlaying both images and measuring the average and Hausdorff distances. The proposed registration method yielded a precision compatible with ear surgery with a 2D mean overlay error of mm for the incus and mm for the round window. The average Hausdorff distance for these 2 targets was mm and mm respectively. An outlier case with higher errors (2.3 mm and 1.5 mm average Hausdorff distance for incus and round window respectively) was observed in relation to a high discrepancy between the projection angle of the reconstructed CT-scan and the video image. The maximum duration for the overall process was 18s. A fully automatic 2D registration method based on a convolutional neural network and applied to ear surgery was developed. The method did not rely on any external fiducial markers nor human intervention for landmark recognition. The method was fast and its precision was compatible with ear surgery.

SOFTWARE AND HARDWARE MODULE FOR AUTOMATED DETECTION AND RECOGNITION OF INTEREST OBJECTS TO INCREASE THE LEVEL OF PROCESSING EFFICIENCY AND RELIABILITY OF AERIAL RECONNAISSANCE DATA

The requirements for the air reconnaissance system using unmanned aircraft systems are being studied. Problematic aspects of the air reconnaissance data processing process are analyzed from the point of view of ensuring the required level of operational efficiency and reliability of intelligence information. The possibility of automating the process of detecting aerial reconnaissance objects on video images is being investigated. A software-hardware module for automated detection and recognition of objects of interest is being developed to increase the level of processing efficiency and reliability of air reconnaissance data. Modern models of detection and recognition of objects of interest based on algorithms of the YOLO platform are studied. The software component of the module is being developed using the YOLOv8 algorithm architecture and a set of data formed in the conditions of air reconnaissance. The analysis of the results of experimental studies shows that: the developed model allows to ensure the required level of operational efficiency of video image processing and the reliability of aerial reconnaissance data; practical implementation of the developed model at ground command and control stations is possible without the use of additional computing equipment. The hardware component of the module is being developed using Raspberry Pi components. The analysis of quantitative evaluations of the efficiency of data processing using the developed software and hardware module on board the UAV of the studied class indicate an increase in time delays compared to the implementation at the command-and-control station, but it allows to ensure the required level of reliability. Further scientific research will be aimed at investigating the peculiarities of the implementation of the conceptual foundations of the developed module for various classes of UAV in order to increase the level of processing efficiency and reliability of aerial reconnaissance data.

DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception

PurposeSemantic segmentation plays a pivotal role in many applications related to medical image and video analysis. However, designing a neural network architecture for medical image and surgical video segmentation is challenging due to the diverse features of relevant classes, including heterogeneity, deformability, transparency, blunt boundaries, and various distortions. We propose a network architecture, DeepPyramid+, which addresses diverse challenges encountered in medical image and surgical video segmentation.MethodsThe proposed DeepPyramid+ incorporates two major modules, namely “Pyramid View Fusion” (PVF) and “Deformable Pyramid Reception” (DPR), to address the outlined challenges. PVF replicates a deduction process within the neural network, aligning with the human visual system, thereby enhancing the representation of relative information at each pixel position. Complementarily, DPR introduces shape- and scale-adaptive feature extraction techniques using dilated deformable convolutions, enhancing accuracy and robustness in handling heterogeneous classes and deformable shapes.ResultsExtensive experiments conducted on diverse datasets, including endometriosis videos, MRI images, OCT scans, and cataract and laparoscopy videos, demonstrate the effectiveness of DeepPyramid+ in handling various challenges such as shape and scale variation, reflection, and blur degradation. DeepPyramid+ demonstrates significant improvements in segmentation performance, achieving up to a 3.65% increase in Dice coefficient for intra-domain segmentation and up to a 17% increase in Dice coefficient for cross-domain segmentation.ConclusionsDeepPyramid+ consistently outperforms state-of-the-art networks across diverse modalities considering different backbone networks, showcasing its versatility. Accordingly, DeepPyramid+ emerges as a robust and effective solution, successfully overcoming the intricate challenges associated with relevant content segmentation in medical images and surgical videos. Its consistent performance and adaptability indicate its potential to enhance precision in computerized medical image and surgical video analysis applications.

Real-time ghost imaging algorithm on the multidimensional vector matrix Walsh transformation with spatiotemporal free-fps.

In this research, we propose a real-time spatiotemporal free-fps algorithm based on the multidimensional vector matrix Walsh transform with an adjustable ghost imaging video quality and a frame rate to address the issues of uneven imaging speed, fixed imaging frame rate, and uncomfortable appearance in real-time ghost imaging videos of moving objects. This algorithm utilizes the temporal and spatial correlation of ghost imaging videos to achieve free and adjustable video frame rates in time without being limited by DMD refresh rates. Improving the spatial information of a single frame in space enhances the smoothness of ghost imaging videos, making the appearance of ghost imaging videos more comfortable. To achieve this, a four-dimensional vector Walsh transform kernel matrix is used to transform and reconstruct the high-quality images of the target object. Then the reconstructed high-quality image is spatially interpolated to enhance spatial information. Reasonable frame rate parameters are set based on the corresponding relationship between the detection values of the adjacent frames and speckle, improving the ghost imaging video in both time and space and achieving a smooth real-time ghost imaging video with an adjustable quality and frame rate. The simulation and experimental results of moving objects show that our algorithm solves the limitation of a DMD refresh rate compared with the existing ghost imaging video methods and makes the ghost imaging video more comfortable and smoother in real time. The PSNR of the objective evaluation index is increased by 12%. Regarding a subjective evaluation, this paper proposes an adaptive parameterless evaluation algorithm (APEA) for images with different resolutions based on the NRSS, which improved the structure retention degree by 13% and the Brisque parameter evaluation by 70%. We propose an adaptive parameterless video comfort evaluation algorithm (APVCEA) to evaluate the subjective comfort of ghost imaging videos by 14% compared with traditional methods.

Automated artificial intelligence–based phase-recognition system for esophageal endoscopic submucosal dissection (with video)

Endoscopic submucosal dissection (ESD) for superficial esophageal cancer is a multistep treatment involving several endoscopic processes. Although analyzing each phase separately is worthwhile, it is not realistic in practice owing to the need for considerable manpower. To solve this problem, we aimed to establish a state-of-the-art artificial intelligence (AI)-based system, specifically, an automated phase-recognition system that can automatically identify each endoscopic phase based on video images. Ninety-four videos of ESD procedures for superficial esophageal cancer were evaluated in this single-center study. A deep neural network-based phase-recognition system was developed in an automated manner to recognize each of the endoscopic phases. The system was trained with the use of videos that were annotated and verified by 2 GI endoscopists. The overall accuracy of the AI model for automated phase recognition was 90%, and the average precision, recall, and F value rates were 91%, 90%, and 90%, respectively. Two representative ESD videos predicted by the model indicated the usability of AI in clinical practice. We demonstrated that an AI-based automated phase-recognition system for esophageal ESD can be established with high accuracy. To the best of our knowledge, this is the first report on automated recognition of ESD treatment phases. Because this system enabled a detailed analysis of phases, collecting large volumes of data in the future may help to identify quality indicators for treatment techniques and uncover unmet medical needs that necessitate the creation of new treatment methods and devices.

Extraction of ocean wave parameters from video images

ABSTRACT In this study, we proposed a method for estimating windsea parameters from the images captured by the surveillance camera installed in coastal areas for practical use. Camera images were acquired in the observation room of the Tokyo Bay Marine Traffic Center at Cape Kannon in Tokyo Bay, Japan for approximately 4 months. Video images of the sea surface were analyzed using in-situ wave observations and wave model data. The brightness spectrum in the frequency domain and the brightness period were defined and introduced from the time series of brightness values in the obtained video images. The novelty of this study lies in the method proposed for estimating the wave parameters using the brightness period. Although this is a simple method, we confirmed that the wave parameters for windsea, such as the significant wave height and significant wave period, can be estimated with high accuracy using the brightness period. Furthermore, we confirmed that the brightness spectrum obtained from camera images and the slope spectrum of wind waves exhibited extremely similar shapes.

Assessing the data quality of AdHawk MindLink eye-tracking glasses.

Most commercially available eye-tracking devices rely on video cameras and image processing algorithms to track gaze. Despite this, emerging technologies are entering the field, making high-speed, cameraless eye-tracking more accessible. In this study, a series of tests were conducted to compare the data quality of MEMS-based eye-tracking glasses (AdHawk MindLink) with three widely used camera-based eye-tracking devices (EyeLink Portable Duo, Tobii Pro Glasses 2, and SMI Eye Tracking Glasses 2). The data quality measures assessed in these tests included accuracy, precision, data loss, and system latency. The results suggest that, overall, the data quality of the eye-tracking glasses was lower compared to that of a desktop EyeLink Portable Duo eye-tracker. Among the eye-tracking glasses, the accuracy and precision of the MindLink eye-tracking glasses were either higher or on par with those of Tobii Pro Glasses 2 and SMI Eye Tracking Glasses 2. The system latency of MindLink was approximately 9 ms, significantly lower than that of camera-based eye-tracking devices found in VR goggles. These results suggest that the MindLink eye-tracking glasses show promise for research applications where high sampling rates and low latency are preferred.