Video Camera Research Articles

Adaptive Processing of Camera Video Stream with Limitations on the Network Data Transmission Bandwidth

Adaptive Processing of Camera Video Stream with Limitations on the Network Data Transmission Bandwidth

Surface Bubbles Emergence as an Indicator for Optimal Concrete Compaction

Compaction quality significantly influences the strength and durability of concrete in structures. Under-compacting can retain entrapped air, reducing strength, while over-compacting can lead to segregation, creating local variances in strength distribution and modulus of elasticity in the concrete structure. This study examines the widely adopted concept that compaction is optimal when bubbles cease to emerge on the concrete surface. We recorded the surface activity of six comparable concrete specimens during the compaction process using a 4K video camera. Four specimens were compacted using a table vibrator and two with a poker vibrator. From the video frames, we isolated the bubbles for analysis, employing digital image processing techniques to distinguish newly risen bubbles per frame. It was found that the bubbles continuously rose to the surface in all specimens throughout the compaction process, suggesting a need for extended compaction, with some specimens showing a slow in the rate of the bubbles’ emergence. However, upon examining the segregation levels, it was discovered that all the specimens were segregated, some severely, despite the continued bubble emergence. These findings undermine the reliability of using bubble emergence as a principle to stop compaction and support the need for developing online measurement tools for evaluating compaction quality.

Deblur-GS: 3D Gaussian Splatting from Camera Motion Blurred Images

Novel view synthesis has undergone a revolution thanks to the radiance field method. The introduction of 3D Gaussian splatting (3DGS) has successfully addressed the issues of prolonged training times and slow rendering speeds associated with the Neural Radiance Field (NeRF), all while preserving the quality of reconstructions. However, 3DGS remains heavily reliant on the quality of input images and their initial camera pose initialization. In cases where input images are blurred, the reconstruction results suffer from blurriness and artifacts. In this paper, we propose the Deblur-GS method for reconstructing 3D Gaussian points to create a sharp radiance field from a camera motion blurred image set. We model the problem of motion blur as a joint optimization challenge involving camera trajectory estimation and time sampling. We cohesively optimize the parameters of the Gaussian points and the camera trajectory during the shutter time. Deblur-GS consistently achieves superior performance and rendering quality when compared to previous methods, as demonstrated in evaluations conducted on both synthetic and real datasets.

Efficient Structure from Motion for Large-Size Videos from an Open Outdoor UAV Dataset

Modern UAVs (unmanned aerial vehicles) equipped with video cameras can provide large-scale high-resolution video data. This poses significant challenges for structure from motion (SfM) and simultaneous localization and mapping (SLAM) algorithms, as most of them are developed for relatively small-scale and low-resolution scenes. In this paper, we present a video-based SfM method specifically designed for high-resolution large-size UAV videos. Despite the wide range of applications for SfM, performing mainstream SfM methods on such videos poses challenges due to their high computational cost. Our method consists of three main steps. Firstly, we employ a visual SLAM (VSLAM) system to efficiently extract keyframes, keypoints, initial camera poses, and sparse structures from downsampled videos. Next, we propose a novel two-step keypoint adjustment method. Instead of matching new points in the original videos, our method effectively and efficiently adjusts the existing keypoints at the original scale. Finally, we refine the poses and structures using a rotation-averaging constrained global bundle adjustment (BA) technique, incorporating the adjusted keypoints. To enrich the resources available for SLAM or SfM studies, we provide a large-size (3840 × 2160) outdoor video dataset with millimeter-level-accuracy ground control points, which supplements the current relatively low-resolution video datasets. Experiments demonstrate that, compared with other SLAM or SfM methods, our method achieves an average efficiency improvement of 100% on our collected dataset and 45% on the EuRoc dataset. Our method also demonstrates superior localization accuracy when compared with state-of-the-art SLAM or SfM methods.

Real-time and Intelligent Moving Targets Tracking based on UAV Remote Sensing Video Camera and Brain-like Computing Chips

Abstract. Moving target tracking technology based on Unmanned Aerial Vehicles (UAV) is widely used in many fields such as automatic inspection and emergency response. The existing moving target tracking methods usually have the problems of large computation and low tracking efficiency. Limited by the computing power of the UAV platform, real-time tracking and analysis of multiple targets based on the video data collected by UAV platform is a difficult task. In this paper, we proposed a novel Target Specific Filtering Tracking with Memory (TSFMTrack) method designed for UAV-based real-time tracking tasks, which involves a Tracklet Filtering Module (TFM) for capturing object appearance features and a Tracklet Matching Module (TMM) for bounding box association in each frame. By experimental comparison with other State-Of-The-Art (SOTA) methods on popular MOT and UAV tracking datasets, the TSFMTrack have shown obvious advantages in accuracy, computational efficiency and reliability. Furthermore, we deployed the TSFMTrack on the brain-inspired chip Lynchip KA200, the experimental results have shown that the TSFMTrack is effective on edge computational platform and suitable for UAV real-time tracking tasks.

Real-Time Deformation Measurement of Long-Span Bridge using Multi-Inertial Camera System

Abstract. Monitoring the deformation of long-span bridges is essential for assessing structural health and safety. The existing single-camera deformation measurement method is unable to meet the high-precision measurement requirement for the deformation of large-span bridges. This paper proposes a real-time deformation measurement method for the long-span bridge using multi-inertial camera system. The method is based on visual measurement which the target deformation is measured by multiple cameras. Meanwhile, inertial measurement is used to estimate the relative pose of the camera to compensate for camera motion errors. It is applied to the health monitoring system of a long-span bridge. The experimental results show that the method proposed in this paper is highly consistent with the hydrostatic leveling measurement, with a RMSE of 4.83mm. The method can accurately and real-time measure the deformation of large-span bridges with a promising engineering value.

Optical Flow-Based Obstacle Detection for Mid-Air Collision Avoidance

The sky may seem big enough for two flying vehicles to collide, but the facts show that mid-air collisions still occur occasionally and are a significant concern. Pilots learn manual tactics to avoid collisions, such as see-and-avoid, but these rules have limitations. Automated solutions have reduced collisions, but these technologies are not mandatory in all countries or airspaces, and they are expensive. These problems have prompted researchers to continue the search for low-cost solutions. One attractive solution is to use computer vision to detect obstacles in the air due to its reduced cost and weight. A well-trained deep learning solution is appealing because object detection is fast in most cases, but it relies entirely on the training data set. The algorithm chosen for this study is optical flow. The optical flow vectors can help us to separate the motion caused by camera motion from the motion caused by incoming objects without relying on training data. This paper describes the development of an optical flow-based airborne obstacle detection algorithm to avoid mid-air collisions. The approach uses the visual information from a monocular camera and detects the obstacles using morphological filters, optical flow, focus of expansion, and a data clustering algorithm. The proposal was evaluated using realistic vision data obtained with a self-developed simulator. The simulator provides different environments, trajectories, and altitudes of flying objects. The results showed that the optical flow-based algorithm detected all incoming obstacles along their trajectories in the experiments. The results showed an F-score greater than 75% and a good balance between precision and recall.

Video camera and seismic monitoring of water bulge explosion at Strokkur Geyser, Iceland

Eruptions of volcanoes and geysers share many fundamental similarities: for example, they are manifestations of Earth’s geothermal energy, involving the pressure-driven expulsion of fluids from the Earth’s interior. However, while volcanoes can produce spectacular lava bubbles that burst, water bubbles are rarely observed on the surface of geysers. It is still unclear why some of these low-viscosity geyser systems produce none, while others produce them regularly. There is no quantification of the size, speed, and height of these bubbles at geysers, which is the gap we fill here. Strokkur creates a water bulge in its surface pool (bulge stage). When the bulge bursts, water is ejected into the air (jet stage). The steam then continues to rise buoyantly and drift away (drift stage). Here we study the evolution of the three stages using records from video camera campaigns and a local seismic network. We find that larger bulges are associated with larger ascent velocities and cause larger jet heights. As energy is channeled into a high jet, small seismic ground motions are recorded. The bulge formation itself is barely visible seismically. Our work suggests that the 0.74±0.27 s-long bulge stage can be used as a first-order proxy for predicting eruption height. This study might also be relevant for understanding fluid dynamics in volcanic systems.

Video-based sequential Bayesian homography estimation for soccer field registration

A novel Bayesian framework is proposed, which explicitly relates the homography of one video frame to the next through an affine transformation while explicitly modelling keypoint uncertainty. The literature has previously used differential homography between subsequent frames, but not in a Bayesian setting. In cases where Bayesian methods have been applied, camera motion is not adequately modelled, and keypoints are treated as deterministic. The proposed method, Bayesian Homography Inference from Tracked Keypoints (BHITK), employs a two-stage Kalman filter and significantly improves existing methods. Existing keypoint detection methods may be easily augmented with BHITK. It enables less sophisticated and less computationally expensive methods to outperform the state-of-the-art approaches in most homography evaluation metrics. Furthermore, the homography annotations of the WorldCup and TS-WorldCup datasets have been refined using a custom homography annotation tool that has been released for public use. The refined datasets are consolidated and released as the consolidated and refined WorldCup (CARWC) dataset.

Non-rigid scene reconstruction of deformable soft tissue with monocular endoscopy in minimally invasive surgery.

The utilization of image-guided surgery has demonstrated its ability to improve the precision and safety of minimally invasive surgery (MIS). Non-rigid scene reconstruction is a challenge in image-guided system duo to uniform texture, smoke, and instrument occlusion, etc. METHODS: In this paper, we introduced an algorithm for 3D reconstruction aimed at non-rigid surgery scenes. The proposed method comprises two main components: firstly, the front-end process involves the initial reconstruction of 3D information for deformable soft tissues using embedded deformation graph (EDG) on the basis of dual quaternions, enabling the reconstruction without the need for prior knowledge of the target. Secondly, the EDG is integrated with isometric nonrigid structure from motion (Iso-NRSFM) to facilitate centralized optimization of the observed map points and camera motion across different time instances in deformable scenes. For the quantitative evaluation of the proposed method, we conducted comparative experiments with both synthetic datasets and publicly available datasets against the state-of-the-art 3D reconstruction method, DefSLAM. The test results show that our proposed method achieved a maximum reduction of 1.6mm in average reconstruction error compared to method DefSLAM across all datasets. Additionally, qualitative experiments were performed on video scene datasets involving surgical instrument occlusions. Our method proved to outperform DefSLAM on both synthetic datasets and public datasets through experiments, demonstrating its robustness and accuracy in the reconstruction of soft tissues in dynamic surgical scenes. This success highlights the potential clinical application of our method in delivering surgeons with critical shape and depth information for MIS.

Effect of video angle on detection of induced front limb lameness in horses

BackgroundLameness examinations are commonly performed in equine medicine. Advancements in digital technology have increased the use of video recordings for lameness assessment, however, standardization of ideal video angle is not available yielding videos of poor diagnostic quality. The objective of this study was to evaluate the effect of video angle on the subjective assessment of front limb lameness. A randomized, blinded, crossover study was performed. Six horses with and without mechanically induced forelimb solar pain were recorded using 9 video angles including horses trotting directly away and towards the video camera, horses trotting away and towards a video camera placed to the left and right side of midline, and horses trotting in a circle with the video camera placed on the inside and outside of the circle. Videos were randomized and assessed by three expert equine veterinarians using a 0–5 point scoring system. Objective lameness parameters were collected using a body-mounted inertial sensor system (Lameness Locator®, Equinosis LLC). Interobserver agreement for subjective lameness scores and ease of grading scores were determined.ResultsInduction of lameness was successful in all horses. There was excellent agreement between objective lameness parameters and subjective lameness scores (AUC of the ROC = 0.87). For horses in the “lame” trials, interobserver agreement was moderate for video angle 2 when degree of lameness was considered and perfect for video angle 2 and 9 when lameness was considered as a binary outcome. All other angles had no to fair agreement. For horses in the “sound” trials, interobserver agreement was perfect for video angle 5. All other video angles had slight to moderate agreement.ConclusionsWhen video assessment of forelimb lameness is required, a video of the horse trotting directly towards the video camera at a minimum is recommended. Other video angles may provide supportive information regarding lameness characteristics.

Spatial and temporal consistency learning for monocular 6D pose estimation

Monocular 6D pose estimation is a challenging task in the field of computer vision and robotics. Many previous works only input the cropped image of single object during training and inference, aiming to remove the noise from non-object regions. However, most of these methods ignore the viewpoint and spatial relationships of objects in the scene, which are crucial for accurate pose estimation of camera. To address this issue, this paper proposes a novel multi-view and multi-object based learning strategy for monocular 6D pose estimation, which involves the consistency of object coordinate for the same object at different viewpoints and the consistency of world coordinate for different objects in the same space. In the proposed method, the spatial and temporal groups are generated to trained the monocular 6D pose estimation network. Due to the camera motion, scene images taken at different times can be regarded as images captured from different viewpoints. Therefore, a temporal consistency loss is designed to constraint the relationship of the same object at different viewpoints, while a spatial consistency loss is designed to constraint the relationship of different objects at the same space. Finally, the proposed method is verified on the public datasets. Experimental results show that the proposed method is accurate, robust, and outperforms similar state-of-the-art approaches.

Moving Object Detection in Freely Moving Camera via Global Motion Compensation and Local Spatial Information Fusion.

Motion object detection (MOD) with freely moving cameras is a challenging task in computer vision. To extract moving objects, most studies have focused on the difference in motion features between foreground and background, which works well for dynamic scenes with relatively regular movements and variations. However, abrupt illumination changes and occlusions often occur in real-world scenes, and the camera may also pan, tilt, rotate, and jitter, etc., resulting in local irregular variations and global discontinuities in motion features. Such complex and changing scenes bring great difficulty in detecting moving objects. To solve this problem, this paper proposes a new MOD method that effectively leverages local and global visual information for foreground/background segmentation. Specifically, on the global side, to support a wider range of camera motion, the relative inter-frame transformations are optimized to absolute transformations referenced to intermediate frames in a global form after enriching the inter-frame matching pairs. The global transformation is fine-tuned using the spatial transformer network (STN). On the local side, to address the problem of dynamic background scenes, foreground object detection is optimized by utilizing the pixel differences between the current frame and the local background model, as well as the consistency of local spatial variations. Then, the spatial information is combined using optical flow segmentation methods, enhancing the precision of the object information. The experimental results show that our method achieves a detection accuracy improvement of over 1.5% compared with the state-of-the-art methods on the datasets of CDNET2014, FBMS-59, and CBD. It demonstrates significant effectiveness in challenging scenarios such as shadows, abrupt changes in illumination, camera jitter, occlusion, and moving backgrounds.

Algorithm for tracking an object observed by а video camera

Algorithm for tracking an object observed by а video camera

MMD-MSD: A Multimodal Multisensory Dataset in Support of Research and Technology Development for Musculoskeletal Disorders

Improper sitting positions are known as the primary reason for back pain and the emergence of musculoskeletal disorders (MSDs) among individuals who spend prolonged time working with computer screens, keyboards, and mice. At the same time, it is well understood that automated technological tools can play an important role in the process of unhealthy habit alteration, so plenty of research efforts are focused on research and technology development (RTD) activities that aim to provide support for the prevention of back pain or the development of MSDs. Here, we report on creating a new resource in support of RTD activities aiming at the automated detection of improper sitting positions. It consists of multimodal multisensory recordings of 100 persons, made with a video recorder, camera, and wrist-attached sensors that capture physiological signals (PPG, EDA, skin temperature), as well as motion sensors (three-axis accelerometer). Our multimodal multisensory dataset (MMD-MSD) opens new opportunities for modeling the body stance (sitting posture and movements), physiological state (stress level, attention, emotional arousal and valence), and performance (success rate on the Stroop test) of people working with a computer. Finally, we demonstrate two use cases: improper neck posture detection from pictures, and task-specific cognitive load detection from physiological signals.

The Importance of an Unmanned Surface Vehicle: A Point of View

In the world there are various types of unmanned vehicles: Unmanned ground vehicle, unmanned aerial vehicle, unmanned surface vehicle and unmanned underwater vehicle. The objective of this paper, however, is to present the importance of an unmanned surface vehicle (USV) in the Black Sea. The safe operations of all USVs are carried out by experienced and qualified operators. These operators can control a USV from a ship or from land. An USV is equipped with various devices (eg. video camera, long range antennae or autopilot box) depending on the mission it has to carry out at sea. These boats are launched through launch and recovery system (LARS). Furthermore, the USVs do not pollute and do not make noise during their activity on sea. Finally, it is expected that the unmanned surface vehicles will further develop, depending on new requirements present in the Black Sea.

Cinematographic Camera Diffusion Model

AbstractDesigning effective camera trajectories in virtual 3D environments is a challenging task even for experienced animators. Despite an elaborate film grammar, forged through years of experience, that enables the specification of camera motions through cinematographic properties (framing, shots sizes, angles, motions), there are endless possibilities in deciding how to place and move cameras with characters. Dealing with these possibilities is part of the complexity of the problem. While numerous techniques have been proposed in the literature (optimization‐based solving, encoding of empirical rules, learning from real examples,…), the results either lack variety or ease of control.In this paper, we propose a cinematographic camera diffusion model using a transformer‐based architecture to handle temporality and exploit the stochasticity of diffusion models to generate diverse and qualitative trajectories conditioned by high‐level textual descriptions. We extend the work by integrating keyframing constraints and the ability to blend naturally between motions using latent interpolation, in a way to augment the degree of control of the designers. We demonstrate the strengths of this text‐to‐camera motion approach through qualitative and quantitative experiments and gather feedback from professional artists. The code and data are available at https://github.com/jianghd1996/Camera-control.

Motion Vector-Based Self-Attention for Real-Time Human Activity Recognition in Compressed Videos: The MVViT Approach

Herein, a novel methodology is proposed for real-time recognition of human activity in a compressed domain of videos based on motion vectors and self-attention mechanism using vision transformers, and it is termed as motion vectors and vision transformers (MVViT). The videos in MPEG-4 and H.264 compression formats are considered for this study. Any video source without any prior setup could be considered by adopting the proposed method to the corresponding video codecs and camera settings. Existing algorithms for recognition of human action in a compressed video have some limitations in this regard, such as (i) requirement of keyframes at a fixed interval, (ii) usage of P frames only, and (iii) normally support single codec only. These limitations are overcome in the proposed method by using arbitrary keyframe intervals, using both P and B frames, and supporting MPEG-4 as well as H.264 codecs. The experimentation is carried out using the benchmark datasets, namely, UCF101, HMDB51, and THUMOS14, and the recognition accuracy in a compressed domain is found to be comparable to that observed in raw video data but at reduced cost of computation. The proposed MVViT method has outperformed other recent methods in terms of a lesser (61.0%) number of parameters and (63.7%) Giga Floating Point Operations Per Second (GFLOPS), while significantly improving accuracy by 0.8%, 5.9% and 16.6% for UCF101, HMDB51 and THUMOS14, respectively. Also, it is observed that the speed is increased by 8% in case of UCF101 when compared to the highest speed reported in the literature on the same dataset. The ablation study of the proposed method has been done using MVViT variants for different codecs and the performance analysis is done in comparison with the state-of-the-art network models.

Bi-State Prediction Network Model for Mixed Traffic Congestion Prediction

Forecasting traffic congestion has become an important issue than ever before. Information provided by traffic congestion prediction methods becomes helpful to traffic management agencies to take suitable actions in due time. On the other hand, with prior information, passengers can plan their trip routes accordingly. In this paper, we tackle the problem of mixed traffic congestion prediction and develop a new Bi-State Prediction Network (BiSPNet). Two heads form BiSPNet. Each head information passed through the layers of three-dimensional convolution (Conv3D), two-dimensional convolutional long short-term memory (ConvLSTM2D), and two-dimensional convolution (Conv2D). Then the dense layer merges both head outputs. These layers capacitate the model to capture the characteristics of multiple inputs. The proposed model is validated by the ground truth data collected by the video cameras installed on the roadways in Delhi, India. Experimental results confirm that the prediction accuracy of the BiSPNet model surpasses that of several state-of-the-art benchmark models. In particular, the BiSPNet reduces mean absolute error by 53.23%, 40.95%, and 0.38%, respectively, in congestion score prediction for 5, 10, and 15-min time horizons compared to the mixed deep learning network (MDLNet). The source data and code can be accessed on GitHub at the following link: https://github.com/amanojup/ .

Asymmetric gap balancing improves knee kinematic following primary total knee arthroplasty

PurposeThe purpose of this study was to demonstrate closer-to-normal knee kinematics following primary total knee arthroplasty (TKA) performed establishing asymmetric gap balancing intraoperatively.Material and methodTwo age-, sex-, BMI-matched groups of patients underwent medially stabilized TKA because of isolated knee disease. Group A (12 patients) underwent “unrestricted” kinematic alignment (uKA) according to Howell while group B (15 patients) received robot-assisted “simplified” KA (sKA) with an alignment goal (Hip-Knee-Ankle axis-HKA) ± 5° respect to the mechanical axis. Intraoperatively, in group B, the flexion gap at 90° was first set at an average of 1.5 mm (0–5 mm; SD 4.4 mm) tighter in the medial compartment with respect to the lateral; in the same way, the extension gap was then set at an average of 2.0 mm (0–4.5 mm; SD 3.1 mm) tighter in the medial compartment with respect to the lateral. All patients, including a non-arthritic cohort (group C: 5 controls) underwent gait analysis using an instrumented treadmill (WalkerView–WV) equipped with an instrumented belt armed with a 3D video camera. The WV software evaluated multiple spatiotemporal and kinematic parameters, including: (1) contact time (s); (2) knee ROM during gait cycle; (3) step length percentage with respect to total gait (%) and pure step length (cm). Statistical analyses included t-Test and ANOVA and were conducted by using SPSS.ResultsAt the final FU, significant differences were noted during gait between the two TKA groups (uKA-sKA) and the controls. Both TKA groups showed superior mean contact time on the surgical knee (uKA 1 s; sKA 0.97 s) as compared to the controls (0.72 s) (P = 0.002) while no differences were found between them (P = 0.11). TKA groups showed a lower, maximum ROM in the surgical knee (mean uKA 36º; mean sKA 49º) relative to the controls (mean 57º) (P < 0.05) but a statistical difference was found between them (P = 0.003). Both TKA groups showed a higher step length percentage with respect to the total gait and a shorter step length on the surgical side (uKA: mean 8.28% and mean step length 35.5 cm; sKA: mean 8.38% and mean step length 34.6 cm) in comparison to the controls (mean 3.38%; mean step length 71.4 cm) (P < 0.05) while no statistical differences were found between them.ConclusionTo our knowledge, this was the first study to exhibit the kinematic advantages of a slightly asymmetric gap balancing during KA TKA. Combining a medially-stabilized implant design and a surgical technique aiming to obtain a tighter medial compartment represents a promising approach to improve outcomes after TKA.Graphical