Video Camera Data Research Articles

Predicting Knee Joint Contact Force Peaks During Gait Using a Video Camera or Wearable Sensors.

Estimating loading of the knee joint may be helpful in managing degenerative joint diseases. Contemporary methods to estimate loading involve calculating knee joint contact forces using musculoskeletal modeling and simulation from motion capture (MOCAP) data, which must be collected in a specialized environment and analyzed by a trained expert. To make the estimation of knee joint loading more accessible, simple input predictors should be used for predicting knee joint loading using artificial neural networks. We trained feedforward artificial neural networks (ANNs) to predict knee joint loading peaks from the mass, height, age, sex, walking speed, and knee flexion angle (KFA) of subjects using their existing MOCAP data. We also collected an independent MOCAP dataset while recording walking with a video camera (VC) and inertial measurement units (IMUs). We quantified the prediction accuracy of the ANNs using walking speed and KFA estimates from (1) MOCAP data, (2) VC data, and (3) IMU data separately (i.e., we quantified three sets of prediction accuracy metrics). Using portable modalities, we achieved prediction accuracies between 0.13 and 0.37 root mean square error normalized to the mean of the musculoskeletal analysis-based reference values. The correlation between the predicted and reference loading peaks varied between 0.65 and 0.91. This was comparable to the prediction accuracies obtained when obtaining predictors from motion capture data. The prediction results show that both VCs and IMUs can be used to estimate predictors that can be used in estimating knee joint loading outside the motion laboratory. Future studies should investigate the usability of the methods in an out-of-laboratory setting.

City-scale Vehicle Trajectory Data from Traffic Camera Videos

Vehicle trajectory data underpins various applications in intelligent transportation systems, such as traffic surveillance, traffic prediction, and traffic control. Traditional vehicle trajectory datasets, recorded by GPS devices or single cameras, are often biased towards specific vehicles (e.g., taxis) or incomplete (typically < 1 km), limiting their reliability for downstream applications. With the widespread deployment of traffic cameras across the city road network, we have the opportunity to capture all vehicles passing by. By collecting city-scale traffic camera video data, we apply a trajectory recovery framework that identifies vehicles across all cameras and reconstructs their paths in between. Leveraging this approach, we are the first to release a comprehensive vehicle trajectory dataset that covers almost full-amount of city vehicle trajectories, with approximately 5 million trajectories recovered from over 3000 traffic cameras in two metropolises. To assess the quality and quantity of this dataset, we evaluate the recovery methods, visualize specific cases, and compare the results with external road speed and flow statistics. The results demonstrate the consistency and reliability of the released trajectories. This dataset holds great promise for research in areas such as unveiling traffic dynamics, traffic network resilience assessment, and traffic network planning.

A Choice-Based Design Approach to Influence Beneficial Use of a Green Space by Family Caregivers at a Tertiary Care Hospital in India.

Principles of behavioral economics are now being used across fields in changing human behavior toward perceived benefit. We studied the effectiveness of simple, cost-effective design cues based on these principles, in encouraging use of a neglected outdoor space in a hospital. Benefits of access to nature and green spaces are established in healthcare; however, we found utilization of an outdoor space to be suboptimal. Presenting choices in different ways is known to influence user choice. We designed four design interventions based to nudge relative-caregivers waiting outside the operation theater toward utilizing an adjoining outdoor balcony. These included making the terrace prominent, reducing indoor sensory stimulation, adding stereotypical elements, and improving usability. We measured usage via video camera data and satisfaction using questionnaires. The number of users and average time per person spent on the balcony increased over the consecutive intervention periods as compared to baseline (trend p < .01 for both), mainly driven by the addition of stereotypical elements. There were no adverse events in the balcony. There was no difference in the satisfaction related outcomes on questionnaire; a larger percentage of positive comments tended to be related to the balcony in the final intervention. Cost-effective design cues nudged patient-caregivers toward spending more time with nature, thus changing behavior toward that with perceived benefits.

Observation and Modeling of Attempted Leaders in a Multibranched Altitude-Triggered Lightning Flash

A multibranched altitude-triggered lightning flash containing 15 attempted leaders (ALs) was observed. These leaders propagated along three paths with different development processes. The average two-dimensional speed of the ALs ranged from 3.8 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> m/s to 29.4 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</sup> m/s, and the terminal height ranged from over 1626 to 505 m above the ground. The progressing characteristics of ALs were investigated in detail based on the high-speed video camera and synchronous electric field data. The electric field waveforms at the termination stage of the AL were divided into two types. Two different termination ways were proposed to describe the process of AL's disappearance. The source charge model was used to simulate the electric field waveform of the ALs, and the results agree well with the measured electric field waveform.

Cycle-Based Estimation on Lane-Level Queue Length at Isolated Signalized Intersection Using License Plate Recognition Data

Queue length is an important parameter to evaluate the congestion level of urban intersections. Abundant methods based on different data sources, such as loop detector data and mobile sensor data, for estimating queue length have received wide attention. A license plate recognition (LPR) system recording vehicle information can be used as a data source to estimate lane-level queue length. In this study, an improved method is proposed for cycle-based queue length estimation at an isolated signalized intersection using LPR data, considering the residual queued vehicles under oversaturated conditions. First, a modified interpolation method is developed to infer the travel time of unmatched vehicles. Then, the complete arrival and departure information are processed as three characteristic parameters of vehicles, which are the input to the maximum probability function for queue length estimation at each lane. Finally, the proposed model is validated using actual cycle maximum queue length collected from a video camera and LPR data in Changsha city, China. The numerical results showed that the proposed model can achieve accurate estimation for lane-level queue length.

Real-time snowy weather detection based on machine vision and vehicle kinematics: A non-parametric data fusion analysis protocol

Introduction: This study introduces a new analysis protocol for detecting real-time snowy weather conditions on freeways by utilizing trajectory-level data extracted from the Second Strategic Highway Research Program (SHRP2) Naturalistic Driving Study (NDS) dataset. The data include parameters reduced from a real-time image feature extraction technique, time series data collected from external sensors, and CANbus data collected by the NDS ego-vehicles. To provide flexibility in winter maintenance, two segmentation types of one-minute and one-mile segments were used to sample snowy trips and their matched clear weather trips. Method: In this study, four non-parametric models were developed using six data assemblies to detect snowy weather on freeways. The data assemblies are arranged based on three data sources, including image database extracted from an in-vehicle video camera, sensors, and CANbus data, to examine the effectiveness of snow detection models for different data types considering real-time availability of data. Results: Overall, the developed models successfully detected snowy weather on freeways with an accuracy ranging between 76% to 89%. Results indicated that high accuracy of estimating snowy weather can be accomplished using the data fusion between external sensors data and texture parameters of images, without accessing to CANbus data. Practical Applications: Practical applications can be driven with respect to the time or distance coordinates, using different data fusion assemblies, and data availability. The study proves the importance of employing vehicles as weather sensors in the Connected Vehicles (CV) applications and Variable Speed Limit (VSL) to improve traffic safety on freeways.

Integrating video and genetic data to estimate annual age‐structured apparent survival of American black bears

AbstractCamera or genetic data are increasingly used to estimate wildlife abundance and density. We integrated video camera data with genetic data over 7 years to estimate annual age‐structured apparent survival of American black bears (Ursus americanus). We identified 70 individuals through meticulous scrutiny of 7531 video captures, cross‐referenced with 721 genetic captures from hair samples concurrently collected from stations in view of cameras. We used the Cormack–Jolly–Seber model in Program Mark to estimate annual age‐structured apparent survival for yearling males, yearling females, 2+ year‐old males, and 2+ year‐old females. We manually calculated cub survival. We compared parameter estimates based on combined video and genetic data with those based on only genetic data. Combining video and genetic data provided a means to test video‐based identification accuracy, which was highest for females (97%–100%). Annual apparent survival was highest for yearling females (φ = 0.92, SE = 0.07), followed by 2+ year‐old females (φ = 0.88, SE = 0.05), 2+ year‐old males (φ = 0.84, SE = 0.06), and yearling males (φ = 0.80, SE = 0.14). Annual cub survival (φ = 0.86, SE = 0.07) was likely biased because we could not account for mortality that occurred in‐den through early spring. Annual apparent survival and recapture probabilities derived from only genetic data were lower than those derived from combined video and genetic data. Our finding that noninvasive data can be used to estimate annual age‐structured apparent survival of a species with relatively indistinct traits is broadly relevant to wildlife research and conservation.

A framework for mode classification in multimodal environments using radar-based sensors

Monitoring traffic at locations where bicyclists and pedestrians are present and studying their interactions with motorized vehicles has the potential to reveal underlying crash mechanisms and, in turn, guide the implementation of suitable countermeasures. Previous research has demonstrated the capabilities of vision-based systems in traffic monitoring; however, as these systems rely on video camera data, they remain constrained in adverse lighting and weather conditions. Although radar-based sensors can be used in traffic monitoring, they have not been tested in multimodal environments. To bridge this gap a novel framework to classify trajectories recorded by radar-based sensors in multimodal traffic environments is developed. The Support Vector Machine is used as the classifier and the following aspects allow to develop a robust, flexible, and transferable classification framework that can be applied in various traffic scenes. The SVM employs (1) a speed and length or speed and acceleration feature vector, (2) trajectory normalization scheme (i.e., select multiple measurements per trajectory), (3) training sample balancing strategy, and (4) cross-validation strategy. The framework is tested and validated using data from two different multimodal intersections. The results suggest that the mode type for every trajectory can be predicted with 95% accuracy using ten speed measurements and the vehicle’s mean length while accounting for the unequal number of observations per class. While these results are subject to change, i.e., both SVM’s input (feature vector and/or balancing approach) and performance may vary across different traffic scenes, the proposed framework is transferable and flexible.

The data conundrum: compression of automotive imaging data and deep neural network based perception

Video compression in automated vehicles and advanced driving assistance systems is of utmost importance to deal with the challenge of transmitting and processing the vast amount of video data generated per second by the sensor suite which is needed to support robust situational awareness. The objective of this paper is to demonstrate that video compression can be optimised based on the perception system that will utilise the data. We have considered the deployment of deep neural networks to implement object (i.e. vehicle) detection based on compressed video camera data extracted from the KITTI MoSeg dataset. Preliminary results indicate that re-training the neural network with M-JPEG compressed videos can improve the detection performance with compressed and uncompressed transmitted data, improving recalls and precision by up to 4% with respect to re-training with uncompressed data.

Heterogeneous sensing and scientific machine learning for quality assurance in laser powder bed fusion – A single-track study

Laser Powder Bed Fusion (LPBF) is the predominant metal additive manufacturing technique that benefits from a significant body of academic study and industrial investment given its ability to create complex geometry parts. Despite LPBF’s widespread use, there still exists a need for process monitoring to ensure reliable part production and reduce post-build quality assessments. Towards this end, we develop and evaluate machine learning-based predictive models using height map-derived quality metrics for single tracks and the accompanying pyrometer and high-speed video camera data collected under a wide range of laser power and laser velocity settings. We extract physically intuitive low-level features representative of the meltpool dynamics from these sensing modalities and explore how these vary with the linear energy density. We find our Sequential Decision Analysis Neural Network (SeDANN) model – a scientific machine learning model that incorporates physical process insights – outperforms other purely data-driven black box models in both accuracy and speed. The general approach to data curation and adaptable nature of SeDANN’s scientifically informed architecture should benefit LPBF systems with an evolving suite of sensing modalities and post-build quality measurements.

An Efficient System for Supporting Bat Swing of Beginners in Baseball Using Wearable Sensors

Recently a support system that improves sports skills using sensor or video camera data is attracting great attention. Since most of these systems are developed for professional athletes, few are available for beginners. In baseball, since hitting skills are generally acquired based on oral pantomimed by baseball experts, it is difficult for beginners to understand the advice based on the skill level of baseball experts. In this paper, we propose a support system through which beginners improve their hitting skills by analyzing the batting stance of baseball players using such wearable sensors as acceleration and angular velocity sensors. In our proposed system, beginners swing based on wearable sensors that compose a triaxial acceleration sensor and an angular velocity sensor to the bat and the body. Next, experts in baseball can analyze the modification of the batting stance of beginners by measuring the sensor values of the swing. In the evaluation result, our proposed system analyzed the differences of batting stance between experts and beginners in baseball and confirmed that we can effectively support the hitting skill of baseball beginners.

Video Camera Data Display Unit

Video camera module are one of the most important components in designing and implementing image/video processing systems. Such kind of systems are widely used all over the world for a long period of time. Due to using in surveillance and guard, they have to have very small size. The aim of this article are following: present method for designing working prototype of a system, that displays data, that is transmitted from digital video camera module; solve the problems and issue, that shows up during projection. In the first chapter explains the reasons of creating video camera data display unit. In the second chapter you can see the process of choosing components, needed for creating prototype, such as microcontroller (STM32F429ZI) and video camera module (OV7670). This module provides visual information, captioned and transmitted to user interface. There are few interesting challenges about choosing OV7670 settings, like frame format and RGB coding. They were questioned and chosen in order to create working and functional prototype. In chapter three are listed detailed theoretical information about features of chosen video camera module, such as purpose of every contact from module, bits placement for RGB coding, and why that is so important in project. In the fourth chapter described block diagram of device, which is projected. Principle of work of projecting system also is given here. You can read there explaining the need for using such communication protocols as I2C, SPI and DCMI bus, that is used very often with video camera modules. Explained reasons of using DMA in the system and short instructions for configuring parameters of DMA. In chapter five described possibility of using different interfaces for transmitting output data to the user – such as Wi-Fi transmitting-receiving technology, based on TCP/IP protocol. It can help in improving projected system to an independent product, which can replicate full-fledged IP camera. Literary data analysis is carried out. It contains examples of similar systems and their detailed descriptions. Some of these details were used in creating both article and prototype. In conclusion, this article shows opportunities for hardware realization of video camera data display unit, based on OV7670 module and STM32F429ZI microcontroller. The features of connecting the video camera module, its synchronization and data transmission via the DCMI bus are considered. The parameters for data transmission, using DMA mode are defined. To visualize the video in the device prototype, was suggested LCD display, connected via the SPI bus. For full designed IP camera, it is necessary to add transceivers (like Wi-Fi) to the hardware and develop software to support the TCP / IP protocol, needed for embedded Web-server realization. Designed hardware and software of video camera data display unit gives a clear representation of all main steps, needed to implement this system.

Cardio-respiratory signal extraction from video camera data for continuous non-contact vital sign monitoring using deep learning

Non-contact vital sign monitoring enables the estimation of vital signs, such as heart rate, respiratory rate and oxygen saturation (SpO2), by measuring subtle color changes on the skin surface using a video camera. For patients in a hospital ward, the main challenges in the development of continuous and robust non-contact monitoring techniques are the identification of time periods and the segmentation of skin regions of interest (ROIs) from which vital signs can be estimated. We propose a deep learning framework to tackle these challenges. Approach: This paper presents two convolutional neural network (CNN) models. The first network was designed for detecting the presence of a patient and segmenting the patient’s skin area. The second network combined the output from the first network with optical flow for identifying time periods of clinical intervention so that these periods can be excluded from the estimation of vital signs. Both networks were trained using video recordings from a clinical study involving 15 pre-term infants conducted in the high dependency area of the neonatal intensive care unit (NICU) of the John Radcliffe Hospital in Oxford, UK. Main results: Our proposed methods achieved an accuracy of 98.8% for patient detection, a mean intersection-over-union (IOU) score of 88.6% for skin segmentation and an accuracy of 94.5% for clinical intervention detection using two-fold cross validation. Our deep learning models produced accurate results and were robust to different skin tones, changes in light conditions, pose variations and different clinical interventions by medical staff and family visitors. Significance: Our approach allows cardio-respiratory signals to be continuously derived from the patient’s skin during which the patient is present and no clinical intervention is undertaken.

A measurement to driving situation awareness in signalized intersections

Drivers get involved easily in Left-Turn Across Path with Opposite Direction Traffic (LTAP-OD) conflicts at signalized intersection with unprotected left-turn phasing. This study classified the left-turn drivers’ performance into four types: (1) the correct acceptance of safe lags/gaps, (2) the correct rejection of dangerous lags/gaps, (3) the false rejection of safe lags/gaps and (4) the false acceptance of dangerous lags/gaps. Based on the high mounted video camera data, the logistic regression model was used to obtain the critical gaps for estimating whether the lags/gaps were safe or not. The results show that the cognition behavior of the left-turn drivers should be improved while the gap/lag is drawn near the critical gap. Furthermore, the conservative drivers are more likely to reject the larger lags/gaps and yield the right-of-way to the opposing vehicles. Simultaneously the poor response execution may cause drivers to have difficulties during the turning performance when the lags/gaps are large enough. These results could be used as the basis for a discussion of the right time to support the drivers.

Contrasting the direct use of data from traffic radars and video-cameras with traffic simulation in the estimation of road emissions and PM hotspot analysis

This study investigates the effect of traffic volume and speed data on the simulation of vehicle emissions and hotspot analysis. Data from a microwave radar and video cameras were first used directly for emission modelling. They were then used as input to a traffic simulation model whereby vehicle drive cycles were extracted to estimate emissions. To reach this objective, hourly traffic data were collected from three periods including morning peak (6–9 am), midday (11–2 pm), and afternoon peak (3–6 pm) on a weekday (June 23, 2016) along a high-volume corridor in Toronto, Canada. Traffic volumes were detected by a single radar and two video cameras operated by the Southern Ontario Centre for Atmospheric Aerosol Research. Traffic volume and composition derived from the radar had lower accuracy than the video camera data and the radar performance varied by lane exhibiting poorer performance in the remote lanes. Radar speeds collected at a single point on the corridor had higher variability than simulated traffic speeds, and average speeds were closer after model calibration. Traffic emissions of nitrogen oxides (NOx) and particulate matter (PM10 and PM2.5) were estimated using radar data as well as using simulated traffic based on various speed aggregation methods. Our results illustrate the range of emission estimates (NOx: 4.0–27.0 g; PM10: 0.3–4.8 g; PM2.5: 0.2–1.3 g) for the corridor. The estimates based on radar speeds were at least three times lower than emissions derived from simulated vehicle trajectories. Finally, the PM10 and PM2.5 near-road concentrations derived from emissions based on simulated speeds were two or three times higher than concentrations based on emissions derived using radar data. Our findings are relevant for project-level emission inventories and PM hot-spot analysis; caution must be exercised when using raw radar data for emission modeling purposes.

Automated Detection of Multiple Pavement Defects

AbstractKnowing the pavement condition is essential for efficiently deciding on maintenance programs. Current practice is predominantly manual with only 0.4% of inspections happening automatically. All methods in the literature aiming at automating condition assessment focus on two defects at most, or are too expensive for practical application. In this paper, the authors propose a low-cost method that automatically detects pavement defects simultaneously using parking camera video data. The types of defects addressed in this paper are two types of cracks (longitudinal and transverse), patches, and potholes. The method uses the semantic texton forests (STFs) algorithm as a supervised classifier on a calibrated region of interest (myROI), which is the area of the video frame depicting only the usable part of the pavement lane. It is validated using data collected from the local streets of Cambridge, U.K. Based on the results of multiple experiments, the overall accuracy of the method is above 82%, with a p...

Acquisition of the field measurement data relating to lane change actions

As a data set for validation of microscopic simulation models, we obtained the fundamental diagram (flux versus density relation), and the relations of velocity versus density, lane usage ratio versus density and lane-changing frequency versus density derived from a single field measurement campaign held at a Japanese urban expressway. The results were drawn from image analysis of video camera data obtained at the site.

English

In this paper, a solution about secure provenance transmission for streaming data is introduced based on Raspberry Pi processor is presented. The embedded system, video capture, short message service (SMS) alarm, and video monitor are introduced. Video 4 Linux is used to get the camera video data, which is transferred to the Web Server, and the data is displayed on the client browser. The main aim of this paper is the designed system have USB camera it will continuously capturing in front of the people then processes the captured images via USB host to the Raspberry Pi to the display device unit, and the parallel connection Ethernet module, web page is created the remote pc using HTTP protocol because HTTP protocol provides secure data transmission. And for more security we are creating some IP address to view the live streaming data on remote pc and when the person entered infront of camera the processor collect the information and the processor will activated the GSM, through GSM send message to the authorized person. In this processes in USB camera captured images face are detected by using LBP accurately and stored temporarily in our database to check or recognize is this person is right person to use or not. Compared with video capture system based on digital signal processor (DSP), this system has the advantage of fewer modules, lower cost, higher intelligence, higher system stability, and higher security. This paper presented design system will integrate ARM11, USB cam, GSM, remote pc, Display monitoring unit, and Ethernet. The entire paper proves continuous, and surveillance secured system.

VideoHandles: Searching through action camera videos by replicating hand gestures

We present a detailed exploration of VideoHandles, a novel interaction technique to support rapid review of wearable video camera data by re-performing gestures as a search query. The availability of wearable video capture devices has led to a significant increase in activity logging across a range of domains. However, searching through and reviewing footage for data curation can be a laborious and painstaking process. We showcase the use of gestures as search queries to support review and navigation of video data. By exploring example action-camera footage across a range of activities, we propose two video data navigation styles using gestures: prospective gesture tagging and retrospective gesture searching. This paper builds on our earlier VideoHandles work, reporting details of our interaction design and implementation, and presenting two additional evaluations. We demonstrate that VideoHandles is a viable interaction technique, returning promising results in its prototype form.

Blockage assessment of buildings during emergency using multiple types of sensors

Abstract In case of an emergency in a building, first responders need to know current blockages in the building (e.g., blocked passageways and exits) and safe evacuating paths so that the occupants can be guided to the unblocked exits and safe paths toward those exits. To automatically determine blockage levels at buildings, a system that fuses data from multiple sensors and video camera was proposed. A prototype was developed and tested on an experimental model of a pilot building's hallway. A series of damage tests were conducted on the hallway model and recorded by the sensors and the video camera. Individual performances of sensors and video camera were evaluated, and a decision tree method was used to fuse sensor and video camera data for estimating the level of blockage in the hallway for different damage combinations applied on building components. The results demonstrated the technical feasibility of the proposed system and the findings of the decision tree highlight that by using less number of sensors, a cost-effective configuration can be achieved. The estimated blockage information can be used to create a topological map of the damaged building, indicating safe paths toward available unblocked exits.