Video Camera Research Articles

Occlusion handling in spatio-temporal object-based image sequence matching

Abstract. Dynamic photogrammetry is an established method for acquiring 3D information of deforming objects or dynamic scenes in various close-range applications. A crucial impact has occlusions caused by object deformations, obstacles or camera movements. Temporal occlusions are highly application-specific and sometimes difficult to predict, resulting in a significant reduction of reconstruction quality or the aborting of image sequence processing. Previous approaches usually model such occlusions as semantic information and consider them using image masks. However, generating these image masks requires complex methods and extensive training data. Due to the unpredictability of the complexity and movements of dynamic scenes, generating training data is challenging in many applications. Therefore, this paper proposes an alternative modelling approach, which can be part of a spatio-temporal matching process. Based on the characteristic high redundancy, occlusions can be detected using robust estimation methods and considered in the optimisation. Therefore, no information about the occlusions and further processing steps are necessary. We evaluate our approach with synthetic and real data of an industrial application regarding the accuracy and ability to detect occlusion simultaneously. The evaluation of the proposed approach shows that the impact of occlusion can be eliminated, and the quality of the results is comparable to conventional methods.

Finding the path of least resistance: An examination of officer communication tactics and their impact on suspect compliance

AbstractResearch SummaryThis study investigated the impact of patrol officer communication tactics on suspect compliance by using Systematic Social Observation (SSO) to code 438 body‐worn camera and dashcam video recordings from two police agencies. Specifically, we examined officer tenor/demeanor, use of noncoercive directives, use of verbal coercion, and accusations, while controlling for a host of suspect and situational factors. Results showed that when officers presented a positive tenor/demeanor or employed noncoercive verbal tactics, suspects were significantly more likely to comply. Conversely, use of coercive verbal tactics and accusatory language did not have a significant impact on suspect compliance. In addition, negative suspect demeanor, suspect impairment, and suspect elevated emotional state negatively affected compliance.Policy ImplicationsRecent calls for police reform have focused on producing less coercive outcomes by explicitly recognizing the role that officers have in managing encounters rather than simply reacting to suspect behaviors. An increased focus on officer communication strategies in both police training and practice is likely to promote suspect compliance while reducing the need for officer use of force. Similarly, it is likely that more coercive verbal strategies have little impact on suspect compliance and may lead to negative perceptions of police interactions and law enforcement in general.

Vision-based Human Activity Recognition Using Local Phase Quantization

Human activity recognition (HAR) has been the most active and interesting area of research in recent years due to its wide range of applications in the field, such as healthcare, security and surveillance, robotics, gaming, entertainment, etc. However, recognizing vision-based human activity is still a challenging as input sequences may have cluttered background, illumination conditions, occlusions, degradation of video quality, blurring, etc. In the literature, several state-of-the-art methods have been trained and tested on different datasets but have yet to perform adequately to a certain extent. Moreover, extracting potential features and combining appropriate methods is one of the most challenging tasks in realistic video. This paper proposes an efficient frequency-based blur-invariance local phase quantization feature extractor and multiclass SVM classifier that overcomes these challenges. The feature is invariant towards camera motion, misfocused optics, movements in the scene, and environmental conditions. The proposed feature vector is then fed to the classifier to recognize human activities. The experiment has conducted on two publicly available datasets, UCF101 and HMDB51, and has achieved 99.79% and 98.67% accuracies, respectively. The approach has also outperformed the existing state-of-the-art approaches in terms of computational cost without compromising the accuracy of HAR.

Effects of Applied Voltage on Branching of Positive Leaders in Laboratory Long Sparks

AbstractPositive leaders branch less frequently than negative counterpart, and the physical processes and properties of positive leader branching remain a mystery. We investigated 10 m laboratory discharges under four positive voltages using a high‐speed video camera. Positive leaders differ from negative leaders by either directly splitting or connecting with floating bidirectional leaders to form branching, and the number of leader branches shows a positive correlation with the applied voltage, that is, the branched channels increased from 1 to 4 when the voltage increased by a factor of 1.5. Grounding points are positioned beneath the electrode and are more concentrated with lower voltage. During the stable progression of the leader, there is a slight increase in its development speed as the applied voltage rises. When the voltage is increased by 70%, the average breakdown time decreases by 40%. These characteristics provide insights into the branching mechanism of positive leaders.

A machine learning-based approach for constructing remote photoplethysmogram signals from video cameras

BackgroundAdvancements in health monitoring technologies are increasingly relying on capturing heart signals from video, a method known as remote photoplethysmography (rPPG). This study aims to enhance the accuracy of rPPG signals using a novel computer technique.MethodsWe developed a machine-learning model to improve the clarity and accuracy of rPPG signals by comparing them with traditional photoplethysmogram (PPG) signals from sensors. The model was evaluated across various datasets and under different conditions, such as rest and movement. Evaluation metrics, including dynamic time warping (to assess timing alignment between rPPG and PPG) and correlation coefficients (to measure the linear association between rPPG and PPG), provided a robust framework for validating the effectiveness of our model in capturing and replicating physiological signals from videos accurately.ResultsOur method showed significant improvements in the accuracy of heart signals captured from video, as evidenced by dynamic time warping and correlation coefficients. The model performed exceptionally well, demonstrating its effectiveness in achieving accuracy comparable to direct-contact heart signal measurements.ConclusionsThis study introduces a novel and effective machine-learning approach for improving the detection of heart signals from video. The results demonstrate the flexibility of our method across various scenarios and its potential to enhance the accuracy of health monitoring applications, making it a promising tool for remote healthcare.

A Light-Field Video Dataset of Scenes with Moving Objects Captured with a Plenoptic Video Camera

A Light-Field Video Dataset of Scenes with Moving Objects Captured with a Plenoptic Video Camera

Computer Vision and Augmented Reality for Human-Centered Fatigue Crack Inspection.

A significant percentage of bridges in the United States are serving beyond their 50-year design life, and many of them are in poor condition, making them vulnerable to fatigue cracks that can result in catastrophic failure. However, current fatigue crack inspection practice based on human vision is time-consuming, labor intensive, and prone to error. We present a novel human-centered bridge inspection methodology to enhance the efficiency and accuracy of fatigue crack detection by employing advanced technologies including computer vision and augmented reality (AR). In particular, a computer vision-based algorithm is developed to enable near-real-time fatigue crack detection by analyzing structural surface motion in a short video recorded by a moving camera of the AR headset. The approach monitors structural surfaces by tracking feature points and measuring variations in distances between feature point pairs to recognize the motion pattern associated with the crack opening and closing. Measuring distance changes between feature points, as opposed to their displacement changes before this improvement, eliminates the need of camera motion compensation and enables reliable and computationally efficient fatigue crack detection using the nonstationary AR headset. In addition, an AR environment is created and integrated with the computer vision algorithm. The crack detection results are transmitted to the AR headset worn by the bridge inspector, where they are converted into holograms and anchored on the bridge surface in the 3D real-world environment. The AR environment also provides virtual menus to support human-in-the-loop decision-making to determine optimal crack detection parameters. This human-centered approach with improved visualization and human-machine collaboration aids the inspector in making well-informed decisions in the field in a near-real-time fashion. The proposed crack detection method is comprehensively assessed using two laboratory test setups for both in-plane and out-of-plane fatigue cracks. Finally, using the integrated AR environment, a human-centered bridge inspection is conducted to demonstrate the efficacy and potential of the proposed methodology.

Student visual attention during group instruction phases in collaborative geometry problem solving

AbstractTeacher gestures support mathematics learning and promote student collaboration. Aligned with speech, gestures can help students to notice the important visual information of geometry tasks. However, students’ visual attention to the teacher’s gestural cues during collaborative problem solving remains a largely unexplored field in mathematics education research. This mixed-method case study investigated relations between students’ visual attention, teacher gestures, and students’ collaborative problem-solving process on a geometry task. The data were collected with video cameras and mobile gaze trackers on four students simultaneously in two Finnish 9th-grade mathematics lessons with the same teacher. The findings show that the students attended to their own papers most of the time during the teacher's gestures, but differences in student attention between the gesture types emerged. The qualitative analysis showed that the teacher’s tracing, pointing, and representational gestures helped in directing student attention to targets relevant to the situational learning process. We conclude that teacher gestures can both convey mathematical contents and direct student attention, which intermediates mathematical thinking in problem solving.

Assessing Table Tennis Technical Proficiency in Individuals with Disabilities: A Scoping Review

Table tennis is a sport that is enjoyed by many, including those with physical and intellectual disabilities. This scoping review summarised the current test protocols for assessing table tennis technical proficiency in individuals with disabilities. Relevant articles were searched through four databases (Scopus, PubMed, SPORTDiscus, and Web of Science) covering three key aspects: disability, table tennis, and technical proficiency. The search resulted in 14 studies included for data extraction, covering physical impairments, intellectual disability, and development coordination disorder. Almost all studies (93%) were conducted on well-trained para-table tennis athletes competing in high-level competitions. There exist protocols to assess service accuracy and stroke accuracy, hand–eye coordination, quality of specific skills and ball control, functional reach, and trunk rotation. The forehand topspin and backhand topspin drives were tested the most. Table tennis robots and video cameras are the common equipment used. Moving forward, future research should develop technical proficiency tests for players across all competency levels. The skill assessment criteria and scoring methods should be standardised and clearly explained. The validity and reliability of tests should be established. Lastly, there is great potential in using artificial intelligence to enhance the assessment of table tennis proficiency in individuals with disabilities.

Non‐extractive fish diversity assessment in Mediterranean rhodolith beds

Abstract Rhodolith beds, recognized as biodiversity hotspots with high ecological and conservation importance, face local anthropic pressures, including trawl fishing. However, monitoring the diversity associated with this sensitive biogenic habitat in the deep waters of the Mediterranean Sea is challenging. Traditional monitoring methods, such as experimental trawling and trammel nets, are extractive and have sampling limitations in providing comprehensive diversity inventories. In this work, we explore three alternative techniques (still video cameras, environmental DNA [eDNA] and bioacoustics) to monitor fish assemblages associated with rhodolith beds in the Menorca Channel at 45–80 m depth. Ten sites were surveyed with eDNA, and 66 fish taxa were identified, including endangered sharks and cryptic species. The molecular operational taxonomic units (MOTUs) richness varied significantly according to habitat heterogeneity, being higher in homogeneous bottoms, and to exposure to trawling, with higher richness in non‐exposed areas. Bioacoustics, though limited by a reduced library of fish sounds, detected differences in sound richness between habitat types, and the highest abundance of sounds was recorded in the areas with the greatest small‐scale habitat heterogeneity. Underwater video revealed limitations in species detection but proved effective for characterizing small‐scale habitat types and heterogeneity. In summary, each technique provides valuable information at different spatial scales and levels of detail. To assess the feasibility of these techniques for long‐term monitoring, we explored their capabilities and limitations in terms of costs, survey logistics and data output.

Eye Strain Expression Classification using Attention Capsule Network for Adapting Screen Vision

Beside the conventional facial expression recognition methods, the research focuses on developing a system for recognizing various eye expressions under different screen conditions. This research deals with the use of Capsule Network (a recent Deep Learning algorithm) to enhance facial expression recognition capabilities and to develop adaptive screen technologies aimed at mitigating digital eye strain. The main objective of this research is to engineer a sophisticated system that employs the capabilities of Capsule Nets to recognize the various expressions that user makes and based on the recognized expression, dynamically modify screen settings, ensuring optimal user visual comfort. The research primarily concentrates on the exploration and application of various Capsule Net architectures designed for the recognition of expressions related to eye strain. The baseline model utilized elementary convolutional layers which feed into subsequent fully connected layers for the task of classification. The model has since been refined by incorporating advanced techniques such as attention mechanisms and more sophisticated network architectures where the classification is done by Capsule Network. Results have demonstrated a modest enhancement in the Capsule Net’s predictive performance, attributed to its superior spatial and hierarchical processing of facial features, in comparison to conventional deep learning approaches. The final model has an accuracy of 82.27%. As a final system the model has been deployed to an application to process frames from video camera in the device and make prediction to prompt the notifications or recommendations.

Does using a bioptic telescope improve visual recognition of the on-road environment?

Individuals with reduced visual acuity (VA) may drive with bioptic telescopes in some jurisdictions. The effect of bioptic telescope use on on-road recognition distances is important for driving safety, as increased recognition distances increase the time available to react to road signs or driving events. This study aimed to investigate the impact of bioptic telescope use on visual recognition of road signs, traffic lights, and hazards in an on-road driving environment in individuals with reduced VA. Ten individuals (mean ± standard deviation age, 39.1 ± 19.1 years) with reduced VA (trained to use bioptic telescopes) participated in a repeated-measures on-road experiment for two viewing conditions, with and without a bioptic telescope (randomized order). Participants underwent a clinical low vision assessment, including high-contrast VA with and without the bioptic telescope aligned, visual fields, and contrast sensitivity testing. For the driving component, participants seated in the front passenger seat of a moving car reported all road signs, traffic lights, and hazards (unpredictable road events that represent a risk to driving safety including other vehicles, cyclists, pedestrians) seen along a route that included suburban roads and highways. Video cameras captured participants' viewing behavior, verbal commentary, and the distance at which three pre-selected road signs were reported. In the eye with the bioptic telescope, high-contrast VA improved from 0.75 ± 0.17 (without) to 0.25 ± 0.1 logMAR with the bioptic telescope. Bioptic telescope use did not affect the percentage of road signs, traffic lights, or hazards correctly recognized but did result in 2.6 times longer recognition distances (49 ± 23 vs. 19 ± 11 m, t9 = 5.02, p<0.001). Road signs were recognized at significantly longer distances when using a bioptic telescope, confirming their positive impact on timely visual recognition of objects within the driving scene. Future work should explore whether this effect generalizes to individuals using bioptic telescopes when driving a vehicle.

Unsupervised Monocular Depth Estimation With Channel and Spatial Attention.

Understanding 3-D scene geometry from videos is a fundamental topic in visual perception. In this article, we propose an unsupervised monocular depth and camera motion estimation framework using unlabeled monocular videos to overcome the limitation of acquiring per-pixel ground-truth depth at scale. The photometric loss couples the depth network and pose network together and is essential to the unsupervised method, which is based on warping nearby views to target using the estimated depth and pose. We introduce the channelwise attention mechanism to dig into the relationship between channels and introduce the spatialwise attention mechanism to utilize the inner-spatial relationship of features. Both of them applied in depth networks can better activate the feature information between different convolutional layers and extract more discriminative features. In addition, we apply the Sobel boundary to our edge-aware smoothness for more reasonable accuracy, and clearer boundaries and structures. All of these help to close the gap with fully supervised methods and show high-quality state-of-the-art results on the KITTI benchmark and great generalization performance on the Make3D dataset.

Chelonia mydas detection and image extraction from noisy field recordings

&lt;span lang="EN-US"&gt;Wildlife videography is an essential data collection method for conducting research on animals. The video recording process of an animal like the &lt;em&gt;Chelonia Mydas&lt;/em&gt; turtle in its natural habitat requires the setting up of special camera traps or by performing complex camera movement to capture the animal in frame whilst the cameraman maneuvers over uneven terrain while filming. The result is hours of footage that only have the presence of the intended subject in it for seconds whilst the rest is background footage; or noisy and blurry footage that has only several usable frames among thousands of noisy and unusable ones. This presents a problem that deep learning models can help to assist, especially in detecting a wildlife subject and extracting usable data from hours of noise and background footage. This paper proposes the use of machine learning models to detect and extract wildlife images of &lt;em&gt;Chelonia Mydas&lt;/em&gt; turtles to help prune through hundreds and thousands of frames from several video footages. Our paper shows that utilizing a custom model with various confidence scores can label and crop out images in noisy field video recordings of &lt;/span&gt;&lt;em&gt;&lt;span lang="EN-US"&gt;Chelonia Mydas &lt;/span&gt;&lt;/em&gt;&lt;span lang="EN-US"&gt;turtles with up to 99.89% of output images correctly cropped and labeled.&lt;/span&gt;

이동용 영상촬영기기 바디캠을 활용한 팀단위 심폐소생술의 교육피드백 효과

This study conducted a team-based CPR simulation with 32 fourth-year emergency rescue students to determine the effectiveness of training feedback using body cameras used at emergency rescue sites, and measured awareness, training feedback effectiveness, and satisfactio+n before and after body camera feedback. , preferences and difficulties in using body camera devices were identified. Data analysis was performed using SPSS 27.0 program, including descriptive statistics, frequency analysis, paried t-test, and Wilcoxon signed rank test. As a result of the study, the perception of body camera use showed a positive change from 3.73±0.62 points to 4.45±0.54 points, and a positive satisfaction level of 3.98±0.51 was shown (p&lt;.001). Additionally, there was a significant increase in self-check accuracy and performance score after body camera feedback (p&lt;.001). Therefore, during team-based simulation resuscitation training, positive feedback effects in improving self-inspection ability and performance can be achieved by watching body camera videos and using self-checklists without direct feedback from the instructor.

A multimodal analysis of the English-to-Arabic adaptation of Disney’s “The Bare Necessities”

Abstract Due to the diverse roles that songs play in films and the growing need for movie distribution in various languages and cultures, the challenge of translating film songs from one language to another has become formidable. This research aims to apply the multimodal three-dimensional model proposed by Cui and Wang (2022. Film song translation: Verbal, vocal, and visual dimensions: On the Chinese translation of Amazing Grace in the film Forever Young. Babel 68(4). 565–585) to analyze the Arabic dubbed version of the 2016 film song “The Bare Necessities.” The analysis revealed that, verbally, the Arabic dubbed version of the song maintained the gay emotional tone and length but made adaptations to the information, metaphors, and images used in the source. Vocally, similar to the English source song, the Arabic dubbed version did not adhere to 1 m and has a 4/4 time signature, aligning with the catchy melody and reinforcing the song’s theme of embracing simplicity and savoring life’s essentials effortlessly. Finally, visually, the analysis showed that the Arabic dubbed version matches the plot, characters and background pictures on the screen. The study contributed by incorporating a visual analysis of songs, adding camera elements, namely camera movement and shot size, to gain a better understanding of the scenes under analysis. The use of over-the-shoulder shots created a sense of conversation that complements the duet, and the pan right and left during dancing added a vividness that matches the verbal gay tone and vocal lightness of the song. This finding, considering visual camera elements, can be regarded as a suggested revision to the original multimodal framework introduced by Cui and Wang. Further implications are discussed.

Visual Fall Detection Analysis Through Computer Vision and Deep Learning – Technology Proposition

Advances in modern medicine has increased humans’ life span. Orderly adults face mobility problems while aging. They feel less fit to continue any activity for short intervals too. This is due to declining fitness levels or muscle strength, diminished dexterity, and loss of balance. These symptoms lead to the fall of the individual and sometimes fatal too, if immediately not attended to. It’s an alarming issue for people staying alone. They may pose significant health risks and need immediate assistance. Fall detection technologies are majorly categorised as wearable sensors and ambient sensors. Fall detection wearable devices like pendant necklaces, watches and wristband devices, and clip-on medical alerts use accelerometers to detect rapid downward movements that can indicate a fall. They often also include manual alert buttons, for an increased accuracy. This requires technology comfort and awareness for usage. Ambient home sensors use video cameras to monitor the user’s movement and detect falls. When the fall is transmitted to a monitoring center, a representative typically will call the user to check on them before notifying contacts or calling for emergency services, but this can depend on the user’s preferences and risk factors. In this paper we propose a technology, using security cameras to record videos and create a video-based fall detection system. The system uses computer vision and deep learning algorithms to accurately recognize fall-related movements and distinguish them from regular activities. This system can be integrated to prompt alerts to emergency contacts, thus assisting in providing immediate aid to individuals who have experienced a fall. For higher accuracy, multiple-angle videos and multi-person tracking is integrated in this system to estimate the intensity of the fall for immediate attention. Thus, this fall detection system can contribute to the safety, well-being and independence of individuals at risk of falling.

Image moment-based visual positioning and robust tracking control of ultra-redundant manipulator

Image moment features can describe more general target patterns and have good decoupling properties. However, the image moment features that control the camera’s rotation motion around the x-axis and y-axis mainly depend on the target image itself. In this paper, the ultra-redundant manipulator visual positioning and robust tracking control method based on the image moments are advocated.First, six image moment features used to control camera motion around the x-axis and around the y-axis are proposed. And then, a novel method is proposed to use to select image features. For tracking a moving target, a kalman filter combined with adaptive fuzzy sliding mode control method is proposed to achieve tracking control of moving targets, which can estimate changes in image features caused by the target’s motion on-line and compensate for estimation errors. Finally, the experimental system based on Labview-RealTime system and ultra-redundant manipulator is used to verify the real-time performance and practicability of the algorithm. Experimental results are presented to illustrate the validity of the image features and tracking method.

Microsaccade-inspired event camera for robotics.

Neuromorphic vision sensors or event cameras have made the visual perception of extremely low reaction time possible, opening new avenues for high-dynamic robotics applications. These event cameras' output is dependent on both motion and texture. However, the event camera fails to capture object edges that are parallel to the camera motion. This is a problem intrinsic to the sensor and therefore challenging to solve algorithmically. Human vision deals with perceptual fading using the active mechanism of small involuntary eye movements, the most prominent ones called microsaccades. By moving the eyes constantly and slightly during fixation, microsaccades can substantially maintain texture stability and persistence. Inspired by microsaccades, we designed an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. In this design, a rotating wedge prism was mounted in front of the aperture of an event camera to redirect light and trigger events. The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion, resulting in a stable texture appearance and high informational output independent of external motion. The hardware device and software solution are integrated into a system, which we call artificial microsaccade-enhanced event camera (AMI-EV). Benchmark comparisons validated the superior data quality of AMI-EV recordings in scenarios where both standard cameras and event cameras fail to deliver. Various real-world experiments demonstrated the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.

Associations between Stress Mindset and Changes in Positive Affect during a Speech Task: A Preliminary Study

A stress mindset is an individual’s belief that stress has either enhancing (stress-is-enhancing mindset: SEM) or debilitating (stress-is-debilitating mindset: SDM) consequences. This study examined the associations between SEM and changes in positive affect during a speech task using a sample of 32 Japanese college students as participants. It was hypothesized that participants with SEM would show a greater increase in positive affect over time than those with SDM. The participants rested for three minutes, prepared for three minutes, delivered a speech about themselves in front of an evaluator and video camera (task period), and again rested for three minutes. Positive affect was measured immediately after the two resting periods and during the speech. The participants were divided into groups of 17 and 15 in the SEM and SDM groups, respectively, based on their stress mindset scores. Both groups were fairly matched concerning age and male/female ratio. Change patterns of positive affect did not differ between the two groups, which did not support the hypothesis. However, positive affect was significantly higher in the SEM group compared with the SDM group. The present results differed from previous findings reported in North America. Inconsistent results are discussed related to differences in the speech task and cross-cultural variations of happiness and coping with stressors between North America and Japan.