Camera Switching Research Articles

EmotionCast: An Emotion-Driven Intelligent Broadcasting System for Dynamic Camera Switching.

Traditional broadcasting methods often result in fatigue and decision-making errors when dealing with complex and diverse live content. Current research on intelligent broadcasting primarily relies on preset rules and model-based decisions, which have limited capabilities for understanding emotional dynamics. To address these issues, this study proposed and developed an emotion-driven intelligent broadcasting system, EmotionCast, to enhance the efficiency of camera switching during live broadcasts through decisions based on multimodal emotion recognition technology. Initially, the system employs sensing technologies to collect real-time video and audio data from multiple cameras, utilizing deep learning algorithms to analyze facial expressions and vocal tone cues for emotion detection. Subsequently, the visual, audio, and textual analyses were integrated to generate an emotional score for each camera. Finally, the score for each camera shot at the current time point was calculated by combining the current emotion score with the optimal scores from the preceding time window. This approach ensured optimal camera switching, thereby enabling swift responses to emotional changes. EmotionCast can be applied in various sensing environments such as sports events, concerts, and large-scale performances. The experimental results demonstrate that EmotionCast excels in switching accuracy, emotional resonance, and audience satisfaction, significantly enhancing emotional engagement compared to traditional broadcasting methods.

Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detection

Learning discriminative features for effectively separating abnormal events from normality is crucial for weakly supervised video anomaly detection (WS-VAD) tasks. Existing approaches, both video and segment level label oriented, mainly focus on extracting representations for anomaly data while neglecting the implication of normal data. We observe that such a scheme is sub-optimal, i.e., for better distinguishing anomaly one needs to understand what is a normal state, and may yield a higher false alarm rate. To address this issue, we propose an Uncertainty Regulated Dual Memory Units (UR-DMU) model to learn both the representations of normal data and discriminative features of abnormal data. To be specific, inspired by the traditional global and local structure on graph convolutional networks, we introduce a Global and Local Multi-Head Self Attention (GL-MHSA) module for the Transformer network to obtain more expressive embeddings for capturing associations in videos. Then, we use two memory banks, one additional abnormal memory for tackling hard samples, to store and separate abnormal and normal prototypes and maximize the margins between the two representations. Finally, we propose an uncertainty learning scheme to learn the normal data latent space, that is robust to noise from camera switching, object changing, scene transforming, etc. Extensive experiments on XD-Violence and UCF-Crime datasets demonstrate that our method outperforms the state-of-the-art methods by a sizable margin.

Camera Switching 및 차량의 주행 방향을 활용한 자율 발렛 주차

Autonomous valet parking(AVP) is a system that allows a vehicle to detect a parking space and park without any human intervention. This paper explores the efficient use of monocular visual simultaneous localization and mapping(visual-SLAM) in AVP scenarios, while also highlighting several problems arising from current AVP methods. These problems include the difficulties of performing a loop closure in narrow places and the unreliability of the estimated scale between the real-world and SLAM maps. This paper explores two methods for resolving these problems. The first solution is to perform AVP without loop closure. This can be done through a camera switching method that uses the front camera while parking, and the rear camera in return driving scenarios. The second solution involves making use of an alternative automatic parking algorithm that utilizes a car’s heading. Most automatic parking algorithms utilize monocular visual-SLAM, but it is difficult to calculate an accurate scale using this method. The alternative automatic parking algorithm using a car’s heading could eliminate these issues with scaling. Each method was tested in three different locations, returning average success rates of 80% in return driving and 93.3% in automatic parking. The methods highlighted in this paper only utilize cameras, which are inexpensive compared to other sensors and are expected to contribute to commercializing the AVP system.

AMTSet: a benchmark for abrupt motion tracking

Since the OTB100 benchmark dataset is released, it has been widely used in a large number of researches on object tracking for performance evaluation. However, the existing datasets are insufficient to evaluate trackers in handling different challenging factors. In this paper, we present the first dataset and benchmark for tracking objects with abrupt motion (AMTSet). The dataset consists of 50 videos of special scenes from our real life, such as camera switching, sudden dynamic change, low frame rate video, etc., which are quite challenging in object tracking. Boundary boxes over 10K frames are marked manually, and all of them are manually labelled for common attributes of object tracking, such as scale variation, illumination variation, occlusion, motion blur, etc. We benchmark the dataset on 36 representative trackers and rank them according to the tracking conditions and results. Furthermore, we propose an evaluation measure for object tracking to better highlight the performances of the trackers against abrupt motion. Our goal is to supplement the existing baseline datasets and provide researchers with more perfect baseline data in order to better evaluate the performance of different trackers.

Visual Multi-Face Tracking Applied to Council Proceedings

Face recognition, face measurement, camera control, measurement of expressions and other tasks can benefit from online visual multi-face tracking. Given the availability of high quality general purpose detectors and tracking-by-detection frameworks, we provide guidance on how to develop a multi-face tracker out of standard components. In this paper, we train common object detectors specifically on faces to understand how well these detectors perform and evaluate different classifier loss functions. Our specific case study tracks faces in the context of council meetings and in parliamentary settings such as the Canadian House of Commons for which we create an annotated video set as a benchmark (see Fig. 1). These meetings in a parliamentary setting are often recorded from multiple cameras with participants and audiences walking around. Fast camera switching and zooming lead to significant scale changes of faces. Therefore, these settings can be characterized as tracking in unconstrained video. This will negatively impact the tracking accuracy and increase the likelihood of identity switches (IDS) between face labels. However, being able to track in unconstrained video enables a wider range of measurement applications. We find that while online tracking based on combining state-of-the-art methods can lead to high-quality tracking results, there is still a large gap between offline and online methods. The discussed method can be adapted to other tracking tasks for which large image databases are available.

Enhanced Unsupervised Change Detector for Industrial Surveillance Systems

Detecting an intruder that is trespassing a prohibited area is a critical task of intelligent surveillance systems. This task requires a change detector to segment an intruder (foreground object) from the background. The task suffers the inherent drawbacks of change detectors due to the dual-camera sensor (color/IR), illumination changes, night time, static, and camouflaged foreground objects. This article proposes an enhanced unsupervised change detector to compensate for the aforementioned challenges for industrial sterile zone monitoring. The camera switch detection based on skewness patterns detects a switch between the dual-camera sensors (color/IR). The optimal color space selection based on the mean squared error will select tolerant color space (RGB/YCbCr) to illumination changes for modeling the background. In addition, the IR camera frames are contrast-enhanced to tackle the camouflaged intruders during the night. The incoming frames are split into respective channels before modeling the background. The background is modeled by Gaussian mixture models. The adaptive background model update scheme is proposed to tackle the various challenges posed by environment and object such as a static foreground object. The comparison is performed on three databases with top-ranked unsupervised change detection algorithms.

Direct-Visual-Operation Support System for Unmanned Construction

Introduction: Unmanned construction through direct visual operation is performed to ensure the safety of workers at construction sites. In direct visual operation, although the equipment is simple and easy to install, the work efficiency and accuracy are reduced because of the lack of view and perspective obtained from boarding a construction machine. For solving this problem, images sent from multiple cameras, Unmanned Aerial Systems (UASs) attached to the construction equipment and the images obtained from boarding a construction machine, as well as blind spots are confirmed by displaying them on the monitor at hand. However, the working efficiency is lowered by the restrictions on the utilization range of the camera monitor, switching operation of the camera, and gaze movement between the construction machine and the monitor. Methods: For solving the problem of low work efficiency and accuracy of the conventional system, this paper proposes a support system for a direct visual operation that does not require monitor installation and gaze movement and enables intuitive camera switching operation by using a transmissive Head Mounted Display (HMD) and a stereo camera robot. Results and Conclusion: The results of the experiment conducted using a remote-controlled backhoe show that unskilled operators can perform the same quality of work as skilled operators, and work efficiency and accuracy was improved by 44.2% and 37.8%, respectively compared to the conventional system. This confirms the usefulness of the proposed system, especially for unskilled operators.

Self-Supervised Multi-View Person Association and its Applications.

Reliable markerless motion tracking of people participating in a complex group activity from multiple moving cameras is challenging due to frequent occlusions, strong viewpoint and appearance variations, and asynchronous video streams. To solve this problem, reliable association of the same person across distant viewpoints and temporal instances is essential. We present a self-supervised framework to adapt a generic person appearance descriptor to the unlabeled videos by exploiting motion tracking, mutual exclusion constraints, and multi-view geometry. The adapted discriminative descriptor is used in a tracking-by-clustering formulation. We validate the effectiveness of our descriptor learning on WILDTRACK T. Chavdarova et al., "WILDTRACK: A multi-camera HD dataset for dense unscripted pedestrian detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018, pp. 5030-5039. and three new complex social scenes captured by multiple cameras with up to 60 people "in the wild". We report significant improvement in association accuracy (up to 18 percent) and stable and coherent 3D human skeleton tracking (5 to 10 times) over the baseline. Using the reconstructed 3D skeletons, we cut the input videos into a multi-angle video where the image of a specified person is shown from the best visible front-facing camera. Our algorithm detects inter-human occlusion to determine the camera switching moment while still maintaining the flow of the action well. Website: http://www.cs.cmu.edu/~ILIM/projects/IM/Association4Tracking.

Research on the forward distance detection algorithm based on the camera switching

In order to promote vehicle active safety and to decrease collision accident, In this paper, an algorithm for detection of vehicle distance in front based on single and double camera switching is presented. This algorithm has done the corresponding difference processing to the day and night two different situations. By switching between single and double cameras, the image of the car bottom shadow, the characteristics of the vertical direction of the vehicle, the characteristics of the car taillight and other features was taken, and the position of the vehicle in front was locked. Finally, the feature information of the vehicle is converted into the distance information. Experiments show that, this system designed in the paper, achieved the anticipated function, can accurately detect multi-lane and short distance vehicle distance, and the efficiency of the system also achieved the real-time requirement.

Optimized camera handover scheme in free-viewpoint video streaming

Free-viewpoint video (FVV) is a promising approach that allows users to control their viewpoint and generate virtual views from any desired perspective. The individual user viewpoints are synthesized from two or more camera streams and correspondent depth sequences. In case of continuous viewpoint changes, the camera inputs of the view-synthesis process must be changed in a seamless way, to avoid the starvation of the viewpoint synthesizer algorithm. Starvation occurs when the desired user viewpoint cannot be synthesized with the currently streamed camera views, and thus, the FVV playout interrupts. In this paper, we propose three different camera handover schemes (TCC, MA, and SA) based on viewpoint prediction to minimize the probability of playout stalls and find the trade-off between the image quality and the camera handover frequency. Our simulation results show that the introduced camera switching methods can reduce the handover frequency with more than 40%, and hence, the viewpoint synthesis starvation and the playout interruption can be minimized. By providing seamless viewpoint changes, the quality of experience can be significantly improved, making the new FVV service more attractive in the future.

Automatic Live Sport Video Streams Curation System from User Generated Media

Emerging Internet of Things (IoT) technologies will allow spectators in a sport game to produce various video streams from various angles. With existing technologies, however, it is difficult to process massive and various data streams for multi-channel contents in real-time. To solve this problem, we aim to construct a software agent (called “Curator”) that compiles video contents automatically according to his/her values. In this paper, we propose a system to automatically switch multiple video streams that general sports spectators have taken using Random Forests classifier. Meta data such as image feature data and game progress data is extracted for each video scene as the input of the classifier. For evaluation, we constructed a camera switching timing estimation model using the live TV broadcast of some baseball game data. A video of another baseball game was curated with the constructed model. As a result, our system predicted the camera switching timing with accuracy (F-measure) of 85.3% on weighted average for the base camera work and 99.7% for the fixed camera work.

Wide Area Object Tracking using Multiple Cameras with Mixed Configuration of Over-lapping and Non-Overlapping FOV

Seamless object tracking in wide area is an important and yet challenging issue of intelligent visual surveillance multi-camera collaboration tracking approach has been becoming attractive. In this work, a flexible and integrated system is constructed for object tracking using multiple cameras either with overlapping fields of views (FOVs), non-overlapping FOVs or mixed configurations for wide areas surveillance. The Kalman filter is applied for object tracking in a single camera and object motion prediction is implemented across blind regions for non-overlapping FOVs. Furthermore, a client and server architecture is constituted using TCP/IP network. The FOV lines are used to guide camera switching. In addition, we integrate various features for object matching. Finally, this system can track objects in a wide area no matter the FOVs are overlapping, non-overlapping or mixed situations.

Dynamic node selection in camera networks based on approximate reinforcement learning

In camera networks, dynamic node selection is an effective technique that enables video stream transmission with constrained network bandwidth, more economical node cooperation for nodes with constrained power supplies, and optimal use of a limited number of display terminals, particularly for applications that need to obtain high-quality video of specific targets. However, the nearest camera in a network cannot be identified by directional measurements alone. Furthermore, errors are introduced into computer vision algorithms by complex background, illumination, and other factors, causing unstable and jittery processing results. Consequently, in selecting camera network nodes, two issues must be addressed: First, a dynamic selection mechanism that can choose the most appropriate node is needed. Second, metrics to evaluate the visual information in a video stream must be modeled and adapted to various camera parameters, backgrounds, and scenes. This paper proposes a node selection method based on approximate reinforcement learning in which nodes are selected to obtain the maximum expected reward using approximate Q-learning. The Q-function is approximated by a Gaussian Mixture Model with parameters that are sequentially updated by a mini-batch stepwise Expectation---Maximization algorithm. To determine the most informative camera node dynamically, the immediate reward in Q-learning integrates the visibility, orientation, and image clarity of the object in view. Experimental results show that the proposed visual evaluation metrics can effectively capture the motion state of objects, and that the selection method reduces camera switching and related errors compared with state-of-the art methods.

Multi-camera interface with novel copositioning

ABSTRACTThis article reports a multi-camera system to determine the position of an object. By using novel copositioning and cameras in a wide range of orientations and viewing angles, the system solved the problem of dead zones and camera switching errors. When an object was close to a charge-coupled device camera, the system switched to that camera to become the focus by the algorithm. When an object left the optimum range of one camera, the system selected the next suitable camera to be the main detector. The position of the target and the grey level thresholds were continuously recorded and the changes were calculated. This interface system had the advantages of high positioning accuracy, fast response time, reduced noise in the light field, and no dead zones.

Bagadus

The importance of winning has increased the role of performance analysis in the sports industry, and this underscores how statistics and technology keep changing the way sports are played. Thus, this is a growing area of interest, both from a computer system view in managing the technical challenges and from a sport performance view in aiding the development of athletes. In this respect, Bagadus is a real-time prototype of a sports analytics application using soccer as a case study. Bagadus integrates a sensor system, a soccer analytics annotations system, and a video processing system using a video camera array. A prototype is currently installed at Alfheim Stadium in Norway, and in this article, we describe how the system can be used in real-time to playback events. The system supports both stitched panorama video and camera switching modes and creates video summaries based on queries to the sensor system. Moreover, we evaluate the system from a systems point of view, benchmarking different approaches, algorithms, and trade-offs, and show how the system runs in real time.

Construction of 3D Substation Monitoring System Based on Unity3D

With the development of the 3D technology, the construction of a 3D substation monitoring system becomes feasible. The Unity3D is a professional and comprehensive engine for providing advantages in fast deployment, architectural visualization, and 3D animation. It can support multiple platforms and make the development much easy with an expansive architecture involved. In the normal use, we build 3D model with tools such as the 3DMax, and import the models to Unity3D. By the means, we can finish making scenes and cartoons as well as integrating business data. We can also design multive functions such as the roaming in first person view, camera switching, real-time monitoring data displaying, real-time alarming, and so on. We write custom Shader to achieve some special effects. For example, we can write vertex and fragment shader to receive the data from the scripts and parse the data to get the information of temperature points. The distance between each UV point and the given temperature points is also computed and mapped to the values of temperature and color.

Focus on lasers and imaging

Focus on lasers and imaging

Effects of Camera Switching on Fine Accuracy in a Motion Capture System

When using optical motion capture systems, increasing the number of cameras improves the visibility. However, the software used to deal with the information fusion from multiple cameras may compromise the accuracy of the system due to camera dropout, which can vary with time. In cadaver studies of radial head motion, increasing the number of cameras used by the motion capture system seemed to decrease the accuracy of the measurements. This study investigates the cause. The hypothesis was that errors in position can be induced when markers are obscured from and then restored to a camera's viewable range, as can happen in biomechanical studies. Accuracy studies quantified the capabilities of the motion capture system with precision translation and rotation movements. To illustrate the effect that abrupt perceived changes in a marker's position can have on the calculation of radial head travel, simulated motion experiments were performed. In these studies, random noise was added to simulated data, which obscured the resultant path of motion. Finally, camera-blocking experiments were performed in which precise movements were measured with a six-camera Vicon system and the errors between the actual and perceived motion were computed. During measurement, cameras were selectively blocked and restored to view. The maximum errors in translation and rotation were 3.7 mm and 0.837 deg, respectively. Repeated measures analysis of variance (ANOVAs) (alpha=0.05) confirmed that the camera-blocking influenced the results. Taken together, these results indicate that camera-switching can affect the observation of fine movements using a motion analysis system with a large number of cameras. One solution is to offer opportunity for user interaction in the software to choose the cameras used for each instant of time.

On Sensor Switching Visual Servoing

This article investigates dynamic camera switching during visual servoing tasks using multi-camera systems considering image-based and position-based Jacobian transpose control schemes. Stability is discussed using a common Lyapunov function assuming ideal target and camera models and multiple Lyapunov functions under parameter perturbations. It is shown that asymptotic stability cannot be achieved for arbitrary switching under parameter perturbations. Therefore, an energy supervised switching scheme is introduced guaranteeing asymptotic stability. The contributions are stable visual servoing strategies which facilitate instantaneous adjustments of control performance and dynamical device switches in case of task requirements or sensor breakdown.

MPEG Splicing: A New Standard for Television — SMPTE 312M

Commercial insertion, promo insertion, studio routing, camera switching, tape editing—each of these basic operations involves switching video and audio. We want to continue to perform these operations even as we move from using uncompressed video to using MPEG-2 compressed video, but they are far from trivial. This paper describes the MPEG bitstream constraints required to facilitate switching—or “splicing”—MPEG bit-streams. It also describes the refinements to these constraints that have led to a new standard, SMPTE 312M, Television, Splice Points for MPEG-2 Transport Streams, which specifies bitstream constraints for both seamless and nonseamless splicing. It is ironic that, although splicing is likely the simplest method of switching between MPEG-2 bitstreams, it is probably the most difficult to understand. This paper intends to alleviate the mystery of splicing by explaining the underlying principles clearly so that this simple yet effective technique may be put into wide use throughout the emerging digital television industry.