Camera Control Research Articles

Automatically Generating Summary Visualizations from Game Logs

In this paper we describe a system called ViGLS (Visualization of Game Log Summaries) that generates summaries of gameplay sessions from game logs. ViGLS automatically produces visualization of the summarized actions that are extracted based on cognitive models of summarization. ViGLS is implemented using a service-oriented architecture, de-coupling the summarization methods from any particular game engine being used. The camera code libraries used in visualization are based on constraint based camera control approaches and, in our implementation, make use of the scripting layer of the Unreal game engine.

Urban dual mode video detection system based on fisheye and PTZ cameras

This work presents an artificial vision-based monitoring system for urban environments. It comprises a fisheye camera monitoring the scenes 180ox360o hemisphere and a Pan-Tilt-Zoom camera capturing narrower regions of interest in high-resolution. The ONVIF protocol standard is used to interface both IP-cameras, allowing for the integration of camera control (image acquisition and movement) and geometric calculations on a single device. The events of interest (motion of vehicles and pedestrians) are assumed to happen on the ground plane. This assumption is required to solve the back-projection, the function that maps coordinates in the highly distorted images of the fisheye camera to the ground plane. A calibration strategy estimates the poses of the cameras without placing restrictions on their orientations or relative distance. It optimizes the back-projection error in the ground plane instead of the re-projection error in the image. Finally, a simple pointing and zoom adjustment strategy controls the Pan-Tilt-Zoom camera. The system is tested in controlled laboratory conditions and shows accurate outdoor performance for pedestrian observation.

A situational understanding enhancer based on augmented visual prompts for teleoperation using a multi-monitor system

This study proposes an operator support system that enhances understandability of multiple monitor for teleoperated disaster response robots. The autonomous environmental camera control system we previously developed could provide operators with various visual information suited to the situation. To enable operators to adequately recognize environmental situations, multiple camera images are projected on multiple monitors, but this may increase the cognitive load of operators. The proposed situational understanding enhancer thus supports selective visual attention, situational awareness, and action decision, based on augmented visual prompts (AVPs). To support both work facilitation and danger avoidance, the AVPs include (i) monitor frame, (ii) obscuration, (iii) end-point vertical arrow, and (iv) distance arrow, with variable color and flashing effect. The system adaptively draws visual attention by changing the types and parameters of the AVPs according to the situations. Debris-removal tasks were conducted using a virtual reality simulator to compare the systems with and without AVP support. The results showed that the AVP support reduced the frequent changes in point of gaze, provided a sense of depth, and drew attention to potential collisions.

Performance of microvascular anastomosis with a new robotic visualization system: proof of concept

Microvascular procedures require visual magnification of the surgical field, e.g. by a microscope. This can be accompanied by an unergonomic posture with musculoskeletal pain or long-term degenerative changes as the eye is bound to the ocular throughout the whole procedure. The presented study describes the advantages and drawbacks of a 3D exoscope camera system. The RoboticScope®-system (BHS Technologies®, Innsbruck, Austria) features a high-resolution 3D-camera that is placed over the surgical field and a head-mounted-display (HMD) that the camera pictures are transferred to. A motion sensor in the HMD allows for hands-free change of the exoscope position via head movements. For general evaluation of the system functions coronary artery anastomoses of ex-vivo pig hearts were performed. Second, the system was evaluated for anastomosis of a radial-forearm-free-flap in a clinical setting/in vivo. The system positioning was possible entirely hands-free using head movements. Camera control was intuitive; visualization of the operation site was adequate and independent from head or body position. Besides technical instructions of the providing company, there was no special surgical training of the surgeons or involved staff upfront performing the procedures necessary. An ergonomic assessment questionnaire showed a favorable ergonomic position in comparison to surgery with a microscope. The outcome of the operated patient was good. There were no intra- or postoperative complications. The exoscope facilitates a change of head and body position without losing focus of the operation site and an ergonomic working position. Repeated applications have to clarify if the system benefits in clinical routine.

Directional Camera Control on High Altitude Balloons

The research reported in this paper examined the design and control of a gimbal for solar eclipse tracking and video recording. The gimbal design required 3 axes of rotation to allow for a full range of motion. Utilizing individual brushless motors for each of the axes ensures minimum rotational requirements on each axes. In controlling the gimbal, both a mathematical and visual method was utilized. The mathematical method is a modeled version of what is currently used for solar array pointing. The visual method looks at where the position of the sun is within the image and determines what angle changes are required. Utilizing a combination of these methods helps to eliminate error that accumulates within the onboard gyros due to the erratic behavior of balloon motion during flight. Elimination of this error ensures accurate video recording of the solar eclipse.

History and Future of Connecting Broadcast Television Cameras: From Multicore to Native IP

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Since the early days of broadcast television, there has been an ongoing evolution of the signal transmission between the camera heads and the camera base stations or camera control units (CCUs). In the 1970s, multicore cables were replaced by triax cables and more recently, with the introduction of ultrahigh-definition (UHD-1)/4K standards, SMPTE hybrid fiber cables became the de facto standard for most applications. At the same time, the signals transmitted over camera cables migrated from analog to digital, and now the most advanced broadcast cameras use Internet Protocol (IP). This paper explains the challenges and the potential solutions for a direct integration of a system camera for live broadcast applications into a full IP infrastructure. This next step in the evolution of camera signal transmission and connectivity will establish new workflows and offers more flexibility for cameras</i> .

An Autonomous System for Efficient Control of PTZ Cameras

This article addresses the research problem of how to autonomously control Pan/Tilt/Zoom (PTZ) cameras in a manner that seeks to optimize the face recognition accuracy or the overall threat detection and proposes an overall system. The article presents two alternative schemes for camera scheduling: Grid-Based Grouping (GBG) and Elevator-Based Planning (EBP). The camera control works with realistic 3D environments and considers many factors, including the direction of the subject’s movement and its location, distances from the cameras, occlusion, overall recognition probability so far, and the expected time to leave the site, as well as the movements of cameras and their capabilities and limitations. In addition, the article utilizes clustering to group subjects, thereby enabling the system to focus on the areas that are more densely populated. Moreover, it proposes a dynamic mechanism for controlling the pre-recording time spent on running the solution. Furthermore, it develops a parallel algorithm, allowing the most time-consuming phases to be parallelized, and thus run efficiently by the centralized parallel processing subsystem. We analyze through simulation the effectiveness of the overall solution, including the clustering approach, scheduling alternatives, dynamic mechanism, and parallel implementation in terms of overall recognition probability and the running time of the solution, considering the impacts of numerous parameters.

Parcel providing and Patrolling Hexacopter with High Payload

The different varieties of drones may be differentiated in terms of the sort (fixed-wing, multirotor, etc.), the degree of autonomy, the dimensions and weight, and also the power supply. These specifications are dynamic, as an example for the drone’s cruising vary, the utmost flight length, and also the loading capability. Apart from the drone itself (i.e., the ‘platform’) varied varieties of payloads may be distinguished, together with freight (e.g., mail parcels, medicines, hearth ending material, flyers, etc.) and differing kinds of sensors (e.g., cameras, sniffers, meteorological sensors, etc.). Applications of various payloads are described. So as to perform a flight, drones have a necessity for (a specific amount of) wireless communication with a pilot on the bottom. During this paper we tend to are introducing drone consists of six blades referred to as hexocoptor that's upraised and propelled by six motor will which may which might carry a load up to ten weight unit or can fly up to thirty minute with payload. It contain twilight vision camera and GPS controller is employed for remote location in night. During this system, brushless DC motors are adjoined with the electronic speed controller.

In Vitro User Assessment of a Novel Wound Protection and Specimen Extraction System for Laparoscopic Procedures Involving Morcellation

Introduction: The risk of unknowingly disseminating leiomyosarcoma by morcellation in women undergoing laparoscopic hysterectomy has massively impacted gynaecological practice. Here, we present the results of an in vitro assessment of a novel protection system developed to mitigate this hazard. Methods: The Tissue Containment System for Manual Morcellation (Guardenia™, Advanced Surgical Concepts, Wicklow, Ireland) is an evolved wound protection/specimen extraction guarded bag system compatible with any 12mm trocar. Device use was assessed by device-naïve gynaecological and general surgeon volunteers (providing expert and inexpert morcellation cohorts, respectively) on a bench model consisting of biological tissue in a custom-built moulded rig with camera control after the operators were instructed in its use. Results: Twenty surgeons (10 gynaecologists/10 general surgeons, median duration of practice experience: 8 years, median annual number of laparoscopic operative procedures: 150 and 80, respectively) completed the user assessment. All subjects understood and correctly performed each step; i.e., (i) placement of the bag through the trocar, (ii) specimen bagging, (iii) incision extension (range 25-60 mm) after tethering the bag through the port, (iv) insertion of the device guard through the mouth of the bag after trocar removal, and (v) sufficient tissue morcellation within the bag to enable complete specimen removal (mean specimen weight 390g, range 201-1800g). There was 100% bag integrity by water-leak testing following use, despite scalpel contact with the guard in 14/20 cases (70%). Conclusion: Among first-time clinical users, this novel device enabled complete containment of morcellation debris and removal of a laparoscopic specimen, which would support further submission for regulatory approval.

ОБЪЕКТНО-ОРИЕНТИРОВАННЫЙ ПОДХОД К ОПИСАНИЮ ВЗАИМОДЕЙСТВИЯ ГРУППЫ РОБОТОТЕХНИЧЕСКИХ СРЕДСТВ НА ОСНОВЕ РАСПРЕДЕЛЕННОЙ ВИРТУАЛЬНОЙ МАШИНЫ

The robotic systems building approaches is analyzed in this paper. Robotics systems can beconsidered as a distributed system which interacted between individual components of the roboticsystem and interacted between robotic systems within a single complex. In first case a roboticsystem can be a collection of individual modules within a single robot. For example, individualmotors, servo drives for camera control, and the camera itself control of the unmanned aerialvehicle (UAV) can be considered as separate modules of the entire UAV within a distributed roboticsystem. In second case robotic system devoted to a massive of connected robotic system. Forexample, each UAV is a robotic system which is a part of whole robotic system that define UAVgroup interaction. An approach that allows a unified way to describe such robotic systems hierarchyis needed. In the field of robotics systems, there are many approaches to building it, which definesthe means of communication and data transfer. This paper describes the existing approaches,their advantages and disadvantages, and suggests another approach for creating distributed roboticsystems. Each nodes network connection in the existing approaches is provided by transferring datawith their subsequent processing The article describes an approach based on the encapsulation ofexecutable code in transmitted network packets. The object interaction is carried out through thecontrol data transfer that interpreted by a distributed virtual machine. The object-oriented programming(OOP) paradigm extension by the concept of a complementary object, allows to create a distributedsystem that abstracted from the network programming difficulty. An object-oriented approachbased on complementary object usage allows to develop a distributed system as a single program,concentrating on the implementation of logic. So we passing from the distributed system as animplementation of separate modules concept to the single distributed program concept without a“syntax breaking”.The proposed approach allows to represent a distributed robotic system in theOOP paradigm as a set of objects interacting over a communication network.

Efficiency in image-guided robotic and conventional camera steering: a prospective randomized controlled trial

Robotic camera steering systems have been developed to facilitate endoscopic surgery. In this study, a randomized controlled trial was conducted to compare conventional human camera control with the AutoLap™ robotic camera holder in terms of efficiency and user experience when performing routine laparoscopic procedures. Novelty of this system relates to the steering method, which is image based. Patients undergoing an elective laparoscopic hemicolectomy, sigmoid resection, fundoplication and cholecystectomy between September 2016 and January 2018 were included. Stratified block randomization was used for group allocation. The primary aim of this study was to compare the efficiency of robotic and human camera control, measured with surgical team size and total operating time. Secondary outcome parameters were number of cleaning moments of the laparoscope and the post-study system usability questionnaire. A total of 100 patients were randomized to have robotic (50) versus human (50) camera control. Baseline characteristics did not differ significantly between groups. In the robotic group, 49/50 (98%) of procedures were carried out without human camera control, reducing the surgical team size from four to three individuals. The median total operative time (60.0 versus 53.0min, robotic vs. control) was not significantly different, p = 0.122. The questionnaire showed a positive user satisfaction and easy control of the robotic camera holder. Image-based robotic camera control can reduce surgical team size and does not result in significant difference in operative time compared to human camera control. Moreover, robotic image-guided camera control was associated with positive user experience.

Anomalous object detection by active search with PTZ cameras

Due to the large amount of visual information generated daily, proposals that automatically analyze and process data are becoming increasingly necessary. This work focuses on the detection of anomalous objects in video sequences captured by PTZ (pan-tilt-zoom) cameras, considering as anomalies the objects which belong to categories that should not appear in a specific scene (e.g. pedestrians on a highway). There is a lack in the previous literature of a principled approach for the control of PTZ cameras that takes advantage of the recent developments in deep learning-based object detection. Our proposal aims to fill this gap by offering a probabilistic framework where the guidance of PTZ cameras is accommodated. The proposed methodology involves three different modules. An object detection stage, where deep learning networks are used to detect the objects that appear in the scene; an anomalous detection module, where a mixture of Dirichlet distributions is considered to detect automatically, and without supervised training, those detected objects which are likely to be anomalous; and finally, a PTZ camera controller which allows to follow and focus on the object considered as the most probably anomalous in the scene. The experimental results show the performance and viability of our proposal, which outperforms several competitors from qualitative and quantitative points of view.

Visual Multi-Face Tracking Applied to Council Proceedings

Face recognition, face measurement, camera control, measurement of expressions and other tasks can benefit from online visual multi-face tracking. Given the availability of high quality general purpose detectors and tracking-by-detection frameworks, we provide guidance on how to develop a multi-face tracker out of standard components. In this paper, we train common object detectors specifically on faces to understand how well these detectors perform and evaluate different classifier loss functions. Our specific case study tracks faces in the context of council meetings and in parliamentary settings such as the Canadian House of Commons for which we create an annotated video set as a benchmark (see Fig. 1). These meetings in a parliamentary setting are often recorded from multiple cameras with participants and audiences walking around. Fast camera switching and zooming lead to significant scale changes of faces. Therefore, these settings can be characterized as tracking in unconstrained video. This will negatively impact the tracking accuracy and increase the likelihood of identity switches (IDS) between face labels. However, being able to track in unconstrained video enables a wider range of measurement applications. We find that while online tracking based on combining state-of-the-art methods can lead to high-quality tracking results, there is still a large gap between offline and online methods. The discussed method can be adapted to other tracking tasks for which large image databases are available.

A learning robot for cognitive camera control in minimally invasive surgery

BackgroundWe demonstrate the first self-learning, context-sensitive, autonomous camera-guiding robot applicable to minimally invasive surgery. The majority of surgical robots nowadays are telemanipulators without autonomous capabilities. Autonomous systems have been developed for laparoscopic camera guidance, however following simple rules and not adapting their behavior to specific tasks, procedures, or surgeons.MethodsThe herein presented methodology allows different robot kinematics to perceive their environment, interpret it according to a knowledge base and perform context-aware actions. For training, twenty operations were conducted with human camera guidance by a single surgeon. Subsequently, we experimentally evaluated the cognitive robotic camera control. A VIKY EP system and a KUKA LWR 4 robot were trained on data from manual camera guidance after completion of the surgeon’s learning curve. Second, only data from VIKY EP were used to train the LWR and finally data from training with the LWR were used to re-train the LWR.ResultsThe duration of each operation decreased with the robot’s increasing experience from 1704 s ± 244 s to 1406 s ± 112 s, and 1197 s. Camera guidance quality (good/neutral/poor) improved from 38.6/53.4/7.9 to 49.4/46.3/4.1% and 56.2/41.0/2.8%.ConclusionsThe cognitive camera robot improved its performance with experience, laying the foundation for a new generation of cognitive surgical robots that adapt to a surgeon’s needs.

Robotic colorectal surgery and ergonomics.

Improved ergonomics for the operating surgeon may be an advantage of robotic colorectal surgery. Perceived robotic ergonomic advantages in visualisation include better exposure, three-dimensional vision, surgeon camera control, and line of sight screen location. Postural advantages include seated position and freedom from the constraints of the sterile operating field. Manipulation benefits include articulated instruments with seven degrees of freedom movement, elimination of fulcrum effect, tremor filtration, and scaling of movement. Potential ergonomic detriments of robotic surgery include lack of haptic feedback, visual, and mental strain from increased operating time and interruptions to workflow from crowding.

Mediated learning materials: visibility checks in telepresence robot mediated classroom interaction

ABSTRACT Videoconferencing is increasingly used in education as a way to support distance learning. This article contributes to the emerging interactional literature on video-mediated educational interaction by exploring how a telepresence robot is used to facilitate remote participation in university-level foreign language teaching. A telepresence robot differs from commonly used videoconferencing set-ups in that it allows mobility and remote camera control. A remote student can thus move a classroom-based robot from a distance in order to shift attention between people, objects and environmental structures during classroom activities. Using multimodal conversation analysis, we focus on how participants manage telepresent remote students’ visual access to classroom learning materials. In particular, we show how visibility checks are accomplished as a sequential and embodied practice in interaction between physically dispersed participants. Moreover, we demonstrate how participants conduct interactional work to make learning materials visible to the remote student by showing them and guiding the ‘seeing’ of materials. The findings portray some ways in which participants in video-mediated interaction display sensitivity to the possibility of intersubjective trouble and the recipient’s visual perspective. Besides increasing understanding of visual and interactional practices in technology-rich learning environments, the findings can be applied in the pedagogical design of such environments.

41 Enhancing Burn Medical Care During a Disaster Using a Novel Augmented-Reality Application

Abstract Introduction In disaster or mass casualty situations, access to remote burn care experts, communication, or resources may be limited. Furthermore, burn injuries are complex and require substantial training and knowledge beyond basic clinical care. Development and use of decision support (DS) technologies may provide a solution for addressing this need. Devices capable of delivering burn management recommendations can enhance the provider’s ability to make decisions and perform interventions in complex care settings. When coupled with merging augmented reality (AR) technologies these tools may provide additional capabilities to enhance medical decision-making, visualization, and workflow when managing burns. For this project, we developed a novel AR-based application with enhanced integrated clinical practice guidelines (CPGs) to manage large burn injuries for use in different environments, such as disasters. Methods We identified an AR system that met our requirements to include portability, infrared camera, gesture and voice control, hands-free control, head-mounted display, and customized application development abilities. Our goal was to adapt burn CPGs to make use of AR concepts as part of an AR-enabled burn clinical decision support system supporting four sub-applications to assist users with specific interventional tasks relevant to burn care. We integrated relevant CPGs and a media library with photos and videos as additional references. Results We successfully developed a clinical decision support tool that integrates burn CPGs with enhanced capabilities utilizing AR technology. The main interface allows input of patient demographics and injuries with step-by-step guidelines that follow typical burn management care and workflow. There are four sub-applications to assist with these tasks, which include: 1) semi-automated burn wound mapping to calculate total body surface area; 2) hourly burn fluid titration and recommendations for resuscitation; 3) medication calculator for accurate dosing in preparation for procedures and 4) escharotomy instructor with holographic overlays. Conclusions We developed a novel AR-based clinical decision support tool for management of burn injuries. Development included adaptation of CPGs into a format to guide the user through burn management using AR concepts. The application will be tested in a prospective research study to determine the effectiveness, timeliness, and performance of subjects using this AR-software compared to standard of care. We fully expect that the tool will reduce cognitive workload and errors, ensuring safety and proper adherence to guidelines.

Running scared: Fear and space in Amnesia: The Dark Descent

Popular horror video game titles such as Outlast, Dead Space, and Amnesia: The Dark Descent are well-known for their effectiveness at evoking negative affects of terror and anxiety. The various camera tricks, control schemes, and visual cues these games deploy to confuse players and limit their sense of control and personal mastery. This article examines how Frictional Games’s Amnesia: The Dark Descent pairs confined spatial layouts with an intentionally vague user interface design to disorient players and heighten the likelihood that they will walk into one of the game’s threatening monsters. This article deploys Marxist and Affect theory conceptualizations of proximity and space to analyse how the game’s use of corners frighten players by narrowing their available field of view. The resulting analysis examines the negative feelings and subjective experiences players are likely to feel when they are unable to properly see the virtual diegetic world with absolute clarity.

$$\text{C}^{3}\text{Net}$$: end-to-end deep learning for efficient real-time visual active camera control

The need for automated real-time visual systems in applications such as smart camera surveillance, smart environments, and drones necessitates the improvement of methods for visual active monitoring and control. Traditionally, the active monitoring task has been handled through a pipeline of modules such as detection, filtering, and control. However, such methods are difficult to jointly optimize and tune their various parameters for real-time processing in resource constraint systems. In this paper a deep Convolutional Camera Controller Neural Network is proposed to go directly from visual information to camera movement to provide an efficient solution to the active vision problem. It is trained end-to-end without bounding box annotations to control a camera and follow multiple targets from raw pixel values. Evaluation through both a simulation framework and real experimental setup, indicate that the proposed solution is robust to varying conditions and able to achieve better monitoring performance than traditional approaches both in terms of number of targets monitored as well as in effective monitoring time. The advantage of the proposed approach is that it is computationally less demanding and can run at over 10 FPS (~4x speedup) on an embedded smart camera providing a practical and affordable solution to real-time active monitoring.

Robot-assisted surgery with Senhance robotic system for colon cancer: our original single-incision plus 2-port procedure and a review of the literature.

The Senhance robotic system provides such advantages as an eye-tracking camera control system, haptic feedback, operator comfort, and reusable endoscopic instruments. The aim of this small study was to assess the feasibility and safety of performing a reduced-port robot-assisted colectomy for colon cancer with the use of a novel robotic system. This was a single-center retrospective study of eight patients with colon cancer who underwent single-incision plus 2-port robot-assisted colectomy with the Senhance robotic system (SILS+2-S) between December 2019 and March 2020 at our hospital. Data on perioperative outcomes, which included operative time, operative blood loss, length of hospitalization, postoperative complications, and histopathological results, were collected prospectively. The mean patient age was 70.9years and the mean body mass index was 24.4kg/m2. One patient was converted to laparoscopy due to a damaged scope holder. The mean operative and console times were 229.1 and 139.1min, respectively. The mean intraoperative blood loss was 49.4ml. The mean length of the umbilical incision was 3.0cm. The mean number of harvested lymph nodes was 18.3. The surgical margins were negative in all eight patients. There was neither morbidity nor mortality associated with the procedure, and no Clavien-Dindo classification Grade II-IV complications occurred. SILS+2-S is a safe and feasible approach for patients with colon cancer. Further studies are needed to validate the advantages of SILS+2-S and to evaluate the long-term oncological outcomes.