Camera Feedback Research Articles

이동용 영상촬영기기 바디캠을 활용한 팀단위 심폐소생술의 교육피드백 효과

This study conducted a team-based CPR simulation with 32 fourth-year emergency rescue students to determine the effectiveness of training feedback using body cameras used at emergency rescue sites, and measured awareness, training feedback effectiveness, and satisfactio+n before and after body camera feedback. , preferences and difficulties in using body camera devices were identified. Data analysis was performed using SPSS 27.0 program, including descriptive statistics, frequency analysis, paried t-test, and Wilcoxon signed rank test. As a result of the study, the perception of body camera use showed a positive change from 3.73±0.62 points to 4.45±0.54 points, and a positive satisfaction level of 3.98±0.51 was shown (p<.001). Additionally, there was a significant increase in self-check accuracy and performance score after body camera feedback (p<.001). Therefore, during team-based simulation resuscitation training, positive feedback effects in improving self-inspection ability and performance can be achieved by watching body camera videos and using self-checklists without direct feedback from the instructor.

Full-colour 3D holographic augmented-reality displays with metasurface waveguides

Emerging spatial computing systems seamlessly superimpose digital information on the physical environment observed by a user, enabling transformative experiences across various domains, such as entertainment, education, communication and training1–3. However, the widespread adoption of augmented-reality (AR) displays has been limited due to the bulky projection optics of their light engines and their inability to accurately portray three-dimensional (3D) depth cues for virtual content, among other factors4,5. Here we introduce a holographic AR system that overcomes these challenges using a unique combination of inverse-designed full-colour metasurface gratings, a compact dispersion-compensating waveguide geometry and artificial-intelligence-driven holography algorithms. These elements are co-designed to eliminate the need for bulky collimation optics between the spatial light modulator and the waveguide and to present vibrant, full-colour, 3D AR content in a compact device form factor. To deliver unprecedented visual quality with our prototype, we develop an innovative image formation model that combines a physically accurate waveguide model with learned components that are automatically calibrated using camera feedback. Our unique co-design of a nanophotonic metasurface waveguide and artificial-intelligence-driven holographic algorithms represents a significant advancement in creating visually compelling 3D AR experiences in a compact wearable device.

Formation Tracking of Mobile Robots Under Obstacles Using Only an Active RGB-D Camera

This paper attempts to achieve leader-follower formation tracking for mobile robots under obstacles with only one RGB-D camera on the follower. In contrast, most existing research employs at least two sensors, including a camera and a LiDAR. However, the limited field-of-view (FOV) of RGB-D cameras introduces three significant challenges: intermittent observation of the leader, partial detection of obstacles, and conflicting observations of the leader and obstacles. To overcome these difficulties, we first propose a controller that enables the follower to achieve continuous observation and formation tracking of the leader without obstacles and depth images. To address the partial detection of obstacles, we design a rotating device that allows the camera to actively extend its observation range and resolve conflicts with leader detection. Based on camera feedback and the proposed controller, a multi-objective controller is presented that enables the follower to achieve formation tracking and obstacle avoidance simultaneously, without requiring communication or leader velocity. Simulations and experiments are conducted to demonstrate the feasibility of our methods.

Perception and Action Augmentation for Teleoperation Assistance in Freeform Telemanipulation

Teleoperation enables controlling complex robot systems remotely, providing the ability to impart human expertise from a distance. However, these interfaces can be complicated to use as it is difficult to contextualize information about robot motion in the workspace from the limited camera feedback. Thus, it is required to study the best manner in which assistance can be provided to the operator that reduces interface complexity and effort required for teleoperation. Some techniques that provide assistance to the operator while freeform teleoperating include: (1) perception augmentation, like augmented reality visual cues and additional camera angles, increasing the information available to the operator; (2) action augmentation, like assistive autonomy and control augmentation, optimized to reduce the effort required by the operator while teleoperating. In this article, we investigate: (1) which aspects of dexterous telemanipulation require assistance; (2) the impact of perception and action augmentation in improving teleoperation performance; and (3) what factors impact the usage of assistance and how to tailor these interfaces based on the operators’ needs and characteristics. The findings from this user study and resulting post-study surveys will help identify task-based and user-preferred perception and augmentation features for teleoperation assistance.

Visual servo control of endoscope-holding robot based on multi-objective optimization: System modeling and instrument tracking

In routine laparoscopic surgeries, the fatigue operation and improper chief-assistant coordination of the assistant surgeon both reduce the stability of the endoscopic field of view (EFOV) and the surgical efficiency. In contrast, the assistant endoscope-holding robot (AEHR) can stably hold the endoscope and adjust the pose of endoscopic image sensor. However, the traditional method simplifies the tracking of surgical instrument tips (SITs) to the tracking of feature points, and lacks the depth optimization of monocular endoscopic image senor. Based on them, this paper proposes a surgical instrument tracking control method based on visual tracking features (VTFs) and hand-eye coordination (HEC) with feedback of multiple sensors, which can control the depth and axial rotational degrees of freedom (DOFs) of endoscope while tracking SITs. Firstly, the motion mapping Jacobian matrix between the endoscopic image sensor and the endoscope-holding manipulator (EHM) end-effector is constructed with joint encoder feedbacks, and the VTFs of endoscopic camera feedback are extracted based on the deep segmentation model of SITs. Secondly, the position adjustment model based on VTFs and the attitude adjustment model based on HEC are established. Thirdly, a complete set of visual servo control (VSC) software and hardware system for AEHRs is designed. Finally, the correctness, effectiveness and operability of the proposed method are comprehensively verified through the simulation and experiment of surgical instrument tracking. The visual tracking distance and radius error of endoscopic image sensor frames are reduced by no less than 34.6% and 42.59% under different experimental backgrounds, respectively.

Accurate holographic light potentials using pixel crosstalk modelling

Arbitrary light potentials have proven to be a valuable and versatile tool in many quantum information and quantum simulation experiments with ultracold atoms. Using a phase-modulating spatial light modulator (SLM), we generate arbitrary light potentials holographically with measured efficiencies between 15 and 40% and an accuracy of <2% root-mean-squared error. Key to the high accuracy is the modelling of pixel crosstalk of the SLM on a sub-pixel scale which is relevant especially for large light potentials. We employ conjugate gradient minimisation to calculate the SLM phase pattern for a given target light potential after measuring the intensity and wavefront at the SLM. Further, we use camera feedback to reduce experimental errors, we remove optical vortices and investigate the difference between the angular spectrum method and the Fourier transform to simulate the propagation of light. Using a combination of all these techniques, we achieved more accurate and efficient light potentials compared to previous studies, and generated a series of potentials relevant for cold atom experiments.

Evaluation of a liquid membrane-based tunable lens and a solid-state LIDAR camera feedback system for presbyopia.

Presbyopia is an age-related loss of accommodation ability of the eye which affects individuals in their late 40s or early 50s. Presbyopia reduces the ability of a person to focus on closer objects at will. In this study, we assessed electronically tunable lenses for their aberration properties as well as for their use as correction lenses. The tunable lenses were evaluated in healthy subjects with cycloplegia by measuring visual acuity and contrast sensitivity for their use in presbyopia correction. Furthermore, we have developed and demonstrated the feasibility of a feedback mechanism for the operation of tunable lenses using a portable solid-state LIDAR camera with a processing time of 40 ± 5 ms.

CameraRoach: A WiFi- and Camera-Enabled Cyborg Cockroach for Search and Rescue

We describe here our design and implementation of a cyborg insect, called CameraRoach, with onboard camera feedback that can be navigated via remote control providing a first-person view. The camera pack is mounted on the Madagascar hissing cockroach, which is small enough to fit into crevices but also can carry a printed circuit boards with power, communication, and sensor components (visual camera). For navigating the cockroach, we implemented a unique electronic backpack neural stimulator, which allows the cockroach to be maneuvered on a desired path with a joystick. A high-resolution wireless camera, also included in the backpack, sends live images via a WiFi (Wireless Fidelity) network. We present the results of an evaluation experiment with the CameraRoach and compare it with the other state of the art systems like the Beetle-Cam.

Heat-Mitigated Design and Lorentz Force-Based Steering of an MRI-Driven Microcatheter toward Minimally Invasive Surgery.

Catheters integrated with microcoils for electromagnetic steering under the high, uniform magnetic field within magnetic resonance (MR) scanners (3–7 Tesla) have enabled an alternative approach for active catheter operations. Achieving larger ranges of tip motion for Lorentz force‐based steering have previously been dependent on using high power coupled with active cooling, bulkier catheter designs, or introducing additional microcoil sets along the catheter. This work proposes an alternative approach using a heat‐mitigated design and actuation strategy for a magnetic resonance imaging (MRI)‐driven microcatheter. A quad‐configuration microcoil (QCM) design is introduced, allowing miniaturization of existing MRI‐driven, Lorentz force‐based catheters down to 1‐mm diameters with minimal power consumption (0.44 W). Heating concerns are experimentally validated using noninvasive MRI thermometry. The Cosserat model is implemented within an MR scanner and results demonstrate a desired tip range up to 110° with 4° error. The QCM is used to validate the proposed model and power‐optimized steering algorithm using an MRI‐compatible neurovascular phantom and ex vivo kidney tissue. The power‐optimized tip orientation controller conserves as much as 25% power regardless of the catheter's initial orientation. These results demonstrate the implementation of an MRI‐driven, electromagnetic catheter steering platform for minimally invasive surgical applications without the need for camera feedback or manual advancement via guidewires. The incorporation of such system in clinics using the proposed design and actuation strategy can further improve the safety and reliability of future MRI‐driven active catheter operations.

Mixed Reality Human–Robot Interface With Adaptive Communications Congestion Control for the Teleoperation of Mobile Redundant Manipulators in Hazardous Environments

Robotic interventions with redundant mobile manipulators pose a challenge for telerobotics in hazardous environments, such as underwater, underground, nuclear facilities, particle accelerators, aerial or space. Communication issues can lead to critical consequences, such as imprecise manipulation resulting in collisions, breakdowns and mission failures. The research presented in this paper was driven by the needs of a real robotic intervention scenario in the Large Hadron Collider (LHC) at the European Organization for Nuclear Research (CERN). The goal of the work was to develop a framework for network optimisation in order to help facilitate Mixed Reality techniques such as 3D collision detection and avoidance, trajectories planning, real-time control, and automatized target approach. The teleoperator was provided with immersive interactions while preserving precise positioning of the robot. These techniques had to be adapted to delays, bandwidth limitation and their volatility in the 4G shared network of the real underground particle accelerator environment. The novel application-layer congestion control with automatic settings was applied for video and point cloud feedback. Twelve automatic setting modes were proposed with algorithms based on the camera frame rate, resolution, point cloud subsampling, network round-trip time and throughput to bandwidth ratio. Each mode was thoroughly characterized to present its specific use-case scenarios and the improvements it brings to the adaptive camera feedback control in teleoperation. Finally, the framework was presented according to which designers can optimize their Human-Robot Interfaces and sensor feedback depending on the network characteristics and task.

UVC-PURGE: A Novel Cost-Effective Disinfection Robot for Combating COVID-19 Pandemic.

During the COVID-19 pandemic, surface disinfection using prevailing chemical disinfection methods had several limitations. Due to cost-inefficiency and the inability to disinfect shaded places, static UVC lamps cannot address these limitations properly. Moreover, the average market price of the prevailing UVC robots is huge, approximately 55,165 USD. In this research firstly, a requirement elicitation study was conducted using a semi-structured interview approach to reveal the requirements to develop a cost-effective UVC robot. Secondly, a semi-autonomous robot named UVC-PURGE was developed based on the revealed requirements. Thirdly, a two-phased evaluation study was undertaken to validate the effectiveness of UVC-PURGE to inactivate the SARS-CoV-2 virus and the capability of semi-autonomous navigation in the first phase and to evaluate the usability of the system through a hybrid approach of SUPR-Q forms and subjective evaluation of the user feedback in the second phase. Pre-treatment swab testing revealed the presence of both Gram-positive and Gram-Negative bacteria at 17 out of 20 test surfaces in the conducted tests. After the UVC irradiation of the robot, the microbial load was detected in only 2 (1D and 1H) out of 17 test surfaces with significant reductions (95.33% in 1D and 90.9% in 1H) of microbial load. Moreover, the usability evaluation yields an above-average SUPR-Q score of 81.91% with significant scores in all the criteria (usability, trust, loyalty, and appearance) and the number of positive themes from the subjective evaluation using thematic analysis is twice the number of negative themes. Additionally, compared with the prevailing UVC disinfection robots in the market, UVC-PURGE is cost-effective with a price of less than 800 USD. Moreover, small form factor along with the real time camera feedback in the developed system helps the user to navigate in congested places easily. The developed robot can be used in any indoor environment in this prevailing pandemic situation and it can also provide cost-effective disinfection in medical facilities against the long-term residual effect of COVID-19 in the post-pandemic era.

Neural 3D holography

Holographic near-eye displays promise unprecedented capabilities for virtual and augmented reality (VR/AR) systems. The image quality achieved by current holographic displays, however, is limited by the wave propagation models used to simulate the physical optics. We propose a neural network-parameterized plane-to-multiplane wave propagation model that closes the gap between physics and simulation. Our model is automatically trained using camera feedback and it outperforms related techniques in 2D plane-to-plane settings by a large margin. Moreover, it is the first network-parameterized model to naturally extend to 3D settings, enabling high-quality 3D computer-generated holography using a novel phase regularization strategy of the complex-valued wave field. The efficacy of our approach is demonstrated through extensive experimental evaluation with both VR and optical see-through AR display prototypes.

Effect of camera monitoring and feedback along with training on hospital infection rate in a neonatal intensive care unit

BackgroundIn terms of pediatric healthcare-associated infections (HAI), neonatal intensive care units (NICU) constitute the greatest risk. Contacting a health care personnel, either directly or indirectly, elevates NICU occurrence rate and risks other infants in the same unit. In this study, it is aimed to retrospectively analyze the effect of the training along with camera monitoring and feedback (CMAF) to control the infection following a small outbreak.MethodsESBL producing Klebsiella pneumoniae was detected on three infants in May 2014 at the isolation room of Sakarya University Hospital NICU. Precautions were taken to prevent further spread of the infection. The infected infants were isolated and the decolonization process was initiated. For this aspect, health care workers (HCWs) in NICU were trained for infection control measures. An infection control committee has monitored the HCWs. Before monitoring, an approval was obtained from the hospital management and HCWs were informed about the CMAF, who were then periodically updated. On a weekly basis, NICU workers were provided with the feedbacks. Epidemic period and post-epidemic control period (June–July–August 2014) were evaluated and p value < 0.05 was considered statistically significant.ResultsHealthcare-associated infection (HAI) density was 9.59% before the onset of the CMAF, whereas it was detected as 2.24% during the CMAF period (p < 0.05). Following the precautions, HAI and HAI density rates have reduced to 76.6% and 74.85%, respectively. Moreover, hand hygiene compliance of health care workers was found 49.0% before the outbreak, whereas this rate has elevated to 62.7% after CMAF.ConclusionsHealthcare workers should be monitored in order to increase their compliance for infection control measures. Here, we emphasized that that CMAF of health workers may contribute reducing the HAI rate in the NICU.

Iterative calibration of a multiview 3D display with linear extrinsic crosstalk using camera feedback.

In this paper, we propose a multiview 3D display calibration algorithm that compensates for misalignment between constituent layers. To minimize distortion of the 3D image due to misalignment, we employ an iterative algorithm using camera feedback. We first establish a mathematical model for the multiview display and predict distortion of a test image pattern due to misalignment. Then, we define two characteristic features of the feedback camera image that indicate the status of the alignment. Based on the mathematical model, we design a compensation algorithm that consists of cascaded one-dimensional root-finding blocks for two-dimensional optimization. Finally, we experimentally verify the image distortion model, convergence of the iterative algorithm, and image quality improvement. The accuracy of the proposed algorithm exceeds the required accuracy for linear extrinsic crosstalk compensation.

Fully automatic visual servoing control for work-class marine intervention ROVs

ROVs with hydraulic manipulators are extensively used for subsea intervention. With camera feedback from the scene, manipulators are teleoperated and slaved to pilot held master arms. While standard for offshore oil and gas, for challenging applications in waves or currents a new approach is required. We present development of robot arm visual servo control approaches used in manufacturing and the transfer and adaption of these to underwater hydraulic manipulators. This is the first time a visual servoing algorithm for automated manipulation has been developed and verified, through subsea trials, on a commercial work-class ROV with industry standard hydraulic manipulators.

Optimal conceptual design and vision-based control of a fruit harvesting robot

The main contribution of this paper is to develop a vision-based control of a robotic arm for the harvesting fruits. The camera fixed in the gripper pad enables to precisely locate the fruit and pluck it from the branch. Rigorous stability analysis is done to ensure the guaranteed performance of the closed loop system. The camera feedback locates the exact position of the fruit; this enables the controller to track a suitable and optimal path to reach the target by performing desirable transformations. The manipulator with five-DOF (RRPRR) is designed and optimised for the formulating simple control strategies. The finger-like built gripper is electrically actuated to provide the necessary force required in harvesting the fruit. Also an additional bellow kind of structure is specially designed and located below the gripper which helps to roll down the harvested fruit on to the storage container without damaging it. Numerical simulation analysis was carried out along with the design realisation to justify the context. The advancement in the field of agrionics has also been a source of inspiration in designing agricultural robots.

Sistema de Visión Subacuático Inalámbrico Usando un Algoritmo de Compresión Progresivo con Región de Interés

&lt;p&gt;La creciente demanda en todo el mundo de sistemas de intervención submarina en diversos dominios de aplicación requiere sistemas más versátiles y económicos. Empleando un sistema de comunicación inalámbrica, los robots semiautónomos supervisados disponen de libertad de movimientos y, al mismo tiempo, permiten al operador obtener información visual y supervisar la intervención. Por otro lado, la velocidad de transmisión de datos típica de los canales inalámbricos submarinos es, en general, muy limitada, siendo necesaria la aplicación de técnicas de compresión avanzadas. En este artículo se presenta principalmente el algoritmo DEBT (Depth Embedded Block Tree) para la compresión progresiva de imágenes con región de interés (ROI). Los resultados demuestran ventajas con respecto a otros algoritmos de compresión, y la posibilidad de ejecución del algoritmo en tiempo real en ordenadores embebidos de bajo consumo basados en ARM.&lt;/p&gt;

Underwater radio frequency image sensor using progressive image compression and region of interest

The increasing demand for underwater robotic intervention systems around the world in several application domains requires more versatile and inexpensive systems. By using a wireless communication system, supervised semi-autonomous robots have freedom of movement; however, the limited and varying bandwidth of underwater radio frequency (RF) channels is a major obstacle for the operator to get camera feedback and supervise the intervention. This paper proposes the use of progressive (embedded) image compression and region of interest (ROI) for the design of an underwater image sensor to be installed in an autonomous underwater vehicle, specially when there are constraints on the available bandwidth, allowing a more agile data exchange between the vehicle and a human operator supervising the underwater intervention. The operator can dynamically decide the size, quality, frame rate, or resolution of the received images so that the available bandwidth is utilized to its fullest potential and with the required minimum latency. The paper focuses first on the description of the system, which uses a camera, an embedded Linux system, and an RF emitter installed in an OpenROV housing cylinder. The RF receiver is connected to a computer on the user side, which controls the camera monitoring parameters, including the compression inputs, such as region of interest (ROI), size of the image, and frame rate. The paper focuses on the compression subsystem and does not attempt to improve the communications physical media for better underwater RF links. Instead, it proposes a unified system that uses well-integrated modules (compression and transmission) to provide the scientific community with a higher-level protocol for image compression and transmission in sub-sea robotic interventions.

Spatial predictive control using a thermal camera as feedback

The sensors being used in industrial settings are becoming increasingly complex and capable of acquiring significantly more data with higher functionality. One of these sensors is the thermal camera, which has the potential of capturing the spatial temperature profile of any surface. Through the extension of the conventional model-based control theory, a new model-based control scheme was formulated verify the novel concept of utilizing the spatial data from the thermal camera. To analyze this new model based control algorithm, a three zone heater barrel apparatus was designed and constructed for closed-loop testing to be performed. The experiments demonstrated the benefit of thermal camera feedback through the alterations in the control scheme. The availability of thermal profile of the system in comparison to single point temperature measurements resulted in the enhanced controller achieving better closed-loop transient responses, reduction in settling times. In addition, the use of spatial thermal images provides the possibility of controlling profiled temperature setpoints, showing the benefit and potential of using more advanced sensors in industry.

Design and Implementation of Dangerous Goods Handling Robot

Design and implementation of intelligent robot for dangerous goods handling. the mechanical arm use is AS-6 DOFB six degrees of freedom of mechanical arm, use and steering gear control board and computer as a way to control system, joints are available in within the prescribed scope of the sport, through the operation PC control software, to servo control instructions sent steering gear control signal, so AS to realize the mechanical arm space in precise work, the task completion in mechanical arm through installation camera feedback image, if install it to mobile platform, and substitute personnel in the harmful environment visible assignments, revise control instruction, accurate completion of each task. Experiments show that, this design is correct and feasible, and can meet the requirement.