Remote Expert Research Articles

A Novel Solution of Using Mixed Reality in Bowel and Oral and Maxillofacial Surgical Telepresence: 3D Mean Value Cloning algorithm.

Most of the Mixed Reality models used in the surgical telepresence are suffering from the discrepancies in the boundary area and spatial-temporal inconsistency due to the illumination variation in the video frames. The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and virtual hand of the remote expertise surgeon. The purpose of the proposed solution is to decrease the processing time and enhance the accuracy of merged video by decreasing the overlay and visualization error and removing occlusion and artefacts. The proposed system enhanced the mean value cloning algorithm that helps to maintain the spatial-temporal consistency of the final composite video. The enhanced algorithm includes the 3D mean value coordinates and improvised mean value interpolant in the image cloning process, which helps to reduce the sawtooth, smudging and discoloration artefacts around the blending region RESULTS: As compared to the state of art solution, the accuracy in terms of overlay error of the proposed solution is improved from 1.01mm to 0.80mm whereas the accuracy in terms of visualization error is improved from 98.8% to 99.4%. The processing time is reduced to 0.173 seconds from 0.211 seconds CONCLUSION: Our solution helps make the object of interest consistent with the light intensity of the target image by adding the space distance that helps maintain the spatial consistency in the final merged video. This article is protected by copyright. All rights reserved.

3DGAM: using 3D gesture and CAD models for training on mixed reality remote collaboration

As Virtual Reality(VR), Augmented Reality(AR), Mixed Reality(MR) technology becomes more accessible, it is important to explore VR/AR/MR technologies that can be used for remote collaboration on physical tasks. Previous research has shown that gesture-based interaction is intuitive and expressive for remote collaboration, and using 3D CAD models can provide clear instructions for assembly tasks. In this paper, therefore, we describe a new MR remote collaboration system which combines the use of gesture and CAD models in a complementary manner. The prototype system enables a remote expert in VR to provide instructions based on 3D gesture and CAD models (3DGAM) for a local worker who uses AR to see these instructions. Using this interface, we conducted a formal user study to explore the effect of sharing 3D gesture and CAD models in an assembly training task. We found that the combination of 3D gesture and CAD models can improve remote collaboration on an assembly task with respect to the performance time and user experience. Finally, we provide some conclusions and directions for future research.

Novel Equine Faecal Egg Diagnostics: Validation of the FECPAKG2.

Faecal egg counts (FECs) are the standard method of diagnosing the level of parasitic helminth egg shedding in horses and other grazing animals. Testing before treatment is an important factor in slowing the appearance of anthelmintic resistance in nematode parasites. The FECPAKG2, optimised for livestock, is reported to allow owners to perform FECs on their own animals without the need for a separate microscope or any specialist knowledge by tapping into remote expertise. However, the performance of the FECPAKG2 has yet to be assessed for equids. Therefore, a comparison of the FECPAKG2 (G2) method with an accepted equine FEC method (FECPAKG1(G1)) was performed, using faecal samples from 57 horses in Wales and 22 horses in New Zealand. There was a significant correlation between the FECs obtained by the two methods (p < 0.001) and no effect of the country of origin on the data (p = 0.157). The mean percentage accuracy compared to the control values (mean G2 count as a percentage of the mean G1 count, ±SStandard Error (SE)) was 101 ± 4%. There was no significant interaction between the method applied and the country of origin of the data (p = 0.814). The relative accuracy of the G2 method compared to the control method (FECPAKG1) was not affected by the level of infection (p = 0.124) and it was concluded that the FECPAKG2 method is a suitable method of performing FECs in horses. It is anticipated that the user-friendliness of the method will increase the uptake of FECs amongst horse owners, either by the direct use of the technology or through their veterinary practice, likely slowing the development of anthelmintic resistance.

In Response.

In Response.

The combination of visual communication cues in mixed reality remote collaboration

Many researchers have studied various visual communication cues (e.g. pointer, sketch, and hand gesture) in Mixed Reality remote collaboration systems for real-world tasks. However, the effect of combining them has not been so well explored. We studied the effect of these cues in four combinations: hand only, hand + pointer, hand + sketch, and hand + pointer + sketch, within the two user studies when a dependent view and independent view are supported respectively. In the first user study with the dependent view, the results showed that the hand gesture cue was the main visual communication cue and adding sketch cue to the hand gesture cue helped participants complete the task faster. In the second study with the independent view, the results showed that the hand gesture had an issue of local worker understanding remote expert’s hand gesture cue and the main visual communication cue was the pointer cue with fast completion time and high level of co-presence.

Offshore telementored ultrasound: a quality assessment study

BackgroundTelementored ultrasound (US) connects experts to novices through various types of communication and network technologies with the overall aim to bridge the medical imaging gap between patients’ diagnostic needs and on-site user experience. The recurrent theme in previous research on remote telementored US is the limited access to US machines and experienced users. This study was conducted to determine whether telementored US was feasible in a remote offshore setting. The aim was to assess if an onshore US expert can guide an offshore nurse through focused US scanning protocols by connecting an US machine to existing videoconference units at the offshore hospitals and to evaluate the diagnostic quality of the images and cineloops procured.ResultsThe diagnostic quality of cineloops was scored on a five-point scale. The percentage of cineloops suitable for interpretation (score 3 ≥) for the FATE and e-FAST protocols was 96.4 and 79.1. Lung sliding and seashore sign could be identified in all volunteers. The scan time for the FAST protocol (n = four scanning positions), FATE protocol (n = six scanning positions) and both lungs (n = two scanning positions) was 1 min 20 s, 4 min 15 s and 32 s, respectively.ConclusionA novice US user can be guided by a remote expert through focused US protocols within an acceptable time frame and with good diagnostic quality using existing communication and network systems found onboard offshore oil rigs.

Unravelling the panel contribution upon peer review evaluation of numerous, unstructured and highly interdisciplinary research proposals

Abstract In this study, we analyze the two-phase bottom-up procedure applied by the Future and Emerging Technologies Program (FET-Open) at the Research Executive Agency (REA) of the European Commission (EC), for the evaluation of highly interdisciplinary, multi-beneficiary research proposals which request funding. In the first phase, remote experts assess the proposals and draft comments addressing the pre-defined (by FET-Open) evaluation criteria. In the second phase, a new set of additional experts (of more general expertise and different from the remote ones), after cross reading the proposals and their remote evaluation reports, they convene in an on-site panel where they discuss the proposals. They complete the evaluation by reinforcing per proposal and per criterion one or another assessment, as assigned remotely during the first phase. We analyze the level of the inter-rater agreement among the remote experts and we identify its relative correlation with the funded proposals resulted after the end of the evaluation. Our study also provides comparative figures of the evolution of the proposals` scores during the two phases of the evaluation process. Finally, by carrying out an appropriate quantitative and qualitative analysis of all scores from the seven past cut-offs, we elaborate on the significant contribution of the panel (the second phase of the evaluation) in identifying and promoting the best proposals for funding.

The effects of spatial auditory and visual cues on mixed reality remote collaboration

Collaborative Mixed Reality (MR) technologies enable remote people to work together by sharing communication cues intrinsic to face-to-face conversations, such as eye gaze and hand gestures. While the role of visual cues has been investigated in many collaborative MR systems, the use of spatial auditory cues remains underexplored. In this paper, we present an MR remote collaboration system that shares both spatial auditory and visual cues between collaborators to help them complete a search task. Through two user studies in a large office, we found that compared to non-spatialized audio, the spatialized remote expert’s voice and auditory beacons enabled local workers to find small occluded objects with significantly stronger spatial perception. We also found that while the spatial auditory cues could indicate the spatial layout and a general direction to search for the target object, visual head frustum and hand gestures intuitively demonstrated the remote expert’s movements and the position of the target. Integrating visual cues (especially the head frustum) with the spatial auditory cues significantly improved the local worker’s task performance, social presence, and spatial perception of the environment.

Remote Transcranial Doppler Monitoring for Carotid Interventions

Transcranial Doppler (TCD) is an important tool in monitoring carotid interventions. One of the limitations is the need for a Neurosonology expert during procedures. This observational study is intended to address two issues: (1) Can the TCD-headframe be placed by a nonexpert with remote advice from the expert and (2) can the remote expert observes the monitoring and communicate with the surgeon effectively.

Assessing the Demand for Teleophthalmology in Florida Emergency Departments.

Background: Teleophthalmology programs are expanding, but have not been adapted into many emergency departments (EDs) in the United States. Introduction: Determining the potential demand for teleophthalmology services in the United States. EDs could enable development of new strategies to improve access to eye care in resource-limited regions. Methods: Telephone surveys were administered to ED physicians and nurses in Florida. Perceptions of ophthalmologist availability, equipment availability, and perceived utility of teleophthalmology services were measured. Results: Responses were from 104 of 207 facilities (50.2%); 88/181 (48.6%) designated as nonrural hospitals (NRHs) and 16/26 (61.5%) as rural hospitals (RHs). NRHs reported a median of 1 ophthalmologist available on call compared with a median of 0 at RHs (p < 0.001). NRHs were more likely to have a slit lamp (98.9% NRH, 50.0% RH; p < 0.001) and tonometer (100% NRH, 75.0% RH; p < 0.001). On a scale from 1 (lowest) to 5, most (68/93; 73.1%) perceived the value of teleophthalmology for remote consults as a 4 or 5. The most common perceived benefit of teleophthalmology use was to provide second/expert opinion (26.5% of responses). The most commonly cited perceived disadvantage was the physical unavailability of an ophthalmologist for examination and follow-up care (35.5% of responses). Discussion: RHs have less access to ophthalmologists and ophthalmic equipment when managing eye-related complaints in the ED. At both RHs and NRHs, providers face limitations in managing eye complaints and perceived teleophthalmology as a potentially valuable tool for remote expert consultation. Conclusions: Results suggest teleophthalmology services may be used to improve access to expert ophthalmic care, particularly in rural communities.

Scalable Virtual Network Video-Optimizer for Adaptive Real-Time Video Transmission in 5G Networks

The increasing popularity of video applications and ever-growing high-quality video transmissions (e.g., 4K resolutions), has encouraged other sectors to explore the growth of opportunities. In the case of health sector, mobile Health services are becoming increasingly relevant in real-time emergency video communication scenarios where a remote medical experts’ support is paramount to a successful and early disease diagnosis. To minimize the negative effects that could affect critical services in a heavily loaded network, it is essential for 5G video providers to deploy highly scalable and priorizable in-network video optimization schemes to meet the expectations of a large quantity of video treatments. This paper presents a novel 5G Video Optimizer Virtual Network Function (vOptimizerVNF) that leverages the latest technologies in 5G and video processing to address this important challenge. Advanced traffic filtering is coupled with Scalable H.265 video coding to enable run-time bandwidth-saving video optimization without compromising Quality of Service (QoS); kernel-space video processing is introduced to achieve further performance gains; and the use of a Virtual Network Function (VNF) facilitates dynamic deployment of virtualized video optimizers to achieve scalability and flexibility in this service. The proposed approach is implemented in a realistic 5G testbed and empirical results demonstrate the superior scalability and performance achieved.

Evaluation of an augmented reality platform for austere surgical telementoring: a randomized controlled crossover study in cricothyroidotomies

Telementoring platforms can help transfer surgical expertise remotely. However, most telementoring platforms are not designed to assist in austere, pre-hospital settings. This paper evaluates the system for telementoring with augmented reality (STAR), a portable and self-contained telementoring platform based on an augmented reality head-mounted display (ARHMD). The system is designed to assist in austere scenarios: a stabilized first-person view of the operating field is sent to a remote expert, who creates surgical instructions that a local first responder wearing the ARHMD can visualize as three-dimensional models projected onto the patient’s body. Our hypothesis evaluated whether remote guidance with STAR could lead to performing a surgical procedure better, as opposed to remote audio-only guidance. Remote expert surgeons guided first responders through training cricothyroidotomies in a simulated austere scenario, and on-site surgeons evaluated the participants using standardized evaluation tools. The evaluation comprehended completion time and technique performance of specific cricothyroidotomy steps. The analyses were also performed considering the participants’ years of experience as first responders, and their experience performing cricothyroidotomies. A linear mixed model analysis showed that using STAR was associated with higher procedural and non-procedural scores, and overall better performance. Additionally, a binary logistic regression analysis showed that using STAR was associated to safer and more successful executions of cricothyroidotomies. This work demonstrates that remote mentors can use STAR to provide first responders with guidance and surgical knowledge, and represents a first step towards the adoption of ARHMDs to convey clinical expertise remotely in austere scenarios.

Structured authoring for AR-based communication to enhance efficiency in remote diagnosis for complex equipment

Remote diagnosis procedures are prone to communication errors due to varying levels of experience and knowledge between expert maintainers and technicians. These result in inefficiencies that delay the diagnosis process. The aim of the paper is to develop a Structured-Message Authoring framework for Augmented Reality (AR) Remote Communication (SMAARRC) and to evaluate its ability to enhance the efficiency of remote diagnosis services. The framework proposes a message structure and automatic AR content creation rules for it that enable data capture and sharing within a remote context. Laboratory experiments present an average time reduction of 56% for remote calls while maintaining same quality compared to traditional remote communication methods (phone calls and emails). Remote experts feedback evidence the usability and feasibility of this framework to work in real-life conditions.

Haptic Feedback Helps Me? A VR-SAR Remote Collaborative System with Tangible Interaction

ABSTRACT Research on Augmented Reality (AR)/Mixed Reality (MR) remote collaboration for physical tasks remains a compelling and dynamic area of study. AR systems have been developed which transmit virtual annotations between remote collaborators, but there has been little research on how haptic feedback can also be shared. In this paper, we present a Virtual Reality (VR)-Spatial Augmented Reality (SAR) remote collaborative system that provides haptic feedback with tangible interaction between a local worker and a remote expert helper. Using this system, we conducted a within-subject user study to compare two interfaces for remote collaboration between a local worker and expert helper, one with mid-air free drawing (MFD) and one with tangible physical drawing (TPD). The results showed that there were no significant differences with respect to performance time and operation errors. However, users felt that the TPD interface supporting passive haptic feedback could significantly improve the remote experts’ user experience in VR. Our research provides useful information on the way for gesture- and gaze-based multimodal interaction supporting haptic feedback in AR/MR remote collaboration on physical tasks.

Rural health issues in rheumatology: a review.

Early access to rheumatology is imperative to achieve appropriate outcomes in rheumatologic diseases. But there seems to be a significant gap and disparity in the access to rheumatology care between urban and rural areas. This review was undertaken to analyze this issue. A significant delay in diagnosis of rheumatic disorder has been correlated to the travel distance to rheumatologist. It is also clear that currently, a significant rheumatology workforce shortage exists and is projected to worsen significantly, thereby making this gap and disparity much bigger. The scope of this gap and disparity in rheumatology care for rural patients remains incompletely defined and quantified. It is felt to be a significant issue and it is important to invest resources to obtain information about its scope. In addition, a number of solutions already exist which can be implemented using current network and infrastructure. These include relatively low-cost interventions such as patient navigator, remote rheumatology experts and if possible tele-rheumatology. These interventions can assist temporarily but a major improvement will require policy change at federal and state government level as well as involvement, buy-in, and incentivization of the providers and health networks providing rheumatology care.

Using a Head Pointer or Eye Gaze: The Effect of Gaze on Spatial AR Remote Collaboration for Physical Tasks

AbstractThis paper investigates the effect of using augmented reality (AR) annotations and two different gaze visualizations, head pointer (HP) and eye gaze (EG), in an AR system for remote collaboration on physical tasks. First, we developed a spatial AR remote collaboration platform that supports sharing the remote expert’s HP or EG cues. Then the prototype system was evaluated with a user study comparing three conditions for sharing non-verbal cues: (1) a cursor pointer (CP), (2) HP and (3) EG with respect to task performance, workload assessment and user experience. We found that there was a clear difference between these three conditions in the performance time but no significant difference between the HP and EG conditions. When considering the perceived collaboration quality, the HP/EG interface was statistically significantly higher than the CP interface, but there was no significant difference for workload assessment between these three conditions. We used low-cost head tracking for the HP cue and found that this served as an effective referential pointer. This implies that in some circumstances, HP could be a good proxy for EG in remote collaboration. Head pointing is more accessible and cheaper to use than more expensive eye-tracking hardware and paves the way for multi-modal interaction based on HP and gesture in AR remote collaboration.

The System for Telementoring with Augmented Reality (STAR): A head-mounted display to improve surgical coaching and confidence in remote areas

BackgroundThe surgical workforce particularly in rural regions needs novel approaches to reinforce the skills and confidence of health practitioners. Although conventional telementoring systems have proven beneficial to address this gap, the benefits of platforms of augmented reality-based telementoring in the coaching and confidence of medical personnel are yet to be evaluated. MethodsA total of 20 participants were guided by remote expert surgeons to perform leg fasciotomies on cadavers under one of two conditions: (1) telementoring (with our System for Telementoring with Augmented Reality) or (2) independently reviewing the procedure beforehand. Using the Individual Performance Score and the Weighted Individual Performance Score, two on-site, expert surgeons evaluated the participants. Postexperiment metrics included number of errors, procedure completion time, and self-reported confidence scores. A total of six objective measurements were obtained to describe the self-reported confidence scores and the overall quality of the coaching. Additional analyses were performed based on the participants’ expertise level. ResultsParticipants using the System for Telementoring with Augmented Reality received 10% greater Weighted Individual Performance Score (P = .03) and performed 67% fewer errors (P = .04). Moreover, participants with lower surgical expertise that used the System for Telementoring with Augmented Reality received 17% greater Individual Performance Score (P = .04), 32% greater Weighted Individual Performance Score (P < .01) and performed 92% fewer errors (P < .001). In addition, participants using the System for Telementoring with Augmented Reality reported 25% more confidence in all evaluated aspects (P < .03). On average, participants using the System for Telementoring with Augmented Reality received augmented reality guidance 19 times on average and received guidance for 47% of their total task completion time. ConclusionParticipants using the System for Telementoring with Augmented Reality performed leg fasciotomies with fewer errors and received better performance scores. In addition, participants using the System for Telementoring with Augmented Reality reported being more confident when performing fasciotomies under telementoring. Augmented Reality Head-Mounted Display–based telementoring successfully provided confidence and coaching to medical personnel.

Effective Collaborative Design of Large Virtual 3D Model using Multiple AR Markers

In the new era of industry, collaboration amongst partners will be the most challenging part of design and manufacturing. There are many ways for companies to collaborate with remote experts or partners in designing, manufacturing, or distributing the new product, depending on how complicated the product is. With the progress in information technology, many techniques can be used for supporting collaborative work. For the design that needs specific or complicates visual appearance, Augmented Reality (AR) is the most promising one. The virtual 3D model can be distributed to the remote partners for evaluation and providing comments. In this research, the technique in generating a stable large 3D model for remote design collaboration using multiple markers was developed. Users from different locations can work on the same virtual model. The platform for the mobile application was also developed. The results from the experiment with this setup confirmed that the system could be used for real collaboration.

Clinical efficacy assessment in severe vernal keratoconjunctivitis: preliminary validation of a new penalties-adjusted corneal fluorescein staining score

ABSTRACT Introduction and objective: Vernal keratoconjunctivitis (VKC) is a rare allergic eye condition that occurs in children and is characterised by a combination of debilitating symptoms. Repeated use of topical corticosteroid rescue therapy is often necessary in severe forms. This study aims to assess the validity of a new composite endpoint: the penalties-adjusted corneal staining score (PACS-S) proposed as primary endpoint in VEKTIS trial evaluating the efficacy of a new corticosteroid-sparing treatment, VERKAZIA® (ciclosporin 1 mg/ml eye drops), in severe VKC patients. Methodology: This research comprised a systematic literature review to identify efficacy endpoints being proposed in clinical trials for pediatric patients with severe VKC, followed by a remote expert advisory board assessing the validity of the PACS-S. Results: While no agreed or validated endpoint for assessing efficacy in VKC was identified when VEKTIS trial started, the experts’ board acknowledged a high face validity of PACS-S as a subjective integrated measure matching the current clinical practice. A fair external validity was considered with regards to VEKTIS trial secondary endpoints. Conclusion: PACS-S appears to be a reliable, valid and clinically meaningful primary endpoint that allows significant improvement over existing endpoints in severe VKC trials. Additional research is needed to validate this endpoint.

Accuracy of Remote Interpretation of Pediatric Emergency Ultrasound Over Third Generation Networks Across Continents.

This study aimed to evaluate live-stream videos for evaluation of pediatric emergency medicine ultrasound using a third generation (3G) network. The interrater reliability of a remote 3G network live-streaming assessment of ultrasound scans for abdominal free fluid, intussusception, and hip effusions was evaluated. Fifty-eight deidentified 6-second video clips in .mp4 format of focused assessment with sonography in trauma scans were downloaded to a laptop. A "bedside" ultrasound expert evaluated each scan and marked it positive or negative for the presence of free fluid. Using an iPhone 3G, Skype was used to connect to the second ultrasound expert located in Addis Ababa, Ethiopia, who reviewed the same scans. When the remote expert returned to the United States, evaluation of the scans was repeated as a second bedside reviewer. Interrater reliability was assessed using the κ statistic and percent agreement. This process was repeated for 44 intussusception scans and 28 hip effusion scans. The κ values (95% confidence interval) between the bedside and remote evaluators for focused assessment with sonography in trauma, intussusception, and hip effusion were 0.748 (0.576-0.92), 0.816 (0.648-0.984), and 0.764 (0.519-1.0), respectively. The percent agreements after adjusting for chance were 86%, 80%, and 88%, respectively. The κ values between the bedside assessments by each evaluator were 0.851 (0.71-0.992), 0.8 (0.616-0.984), and 0.747 (0.479-0.884), respectively. The percent agreements after adjusting for chance were 92%, 81%, and 88%, respectively. Live-stream video conference using cellphones over 3G network is a feasible and accurate method for consultation of ultrasound scans from a remote location.