Objective Assessment Metrics Research Articles

Acquiring 360° Light Field by a Moving Dual-Fisheye Camera.

In this paper, we propose an efficient deep learning pipeline for light field acquisition using a back-to-back dual-fisheye camera. The proposed pipeline generates a light field from a sequence of 360° raw images captured by the dual-fisheye camera. It has three main components: a convolutional network (CNN) that enforces a spatiotemporal consistency constraint on the subviews of the 360° light field, an equirectangular matching cost that aims at increasing the accuracy of disparity estimation, and a light field resampling subnet that produces the 360° light field based on the disparity information. Ablation tests are conducted to analyze the performance of the proposed pipeline using the HCI light field datasets with five objective assessment metrics (MSE, MAE, PSNR, SSIM, and GMSD). We also use real data obtained from a commercially available dual-fisheye camera to quantitatively and qualitatively test the effectiveness, robustness, and quality of the proposed pipeline. Our contributions include: 1) a novel spatiotemporal consistency loss that enforces the subviews of the 360° light field to be consistent, 2) an equirectangular matching cost that combats severe projection distortion of fisheye images, and 3) a light field resampling subnet that retains the geometric structure of spherical subviews while enhancing the angular resolution of the light field.

Analysis of quality objective assessment metrics for visible and infrared image fusion

Analysis of quality objective assessment metrics for visible and infrared image fusion

Mixed Reality Annotation of Robotic-Assisted Surgery videos with real- time tracking and stereo matching

Robotic-Assisted Surgery (RAS) is beginning to unlock its potential. However, despite the latest advances in RAS, the steep learning curve of RAS devices remains a problem. A common teaching resource in surgery is the use of videos of previous procedures, which in RAS are almost always stereoscopic. It is important to be able to add virtual annotations onto these videos so that certain elements of the surgical process are tracked and highlighted during the teaching session. Including virtual annotations in stereoscopic videos turns them into Mixed Reality (MR) experiences, in which tissues, tools and procedures are better observed. However, an MR-based annotation of objects requires tracking and some kind of depth estimation. For this reason, this paper proposes a real-time hybrid tracking–matching method for performing virtual annotations on RAS videos. The proposed method is hybrid because it combines tracking and stereo matching, avoiding the need to calculate the real depth of the pixels. The method was tested with six different state-of-the-art trackers and assessed with videos of a sigmoidectomy of a sigma neoplasia, performed with a Da Vinci® X surgical system. Objective assessment metrics are proposed, presented and calculated for the different solutions. The results show that the method can successfully annotate RAS videos in real-time. Of all the trackers tested for the presented method, the CSRT (Channel and Spatial Reliability Tracking) tracker seems to be the most reliable and robust in terms of tracking capabilities. In addition, in the absence of an absolute ground truth, an assessment with a domain expert using a novel continuous-rating method with an Oculus Quest 2 Virtual Reality device was performed, showing that the depth perception of the virtual annotations is good, despite the fact that no absolute depth values are calculated.

Objective Quality Assessment Metrics for Light Field Image Based on Textural Features

Light Field (LF) imaging is a plenoptic data collection method enabling a wide variety of image post-processing such as 3D extraction, viewpoint change and digital refocusing. Moreover, LF provides the capability to capture rich information about a scene, e.g., texture, geometric information, etc. Therefore, a quality assessment model for LF images is needed and poses significant challenges. Many LF Image Quality Assessment (LF-IQA) metrics have been recently presented based on the unique characteristics of LF images. The state-of-the-art objective assessment metrics have taken into account the image content and human visual system such as SSIM and IW-SSIM. However, most of these metrics are designed for images and video with natural content. Additionally, other models based on the LF characteristics (e.g., depth information, angle information) trade high performance for high computational complexity, along with them possessing difficulties of implementation for LF applications due to the immense data requirements of LF images. Hence, this paper presents a novel content-adaptive LF-IQA metric to improve the conventional LF-IQA performance that is also low in computational complexity. The experimental results clearly show improved performance compared to conventional objective IQA metrics, and we also identify metrics that are well-suited for LF image assessment. In addition, we present a comprehensive content-based feature analysis to determine the most appropriate feature that influences human visual perception among the widely used conventional objective IQA metrics. Finally, a rich LF dataset is selected from the EPFL dataset, allowing for the study of light field quality by qualitative factors such as depth (wide and narrow), focus (background or foreground) and complexity (simple and complex).

Adaptive Surgical Robotic Training Using Real-Time Stylistic Behavior Feedback Through Haptic Cues.

Surgical skill directly affects surgical procedure outcomes; thus, effective training is needed to ensure satisfactory results. Many objective assessment metrics have been developed that provide the trainee with descriptive feedback about their performance however, often lack feedback on how to improve performance. The most effective training method is one that is intuitive, easy to understand, personalized to the user,and provided in a timely manner. We propose a framework to enable user-adaptive training using near real-time detection of performance, based on intuitive styles of surgical movements, and design a haptic feedback framework to assist with correcting styles of movement. We evaluate the ability of three types of force feedback (spring, damping, and spring plus damping feedback), computed based on prior user positions, to improve different stylistic behaviors of the user during kinematically constrained reaching movement tasks. The results indicate that five out of six styles studied here were improved using at least one of the three types of force feedback. Task performance metrics were compared in the presence of the three types of feedback. Task time was statistically significantly lower when applying spring feedback, compared to the other two types of feedback. Path straightness and targeting error were statistically significantly improved when using spring-damping feedback compared to the other two types of feedback. This study presents a groundwork for adaptive training in robotic surgery based on near real-time human-centric models of surgical behavior.

OBJECTIVE ASSESSMENT METRICS FOR CENTRAL LINE SIMULATORS: AN EXPLORATION OF CAUSAL FACTORS.

The Dynamic Haptic Robotic Trainer (DHRT) was developed to minimize the up to 39% of adverse effects experienced by patients during Central Venous Catheterization (CVC) by standardizing CVC training, and provide automated assessments of performance. Specifically, this system was developed to replace manikin trainers that only simulate one patient anatomy and require a trained preceptor to evaluate the trainees' performance. While the DHRT system provides automated feedback, the utility of this system with real-world scenarios and expertise has yet to be thoroughly investigated. Thus, the current study was developed to determine the validity of the current objective assessment metrics incorporated in the DHRT system through expert interviews. The main findings from this study are that experts do agree on perceptions of patient case difficulty, and that characterizations of patient case difficulty is based on anatomical characteristics, multiple needle insertions, and prior catheterization.

Vocal Tract Contour Tracking in rtMRI Using Deep Temporal Regression Network

Recent advances in real-time Magnetic Resonance Imaging (rtMRI) provide an invaluable tool to study speech articulation. In this paper, we present an effective deep learning approach for supervised detection and tracking of vocal tract contours in a sequence of rtMRI frames. We train a single input multiple output deep temporal regression network (DTRN) to detect the vocal tract (VT) contour and the separation boundary between different articulators. The DTRN learns the non-linear mapping from an overlapping fixed-length sequence of rtMRI frames to the corresponding articulatory movements, where a blend of the overlapping contour estimates defines the detected VT contour. The detected contour is refined at a post-processing stage using an appearance model to further improve the accuracy of VT contour detection. The proposed VT contour tracking model is trained and evaluated over the USC-TIMIT dataset. Performance evaluation is carried out using three objective assessment metrics for the separating landmark detection, contour tracking and temporal stability of the contour landmarks in comparison with three baseline approaches from the recent literature. Results indicate significant improvements with the proposed method over the state-of-the-art baselines.

Light Field Synthesis by Training Deep Network in the Refocused Image Domain.

Light field imaging, which captures spatial-angular information of light incident on image sensors, enables many interesting applications such as image refocusing and augmented reality. However, due to the limited sensor resolution, a trade-off exists between the spatial and angular resolutions. To increase the angular resolution, view synthesis techniques have been adopted to generate new views from existing views. However, traditional learning-based view synthesis mainly considers the image quality of each view of the light field and neglects the quality of the refocused images. In this paper, we propose a new loss function called refocused image error (RIE) to address the issue. The main idea is that the image quality of the synthesized light field should be optimized in the refocused image domain because it is where the light field is viewed. We analyze the behavior of RIE in the spectral domain and test the performance of our approach against previous approaches on both real (INRIA) and software-rendered (HCI) light field datasets using objective assessment metrics such as MSE, MAE, PSNR, SSIM, and GMSD. Experimental results show that the light field generated by our method results in better refocused images than previous methods.

Application of HDR algorithms to solve direct sunlight problems when autonomous vehicles using machine vision systems are driving into sun

Just after sunrise and before sunset the sun can shine directly into driver's eyes making it more difficult to see the road ahead. The direct sunlight problem creates a serious risk for human drivers as well as machine vision systems for autonomous vehicles. In 2016, the on-board cameras of a Tesla level 2 autonomy vehicle were unable to detect the side of a white truck due to bright sunlight. In order to solve the direct sunlight problem, we apply high dynamic range (HDR) imaging algorithms to the machine vision system of Lawrence Technological University's autonomous electric research vehicle and measure how well the system performs in direct sunlight. A Raspberry Pi is used to capture several manual exposure frames and compose HDR frames using three HDR algorithms provided by OpenCV 3. The resulting frames are supplied as input to the autonomous vehicle. Standard dynamic range (SDR) images captured using auto exposure are compared to equivalent HDR frames using various quality metrics. Objective assessment metrics are also performed to compare the color and contrast of HDR and SDR frames. Results show the system performs best when using HDR frames generated by the Mertens algorithm.

Design Property Network-Based Change Propagation Prediction Approach for Mechanical Product Development

Design changes are unavoidable during mechanical product development; whereas the avalanche propagation of design change imposes severely negative impacts on the design cycle. To improve the validity of the change propagation prediction, a mathematical programming model is presented to predict the change propagation impact quantitatively. As the foundation of change propagation prediction, a design change analysis model(DCAM) is built in the form of design property network. In DCAM, the connections of the design properties are identified as the design specification, which conform to the small-world network theory. To quantify the change propagation impact, change propagation intensity(CPI) is defined as a quantitative and much more objective assessment metric. According to the characteristics of DCAM, CPI is defined and indicated by four assessment factors: propagation likelihood, node degree, long-chain linkage, and design margin. Furthermore, the optimal change propagation path is searched with the evolutionary ant colony optimization(ACO) algorithm, which corresponds to the minimized maximum of accumulated CPI. In practice, the change impact of a gear box is successfully analyzed. The proposed change propagation prediction method is verified to be efficient and effective, which could provide different results according to various the initial changes.

Efficient Objective Metric Tool for Medical Electrical Device Development: Eye Phantom for Glaucoma Diagnosis Device

Developing electronic medical devices is challenging. Simulations or in vivo experiments are not sufficient to obtain pertinent comparisons between potential design options. This paper presents a new artificial tool allowing objective comparisons between electronic device topologies. The main idea is to build a tool which is sensitive to targeted biological parameters only. These tools are generally called phantoms. The phantom presented in this paper is dedicated to IntraOcular Pressure (IOP) Measurement devices used in glaucoma diagnosis and treatment. It is called Biomechanical Eye Emulator (BEE). The BEE emulates the main biomechanical parameters influencing the IOP measurements. Because it is not sensitive to the living context, the BEE is the most efficient tool to investigate the best sensor design. BEE specifications are defined to be as close as possible to chosen models (humans or animals). Its efficiency is shown with a case study on rabbits. The results clearly demonstrate the BEE phantoms efficacy in providing objective assessment metrics during the sensor design process.

SS10. Advanced Vascular Surgery Surgical Skills and Simulated Assessment Program for Senior Vascular Surgery Trainees

Cognitive competence in vascular surgery is determined by oral and written examinations administered by the Vascular Surgery Board. Technical competence of vascular surgery trainees is determined solely by subjective assessment from faculty members of the trainee's institution. This program represents the first attempt to objectively evaluate the advanced vascular surgical skills of senior vascular trainees using high-fidelity simulation models by an independent faculty proctor. Nineteen senior vascular trainees were evaluated over a 3-day period. Twenty-five board-certified vascular surgeons served as assessment proctors. Assessment stations included thoracic and abdominal aortic aneurysm endograft repair, open abdominal aortic aneurysm repair, carotid endarterectomy, femoral-popliteal bypass, arteriovenous fistula creation, mismatched end-to-side anastomosis at depth, carotid, renal, iliac, and superficial femoral artery angioplasty and stenting, and use of a percutaneous closure device. Trainees were given 1 hour to complete each station. Technical performances were evaluated by a single faculty member using global rating scales, critical task and time assessment metrics, and a global rating summary. Competency was defined as achieving >80% global rating scale score, 100% achievement of all critical task metrics, or attainment of a level 4 (proficient) or level 5 (advanced) global rating summary score. Immediate feedback was provided after each performance. Six simulation stations were videotaped for future assessment metrics validation. A total of 202 skill and simulation assessments were performed. Vascular trainees were graded as being proficient for 83% (global rating scale), 75% (global rating summary), and 49% (critical task metrics) of all surgical skills and simulated performances. Similar patterns of trainee technical and procedural errors were noted to occur for each simulated procedure. The performance results identified a range of technical skills among the trainees. Faculty assessment was inconsistent with reporting errors or omissions occurring in 18%. Endovascular simulator malfunction occurred in just 4% of procedures. No malfunctions occurred in the open vascular simulators. These preliminary results indicate that it is feasible to perform objective assessment of senior vascular trainees by board-certified vascular surgery faculty using multiple objective assessment metrics. Simulation models overall exhibited good durability and consistency, and therefore appear to be useful in the assessment of advanced vascular surgical skills. Data from this initial feasibility study can be used to determine reproducible objective assessment metrics. These metrics, once validated, can than serve as a foundation for a technical competence examination in vascular surgery.

Implementation and Comparison of Image Enhancement Techniques

Image enhancement is the key and important step in digital image processing. If the image is not clear, it cannot be able to perform precisely edge detection, segmentation and other image processing steps. For enhancement of images that are multiresolution, image fusion provides the best output results. Image fusion is the technique of combining relevant information from source images, to get the fused image having most of the information from the source images. This technique can be used in various application areas like aerial images, forensic, flash photography, real life photographs and etc. In this paper, authors discusses the implementation of three categories of image fusion algorithms – basic fusion algorithms, pyramid based algorithms and the basic DWT algorithms and these algorithm are assessed using various objective assessment metrics for image enhancement .These fusion algorithms are compared against the general image enhancement methods for different images with the help of error analysis techniques i.e. Average Difference(AD), Normalized Mean Square Error (NMSE) and the Peak Signal to Noise Ratio (PSNR) and etc.The image fusion methods provide better results than the general image enhancement methods. General Terms Digital Image Processing, Image Enhancement

Visual comfort improvement in stereoscopic 3D displays using perceptually plausible assessment metric of visual comfort

Visual discomfort is caused by several factors such as accommodation-vergence conflict and window violation in conventional stereoscopic displays. To ensure comfortable conditions in the stereoscopic viewing, it is essential to develop an effective way of improving visual comfort in stereoscopic displays. This paper proposes a new method for visual comfort improvement that adjusts the disparity of a scene under the guidance of an objective assessment metric of visual comfort. The proposed method employs a visual comfort assessment metric that takes into account various discomfort factors in the stereoscopic viewing. The results of subjective assessment experiments demonstrate that the proposed method can efficiently improve visual comfort in the stereoscopic viewing.

Image Retargeting Quality Assessment

AbstractContent‐aware image retargeting is a technique that can flexibly display images with different aspect ratios and simultaneously preserve salient regions in images. Recently many image retargeting techniques have been proposed. To compare image quality by different retargeting methods fast and reliably, an objective metric simulating the human vision system (HVS) is presented in this paper. Different from traditional objective assessment methods that work in bottom‐up manner (i.e., assembling pixel‐level features in a local‐to‐global way), in this paper we propose to use a reverse order (top‐down manner) that organizes image features from global to local viewpoints, leading to a new objective assessment metric for retargeted images. A scale‐space matching method is designed to facilitate extraction of global geometric structures from retargeted images. By traversing the scale space from coarse to fine levels, local pixel correspondence is also established. The objective assessment metric is then based on both global geometric structures and local pixel correspondence. To evaluate color images, CIE L*a*b* color space is utilized. Experimental results are obtained to measure the performance of objective assessments with the proposed metric. The results show good consistency between the proposed objective metric and subjective assessment by human observers.