Stereoscopic Data Research Articles

Geometric vs spectral content of Remotely Piloted Aircraft Systems images in the Precision agriculture context

In the last years the agricultural sector has been evolving and new technologies, like Unmanned Aerial Vehicles (UAV) and satellites, were introduced to increase crop management efficiency, reducing environmental costs and improving farmers’ income. MAIA-S2 sensor is presently one of the most performing optical sensors operating on a Remotely Piloted Aircraft Systems (RPAS); given its spectral features, it aims at supporting a scaling process where monoscopic satellite data (namely Copernicus S2) with high temporal and limited geometric resolution can be integrated with stereoscopic data from RPAS having a very high spatial resolution. In this work, data from MAIA-S2 sensor were used to detect the effects of different fertilization types on corn with reference to a test field located in Carignano (Piemonte region, NW-Italy). Different amounts of top dressing fertilization were applied on corn and an RPAS acquisition operated on 14th June 2021 (corresponding date to the corn stem elongation stage) to explore if any effects could be detectable. Three spectral indices, namely Normalized Difference Vegetation Index, Normalized Difference Red Edge index and Canopy Height Model, computed from at-the-ground reflectance calibrated MAIA-S2 data, were compared to evaluate the correspondent response to the different fertilization rates. Results show that: (i) NDVI poorly detect N-related differences zones; (ii) NDRE and CHM reasonably reflect the different N fertilization doses; (iii) Only CHM proved to be able to detect crop height and, consequently, biomass differences that are known to be induced by different rates of fertilization.

Visualization of surgical maneuvers using intraoperative real-time volumetric optical coherence tomography.

Ophthalmic microsurgery is traditionally performed using stereomicroscopes and requires visualization and manipulation of sub-millimeter tissue structures with limited contrast. Optical coherence tomography (OCT) is a non-invasive imaging modality that can provide high-resolution, depth-resolved cross sections, and has become a valuable tool in clinical practice in ophthalmology. While there has been substantial progress in both research and commercialization efforts to bring OCT imaging into live surgery, its use is still somewhat limited due to factors such as low imaging speed, limited scan configurations, and suboptimal data visualization. In this paper we describe, to the best of our knowledge, the translation of the fastest swept-source intraoperative OCT system with real-time volumetric imaging with stereoscopic data visualization provided via a heads-up display into the operating room. Results from a sampling of human anterior segment and retinal surgeries chosen from 93 human surgeries using the system are shown and the benefits that this mode of intrasurgical OCT imaging provides are discussed.

A Robust Edge-Preserving Stereo Matching Method for Laparoscopic Images.

Stereo matching has become an active area of research in the field of computer vision. In minimally invasive surgery, stereo matching provides depth information to surgeons, with the potential to increase the safety of surgical procedures, particularly those performed laparoscopically. Many stereo matching methods have been reported to perform well for natural images, but for images acquired during a laparoscopic procedure, they are limited by image characteristics including illumination differences, weak texture content, specular highlights, and occlusions. To overcome these limitations, we propose a robust edge-preserving stereo matching method for laparoscopic images, comprising an efficient sparse-dense feature matching step, left and right image illumination equalization, and refined disparity optimization. We validated the proposed method using both benchmark biological phantoms and surgical stereoscopic data. Experimental results illustrated that, in the presence of heavy illumination differences between image pairs, texture and textureless surfaces, specular highlights and occlusions, our proposed approach consistently obtains a more accurate estimate of the disparity map than state-of-the-art stereo matching methods in terms of robustness and boundary preservation.

Uncovering the Locus of Object-Context-Based Modulations in Depth Processing Using Repetitive Transcranial Magnetic Stimulation.

Neural responses of dorsal visual area V7 and lateral occipital complex (LOC) have been shown to correlate with changes in behavioral metrics of depth sensitivity observed as a function of object context, although it is unclear as to whether the behavioral manifestation results from an alteration of early depth-specific responses in V7 or arises as a result of alterations of object-level representations at LOC that subsequently feed back to affect disparity readouts in dorsal cortex. Here, we used online transcranial magnetic stimulation (TMS) to examine the roles of these two areas in giving rise to context–disparity interactions. Stimuli were disparity-defined geometric objects rendered as random-dot stereograms, presented in geometrically plausible and implausible variations. Observers’ sensitivity to depth (depth discrimination) or object identity (plausibility discrimination) was indexed while receiving repetitive TMS at one of the two sites of interest (V7, LOC) along with a control site (Cz). TMS over LOC produced results no different from TMS over baseline Cz (and prior no-TMS behavioral work). That is, depth sensitivity was higher for implausible versus plausible objects. Strikingly, TMS over V7 abolished differences in depth sensitivity for implausible versus plausible objects. V7 serves as a key locus in bringing stereosensitivity changes because of object context, perhaps by reweighing stereoscopic data en route to informing object–motoric interactions.

A Novel Technique of Mixed Reality Systems in the Treatment of Spinal Cord Tumors

Several reports have compared spinal cord tumor removal techniques but none have clearly described the appropriate site and level of indication for laminectomy or laminoplasty. The approach method for tumor removal depends on the type and localization of the tumor and the surgeon's skill. Therefore, a system that can suggest various surgical techniques is useful for spinal cord tumor surgery. The mixed reality system introduced in this paper is an excellent system that can suggest various surgical procedures. Using this system for spinal cord tumor removal, we made the surgery less invasive; therefore, we introduced this system and demonstrated its usefulness. Stereoscopic data of the patients with spinal cord tumors were obtained from preoperative myelogram-CT data. Stereoscopic laminectomy models including tumors were created using Blender, a free three-dimensional (3D) image editing software. We observed these data as 3D object images using a head-mounted display (HMD). This HMD is commercially available and relatively inexpensive. The surgical procedure is determined by considering those 3D images, radiological diagnosis, and the skill of surgeons. Intraoperative confirmation of the laminectomy site could be performed using the HMD. The 3D visualization of pathological conditions resulted in correct preoperative surgical planning and less invasive surgery in all five cases. Stereoscopic images using HMDs allow us a more intuitive understanding of the positional relationship between the tumor and spinal structure. These 3D object images can bring us more accurate preoperative planning and proper determination of surgical methods.

Design and Research of Interactive Animation of Immersive Space Scene Based on Computer Vision Technology

With the development of computational simulation technology, the need for stable and immersive display effects of space scene animation in the field of life experience and visual art has gradually increased. In this case, the requirements for immersive characteristics of space scene animation have also been strengthened. The existing 3D space scene animation has a limited degree of stereoscopic display model and data visualization. However, the current space scene interactive animation adopts a plane layout as a whole, and the size of the view interface is generally fixed, including the size of virtual elements. However, the immersion in this paper interactive animation of spatial scene can effectively solve the problems of incomplete display of 3D effects and unstable view simulation in 3D effects. This paper takes immersive space scene animation as the research object and studies the 3D and characteristics of space scene animation based on computer parallel computing in different immersive space scenes and different virtual space technologies, as well as the animation effects of different scene transformations and art forms. The result of research shows that, with the continuous increase of the degree of virtuality within a certain range, the immersive effect of spatial scene interactive animation gradually becomes better. When the color of the space scene animation is below 15, the virtual immersive effect changes less. When the space scene is in the range of 15–20, the space scene will make people feel the atmosphere of a beautiful and mysterious illusion.

Laser Doppler for Accurate Diagnosis of Oehler’s Type III Dens Invaginatus: A Case Report

In vital pulp therapy, pulp testing aids in the differential diagnosis and influences the treatment plan. Different from previous doppler flowmetry studies that only recorded waves and data, we report the first time, the use of a laser doppler blood flow monitor and imager with a colored image to present pulp vitality information in the dental field. We report here a case of Oehlers type IIIa dens invaginatus of the right upper lateral incisor in a 31-year-old girl, with the opening of the invagination near CDJ and a peri-invagination lesion sized 8 × 9 mm2. Cone beam computed tomography was used to verify specific spatial and stereoscopic data. After removing the source of infection, mineral trioxide aggregate was selected to fill the invagination. Laser doppler was scheduled for further assessment. After three years of regular follow-up, the patient’s clinical condition improved, the peri-invagination lesion healed, and the laser doppler and electric pulp test both showed a positive reaction. Laser doppler has successfully improved the diagnostic accuracy Thus, unnecessary interventions can be avoided while reducing the treatment time by preserving the vitality. However, further work is needed to resolve the limitations of laser doppler.

Anatomical calcaneal external fixator self-designed according to the morphology of heel

The anatomical calcaneal external fixator was designed by measuring and calculating the morphological data of the heel. A total of 100 normal people were randomly selected to obtain 200 hind foot data, including 45 males and 55 females, with an average age of 43.9 years (range, 19-67 years). According to the principles of human engineering and local anatomy, the morphological data of the heel in the weight-bearing standing position and supine position were measured with the direct measurement mode. The heel length, heel width, heel height, medial ankle height, lateral ankle height, and calcaneal pitch angle (CPA) were measured by vernier calipers and ulnar markers in weight-bearing standing position, and the gender groups and left and right foot groups were compared; the shape of the hind foot in the supine position was measured by three-dimensional (3D) dot matrix inverse model method. According to the stereoscopic data of the comprehensive anatomical morphology of the heel, the anatomical calcaneal external fixator was designed with AutoCAD 2019 and other 3D industrial design softwares. The measurements of shoe size, heel length, heel width, heel height, medial ankle height, lateral ankle height, and CPA in male were significantly higher than those in female ( P<0.05). There was no significant difference between the left and right feet in the other indexes except that the height of the medial malleolus of the left foot was significantly lower than that of the right foot ( t=-2.827, P=0.005). The measurement of 3D dot matrix inverse model in supine position showed that the heel part was non-circular arc edge, and many groups of arc edges fluctuate in a limited range. Based on the above data, an anatomical calcaneal external fixator was designed, which could fit the anatomic radian in theory, so as to be flexible in configuration. On this basis, the ordinary configuration, compression configuration, and orthodontic configuration were designed to meet the treatment needs of calcaneal fractures in different degrees. The ordinary configuration was suitable for patients with Sanders Ⅰ, ⅡA, and ⅡB calcaneal fractures with no or slight displacement of intra-articular fractures; the ordinary configuration was mainly used for simple fixing. The compression configuration was suitable for patients with Sanders ⅡC, ⅢA, and ⅢB, tongue fractures, and avulsion fractures with severe displacement of intra-articular fractures; the compression configuration used obliquely drawn console wires to fix the displaced bones. The orthodontic configuration was suitable for patients with Sanders ⅢC and Ⅳ calcaneal fractures with severe displacement of intra-articular fractures or severe calcaneal bone defects; the orthodontic configuration was a multi-module design, which took into account the stable fixation of the fracture and the arbitrary adjustment of the joint fixation angle. The hind foot is special for morphology, so the external fixator designed based on the vernier caliper measurement method and 3D dot matrix measuring plate measurement method is an anatomical type and its configuration can theoretically meet stable and flexible clinical needs.

Disparity tuning guided stereoscopic saliency detection for eye fixation prediction

The development of emerging 3D display brings increasing attention of stereoscopic techniques such as quality assessment, re-targeting, and compression of 3D image, that require new saliency detection methods to deal with stereoscopic data. In this paper, we present a disparity tuning guided stereoscopic saliency (DTSS) model which apply the disparity tuning mechanism of visual cortical neurons into visual attention modeling. First, we investigate the rationality of converting features from physical quantity into perception quantity before computing saliency. Second, we discuss the biological principles that depth affects visual attention and consider both absolute and relative depth tuning mechanisms to model visual attention. Then, we propose a diffusion saliency feature combining depth and RGB features. Specifically, we use RGB contrast as primitive labels to diffuse a saliency map for depth map and using depth contrast as primitive labels to diffuse a saliency map for RGB image. Finally, an adaptively weighted fusion method is proposed for the integration of feature maps. Experiments demonstrate that the proposed model performs well against to the state-of-art methods on the task of 3D eye fixation prediction.

Band registration of tuneable frame format hyperspectral UAV imagers in complex scenes

A recent revolution in miniaturised sensor technology has provided markets with novel hyperspectral imagers operating in the frame format principle. In the case of unmanned aerial vehicle (UAV) based remote sensing, the frame format technology is highly attractive in comparison to the commonly utilised pushbroom scanning technology, because it offers better stability and the possibility to capture stereoscopic data sets, bringing an opportunity for 3D hyperspectral object reconstruction. Tuneable filters are one of the approaches for capturing multi- or hyperspectral frame images. The individual bands are not aligned when operating a sensor based on tuneable filters from a mobile platform, such as UAV, because the full spectrum recording is carried out in the time-sequential principle. The objective of this investigation was to study the aspects of band registration of an imager based on tuneable filters and to develop a rigorous and efficient approach for band registration in complex 3D scenes, such as forests. The method first determines the orientations of selected reference bands and reconstructs the 3D scene using structure-from-motion and dense image matching technologies. The bands, without orientation, are then matched to the oriented bands accounting the 3D scene to provide exterior orientations, and afterwards, hyperspectral orthomosaics, or hyperspectral point clouds, are calculated. The uncertainty aspects of the novel approach were studied. An empirical assessment was carried out in a forested environment using hyperspectral images captured with a hyperspectral 2D frame format camera, based on a tuneable Fabry-Pérot interferometer (FPI) on board a multicopter and supported by a high spatial resolution consumer colour camera. A theoretical assessment showed that the method was capable of providing band registration accuracy better than 0.5-pixel size. The empirical assessment proved the performance and showed that, with the novel method, most parts of the band misalignments were less than the pixel size. Furthermore, it was shown that the performance of the band alignment was dependent on the spatial distance from the reference band.

Stereoscopic medical data video quality issues.

Stereoscopic medical videos are recorded, e.g., in stereo endoscopy or during video recording medical/dental operations. This paper examines quality issues in the recorded stereoscopic medical videos, as insufficient quality may induce visual fatigue to doctors. No attention has been paid to stereo quality and ensuing fatigue issues in the scientific literature so far. Two of the most commonly encountered quality issues in stereoscopic data, namely stereoscopic window violations and bent windows, were searched for in stereo endoscopic medical videos. Furthermore, an additional stereo quality issue encountered in dental operation videos, namely excessive disparity, was detected and fixed. The conducted experiments prove the existence of such quality issues in stereoscopic medical data and highlight the need for their detection and correction.

First broadband characterization and redshift determination of the VHE blazar MAGIC J2001+439

We aim to characterize the broadband emission from 2FGL J2001.1+4352, which has been associated with the unknown-redshift blazar MG4 J200112+4352. Based on its gamma-ray spectral properties, it was identified as a potential very high energy (VHE; E > 100 GeV) gamma-ray emitter. The source was observed with MAGIC first in 2009 and later in 2010 within a multi-instrument observation campaign. The MAGIC observations yielded 14.8 hours of good quality stereoscopic data. The object was monitored at radio, optical and gamma-ray energies during the years 2010 and 2011. The source, named MAGIC J2001+439, is detected for the first time at VHE with MAGIC at a statistical significance of 6.3 {\sigma} (E > 70 GeV) during a 1.3-hour long observation on 2010 July 16. The multi-instrument observations show variability in all energy bands with the highest amplitude of variability in the X-ray and VHE bands. We also organized deep imaging optical observations with the Nordic Optical Telescope in 2013 to determine the source redshift. We determine for the first time the redshift of this BL Lac object through the measurement of its host galaxy during low blazar activity. Using the observational evidence that the luminosities of BL Lac host galaxies are confined to a relatively narrow range, we obtain z = 0.18 +/- 0.04. Additionally, we use the Fermi-LAT and MAGIC gamma-ray spectra to provide an independent redshift estimation, z = 0.17 +/- 0.10. Using the former (more accurate) redshift value, we adequately describe the broadband emission with a one-zone SSC model for different activity states and interpret the few-day timescale variability as produced by changes in the high-energy component of the electron energy distribution.

MAGIC reveals a complex morphology within the unidentified gamma-ray source HESS J1857+026

HESS J1857+026 is an extended TeV gamma-ray source that was discovered by H.E.S.S. as part of its Galactic plane survey. Given its broadband spectral energy distribution and its spatial coincidence with the young energetic pulsar PSR J1856+0245, the source has been put forward as a pulsar wind nebula (PWN) candidate. MAGIC has performed follow-up observations aimed at mapping the source down to energies approaching 100 GeV in order to better understand its complex morphology. HESS J1857+026 was observed by MAGIC in 2010, yielding 29 hours of good quality stereoscopic data that allowed us to map the source region in two separate ranges of energy. We detected very-high-energy gamma-ray emission from HESS J1857+026 with a significance of $12 \sigma$ above $150$ GeV. The differential energy spectrum between $100$ GeV and $13$ TeV is well described by a power law function $dN/dE = N_0(E/1\textrm{TeV})^{-\Gamma}$ with $N_0 = (5.37 \pm0.44_{stat} \pm1.5_{sys}) \times 10^{-12} (\textrm{TeV}^{-1} \textrm{cm}^{-2}$ $\textrm{ s}^{-1})$ and $\Gamma = 2.16\pm0.07_{stat} \pm0.15_{sys}$, which bridges the gap between the GeV emission measured by Fermi-LAT and the multi-TeV emission measured by H.E.S.S.. In addition, we present a detailed analysis of the energy-dependent morphology of this region. We couple these results with archival multi-wavelength data and outline evidence in favor of a two-source scenario, whereby one source is associated with a PWN, while the other could be linked with a molecular cloud complex containing an HII region and a possible gas cavity.

Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction.

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper.

Research on Information Applied Technology with Video Compression Algorithms Based on the Optimal Multi-Band Haar Wavelet Transform

Video playback has been one of the most important online communication ways. With the application of stereo video, large amount of video data need to be stored and transported so that fluency and clarity of demand system, and how to efficiently conduct compressed encoding for stereoscopic video data becomes a hot topic currently. In view of this problem, this paper puts forward the video-on-demand compression algorithm based on the optimal multi-band Haar wavelet transform, through the research on wavelet transform algorithm model to reinforce the algorithm secondly, strengthening from the binary wavelet theory into octal wavelet system theory to get better compression capability. The simulation experiments show that video-on-demand compression algorithm based on the optimal multi-band Haar wavelet transform proposed in this paper has a good compression performance not only under medium and high bit- rate conditions, and also reaches the H. 263 under low bit-rate condition.

NEW LIGHT-WEIGHT STEREOSOPIC SPECTROMETRIC AIRBORNE IMAGING TECHNOLOGY FOR HIGH-RESOLUTION ENVIRONMENTAL REMOTE SENSING – CASE STUDIES IN WATER QUALITY MAPPING

Abstract. A new Fabry-Perot interferometer (FPI) based light-weight spectrometric camera provides new possibilities for environmental remote sensing applications. The sensor collects spectral data cubes with adjustable spectral properties in a rectangular image format, and so stereoscopic data can be obtained by gathering images in block structures with overlapping images. The FPI camera thus enables stereoscopic, spectrometric remote sensing applications with light-weight, low-cost airborne imaging systems. Our objective is to investigate the processing and use of this new imaging technology in a water quality mapping. We carried out imaging campaigns over a small lake in summer and autumn 2012 using a light-weight unmanned airborne vehicle (UAV) and a small manned airborne vehicle (MAV). We present the preliminary results of these campaigns.

Phase-resolved energy spectra of the Crab pulsar in the range of 50–400 GeV measured with the MAGIC telescopes

We use 73 h of stereoscopic data taken with the MAGIC telescopes to investigate the very high-energy (VHE) gamma-ray emission of the Crab pulsar. Our data show a highly significant pulsed signal in the energy range from 50 to 400 GeV in both the main pulse (P1) and the interpulse (P2) phase regions. We provide the widest spectra to date of the VHE components of both peaks, and these spectra extend to the energy range of satellite-borne observatories. The good resolution and background rejection of the stereoscopic MAGIC system allows us to cross-check the correctness of each spectral point of the pulsar by comparison with the corresponding (strong and well-known) Crab nebula flux. The spectra of both P1 and P2 are compatible with power laws with photon indices of 4.0 \pm 0.8 (P1) and 3.42 \pm 0.26 (P2), respectively, and the ratio P1/P2 between the photon counts of the two pulses is 0.54 \pm 0.12. The VHE emission can be understood as an additional component produced by the inverse Compton scattering of secondary and tertiary e\pm pairs on IR-UV photons.

DIGITAL ELEVATION MODEL FROM PRISM-ALOS AND ASTER STEREOSCOPIC DATA

Digital Elevation Model (DEM) is a source to produce contour map, slope, and aspect information, which is needed for other information such as disaster and water resources management. DEM can be generated by several methods. One of them is parallax calculations from stereoscopic data of optical sensor. Panchromatic Remote-Sensing Instrument for Stereo Mapping (PRISM) sensor from Advanced LAnd Observation Satellite (ALOS) satellite and advance space borne Thermal Emission and Reflection Radiometer (ASTER) sensor from Terra Satellite is Japanese optical satellite sensor which have abilityto produce stereoscopic data. This study showed DEM generations from PRISM (2.5 m spatial resolution) and ASTER (15m spatial resolution) stereoscopic data using image matching and collinear model based on Orthobase-pro software. The Generated DEM from each sensor was compared to the DEM from Shuttle Radar Topography Mission (SRTM) X-C band with 30 m spatial resolution. The dependent on the pixel size from the DEM produced were also discussed. The result showed that both DEMs have similiar elevation and distribution pattern to the referenced DEM, but DEM for PRISM had higher relative accuracy (RMSE is 6.5 m) and Smoother pattern comparing to DEM from ASTER (RMSE is 10.2 m) Keyword : ASTER, DEM, PRISM, SRTM, Stereoscopic satellite data

Landslide Volumetric Analysis Using Cartosat-1-Derived DEMs

The monitoring of landscape changes can lead to the identification of environmental hot spots, improve process understanding, and provide means for law enforcement. Digital elevation models (DEMs) derived from stereoscopic satellite data provide a systematic synoptic framework that is potentially useful to support these issues. Along-track high-resolution stereoscopic data, provided with rational polynomial coefficients (RPCs), are ideal for the fast and accurate extraction of DEMs due to the reduced radiometric differences between images. In this letter, we assess the suitability of data from the relatively new Cartosat-1 satellite to quantify large-scale geomorphological changes, using the volume estimation of the 2007 Salna landslide in the Indian Himalayas as a test case. The depletion and accumulation volumes, estimated as 0.55 × 106 and 1.43 × 106 m3, respectively, showed a good match with the volumes calculated using DEMs generated only with RPCs and without ground control points (GCPs), indicating that the volume figures are less sensitive to GCP support. The result showed that these data can provide an important input for disaster-management activities.

Stereoscopic Analysis of the 19 May 2007 Erupting Filament

A filament eruption, accompanied by a B9.5 flare, coronal dimming and an EUV wave, was observed by the Solar TERrestrial Relations Observatory (STEREO) on 19 May 2007, beginning at about 13:00 UT. Here, we use observations from the SECCHI/EUVI telescopes and other solar observations to analyze the behavior and geometry of the filament before and during the eruption. At this time, STEREO A and B were separated by about 8.5 degrees, sufficient to determine the three-dimensional structure of the filament using stereoscopy. The filament could be followed in SECCHI/EUVI 304 A stereoscopic data from about 12 hours before to about 2 hours after the eruption, allowing us to determine the 3D trajectory of the erupting filament. From the 3D reconstructions of the filament and the chromospheric ribbons in the early stage of the eruption, simultaneous heating of both the rising filamentary material and the chromosphere directly below is observed, consistent with an eruption resulting from magnetic reconnection below the filament. Comparisons of the filament during eruption in 304 A and Halpha show that when it becomes emissive in He II, it tends to disappear in Halpha, indicating that the disappearance probably results from heating or motion, not loss, of filamentary material.