Fisheye Lens Research Articles

Exploratory investigation of virtual lesions in gastrointestinal endoscopy using a novel phase‐shift method for three‐dimensional shape measurement

AbstractAccurate measurement of the size of lesions or distances between any two points during endoscopic examination of the gastrointestinal tract is difficult owing to the fisheye lens used in endoscopy. To overcome this issue, we developed a phase‐shift method to measure three‐dimensional (3D) data on a curved surface, which we present herein. Our system allows the creation of 3D shapes on a curved surface by the phase‐shift method using a stripe pattern projected from a small projecting device to an object. For evaluation, 88 measurement points were inserted in porcine stomach tissue, attached to a half‐pipe jig, with an inner radius of 21 mm. The accuracy and precision of the measurement data for our shape measurement system were compared with the data obtained using an Olympus STM6 measurement microscope. The accuracy of the path length of a simulated protruded lesion was evaluated using a plaster model of the curved stomach and graph paper. The difference in height measures between the measurement microscope and measurement system data was 0.24 mm for the 88 measurement points on the curved surface of the porcine stomach. The error in the path length measurement for a lesion on an underlying curved surface was <1% for a 10‐mm lesion. The software was developed for the automated calculation of the major and minor diameters of each lesion. The accuracy of our measurement system could improve the accuracy of determining the size of lesions, whether protruded or depressed, regardless of the curvature of the underlying surface.

Retinal Image Quality Through an Operating Microscope With Wavefront Shaping Extended Depth of Focus Intraocular Lens in Model Eye

To compare the quality of images of gratings placed in a model eye viewed through an extended depth of focus (EDoF) intraocular lens (IOL) to that of diffractive bifocal IOL or monofocal IOL. Experimental laboratory investigation. Nondiffractive wavefront shaping EDoF (CNAET0, Alcon Laboratories), echelette-designed EDoF (ZXR00V, Johnson & Johnson Vision), diffractive bifocal IOL with low power addition (SV25T, Alcon Laboratories), or monofocal IOL (CNA0T0, Alcon Laboratories) was placed in a fluid-filled model eye. A United States Air Force Resolution Grating Target was glued to the posterior surface of the model eye and viewed through a flat or a wide-angle contact lens. The contrast of the gratings viewed through the EDoF or multifocal IOLs was compared to that through the monofocal IOL. A wavefront analyzer was used to measure the spherical power of the central 4.5 mm optics of the EDoF, multifocal, and monofocal IOLs. The distribution of the dioptric power and the dioptric power map were compared. The gratings observed through the flat contact lens with CNAET0, ZXR00V, or SV25T were slightly blurred when viewed through the multifocal optics. The blurred area was in the circumferential area of CNAET0, the central area of SV25T, and the peripheral area of ZXR00V. The mean contrast was 0.258 ± 0.020 for CNAET0, 0.227 ± 0.025 for ZXR00V, and 0.221 ± 0.020 for SV25T for the 16.0 cyc/mm grating. The contrast was significantly lower for ZXR00V (P = .004) and SV25T (P = .004) than 0.303 ± 0.015 for CNA0T0 but the differences were not significant for CNAET0. For the wide-angle contact lens, the contrast for CNAET0 was 0.182 ± 0.009, for ZXR00V was 0.162 ± 0.011, and for SV25T was 0.163 ± 0.007 for the 16.0 cyc/mm grating, and none was significantly different from 0.188 ± 0.012 for CNA0T0. The dioptric variations of CNAET0 indicated a ring-shaped area of higher power corresponding to the circumferential blurred zone observed through the flat contact lens. The wavefront shaping and echelette-designed EDoF-IOLs reduce the contrast of the grating more than the monofocal IOL when viewed through the flat contact lens. The degree of reduction depended on the design of the extended-focus optics. The difference was less through the wide-angle contact lens.

A CNN-LSTM model using elliptical constraints for temporally consistent sun position estimation

More accurate sun position estimation could transform the design and operation of solar power systems, weather forecasting services, and outdoor augmented reality systems. Although several image-based approaches to sun position estimation have been proposed, their performance is significantly affected by momentary disruptions in cloud cover because they use only a single image as input. This study proposes a deep learning-based sun position estimation system that leverages spatial, temporal, and geometric features to accurately regress sun positions even when the sun is partially or entirely occluded. In the proposed approach, spatial features are extracted from an input image sequence by applying a separate Resnet-based convolution network to each frame. To ensure that the temporal changes in the brightness distribution across frames are considered, the spatial features are concatenated and passed on to a stack of LSTM layers prior to regressing the final sun position. The proposed network is also trained with elliptical (geometric) constraints to ensure that predicted sun positions are consistent with the natural elliptical path of the sun in the sky. The proposed approach's performance was evaluated on the Sirta and Laval datasets along with a custom dataset, and an R2 Score of 0.98 was achieved, which is at least 0.1 higher than that of previous approaches. The proposed approach is capable of identifying the position of the sun even when occluded and was employed in a novel sky imaging system consisting of only a camera and fisheye lens in place of a complex array of sensors.

A photogrammetric approach for real‐time visual SLAM applied to an omnidirectional system

AbstractThe problem of sequential estimation of the exterior orientation of imaging sensors and the three‐dimensional environment reconstruction in real time is commonly known as visual simultaneous localisation and mapping (vSLAM). Omnidirectional optical sensors have been increasingly used in vSLAM solutions, mainly for providing a wider view of the scene, allowing the extraction of more features. However, dealing with unmodelled points in the hyperhemispherical field poses challenges, mainly due to the complex lens geometry entailed in the image formation process. To address these challenges, the use of rigorous photogrammetric models that appropriately handle the geometry of fisheye lens cameras can overcome these challenges. Thus, this study presents a real‐time vSLAM approach for omnidirectional systems adapting ORB‐SLAM with a rigorous projection model (equisolid‐angle). The implementation was conducted on the Nvidia Jetson TX2 board, and the approach was evaluated using hyperhemispherical images captured by a dual‐fisheye camera (Ricoh Theta S) embedded into a mobile backpack platform. The trajectory covered a distance of 140 m, with the approach demonstrating accuracy better than 0.12 m at the beginning and achieving metre‐level accuracy at the end of the trajectory. Additionally, we compared the performance of our proposed approach with a generic model for fisheye lens cameras.

Simple calibration of fisheye lenses for hemispherical photography of the forest canopy

Canopy hemispherical photography (HP) is a technology for vegetation surveying that usually employs consumer-oriented devices. Variations on those devices are caused by technological advances and design. These variations, despite providing the potential for more reliable surveying, make the measurements challenging. The knowledge about how to standardize devices output is available, what is missing is a simple and inexpensive methodology to accomplish it. This is needed since the use of consumer-oriented devices creates an expectation of simplicity and affordability. To fill this methodology gap, we developed an easy-to-assemble calibration board of near-zero material cost (simple board) and outlined how to use it to develop device-specific functions: the projection function (fp) and the vignetting function (fv). The former relates view zenith angle (θ) with distance from image center, while fv models the radial decreasing of brightness known as vignetting effect. Explanations are accompanied by tailored R code. Altogether, we call it “the simple method”. Three devices were used to test it: a Nikon Coolpix 5700 with a FC-E9 converter (D1), a Nikon D610 with a Nikkor 8 mm fisheye lens (D2), and an iPhone 6 Plus with an Olloclip fisheye converter (D3). The reference data was taken with D1, corrected for vignetting effect with a photometric sphere and reprojected to the standard equidistant projection with a fv developed by Delta-T Devices (www.delta-t.co.uk). The root mean square error (RMSE) of the simple method were 0.55° θ and 0.021 relative radiance (RR) for D1; 1.8° θ and 0.1 RR for D2; and 2.1° θ and 0.066 RR for D3. This level of standardization will help to simplify the classification of canopy gaps, a key processing step in HP. And, if it is broadly adopted, will facilitate comparisons between independent studies and large time series.

Design and Application of Liquid Silicone Rubber Light Guide in Compact Automotive Headlamps

Traditional car headlights use halogen or high-intensity discharge (HID) lamps paired with a reflector cup, fisheye lens, and shading plate to comply with ECE112B (Headlamps with an Asymmetrical Passing Beam) regulations. This design has issues such as large and bulky volumes, and it sacrifices optical efficiency due to the shading plate. This paper proposes a solution using an LED light source coupled with a Liquid Silicone Rubber-Light Guide (LSR-LG) for compact automotive low-beam headlights. By employing the principle of total internal reflection, the light beam is confined within the LSR-LG to reduce the volume of the car lamp. Simultaneously, optimized the structure and adjusted the light distribution. In the prototype measurements, the LED combined with the LSR-LG in the compact automotive headlamps module exhibited light intensities of 11,942 cd, 12,898 cd, and 298.2 cd at measurement points 75 R, 50 R, and BR, respectively, by ECE R112B standards. Simultaneously, the design effectively reduced the car lamp module’s volume to 90.63 mm × 164.53 mm × 73.37 mm. This design offers advantages such as energy efficiency, lightweight construction, and compact size. It also complies with ECE R112B regulations, showing promising potential as an excellent choice for the next generation of automotive low-beam headlights.

The Application of a Fisheye Lens in the Automatic Alignment of Wireless Optical Communication

To achieve a lightweight design and wide field of view for the automatic alignment system in wireless optical communication, in the receiving antenna—a fisheye lens is incorporated as the receiving optical system. This feature enables the detection and tracking of lasers. The system utilizes a micro motor as the control servo system and a four-quadrant detector as the detection unit. Sequential and non-sequential ray tracing techniques were used to simulate the analysis of the fish-eye lens. Point array diagrams, ray trace diagrams, and encircled energy analysis were utilized to evaluate the spot’s quality. The simulation results demonstrate that the fish-eye lens has a field of view of 120°, and the spot with 80% energy has a diameter smaller than 30 μm. The experimental results indicate that the fish-eye lens effectively captures the light beam within the range of ±30°. The simulated and experimental results for the sum and difference frequency amplitudes show good agreement. The outdoor experiments have demonstrated a tracking error of 22.757 μrad in this system. After alignment, the average output optical power of the detector is 3.23 μW, and the detected waveform amplitude is 12.48 mV. These findings demonstrate the system’s potential for automatic alignment in wireless optical communication. Additionally, the system is relatively simple and practical.

Analysis of the Photogrammetric Use of 360-Degree Cameras in Complex Heritage-Related Scenes: Case of the Necropolis of Qubbet el-Hawa (Aswan Egypt).

This study shows the results of the analysis of the photogrammetric use of 360-degree cameras in complex heritage-related scenes. The goal is to take advantage of the large field of view provided by these sensors and reduce the number of images used to cover the entire scene compared to those needed using conventional cameras. We also try to minimize problems derived from camera geometry and lens characteristics. In this regard, we used a multi-sensor camera composed of six fisheye lenses, applying photogrammetric procedures to several funerary structures. The methodology includes the analysis of several types of spherical images obtained using different stitching techniques and the comparison of the results of image orientation processes considering these images and the original fisheye images. Subsequently, we analyze the possible use of the fisheye images to model complex scenes by reducing the use of ground control points, thus minimizing the need to apply surveying techniques to determine their coordinates. In this regard, we applied distance constraints based on a previous extrinsic calibration of the camera, obtaining results similar to those obtained using a traditional schema based on points. The results have allowed us to determine the advantages and disadvantages of each type of image and configuration, providing several recommendations regarding their use in complex scenes.

Rotation of reference frame dependent polarimetric variables for equidistant fisheye lens projections.

The effect of a linear polarizer is conventionally well defined when viewed along the axis normal to its face. However, even for ideal linear polarizers, the non-normal viewing geometries attainable with wide-angle lenses require further considerations. A method to describe the polarization state of light passed through a linear polarizer and observed with an equidistant fisheye lens is described.

Bootstrap geometric ground calibration method for wide angle star sensors.

Wide angle star sensors are becoming more prevalent in aeronautics. A wide angle lens provides a greater field of view for star detection, but consequently incurs significant lens distortion. The effects of distortion complicate star identification, causing algorithms to fail or report false identifications. We address the issue of calibrating a wide angle star sensor without any specialized equipment, by analyzing two time-separated images captured from a static camera. An initial estimate of the focal length is obtained by observing the displacement of stars between the images. The focal length is subsequently used to build an initial estimate of camera intrinsics, and to identify stars in the image. A RANSAC-augmented Kabsch algorithm is implemented to determine camera orientation, while simultaneously removing false identifications. The identified stars are used to provide a precise estimate of camera focal length, before applying non-linear optimization in a radial search algorithm. The methodology was tested on two cameras, demonstrating the effectiveness of this algorithm in achieving a precise geometric calibration using real hardware, without any specialized calibration equipment.

FishSegSSL: A Semi-Supervised Semantic Segmentation Framework for Fish-Eye Images.

The application of large field-of-view (FoV) cameras equipped with fish-eye lenses brings notable advantages to various real-world computer vision applications, including autonomous driving. While deep learning has proven successful in conventional computer vision applications using regular perspective images, its potential in fish-eye camera contexts remains largely unexplored due to limited datasets for fully supervised learning. Semi-supervised learning comes as a potential solution to manage this challenge. In this study, we explore and benchmark two popular semi-supervised methods from the perspective image domain for fish-eye image segmentation. We further introduce FishSegSSL, a novel fish-eye image segmentation framework featuring three semi-supervised components: pseudo-label filtering, dynamic confidence thresholding, and robust strong augmentation. Evaluation on the WoodScape dataset, collected from vehicle-mounted fish-eye cameras, demonstrates that our proposed method enhances the model's performance by up to 10.49% over fully supervised methods using the same amount of labeled data. Our method also improves the existing image segmentation methods by 2.34%. To the best of our knowledge, this is the first work on semi-supervised semantic segmentation on fish-eye images. Additionally, we conduct a comprehensive ablation study and sensitivity analysis to showcase the efficacy of each proposed method in this research.

Wide-Angle Beam-Switching Antenna with Stable Gain Based on a Virtual Image Lens

Beam-switching antennas based on quasi-optical beamformers can provide cost-effective solutions for high-frequency communication applications. Here, we propose a wide-angle beam-switching planar lens antenna based on the recently presented virtual image lens. The antenna operates from 24 to 28 GHz and produces a beam that can be steered in a 100-degrees range in one plane with less than 2 dB simulated gain variation over the angular range and operational band. The performance of the presented antenna is similar to reported lens antennas with stable gain, but the proposed lens requires a smaller refractive index range to be realized, which alleviates the manufacturing.

Improvements in digital meteor spectra reduction

This study addresses the complexity and importance of developing a method of calibrating digital observations of meteor spectra with all-sky cameras. It aims to present novel approaches to spectral sensitivity, atmospheric extinction and flat-field corrections. Images of a known line emission spectrum were captured at various positions within the field of view using a camera with a fish-eye lens and plastic holographic grating. The flat-field correction was separated into a wavelength-independent and wavelength-dependent component, both dependent on the position of the spectral line in the field of view (FoV). Total profile intensities of spectra obtained from the images were compared throughout the spectral range at different positions in the FoV. The flat-field was constructed by fitting those dependencies with high-degree polynomial functions. Using a simplified atmospheric model, a novel approach was constructed to determine the atmospheric extinction curve throughout the spectral range, allowing it to be separately considered from the spectral sensitivity which was previously not the case. A comparison of the newly developed and previously used methodology was tested on several meteor spectra of the same meteor captured from different stations of the European Fireball Network. It revealed a significantly improved correspondence of the analysed spectra in the part of the spectral range unaffected by the limitations imposed by the newly developed methodology. Failing to follow the correct calibration methodology precisely may introduce varying degrees of uncertainty in computations of elemental abundances and other physical properties, depending on the equipment’s specific effect magnitude.

Cloud Identification and Reconstruction from All-sky Camera Images Based on Star Photometry Estimation

Cloud cover significantly influences ground-based optical astronomical observations, with nighttime astronomy often relying on visible light all-sky cameras for cloud detection. However, existing algorithms for processing all-sky cloud images typically require extensive manual intervention, posing challenges in identifying clouds with pronounced extinction characteristics. Furthermore, there is a lack of effective means for detailed visualization of cloud cover. To address these issues, this paper proposes a method that reconstructs the cloud distribution and thickness from all-sky images through star identification and photometry. Specifically, a high-precision star coordinate to the pixel position imaging model calibration method based on the star recognition for fisheye lenses is investigated, resulting in an all-sky rms error of less than 0.87 pixels. Based on the comprehensive reference star catalog, an optimized star extraction method based on SExtractor is developed to handle the difficulty of image source detection in all-sky cloud images. The optical thickness and distribution of cloud layers is calculated through star matching and extinction measurements. Finally, contingent upon the capability of camera and catalog star density, seven cloud layer reconstruction methods are proposed based on meshing and machine learning techniques, achieving a reconstruction accuracy of up to 1.°8. The processing results from real observed images indicate that the proposed method offers a straightforward calibration process and delivers excellent cloud cover extraction and reconstruction outcomes, thereby providing practical value in telescope dynamic scheduling, site characterization and the development of observation strategies.

Wide-angle passive beam steering using 3D modified partial Maxwell fisheye lens

This study presents a broadband, 3D gradient index beam-steering lens, derived from an optimized modification of the partial Maxwell fisheye (PMFE) design, achieving a boresight gain of 23 dBi, -80° to 80° beam steering, and <10 dB gain roll-off. Utilizing fused filament fabrication (FFF) to realize its intricate geometry, the design employs a novel polar space-filling curve (PSFC) to establish a 3D varying, effective permittivity distribution. Rigorous simulations and experimental validation attest to its effectiveness, marking the first 3D implementation of a PMFE-type lens to our knowledge. This research underscores the feasibility and diverse applications of a low-cost, wide-angle passive beam-steering dielectric lens.

Accreditation of the handheld 3-dimensional scanner and conventional photo images for area measurement.

Accurate assessment of wound areas is crucial in making therapeutic decisions, as the prognosis and changes in the size of the wound over time play a significant role. An ideal assessment method should possess qualities such as speed, affordability, accuracy, user-friendliness for both patients and healthcare professionals, and suitability for daily clinical practice. This study aims to introduce a handheld 3-dimensional (3D) scanner and evaluate its accuracy in measuring wound areas. Engineers from the Industrial Technology Research Institute in Taiwan developed a handheld 3D scanner with the intention of extending its application to the medical field. A project was conducted to validate the accuracy of this 3D scanner. We utilized a smartphone (Asus ZenFone 2 with a 13-million-pixel rear camera), a digital single-lens reflex digital camera (Nikon, D5000, Tokyo, Japan), and the 3D scanner to repeatedly measure square papers of known size that were affixed to the curved surface of life-size facial mask or medical teaching breast models. The "Image J" software was employed for 2-dimensional image measurements, while the "3D Edit" software was used to assess the "area of interest" on 3D objects. By using square papers with predetermined dimensions, the measurement-associated error rate (ER) could be calculated for each image. Three repeated measurements were performed using the "Image J" software for each square paper. The ERs of the 3D scan images were all below 3%, with an average ER of 1.64% in this study. The close-up mode of the smartphone exhibited the highest ER. It was observed that as the area increased, the ER also increased in the digital single-lens reflex camera group. The extension distortion effect caused by the wide-angle lens on the smartphone may increase the ER. However, the definition of a healthy skin edge may vary, and different algorithms for calculating the measurement area are employed in various 3D measurement software. Therefore, further validation of their accuracy for medical purposes is necessary. Effective communication with software engineers and discussions on meeting clinical requirements are crucial steps in enhancing the functionality of the 3D scanner.

Super-resolution imaging based on modified Maxwell's fish-eye lens

In this paper, a modified Maxwell's fish-eye lens is proposed in order to achieve super-resolution imaging. This lens possesses elevated refractive index profile compared with the traditional Maxwell's fish-eye lens. The refractive index profile is achieved with variable thickness configuration defined in a sheet plate structure, to realise desired changes in refractive indices. The wave propagation behaviours and the full width at half maximum (FWHM) are obtained from numerical simulations and experimental studies at the focal region of the lens. Super-resolution imaging is observed in a broadband frequency scope, with the FWHM around 0.2λ from 5 to 10 kHz. This work provides a straightforward and flexible approach to the engineering of the MFEL imaging characteristics and energy distributions for related applications.

Design of a novel wide-angle Rotman lens beamformer for 5G mmWave applications

The attachment of 5G with millimeter wave (mmWave) frequencies offers massive capacity and low latency to reveal the full 5G experiences. High directive gain and beamforming are considered essential for mmWave 5G systems. The main requirements of the beamforming network for 5G mmWave applications are the scanning coverage of ± 60° and SLL > 10 dB in a wide operational bandwidth over the standard 5G frequency bands. In this paper, a novel PCB-based wide-angle Rotman lens beamformer is designed, simulated, and successfully measured to meet the mentioned requirements for 5G mmWave applications. A comprehensive improved design methodology is provided for all components of the Rotman lens to reach a wide scanning angle, enhanced sidelobe level, and low scan loss. The end-fire Vivaldi antenna is selected as an array element for easy integration to the beamforming network as well as its capability to use in dual-polarization configuration. The proposed Rotman lens is operational in the 24–30 GHz frequency band covering 5G n257, n258, and n261 frequency bands. The results show a nearly constant 8 beams across the whole bandwidth steering from − 53° to 53° in 15° increments to provide ± 60° coverage with the SLL > 10 dB and scan loss < 1.9 dB. The retrieved novelties from this work contain an effective design methodology for an optimized Rotman lens with wide-scan angle and low phase and amplitude error, non-uniform distribution based array ports, and integration with end-fire antenna for possible dual polarization and 2-D beamforming capabilities. The comparison of the proposed beamformer with the most recent works shows several advantages in terms of integrated structure and performances including bandwidth, wide scanning angle, SLL, and scan loss. With such performances, this beamformer can be used for various mmWave and 5G applications such as advanced antenna systems, massive MIMO systems, and hybrid beamforming systems.

Key point calibrating and clustering for hard example mining of dense analogs in the fish-eye lens.

Recent years have witnessed widespread applications of the fish-eye lens with a wide field-of-view. However, its inherent distortion poses a big challenge to the intelligent recognition of dense analogs (IRDA) by convolutional neural networks (CNN). The major bottleneck of existing CNN models lies in their limited modeling capacity for distorted objects in fish-eye images, leading to the misclassification of hard examples. To further improve the accuracy of IRDA, we propose a novel key point calibrating and clustering (KPCC) algorithm based on the hemispherical projection model. Our method can effectively correct the hard example misclassification predicted by the CNN, significantly enhancing the performance of the IRDA. The experiments show that, as a light-weight computation calibrating and stable adaptive clustering method, the KPCC increases the precision and recall rate of IRDA on the intelligent retail dataset by 8.55% and 8.07%, respectively; compared with the classic Focalloss, QFocalloss, and OHEM (online hard example mining), it can mine hard examples more sufficiently, especially in the scene of distorted dense analog detection.

Ultrafast Excitation Exchange in a Maxwell Fish-Eye Lens.

The strong coupling of quantum emitters to a cavity mode has been of paramount importance in the development of quantum optics. Recently, also the strong coupling to more than a single mode of an electromagnetic resonator has drawn considerable interest. We investigate how this multimode strong coupling regime can be harnessed to coherently control quantum systems. Specifically, we demonstrate that a Maxwell fish-eye lens can be used to implement a pulsed excitation exchange between two distant quantum emitters. This periodic exchange is mediated by single-photon pulses and can be extended to a photon-exchange between two atomic ensembles, for which the coupling strength is enhanced collectively.