Light Plane Research Articles

Research on Calibration Method of Line-Structured Light Based on Multiple Geometric Constraints

Line-structured light calibration is an important step in the process of the visual measurement of structured light, and in the calibration process, the light plane equation is fitted by the camera coordinates of the data points on the light plane. There is error in solving the spatial coordinates of the data points, and the existing methods generally cannot effectively find the deviation points in the fitted data points, which leads to deviation when solving the parameters of the light plane equation. To correct the error caused by the deviation point in the calibration, this paper established a calibration method based on geometric constraints. In the method, the first constraint was that all fit points involved in the fitting had a certain consistency, and a part of the error points could be filtered by the distance from the data point to the light plane. In order to further improve the calibration accuracy, the laser was moved on a linear guide, the optimized objective function was established by the second geometric constraint, and the constraint was the relationship that the light plane at each position was parallel. In the experiment, a high-precision gauge block was used as the measured object, and the measurement results could be obtained. The calibration method can improve the calibration accuracy according to the experimental results.

Precise measurement for line structure light vision sensor with large range

High precision and large measurement range are the target of any one three-dimensional scanner. For a line structure light vision sensor, measurement precision depends on its calibration results, i.e., determining mathematical expression of the light plane in camera coordinate system. However, as calibration results are locally optimal solutions, high precise measurement in a large range is difficult. In this paper, we give a precise measurement method and the corresponding calibration procedure for a line structure light vision sensor with a large measurement range. A motorized linear translation stages with a travel range of 150 mm and a planar target which is a surface plate with a machining precision of 0.05 mm are utilized. With the help of the linear translation stage and the planar target, functions which gives the relationship between center point of the laser stripe and the perpendicular/ horizontal distance are obtained. Once image of light stripe is captured, we can get a precise measurement result from the normalized feature points. Compared with a traditional measurement method, distortion compensation is not necessary and precision of measurement is improved significantly. Experiments show that root mean square error of measurement results according to our proposed method is reduced by 64.67% related to the traditional method.

Image Stitching Techniques Applied to Plane or 3-D Models: A Review

Image stitching is a technique in which multiple overlapping images of the scene are stitched together to generate an image with wide view and high resolution. Image stitching methods can be broadly classified into feature-based methods and deep learning methods. Feature-based methods use manually designed features to establish transformation relationships between multiple images. This technology has played an important role in medical, industrial, military, and other fields. With the rise of deep learning in computer vision, it has also become the mainstream method in the field of image stitching. This paper provides a systematic literature review of image stitching techniques applied on the plane and 3D models for both feature-based and deep learning methods. We divide the stitching methods into two categories, namely mosaic stitching methods for generating stitched plane images and panoramic stitching methods for generating stitched panoramic images. Based on the camera type, it is further divided into pinhole camera plane stitching methods, pinhole camera panoramic stitching methods, fisheye camera panoramic stitching methods, and light field camera plane stitching methods. Extensive search was conducted in ICIP, TIP, ICCV, ECCV, CVPR, BMVC, ICPR, IJCV, AIM, ITS, TPAMI, SIGGRAFH to summarize related image stitching techniques. 89 articles are selected for systematic literature review and are presented in the table. The objective of this systematic literature review is to provide detailed technical progress in image stitching techniques and to identify the research gap in this field.

Linear photogalvanic effect in surface states of topological insulators

The theory of the linear photogalvanic effect is developed for direct optical transitions between surface states of three-dimensional topological insulators. The photocurrent governed by the orientation of the polarization plane of light and caused by the warping of the energy dispersion of two-dimensional carriers is calculated. It is shown that both the shift contribution caused by coordinate shifts of the particle wave packets during the optical transitions and the ballistic contribution caused by interference of the optical absorption and scattering by disorder have generally the same order of magnitude. The ballistic contribution is present owing to electron-hole asymmetry of topological surface states and has a frequency dependence in contrast to the shift photocurrent. In the nonlinear-in-the-light-intensity regime appearing due to saturation of the direct optical transitions, the ballistic contribution dominates. The nontrivial dependence of the photocurrent on the polarization plane orientation in the nonlinear regime appears solely due to the ballistic contribution. Our findings allow separating the ballistic and shift contributions to the Linear photogalvanic current in experiments.

An unusual case of delayed recovery from spinal anesthesia in a patient with Lambert-Eaton myasthenic syndrome—a case report

BackgroundThe Lambert-Eaton myasthenic syndrome (LEMS) is a rare disorder characterized by proximal muscle weakness and autonomic dysfunction due to involvement of the neuromuscular junction A case of delayed unilateral recovery from spinal anesthesia in a patient with LEMS undergoing open cholecystectomy is described.Case presentationA 42-year-old lady, on treatment for LEMS with pyridostigmine, prednisolone, and azathioprine presented with cholelithiasis and was scheduled for open cholecystectomy. Neuraxial anesthesia combined with a light plane of general anesthesia provided adequate muscle relaxation without the use of neuromuscular blocking drugs. In spite of perioperative continuation of anticholinesterases, the patient required 12 h post-surgery for complete bilateral motor recovery from spinal anesthesia.ConclusionsA combination of spinal anesthesia with a light plane of general anesthesia appears to be a safe anesthetic technique for managing patients with Lambert-Eaton myasthenic syndrome undergoing open cholecystectomy. However, neurologically asymptomatic patients with good treatment compliance may also show an unusual delay in recovery from neuraxial anesthesia.

High magnetic, transport, and optical uniaxial anisotropies generated by controlled directionally grown nano-sheets in Fe thin films

Fe films with thicknesses between 17 and 95 nm were grown with a nano-sheet morphology, which enabled their high uniaxial magnetic, transport, and optical in-plane anisotropies. The top edge of the nano-sheets was directly visualized as nano-string-like structures of approximately 12.5–14 nm width and 100–300 nm length. The hysteresis loops showed a clear easy direction of magnetization in the longitudinal direction of the nano-sheets, whereas the hard direction loops were anhysteretic, with no remanence and zero coercive field. The anisotropy field exhibited values between 70 and 111 kA/m depending on the thickness of the films, with the maximum value corresponding to a 34 nm thick sample. The resistance of the films was also found to be highly anisotropic. The ratio (R⊥–R||)/R|| was ≈86%, with R|| and R⊥ being the resistances in the parallel and perpendicular directions of the nano-sheets, respectively. Likewise, the reflectivity of the samples behaved anisotropically; the ratio (IReflmax–IReflmin)/IReflmax of the intensity of reflected light by the films reached up to 61% for 34 nm thick samples, achieving the maximum value, IReflmax, when the plane of the incident light coincided with the direction of the nano-sheets and the minimum, IReflmin, when this plane was perpendicular to the direction of the nano-sheets. The origin of these anisotropic behaviors was established. These anisotropic films with high magnetization and high uniaxial anisotropies at the nanoscale can be useful for microelectronics applications, for devices such as magnetic sensors and transducers, or for ultrahigh frequency inductors.

Stereo Jones Matrix Holography with Longitudinal Polarization Transformation

AbstractMetasurfaces have exhibited powerful abilities for manipulating multiple fundamental properties of light including amplitude, phase, polarization, and so on. However, these strategies are commonly concentrated on the modulations at a single transverse plane of output light. The spatial evolutions of these properties, especially the polarizations along longitudinal direction, are rarely investigated. Here, a stereo Jones matrix holography method is presented for understanding the spatial evolution including polarization, amplitude, and phase variations along the longitudinal direction. Stereo holographic algorithms in matrix framework are developed to generate multiplane and even continuously varied vectorial holographic images that exhibit distinct polarization states at each transverse plane. This method provides a benchmark of longitudinal polarization transformations as well as beam modulations by simply using a single planar metasurface without extra burdens on optical path. In addition, the obtained propagation‐dependent features can favor the realizing of on‐demand transverse and longitudinal spatial evolution from the perspective of the holographic method. Furthermore, it may also promote the development of related areas including polarization‐switchable devices, optical trapping, microscopy, laser processing, etc.

Calibration method of line-structured light sensors based on a hinge-connected target with arbitrary pinch angles.

Line-structured light 3D measurement is often used for 3D contour reconstruction of objects in complex industrial environments, where light plane calibration is a key step. In this paper, we propose a calibration method for a line-structured optical system based on a hinge-connected double-checkerboards stereo target. First, the target is moved randomly in multiple positions at any angle within the camera measurement space. Then, by acquiring any one image of the target with line-structured light, the 3D coordinates of the light stripes feature points are solved with the help of the external parameter matrix of the target plane and the camera coordinate system. Finally, the coordinate point cloud is denoised and used to quadratically fit the light plane. Compared with the traditional line-structured measurement system, the proposed method can acquire two calibration images at once; thus, only one image of line-structured light is needed to complete the light plane calibration. There is no strict requirement for the target pinch angle and placement, which improve system calibration speed with high accuracy. The experimental results show that the maximum RMS error of this method is 0.075mm, and the operation is simpler and more effective to meet the technical requirements of industrial 3D measurement.

Flexible and fast calibration method for uni-directional multi-line structured light system

Multi-line Shift Profilometry (MLSP) is widely used to measure the high-reflectivity object by projecting high-contrast binary stripe image. However, in the calibration of a MLSP system, the lens distortion of the projector makes the most existing calibration methods fail in obtaining high accurate result. Therefore, to avoid the influence of lens distortion, we propose a novel flexible and fast calibration method for MLSP, which requiring only uni-directional patterns. In this method, for each bending light plane, lens distortion is avoided by establishing a polynomial model between three-dimensional (3D) coordinates and camera sub-pixel coordinates. Because the position of calibration planar targets is placed freely without any constraint, this method is flexible and convenient to operate. Experimental results demonstrate that the proposed method achieves high-precision 3D reconstruction.

Large-scale calibration method for MEMS-based projector 3D reconstruction.

Projectors based on Micro-Electro-Mechanical System (MEMS) have the advantages of small size and low cost. Moreover, uniaxial MEMS projectors have high projection accuracy, and have been widely used in structured light 3D reconstruction. However, the existing calibration methods for uniaxial MEMS projectors are not effective in large-scale scenes. To solve this problem, this paper proposes a novel efficient large-scale calibration method, which is easily implemented. The proposed method first calibrates a partial light plane for a fixed sampling period, then obtains the rest of the light plane by exploiting a non-fixed rotating shaft linear interpolation method. Experimental results verify that the proposed method attains high accuracy in a large depth field with only 11 sets of calibration data. Specifically, at a distance of 3000mm, the standard deviation of the plane fitting error reaches 0.2584mm on the standard plane, and the measurement accuracy attains 0.9124mm on the standard step object with 200mm interval.

High-accuracy calibration method for an underwater one-mirror galvanometric laser scanner.

Three-dimensional (3D) perception of deep-sea targets is the key to autonomous operation of underwater equipment (e.g., underwater robots). Underwater one-mirror galvanometric line-laser scanner has advantages for short-range measurement, but it is difficult to achieve high calibration accuracy due to installation errors and refraction effects. For this reason, in this paper, a high-accuracy refraction-considered and installation-error-independent calibration method is proposed for the vision system. Firstly, to address the difficulty of aligning the incident light plane with the galvanometer shaft, a high-accuracy land-based installation-error-independent model is proposed, which avoids the influence of the installation errors and allows the real shaft axis and the light-plane cluster poses to be calculated using only three light planes. Subsequently, considering the underwater refraction, a 3D model is established for simulating refractive behaviors of the light-plane cluster, and then a partition-based method is proposed for calibrating the underwater light-plane cluster, which further improves the calibration accuracy of the scanner in underwater measurement scenarios. Finally, a one-mirror galvanometric laser scanner is developed in the laboratory to verify the calibration accuracy and to perform the 3D measurement experiments of underwater targets. The results show that the calibration accuracy of the proposed land-based installation-error-independent model is improved 2 times more compared with the traditional installation-error-dependent model. Additionally, the measurement accuracy of the scanner for the standard sphere is 11.98 µm and 12.75 µm in the air and underwater measurement scenarios, and the two measurements are in good agreement. The above results comprehensively verify the high accuracy of the calibration method proposed in this paper.

Dithered single-objective light sheet for improved super-resolution imaging throughout mammalian cells.

Single-molecule super-resolution fluorescence microscopy is a powerful method for imaging detailed biological structures at the nanoscale. However, imaging single molecules precisely in thick samples such as mammalian cells is notoriously difficult due to increased fluorescence background. Light sheet fluorescence microscopy in which the sample is illuminated with a thin plane of light mitigates the problem of increased fluorescence background in thick samples and results in increased signal-to-background ratio and improved localization precision. However, light sheet illumination is sensitive to shadowing artifacts from imperfections in the optical path and scattering throughout the sample. Additionally, most light sheet systems employ two objectives, which may suffer from steric hindrance and drift between the sample and illumination objective. To overcome these disadvantages, we have developed a single-objective tilted light sheet illumination system which employs dithering to reduce stripe artifacts and achieve homogeneous illumination of the sample. Our light sheet has been designed with a width, thickness, and confocal parameter specifically suited for cellular imaging. We further improve the imaging capabilities of our system with a microfluidic device enabling Exchange-PAINT for multi-target imaging without chromatic offsets, using engineered point spread functions for 3D imaging, and employing deep learning algorithms for high-density localization of single molecules. Our approach achieves improved localization precision, imaging speeds, and multi-target accuracy, enabling fast and precise multi-target 3D single-molecule super-resolution cellular imaging. Here, we focus on the design and construction of our single-objective dithered light sheet and its range of applications from nuclear protein investigation to whole cell imaging. We think the improved cellular imaging resulting from this system will impact our understanding of the relationship between organization and function of subcellular structures and help us and others answer relevant biomedical questions involved in disease pathogenesis.

Petrographic Microscopy with Ray Tracing and Segmentation from Multi-Angle Polarisation Whole-Slide Images

‘Slide scanners’ are rapid optical microscopes equipped with automated and accurate x-y travel stages with virtual z-motion that cannot be rotated. In biomedical microscopic imaging, they are widely deployed to generate whole-slide images (WSI) of tissue samples in various modes of illumination. The availability of WSI has motivated the development of instrument-agnostic advanced image analysis software, helping drug development, pathology, and many other areas of research. Slide scanners are now being modified to enable polarised petrographic microscopy by simulating stage rotation with the acquisition of multiple rotation angles of the polariser–analyser pair for observing randomly oriented anisotropic materials. Here we report on the calibration strategy of one repurposed slide scanner and describe a pilot image analysis pipeline designed to introduce the wider audience to the complexity of performing computer-assisted feature recognition on mineral groups. The repurposed biological scanner produces transmitted light plane- and cross-polarised (TL-PPL and XPL) and unpolarised reflected light (RL) WSI from polished thin sections or slim epoxy mounts at various magnifications, yielding pixel dimensions from ca. 2.7 × 2.7 to 0.14 × 0.14 µm. A data tree of 14 WSI is regularly obtained, containing two RL and six of each PPL and XPL WSI (at 18° rotation increments). This pyramidal image stack is stitched and built into a local server database simultaneously with acquisition. The pyramids (multi-resolution ‘cubes’) can be viewed with freeware locally deployed for teaching petrography and collaborative research. The main progress reported here concerns image analysis with a pilot open-source software pipeline enabling semantic segmentation on petrographic imagery. For this purpose, all WSI are post-processed and aligned to a ‘fixed’ reflective surface (RL), and the PPL and XPL stacks are then summarised in one image, each with ray tracing that describes visible light reflection, absorption, and O- and E-wave interference phenomena. The maximum red-green-blue values were found to best overcome the limitation of refractive index anisotropy for segmentation based on pixel-neighbouring feature maps. This strongly reduces the variation in dichroism in PPL and interference colour in XPL. The synthetic ray trace WSI is then combined with one RL to estimate modal mineralogy with multi-scale algorithms originally designed for object-based cell segmentation in pathological tissues. This requires generating a small number of polygonal expert annotations that inform a training dataset, enabling on-the-fly machine learning classification into mineral classes. The accuracy of the approach was tested by comparison with modal mineralogy obtained by energy-dispersive spectroscopy scanning electron microscopy (SEM-EDX) for a suite of rocks of simple mineralogy (granulites and peridotite). The strengths and limitations of the pixel-based classification approach are described, and phenomena from sample preparation imperfections to semantic segmentation artefacts around fine-grained minerals and/or of indiscriminate optical properties are discussed. Finally, we provide an outlook on image analysis strategies that will improve the status quo by using the first-pass mineralogy identification from optical WSI to generate a location grid to obtain targeted chemical data (e.g., by SEM-EDX) and by considering the rock texture.

Decision Method of Optimal Needle Insertion Angle for Dorsal Hand Intravenous Robot.

In the context of COVID-19, the research on various aspects of the venipuncture robot field has become increasingly hot, but there has been little research on robotic needle insertion angles, primarily performed at a rough angle. This will increase the rate of puncture failure. Furthermore, there is sometimes significant pain due to the patients' differences. This paper investigates the optimal needle entry angle decision for a dorsal hand intravenous injection robot. The dorsal plane of the hand was obtained by a linear structured light scan, which was used as a basis for calculating the needle entry angle. Simulation experiments were also designed to determine the optimal needle entry angle. Firstly, the linear structured optical system was calibrated and optimized, and the error function was constructed and solved iteratively by the optimization method to eliminate measurement error. Besides, the dorsal hand was scanned to obtain the spatial point clouds of the needle entry area, and the least squares method was used to fit it to obtain the dorsal hand plane. Then, the needle entry angle was calculated based on the needle entry area plane. Finally, the changes in the penetration force under different needle entry angles were analyzed to determine the optimal needle insertion angle. According to the experimental results, the average error of the optimized structured light plane position was about 0.1 mm, which meets the needs of the project, and a large angle should be properly selected for needle insertion during the intravenous injection.

A Three-Dimensional Structured Light Vision System by Using a Combination of Single-Line and Three-Line Lasers

A multi-line structured light measurement method that combines a single-line and a three-line laser, in which precision sliding rails and displacement measurement equipment are not required, is proposed in this paper. During the measurement, the single-line structured light projects onto the surface of an object and the three-line structured light remains fixed. The single-line laser is moved and intersects with the three-line laser to form three intersection points. The single-line light plane can be solved using the camera coordinates of three intersection points, thus completing the real-time calibration of the scanned light plane. The single-line laser can be scanned at any angle to determine the overall complete three-dimensional (3D) shape of the object during the process. Experimental results show that this method overcomes the difficulty of obtaining information about certain angles and locations and can effectively recover the 3D shape of the object. The measurement system's repetition error is under 0.16 mm, which is sufficient to measure the 3D shapes of complicated workpieces.

Polarization effects on the fluorescence emission of zebrafish neurons using light-sheet microscopy.

Light-sheet fluorescence microscopy (LSFM) makes use of a thin plane of light to optically section and image transparent tissues or organisms in vivo, which has the advantages of fast imaging speed and low phototoxicity. In this paper, we have employed light-sheet microscopy to investigate the polarization effects on fluorescence emission of zebrafish neurons via modifying the electric oscillation orientation of the excitation light. The intensity of the fluorescence emission from the excited zebrafish larvae follows a cosine square function with respect to the polarization state of the excitation light and reveals a 40% higher fluorescence emission when the polarization orientation is orthogonal to the illumination and detection axes. Through registration and subtraction of fluorescence images under different polarization states, we have demonstrated that most of the enhanced fluorescence signals are from the neuronal cells rather than the extracellular substance. This provides us a way to distinguish the cell boundaries and observe the organism structures with improved contrast and resolution.

Single-Shot Three-Dimensional Reconstruction Using Grid Pattern-Based Structured-Light Vision Method

Structured-light vision methods are widely employed for three-dimensional reconstruction. As a typical structured light pattern, grid pattern is extensively applied in single-shot three-dimensional reconstruction. The uniqueness of the grid feature retrieval is critical to the reconstruction. Most methods using grid pattern utilize the epipolar constraint to retrieve the correspondence. However, the low calibration accuracy of the camera–projector stereo system may impact the correspondence retrieval. An approach using grid pattern-based structured-light vision method is proposed. The grid pattern-based structured-light model was combined with the camera model and the multiple light plane equations. An effective extraction method of the grid stripe features was investigated. The system calibration strategy, based on coplanar constraint, is presented. The experimental setup consisted of a camera and an LED projector. Experiments were carried out to verify the accuracy of the proposed method.

Photoinduced Split of the Cavity Mode in Photonic Crystals Based on Porous Silicon Filled with Photochromic Azobenzene-Containing Substances

Phototunable photonic structures based on electrochemically etched silicon filled with four photochromic azobenzene-containing compounds, a bent-shaped low-molar-mass substance and side-chain polymethacrylates and a copolyacrylate, were prepared, and their photo-optical properties were studied. It was found that irradiation of these composites with polarized blue light results in spectral changes in the photonic band gap (split of the cavity mode) associated with cooperative photo-orientation of azobenzene moieties inside silicon pores in the direction perpendicular to the polarization plane of the incident light. Kinetics of the photoinduced split is studied. The observed photoinduced split is completely reversible, and heating of the composites to temperatures above isotropization or glass transitions fully recovers the initial spectral shape of the photonic band gap. Thermal and temporal stabilities of the obtained photoinduced split were comparatively studied, and it was found that for composites with a bent-shaped substance and polymethacrylate shape of the reflectance spectra do not change over time at room temperature. The prepared composites have a high potential for the different applications in photonics.

A method to reduce the systematic error of line-structured light sensors based on light plane correction

Line-structured light sensors are widely used in 3-dimentional (3D) reconstruction, however, there are few studies on the systematic error and the correction method. To further improve the measurement accuracy, this paper establishes the systematic error model of the line-structured light sensor, and the distribution function of the systematic error caused by the inaccurate light plane equation is derived. Then, the measurement method of the systematic error is proposed, the distance error and the orientation error of the light plane equation are corrected in two steps according to the distribution characteristics of the systematic error. The experimental results show that the systematic error of the sensor is reduced by at least 80% after correction. This method effectively reduces the systematic error of the line-structured light sensor and has good robustness.

Vision occlusion solution for line-structured light measurement system based on a plane mirror

To solve the problem of the measurement information loss caused by occlusion of the field of view in line-structured light measurement, a vision occlusion solution of the line-structured light measurement system based on a plane mirror is proposed in this work. In the system, a symmetrical structure between real and virtual images was created by the mirror. A checkerboard with a light stripe and its mirror image were captured by the camera in an image. The centers of the light stripe and its mirror image were corrected by the vanishing point constraint, and a calibration method of the free plane target was used to calibrate the real and virtual light planes. The real and virtual point clouds of the object were obtained by scanning with a linear guide. According to the ideal reflection model of the mirror, the real and virtual point clouds of the object were transformed into a common coordinate system, and an iterative closest point (ICP) algorithm was used to realize fine registration. As a result, the occluded parts can be compensated for each other. Compared with a standard ball, the mean/maximum absolute error of the measurement was 0.2553/0.3276 mm. The experimental results revealed that the proposed method can obtain the complete surface of the object, thus effectively solving the occlusion problem in the line-structured light measurement, and reducing the loss of the measurement information, which can meet the industrial measurement requirements.