Real-time Optical Processing Research Articles

High-frame-rate reconfigurable diffractive neural network based on superpixels.

The existing implementations of reconfigurable diffractive neural networks rely on both a liquid-crystal spatial light modulator and a digital micromirror device, which results in complexity in the alignment of the optical system and a constrained computational speed. Here, we propose a superpixel diffractive neural network that leverages solely a digital micromirror device to control the neuron bias and connection. This approach considerably simplifies the optical system and achieves a computational speed of 326 Hz per neural layer. We validate our method through experiments in digit classification, achieving an accuracy of 82.6%, and action recognition, attaining a perfect accuracy of 100%. Our findings demonstrate the effectiveness of the superpixel diffractive neural network in simplifying the optical system and enhancing computational speed, opening up new possibilities for real-time optical information processing applications.

Time scrambling and decoy-state quantum key distribution for optical physical layer encryption

We propose an optical physical layer encryption method based on real-time optical signal temporal scrambling and decoy-state quantum key distribution. The real-time optical signal processing is operating with an array of tunable Fabry-Perot cavities by using the updating and synchronization keys. Based on this scheme, an experimental demonstration is given to realize secure communication for optical physical layer in a standard 10 Gbps data transmission with quantum key distribution in real time. An on-line transmitted data over 100 km of a standard single mode fiber is realized with power penalty of as low as 3.45 dB, where the average key generation rate is better than 800 bps. The experimental results indicate that, the proposed encryption scheme of dynamic scrambling with quantum keys exhibits significantly potential for improving the confidentiality of optical physical layer.

Real-time optical flow processing on embedded GPU: an hardware-aware algorithm to implementation strategy

Determining the optical flow of a video is a compute-intensive task essential for computer vision. For achieving this processing in real time, the whole algorithm deployment chain must be thought of for efficiency first. The development is usually divided into two parts: first, designing an algorithm that meets precision constraints, then, implementing and optimizing its execution on the targeted platform. We argue that unifying those operations enhances performance on the embedded processor. This paper is based on an industrial use case of computer vision. The objective is to determine dense optical flow in real time on an embedded GPU platform: the Nvidia AGX Xavier. The CLG (combined local–global) optical flow method, initially chosen, is analyzed to understand the convergence speed of its underlying optimization problem. The Jacobi solver is selected for implementation because of its parallel nature. The whole multi-level processing is then ported to the GPU, using several specific optimization strategies. In particular, we analyze the impact of fusing the solver’s iterations with the roofline model. As a result, with a 30 W power budget, our implementation runs at 60FPS, on $$640 \times 512$$ images, with a four-level processing. Hopefully, this example should provide feedback on the issues that arise when trying to port a method to a parallel platform and serve for further implementations of computer vision algorithms on specialized hardware.

Parallel Analog Computing Based on a 2×2 Multiple-Input Multiple-Output Metasurface Processor With Asymmetric Response

We present a polarization-insensitive metasurface processor to perform spatial asymmetric filtering of an incident optical beam, thereby allowing for real-time parallel optical processing. To enable massive parallel processing, we introduce a novel Multi Input-Multi Output (MIMO) computational metasurface with an asymmetric optical response that can perform spatial differentiation on two distinct input signals regardless of their polarization. In our scenario, two distinct signals set in x and y directions, parallel and perpendicular to the incident plane, illuminate simultaneously the metasurface processor, and the resulting differentiated signals are separated from each other via appropriate Spatial Low Pass Filters (SLPF). By leveraging Generalized Sheet Transition Conditions (GSTCs) and surface susceptibility tensors, we design an asymmetric meta-atom augmented with normal susceptibilitiesto reach asymmetric optical response at normal beam illumination. Proof-of-principle simulations are also reported along with the successful realization of signal processing functions. The proposed metasurface overcomes major shortcomings imposed by previous studies such as large architectures arising from the need of additional subblocks, slow responses, and most importantly, supporting only a single input with a given polarization. Our results set the path for future developments of material-based analog computing using efficient and easy-to-fabricate MIMO processors for compact, fast, and integrable computing elements without any Fourier lens.

Research on multi-targets real-time ballistic recognition method based on movement multiple feature correlation

In the optical measurement of burst multi-targets such as the rocket, missile, bullet and so on, in order to solve the poor real-time reliability problem of the multi-targets ballistic recognition caused by the invalid ballistic recognition algorithm variable and less movement feature quantity, a multi-targets ballistic recognition method based on movement multiple feature correlation has been proposed, and the mathematical model algorithm and multi-targets real-time ballistic recognition strategy has been established at the same time. It has been proved that the method can improve the adaptability, real-time performance and reliability of multi-targets real-time ballistic recognition, to provide technical support to real-time multi-targets optical measurement and processing.

A Novel Wireless Mobile Platform to Locate and Gather Data From Optical Fiber SensorsIntegrated Into a WSN

This paper presents a novel design for a wireless mobile platform to locate and gather data from different types of optical fiber sensors, thereby enabling the more effective integration of a number of such optical fiber sensors with an advanced mobile wireless sensor network (WSN). This then more readily permits potential applications, such as monitoring in remote and harsh environments and tracking, exploiting fully the combined advantages offered both by the mobile WSN and the advanced optical fiber sensing technologies. The platform which was designed and implemented consists of an optical fiber sensor module and a smart mobile WSN module, which shows important advantages for mobile sensing and tracking and mesh networking. In this paper, a fiber Bragg grating-based temperature sensor and an intrinsic pH optical fiber sensor were specially designed and integrated successfully into the optical fiber sensor module as an exemplar to investigate the performance of the integrated system based on the mobile WSN platform. The positive experimental results obtained have confirmed the functionality of the platform designed and demonstrated its capacity for real-time optical fiber sensor data monitoring, processing, and wireless transmission. The successful creation of this type of wireless mobile platform with optical fiber sensors can thus be expected to make an important impact on a number of sectors, where either conventional optical sensor designs or WSNs alone cannot meet the systems requirements.

Wireless Sensor Network Platform for Intrinsic Optical Fiber pH Sensors

This paper explores the potential of creating a hybrid wireless sensor network (WSN) through the establishment of a generic WSN platform, which allows for the effective integration of a number of novel optical fiber (wired) sensors. The aim is to maximize the potential of a sensor system created using advanced optical sensor techniques coupled to the WSNs. In this design, each wireless mote in a WSN is configured as a generic platform for ease of integration of the optical fiber sensor modules. In the work carried out, a typical configuration is used comprising a LED light source, a filter, a photodiode, and an amplifying circuit. Thus, each mote is able to record and process the signals generated from the optical fiber sensors and transmit the data obtained on the measurements made wirelessly to a base station. To verify both the hybrid concept and the effectiveness of a generic platform established, the system is tested using an intrinsic pH optical fiber sensor developed by some of the authors and especially chosen to be integrated into the platform. When the sensor is subjected to a range of solutions with varying pH values from 1 to 9, the sensor data generated from the system are collected at the base station. The experimental results obtained confirm the functionality of the platform designed, and its ability to allow real-time optical fiber sensor data monitoring, processing, and transmission. The success of the creation of this type of hybrid WSN platforms thus opens up new sensing opportunities using a wide range of optical fiber sensors, thus showing a potential impact on some measurement applications where conventional WSNs cannot meet the systems requirements.

A real-time optical tracking and measurement processing system for flying targets.

Optical tracking and measurement for flying targets is unlike the close range photography under a controllable observation environment, which brings extreme conditions like diverse target changes as a result of high maneuver ability and long cruising range. This paper first designed and realized a distributed image interpretation and measurement processing system to achieve resource centralized management, multisite simultaneous interpretation and adaptive estimation algorithm selection; then proposed a real-time interpretation method which contains automatic foreground detection, online target tracking, multiple features location, and human guidance. An experiment is carried out at performance and efficiency evaluation of the method by semisynthetic video. The system can be used in the field of aerospace tests like target analysis including dynamic parameter, transient states, and optical physics characteristics, with security control.

不同波长光源照明的水下成像及光学图像实时处理

不同波长光源照明的水下成像及光学图像实时处理

Digital Micromirror Devices: Principles and Applications in Imaging

A digital micromirror device (DMD) is an array of individually switchable mirrors that can be used in many advanced optical systems as a rapid spatial light modulator. With a DMD, several implementations of confocal microscopy, hyperspectral imaging, and fluorescence lifetime imaging can be realized. The DMD can also be used as a real-time optical processor for applications such as the programmable array microscope and compressive sensing. Advantages and disadvantages of the DMD for these applications as well as methods to overcome some of the limitations will be discussed in this article. Practical considerations when designing with the DMD and sample optical layouts of a completely DMD-based imaging system and one in which acousto-optic deflectors (AODs) are used in the illumination pathway are also provided.

Highly Efficient Photorefractive Organic Polymers Based on Benzonitrile Shiff Bases Nonlinear Chromophores

Photorefractive (PR) materials are ideal for potential applications in real-time optical processing. We report new highly efficient organic photorefractive polymers based on the nonlinear chromopho...

Speckle-illuminated fluorescence confocal microscopy, using a digital micro-mirror device

An implementation of a speckle-illuminated fluorescence confocal microscope using a digital micro-mirror device (DMD) is described. The DMD not only projects a sequence of imaged binary speckle patterns onto the specimen at a very high frame rate but also operates as a spatial light modulator to perform real-time optical data processing. Frame averaging is accomplished by CCD charge accumulation during a single exposure. The recorded time-averaged image is a confocal image plus an unwanted non-confocal image which can be removed by recording a separate image. Experimental results with image acquisition within a fraction of a second are shown. Images of a thin biological sample are also shown to demonstrate practical application of the technique.

Laser frequency stabilization at 1.5 microns using ultranarrow inhomogeneous absorption profiles in Er 3+:LiYF 4

Single-frequency diode lasers have been frequency stabilized to 1.5 kHz Allan deviation over 0.05–50 s integration times, with laser frequency drift reduced to less than 1.4 kHz/min, using the frequency reference provided by an ultranarrow inhomogeneously broadened Er 3+: 4I 15/2→ 4I 13/2 optical absorption transition at a vacuum wavelength of 1530.40 nm in a low-strain LiYF 4 crystal. The 130 MHz full-width at half-maximum (FWHM) inhomogeneous line width of this reference transition is the narrowest reported for a solid at 1.5 μm. Strain-induced inhomogeneous broadening was reduced by using the single isotope 7Li and by the very similar radii of Er 3+ and the Y 3+ ions for which it substitutes. To show the practicability of cryogen-free cooling, this laser stability was obtained with the reference crystal at 5 K; moreover, this performance did not require vibrational isolation of either the laser or crystal frequency reference. Stabilization is feasible up to T=25 K where the Er 3+ absorption thermally broadens to ∼500 MHz. This stabilized laser system provides a tool for interferometry, high-resolution spectroscopy, real-time optical signal processing based on spatial spectral holography and accumulated photon echoes, secondary frequency standards, and other applications such as quantum information science requiring narrow-band light sources or coherent detection.

OPTICAL FLOW NAVIGATION FOR AN OUTDOOR UAV USING A WIDE ANGLE MONO CAMERA AND DEM MATCHING

Visual navigation for low altitude unmanned aerial vehicles (UAV) shows many scientific and technical challenges in terms of navigation performances, complex 3D environment, high computational requirements for real-time image processing and very limited onboard mass and power resources.The current paper presents a concept of a visual navigation system, based on a wide angle mono camera as vision sensor and matching of the recovered and reference 3D environment, modeled by a digital elevation map (DEM). The recovered 3D models are being produced by real-time optical flow processing of the navigation camera images. An embedded optical correlator is introduced, which allows a robust and ultra high-speed optical flow processing under different and even unfavorable illumination conditions. The proposed concept can be augmented by any state-of-the art navigation aids, e.g. GPS, IMU, magnetometers. The current paper discusses the general principles and structure of the navigation system as a vision only system, i.e. without any additional navigation aids, and with instantaneous determination of the vehicle state, i.e. without time filtering. The paper gives also detailed end-to-end visual navigation performance results, based on a detailed software simulation model of the visual navigation system, including the optical correlator as the key component for ultra-high speed image processing.

A real-time optical signal and image processing p-i-n/p-i-n device

ABSTRACTLarge area (4x4 cm2) optical signal and image processing (OSIP) devices were produced at low temperatures (110°C) by PE-CVD. The OSIP device consists of two stacked sensing/switching diodes (p(SiC:H)/i(Si:H)/n(SiC:H)) with or without an internal blocking layer between them and two semitransparent contacts. An optical scanner is used for charge readout. In this work the main emphasis will be put on the analysis of the optical characteristics. The use of a metal grid (290x290 μm2 Cr pixels with 40 m m spacing) between the two diodes, working as light screening layer or as floating anode via an a-SiN insulator layer, is analyzed. Its influence on the transfer functions, resolution, responsivity and response time of the sensor is presented. The various design parameters trade-offs are discussed. The optical-to-electrical transfer characteristics show high quantum efficiency, broad spectral response, and reciprocity between the optical and the electrical images. When the light screening floating anode is present an effective optical decoupling from both photodiodes is achieved while maintaining a good electrical conductivity and an increased light-to-dark sensitivity. A trade-off is established between sensor design and light pattern and scanner wavelengths in order to minimize the cross talk between the write and the read beams and to improve the light to dark sensitivity.

Laser stabilization at 1536 nm using regenerative spectral hole burning

Laser frequency stabilization giving a 500-Hz Allan deviation for a 2-ms integration time with drift reduced to 7 kHz/min over several minutes was achieved at 1536 nm in the optical communication band. A continuously regenerated spectral hole in the inhomogeneously broadened ${}^{4}{I}_{15/2}{(1)}^{4}{I}_{13/2}(1)$ optical absorption of an ${\mathrm{Er}}^{3+}:{\mathrm{Y}}_{2}{\mathrm{SiO}}_{5}$ crystal was used as the short-term frequency reference, while a variation on the locking technique allowed simultaneous use of the inhomogeneously broadened absorption line as a long-term reference. The reported frequency stability was achieved without vibration isolation. Spectral hole burning frequency stabilization provides ideal laser sources for high-resolution spectroscopy, real-time optical signal processing, and a range of applications requiring ultra-narrow-band light sources or coherent detection; the time scale for stability and the compatibility with spectral hole burning devices make this technique complementary to other frequency references for laser stabilization.

Processing of medical images using real-time optical Fourier processing.

Optical image processing techniques are inherently fast in view of parallel processing. A self-adaptive optical Fourier processing system using photoinduced dichroism in a bacteriorhodopsin film was experimentally demonstrated for medical image processing. Application of this powerful analog all-optical interactive technique for cancer diagnostics is illustrated with two mammograms and a Pap smear. Microcalcification clusters buried in surrounding tissue showed up clearly in the processed image. By playing with one knob, which rotates the analyzer in the optical system, either the microcalcification clusters or the surrounding dense tissue can be selectively displayed. Bacteriorhodopsin films are stable up to 140 degrees C and environmentally friendly. As no interference is involved in the experiments, vibration isolation and even a coherent light source are not required. It may be possible to develop a low-cost rugged battery operated portable signal-enhancing magnifier.

Enhancement of the measurement sensitivity in real-time optical image processing for objects with a periodic structure

A possible method of real-time visualization of large-scale objects giving enhanced measurement sensitivity using incoherent illumination is reported.

Implementation of a photorefractive binary joint transform correlator

The use of photorefractive materials such as Bi12SiO20 as dynamic holographic media is becoming an interesting alternative to that of the current liquid-crystal-based modulators in real-time optical image processing. We present an experimental realization of optical correlation for pattern recognition by means of a photorefractive joint transform correlator. The correlator operates with a liquid-crystal television as the input and a photorefractive crystal at the recording plane. We consider two possible ways of registering the Fourier plane information: conventional detection of the joint power spectrum, and utilization of only phase information at the Fourier plane by suitable preprocessing of the scene and the reference at the object plane. We compare the latter case with the performance of a binary joint transform correlator. Analysis, simulations, and experimental results are presented.

Volume holographic storage demonstrator based on phase-coded multiplexing

We present the design and realization of a compact volume holographic memory based on phase-coded multiplexing. Due to the use of nonmechanical reference beam changes in this multiplexing technique, rapid access of the stored data pages is achieved. Our system can reach a maximum storage capacity up to 480 data pages with a resolution of 640/spl times/480 pixels in a single interaction region of a photorefractive LiNbO/sub 3/ crystal. We present the implementation of the system and first experimental results of both, high-resolution analog image storage and digital data storage. The storage of 180 data pages is demonstrated as well as the application of the system to real-time optical image processing.