Computer Image Research Articles

Swept coded aperture real-time femtophotography

Single-shot real-time femtophotography is indispensable for imaging ultrafast dynamics during their times of occurrence. Despite their advantages over conventional multi-shot approaches, existing techniques confront restricted imaging speed or degraded data quality by the deployed optoelectronic devices and face challenges in the application scope and acquisition accuracy. They are also hindered by the limitations in the acquirable information imposed by the sensing models. Here, we overcome these challenges by developing swept coded aperture real-time femtophotography (SCARF). This computational imaging modality enables all-optical ultrafast sweeping of a static coded aperture during the recording of an ultrafast event, bringing full-sequence encoding of up to 156.3 THz to every pixel on a CCD camera. We demonstrate SCARF’s single-shot ultrafast imaging ability at tunable frame rates and spatial scales in both reflection and transmission modes. Using SCARF, we image ultrafast absorption in a semiconductor and ultrafast demagnetization of a metal alloy.

Research on wireless multimedia big data transmission model based on non-linear algebraic equation with the adaptive scheduling algorithm

With the rapid development of computer technology, image processing and automatic book recognition has become a very important and practical field, and this discipline has a broad prospect and great potential. The digital visión system is a multi-plane information análisis method basad on the human brain, which is applied to library identification problems by studying several regular changes in the distribution of objects themselves in two-dimensional space and three-dimensional position, in order to improve book quality and reduce costs. In addition, digital Reading devices have become an essential part of people’s daily learning life, and multimedia retrieval methods have also undergone radical changes, image processing is becoming more and more complex, and algorithms must become more efficient and reliable as the amount of computing increases to a certain level, so as to adapt to this fast and powerful data storage and query needs. Therefore, the tren is to improve the speed and reliability of computer systems. This paper discusses and proposes measures for the analysis and extraction, classification and sorting of library images.

A Virtual Staining Method Based on Self-Supervised GAN for Fourier Ptychographic Microscopy Colorful Imaging

Fourier ptychographic microscopy (FPM) is a computational imaging technology that has endless vitality and application potential in digital pathology. Colored pathological image analysis is the foundation of clinical diagnosis, basic research, and most biomedical problems. However, the current colorful FPM reconstruction methods are time-inefficient, resulting in poor image quality due to optical interference and reconstruction errors. This paper combines coloring and FPM to propose a self-supervised generative adversarial network (GAN) for FPM color reconstruction. We design a generator based on the efficient channel residual (ECR) block to adaptively obtain efficient cross-channel interaction information in a lightweight manner, and we introduce content-consistency loss to learn the high-frequency information of the image and improve the image quality of the staining. Furthermore, the effectiveness of our proposed method is demonstrated through objective indicators and visual evaluations.

Multi‐Domain Detection Computational Imaging via Reconfigurable Metasurfaces MIMO System

AbstractA reconfigurable transmission metasurfaces (RTMs) excited by a multiple‐input multiple‐output (MIMO)( RTMs‐MIMO) antenna is designed to implement multi‐domain metamaterial computational imaging (MCI). Based on the receiver‐transmitter metasurfaces structure, four PIN diodes are applied to control the operating state of the RTMs. It enables arbitrary switching among four polarization transmission waves. A reverse‐induced current can be generated by adjusting the PIN diodes in the symmetrical position, resulting in a perfect 1‐bit phase modulation with a 3dB‐loss bandwidth of 15.38~19.89 GHz. A microstrip antenna with mushroom‐shaped parasitic patches is employed to implement the MIMO feeds. The ‐10dB bandwidth of MIMO antenna is compatible with the RTMs, and the maximum envelope correlation coefficient of 0.16 demonstrates good isolation between the MIMO feeds. In the MCI system constructed by the RTMs‐MIMO antennas, a transmission matrix H can be achieved by MIMO space beams, metasurfaces phase, metasurfaces polarization, and frequency modes. The correlation coefficient µg = 0.018 is achieved by the multi‐domain integration, allowing H to be highly non‐correlated. Based on the detection results of H, an imaging result with a relative reconstruction error of 0.372 is achieved. The designed MCI system therefore has unparalleled application prospects in the broad field of medical and military detection.

A compressive hyperspectral video imaging system using a single-pixel detector

Capturing fine spatial, spectral, and temporal information of the scene is highly desirable in many applications. However, recording data of such high dimensionality requires significant transmission bandwidth. Current computational imaging methods can partially address this challenge but are still limited in reducing input data throughput. In this paper, we report a video-rate hyperspectral imager based on a single-pixel photodetector which can achieve high-throughput hyperspectral video recording at a low bandwidth. We leverage the insight that 4-dimensional (4D) hyperspectral videos are considerably more compressible than 2D grayscale images. We propose a joint spatial-spectral capturing scheme encoding the scene into highly compressed measurements and obtaining temporal correlation at the same time. Furthermore, we propose a reconstruction method relying on a signal sparsity model in 4D space and a deep learning reconstruction approach greatly accelerating reconstruction. We demonstrate reconstruction of 128 × 128 hyperspectral images with 64 spectral bands at more than 4 frames per second offering a 900× data throughput compared to conventional imaging, which we believe is a first-of-its kind of a single-pixel-based hyperspectral imager.

Computational optical imaging: challenges, opportunities, new trends, and emerging applications

Computational imaging technology (CIT), with its many variations, addresses the limitations of industrial design. CIT can effectively overcome the bottlenecks in physical information acquisition, model development, and resolution by being tightly coupled with mathematical calculations and signal processing in information acquisition, transmission, and interpretation. Qualitative improvements are achieved in the dimensions, scale, and resolution of the information. Therefore, in this review, the concepts and meaning of CIT are summarized before establishing a real CIT system. The basic common problems and relevant challenging technologies are analyzed, particularly the non-linear imaging model. The five typical imaging requirements–distance, resolution, applicability, field of view, and system size–are detailed. The corresponding key issues of super-large-aperture imaging systems, imaging beyond the diffraction limit, bionic optics, interpretation of light field information, computational optical system design, and computational detectors are also discussed. This review provides a global perspective for researchers to promote technological developments and applications.

Parallel computational ghost imaging with modulation patterns multiplexing and permutation inspired by compound eyes

Real-time computational ghost imaging (CGI) has received significant attention in recent years to overcome the trade-off between long acquisition time and high reconstructed image quality of CGI. Inspired by compound eyes, we propose a parallel computational ghost imaging with modulation patterns multiplexing and permutation to achieve a faster and high-resolution CGI. With modulation patterns multiplexing and permutation, several small overlapping fields-of-view can be obtained; meanwhile, the difficulty in alignment of illumination light field and multiple detectors can be well resolved. The method combining compound eyes with multi-detectors to capture light intensity can resolve the issue of a gap between detector units in the array detector. Parallel computation facilitates significantly reduced acquisition time, while maintaining reconstructed quality without compromising the sampling ratio. Experiments indicate that using m × m detectors reduce modulation pattern count, projector storage, and projection time to around 1/m2 of typical CGI methods, while increasing image resolution to m2 times. This work greatly promotes the practicability of parallel computational ghost imaging and provides optional solution for real-time computational ghost imaging.

Single-Pixel Infrared Hyperspectral Imaging via Physics-Guided Generative Adversarial Networks

A physics-driven generative adversarial network (GAN) was utilized to demonstrate a single-pixel hyperspectral imaging (HSI) experiment in the infrared spectrum, eliminating the need for extensive dataset training in most data-driven deep neural networks. Within the GAN framework, the physical process of single-pixel imaging (SPI) was integrated into the generator, and its estimated one-dimensional (1D) bucket signals and the actual 1D bucket signals were employed as constraints in the objective function to update the network’s parameters and optimize the generator with the assistance of the discriminator. In comparison to single-pixel infrared HSI methods based on compressive sensing and physics-driven convolution neural networks, our physics-driven GAN-based single-pixel infrared HSI exhibits superior imaging performance. It requires fewer samples and achieves higher image quality. We believe that our physics-driven network will drive practical applications in computational imaging, including various SPI-based techniques.

Metasurface-based computational imaging: a review

Metasurface-based computational imaging: a review

Semantic representation learning for a mask-modulated lensless camera by contrastive cross-modal transferring.

Lensless computational imaging, a technique that combines optical-modulated measurements with task-specific algorithms, has recently benefited from the application of artificial neural networks. Conventionally, lensless imaging techniques rely on prior knowledge to deal with the ill-posed nature of unstructured measurements, which requires costly supervised approaches. To address this issue, we present a self-supervised learning method that learns semantic representations for the modulated scenes from implicitly provided priors. A contrastive loss function is designed for training the target extractor (measurements) from a source extractor (structured natural scenes) to transfer cross-modal priors in the latent space. The effectiveness of the new extractor was validated by classifying the mask-modulated scenes on unseen datasets and showed the comparable accuracy to the source modality (contrastive language-image pre-trained [CLIP] network). The proposed multimodal representation learning method has the advantages of avoiding costly data annotation, being more adaptive to unseen data, and usability in a variety of downstream vision tasks with unconventional imaging settings.

Photonic neural probe enabled microendoscopes for light-sheet light-field computational fluorescence brain imaging.

Light-sheet fluorescence microscopy is widely used for high-speed, high-contrast, volumetric imaging. Application of this technique to in vivo brain imaging in non-transparent organisms has been limited by the geometric constraints of conventional light-sheet microscopes, which require orthogonal fluorescence excitation and collection objectives. We have recently demonstrated implantable photonic neural probes that emit addressable light sheets at depth in brain tissue, miniaturizing the excitation optics. Here, we propose a microendoscope consisting of a light-sheet neural probe packaged together with miniaturized fluorescence collection optics based on an image fiber bundle for lensless, light-field, computational fluorescence imaging. Foundry-fabricated, silicon-based, light-sheet neural probes can be packaged together with commercially available image fiber bundles to form microendoscopes for light-sheet light-field fluorescence imaging at depth in brain tissue. Prototype microendoscopes were developed using light-sheet neural probes with five addressable sheets and image fiber bundles. Fluorescence imaging with the microendoscopes was tested with fluorescent beads suspended in agarose and fixed mouse brain tissue. Volumetric light-sheet light-field fluorescence imaging was demonstrated using the microendoscopes. Increased imaging depth and enhanced reconstruction accuracy were observed relative to epi-illumination light-field imaging using only a fiber bundle. Our work offers a solution toward volumetric fluorescence imaging of brain tissue with a compact size and high contrast. The proof-of-concept demonstrations herein illustrate the operating principles and methods of the imaging approach, providing a foundation for future investigations of photonic neural probe enabled microendoscopes for deep-brain fluorescence imaging in vivo.

Exploring deep learning techniques for illuminance estimation

AbstractIn recent years, deep learning techniques had a revolutionary impact on several domains, including computer vision and image processing. This research paper focuses on exploring deep learning methods to achieve precise illuminance estimation, which holds significant importance in applications such as augmented reality, virtual reality, and photography. However, accurately estimating illuminance in complex scenes continues to pose challenges due to the intricate interplay between light sources, objects, and surfaces. The results of extensive experimentation demonstrate the immense potential of deep learning techniques in illuminance estimation. These techniques exhibit promising accuracy and robustness, enabling them to handle diverse scenarios effectively. The valuable insights derived from this study can serve as a guiding framework for future research endeavours and contribute to the development of efficient and precise methodologies for illuminance estimation across a wide range of practical applications.

Scalable Bright and Pure Single Photon Sources by Droplet Epitaxy on InP Nanowire Arrays.

High-quality quantum light sources are crucial components for the implementation of practical and reliable quantum technologies. The persistent challenge, however, is the lack of scalable and deterministic single photon sources that can be synthesized reproducibly. Here, we present a combination of droplet epitaxy with selective area epitaxy to realize the deterministic growth of single quantum dots in nanowire arrays. By optimization of the single quantum dot growth and the nanowire cavity design, single emissions are effectively coupled with the dominant mode of the nanowires to realize Purcell enhancement. The resonance-enhanced quantum emitter system boasts a brightness of millions of counts per second with nanowatt excitation power, a short radiation lifetime of 350 ± 5 ps, and a high single-photon purity with g(2)(0) value of 0.05 with continuous wave above-band excitation. Finite-difference time-domain (FDTD) simulation results show that the emissions of single quantum dots are coupled into the TM01 mode of the nanowires, giving a Purcell factor ≈ 3. Our technology can be used for creating on-chip scalable single photon sources for future quantum technology applications including quantum networks, quantum computation, and quantum imaging.

Spatial perception in stereoscopic augmented reality based on multifocus sensing

In many areas ranging from medical imaging to visual entertainment, 3D information acquisition and display is a key task. In this regard, in multifocus computational imaging, stacks of images of a certain 3D scene are acquired under different focus configurations and are later combined by means of post-capture algorithms based on image formation model in order to synthesize images with novel viewpoints of the scene. Stereoscopic augmented reality devices, through which is possible to simultaneously visualize the three dimensional real world along with overlaid digital stereoscopic image pair, could benefit from the binocular content allowed by multifocus computational imaging. Spatial perception of the displayed stereo pairs can be controlled by synthesizing the desired point of view of each image of the stereo-pair along with their parallax setting. The proposed method has the potential to alleviate the accommodation-convergence conflict and make augmented reality stereoscopic devices less vulnerable to visual fatigue.

Optimal design of the computational flat diffractive optical system

A design method of the computational flat diffractive computational flat diffractive optical system is presented to simplify the optical system structure and achieve high image quality. The aberration expression of the flat diffractive optical element (FDOE) is derived, and then computational imaging methods are used to eliminate the influence of off-axis aberration on image quality, so the field of view is expanded. Based on theoretical analysis, the FDOE is designed, and the field of view has been expanded from 2° to 5°. The results show that the detail resolution of the edge field of view is enhanced after restoration, and the modulation transfer function (MTF) of different subareas calculated using the slanted-edge method improved by an average of 0.17. The diffraction efficiency of the FDOE is greater than 95.75%. This method realizes the miniaturization and lightweight of the optical system, and provides new ideas for the integration of optical systems.

Computed tomography/magnetic resonance imaging for mandibular boundary invasion of oral squamous cell carcinoma assessment

BackgroundThe range of mandibular invasion by a tumour needs to be determined accurately to minimize unnecessary damage to the mandible. This study aimed to compare tumour boundary lines on computed tomography/magnetic resonance (CT/MR) images with those from pathological findings during the preoperative assessment of mandibular invasion by oral squamous cell carcinoma (OSCC). By comparing the methods, the potential of CT/MR for this application could be further elucidated.MethodsEight patients with OSCC were imaged with CT/MR, mandibular specimens were collected, and the material site was measured. Haematoxylin–eosin staining was used for histopathological assessment. The presence and boundaries of bone invasion were evaluated. The CT/MR and histopathological boundaries of bone invasion were delineated and merged to compare and calculate the deviation of CT/MR and histopathological boundaries using the Fréchet distance.ResultsThe mean Fréchet distance between the CT and pathological tumour boundaries was 2.69 mm (standard error 0.46 mm), with a minimum of 1.18 mm, maximum of 3.64 mm, median of 3.10 mm, and 95% confidence interval of 1.40–3.97 mm. The mean Fréchet distance between the tumour boundaries on the MR and pathological images was 3.07 mm (standard error 0.56 mm), with a minimum of 1.53 mm, maximum of 4.74 mm, median of 2.90 mm, and 95% confidence interval of 1.53–4.61 mm.ConclusionsCT/MR imaging can provide an effective preoperative assessment of mandibular invasion of OSCC. Pathology images can be positioned on CT/MR scans with the help of computer software to improve the accuracy of the findings. The introduction of the Fréchet distance to compare tumour boundary lines is conducive to computer image diagnosis of tumour invasion of jaw boundaries.

Photomultiplier Tubes for Programmable Logic Controller Foreign Object Sorting System and Its Performance Analysis

The photoelectric foreign object sorting system is a comprehensive device developed to remove foreign objects from target substances, integrating photoelectric, computer, and mechanical technologies. This system combines photoelectric, computer image, and mechanical technologies to identify materials based on color and appearance, utilizing the scattered light returned from the scanned object for detection and recognition. This work designs a photoelectric foreign object sorting system for food and tobacco. This system takes photomultiplier tubes as the main photoelectric devices and a programmable logic controller (PLC) as the control unit. The hardware components of this system include the design of a low-noise preamplifier, conditioning circuit, ADC data acquisition module, and PLC-external circuit connections. In simulation experiments, testing is conducted on the low-noise preamplifier, considering the significant impact of the collected signals on subsequent analysis. Amplitude-frequency and phase-frequency tests show that the circuit signal output of this module is stable. There are no oscillations in the time domain, and no occurrences of overshooting or ringing in frequency. The noise indicates that the signal noise of this module is relatively low. During the debugging process, ADA4896 is selected, and a black-and-white alternating stripe with a 3 mm spacing is used as the scanning object. As expected, the first-stage amplification results of the low-noise preamplifier show a range of 2 mV with some DC components in the output voltage. In its second-stage amplification, the fine scale reaches 10 mV, and the output signal can represent the physical characteristics of the scanned object.

Real time data monitoring of water resources environment based on computer remote data collection and image analysis

Real time data monitoring of water resources environment based on computer remote data collection and image analysis

Diagnostics and characterization of a novel multi gas layer RF atmospheric pressure plasma jet for polymer processing

The quest to employ cold plasma sources at atmospheric pressure in polymer processing has emerged as a potent driving force behind their development. Atmospheric pressure operation of plasma jets provides potential cost reductions as well as easier handling and maintenance. In addition, their unique advantage of remote operation allows the substrate to be placed outside the source boundaries. This latter feature makes it easier to process complex three-dimensional objects and to integrate plasma jets into existing production lines. Although conventional atmospheric pressure plasma jet (APPJ) sources have undergone significant advancements in their design and construction, they have reached their technical and technological thresholds in several domains, thereby also impeding further enhancements in material processing applications. To cope with this issue, this work introduces a promising APPJ (named MPPJ3) working in a three co-axial gas layer geometry, incorporating the capability of aerosol and shield gas introduction leading to a configuration rich in reactive plasma species with controllable size and suitable temperature for polymer processing. A parametric study on the novel MPPJ3 device is carried out and plasma characteristics, such as reactive plasma species and temperatures, are determined by means of optical emission spectroscopy (OES), laser scattering, and infrared (IR) camera imaging whereas the fluid dynamics are analyzed using computational fluid dynamics (CFDs) and Schlieren imaging. The obtained promising results clearly show the flexibility and adaptability of the MPPJ3 device for polymer processing applications.

Partitioning intensity inhomogeneity colour images via Saliency-based active contour

Partitioning or segmenting intensity inhomogeneity colour images is a challenging problem in computer vision and image shape analysis. Given an input image, the active contour model (ACM) which is formulated in variational framework is regularly used to partition objects in the image. A selective type of variational ACM approach is better than a global approach for segmenting specific target objects, which is useful for applications such as tumor segmentation or tissue classification in medical imaging. However, the existing selective ACMs yield unsatisfactory outcomes when performing the segmentation for colour (vector-valued) with intensity variations. Therefore, our new approach incorporates both local image fitting and saliency maps into a new variational selective ACM to tackle the problem. The euler-lagrange (EL) equations were presented to solve the proposed model. Thirty combinations of synthetic and medical images were tested. The visual observation and quantitative results show that the proposed model outshines the other existing models by average, with the accuracy of 2.23% more than the compared model and the Dice and Jaccard coefficients which were around 12.78% and 19.53% higher, respectively, than the compared model.