Two-dimensional Spatial Information Research Articles

CTIS spectral image reconstruction technology based on slit-scanning architecture

The computed tomography imaging spectrometer (CTIS) is a snapshot imaging spectrometer, excelling in dynamic detection tasks. It can capture two-dimensional spatial information and spectrally compressed information of a target within a single exposure time. However, traditional CTIS image reconstruction algorithms suffer from missing-cone problem, which reduces the accuracy of spectral reconstruction. In recent years, deep learning has been applied to CTIS spectral image reconstruction, significantly improving spectral reconstruction accuracy compared to traditional algorithms. However, due to the missing-cone problem, it is difficult to accurately recover the truth of spectral data cube in the real scene. Currently, most CTIS neural network reconstruction models are trained using simulated datasets of spectral data cubes and diffractive images. Because these data differ significantly from real data under actual application conditions, the established models may not be effectively applicable to real-world scenes. Therefore, we propose a new CTIS system based on slit-scanning architecture utilizing an adjustable slit aperture to obtain the real spectral data cube of the target while maintaining the simplicity of the CTIS structure. By limiting the field of view (FOV) through the slit, the area of diffraction overlap can be reduced, thereby enhancing the accuracy of CTIS spectral reconstruction using the expectation-maximization (EM) algorithm. This architecture allows us to obtain accurate spectral cubes that match the CTIS diffractive image of real-world scenes, providing a real dataset for training the reconstruction network. A prototype has been built to demonstrate the feasibility of our proposed solution. Furthermore, we also constructed a residual network based on multi-scale and attention mechanism. This network is trained using a combination of simulated and real spectral imaging data. Compared to the reconstruction performance of the EM algorithm and convolutional neural networks, our approach demonstrates superior spectral reconstruction accuracy, validating the importance of real spectral data in CTIS spectral reconstruction tasks.

Concrete crack segmentation based on multi-dimensional structure information fusion-based network

The state-of-the-art (SOTA) crack segmentation methods still face the challenges in low-contrast shallow crack recognition and multi-scene compatibility. Therefore, this paper proposes a solution based on a multi-dimensional structure information fusion-based network (MDSIFNet). This network consists of two branches, one for extracting two-dimensional (2D) spatial feature information and another for acquiring three-dimensional (3D) geometric one. In the former, the designed 2D curve structure constraint module based on prior knowledge combined with a pretrained 2D feature extraction module can reduce learning samples and realize crack segmentation in unseen scenes. In the latter, a one-hot transformation and a 3D geometric structure information extraction module are designed to make the network pay attention to the geometric features of shallow cracks and improve their segmentation accuracy. Finally, a fusion module couples the multi-dimensional structure feature information of these two branches and outputs pixel-wise segmentation results that can be used for crack measurement and quantitative analysis. The experimental results show that the designed model network MDSIFNet performs better than the SOTA methods in terms of performance and visualized results.

Collaborative networks of transformers and convolutional neural networks are powerful and versatile learners for accurate 3D medical image segmentation

Integrating transformers and convolutional neural networks represents a crucial and cutting-edge approach for tackling medical image segmentation problems. Nonetheless, the existing hybrid methods fail to fully leverage the strengths of both operators. During the Patch Embedding, the patch projection method ignores the two-dimensional structure and local spatial information within each patch, while the fixed patch size cannot capture features with rich representation effectively. Moreover, the calculation of self-attention results in attention diffusion, hindering the provision of precise details to the decoder while maintaining feature consistency. Lastly, none of the existing methods establish an efficient multi-scale modeling concept. To address these issues, we design the Collaborative Networks of Transformers and Convolutional neural networks (TC-CoNet), which is generally used for accurate 3D medical image segmentation. First, we elaborately design precise patch embedding to generate 3D features with accurate spatial position information, laying a solid foundation for subsequent learning. The encoder–decoder backbone network is then constructed by TC-CoNet in an interlaced combination to properly incorporate long-range dependencies and hierarchical object concepts at various scales. Furthermore, we employ the constricted attention bridge to constrict attention to local features, allowing us to accurately guide the recovery of detailed information while maintaining feature consistency. Finally, atrous spatial pyramid pooling is applied to high-level feature map to establish the concept of multi-scale objects. On five challenging datasets, including Synapse, ACDC, brain tumor segmentation, cardiac left atrium segmentation, and lung tumor segmentation, the extensive experiments demonstrate that TC-CoNet outperforms state-of-the-art approaches in terms of superiority, migration, and strong generalization. These illustrate in full the efficacy of the proposed transformers and convolutional neural networks combination for medical image segmentation. Our code is freely available at: https://github.com/YongChen-Exact/TC-CoNet.

High-precision single-pixel 3D calibration method using pseudo-phase matching.

Compressive sensing makes it possible to explore two-dimensional spatial information using a single-point detector. However, the reconstruction of the three-dimensional (3D) morphology using a single-point sensor is largely limited by the calibration. Here we demonstrate a pseudo-single-pixel camera calibration (PSPC) method using pseudo phase matching in stereo, which can perform 3D calibration of low-resolution images with the help of a high-resolution digital micromirror device (DMD) in the system. In this paper, we use a high-resolution CMOS to pre-image the DMD surface and successfully calibrate the spatial position of a single-point detector and the projector with the support of binocular stereo matching. Our system achieved sub-millimeter reconstructions of spheres, steps, and plaster portraits at low compression ratios with a high-speed digital light projector (DLP) and a highly sensitive single-point detector.

Polarization Super-Resolution Imaging Method Based on Deep Compressed Sensing.

The division of focal plane (DoFP) polarization imaging sensors, which can simultaneously acquire the target's two-dimensional spatial information and polarization information, improves the detection resolution and recognition capability by capturing the difference in polarization characteristics between the target and the background. In this paper, we propose a novel polarization imaging method based on deep compressed sensing (DCS) by adding digital micromirror devices (DMD) to an optical system and simulating the polarization transmission model of the optical system to reconstruct high-resolution images under low sampling rate conditions. By building a simulated dataset, training a polarization super-resolution imaging network, and showing excellent reconstructions on real shooting scenes, compared to current algorithms, our model has a higher peak signal-to-noise ratio (PSNR), which validates the feasibility of our approach.

Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery.

The study of brain science is vital to human health. The application of hyperspectral imaging in biomedical fields has grown dramatically in recent years due to their unique optical imaging method and multidimensional information acquisition. Hyperspectral imaging technology can acquire two-dimensional spatial information and one-dimensional spectral information of biological samples simultaneously, covering the ultraviolet, visible and infrared spectral ranges with high spectral resolution, which can provide diagnostic information about the physiological, morphological and biochemical components of tissues and organs. This technology also presents finer spectral features for brain imaging studies, and further provides more auxiliary information for cerebral disease research. This paper reviews the recent advance of hyperspectral imaging in cerebral diagnosis. Firstly, the experimental setup, image acquisition and pre-processing, and analysis methods of hyperspectral technology were introduced. Secondly, the latest research progress and applications of hyperspectral imaging in brain tissue metabolism, hemodynamics, and brain cancer diagnosis in recent years were summarized briefly. Finally, the limitations of the application of hyperspectral imaging in cerebral disease diagnosis field were analyzed, and the future development direction was proposed.

All-fiber high-speed image detection enabled by deep learning

Ultra-high-speed imaging serves as a foundation for modern science. While in biomedicine, optical-fiber-based endoscopy is often required for in vivo applications, the combination of high speed with the fiber endoscopy, which is vital for exploring transient biomedical phenomena, still confronts some challenges. We propose all-fiber imaging at high speeds, which is achieved based on the transformation of two-dimensional spatial information into one-dimensional temporal pulsed streams by leveraging high intermodal dispersion in a multimode fiber. Neural networks are trained to reconstruct images from the temporal waveforms. It can not only detect content-aware images with high quality, but also detect images of different kinds from the training images with slightly reduced quality. The fiber probe can detect micron-scale objects with a high frame rate (15.4 Mfps) and large frame depth (10,000). This scheme combines high speeds with high mechanical flexibility and integration and may stimulate future research exploring various phenomena in vivo.

Imaging spectrum reconstruction of a spatial heterodyne imaging spectrometer.

Spatial heterodyne scanning imaging combines the advantages of spatial heterodyne spectroscopy and imaging technology. It can obtain images and spectral information simultaneously with the characteristics of high spectral resolution, high flux, and small volume and can achieve spectral restoration of all target points in the measured area through scan imaging only once. When two-dimensional spatial information and one-dimensional spectral information are obtained, the imaging spectrum data cube is generated. Using MATLAB and ZEMAX as simulation software, image information with interference fringes was constructed, and the distribution of interference information was explored. A potassium lamp, xenon lamp, and tungsten lamp were used as different light sources to perform validation. Because of the influence of the scanning displacement error on the accuracy of interference information extraction, an algorithm for image spectrum reconstruction is proposed based on the principle of distribution of interference information. The speed-up robust features algorithm was used for image registration to improve accuracy. The experimental results show that the spectral information under the three light sources can be reconstructed, which corresponds to the standard spectrum from the optical fiber spectrometer. The feasibility of this imaging spectrum reconstruction has been preliminarily verified, and the spectral information of the desired target can be directly extracted from the data cube.

Fast Evaluation of Unhealthy and Healthy Neonates Using Hyperspectral Features on 700-850 Nm Wavelengths, ROI Extraction, and 3D-CNN

ObjectivesHyperspectral imaging (HSI) has great potential in detecting the health conditions of neonates as it provides diagnostic information about the tissue by avoiding tissue biopsy. HSI gives more features than thermal imaging, which can obtain images in a single wavelength, as it can obtain images in a large number of wavelengths. The data obtained with hyperspectral sensors are 3-dimensional data called hypercube including first two-dimensional spatial information and third-dimensional spectral information. Material and methodsIn this study, hyperspectral data were obtained from 19 different neonates in the Neonatal Intensive Care Unit (NICU) of Selcuk University, Medical Faculty. There are 16 hypercubes from 16 unhealthy neonates, 16 hypercubes from 3 healthy neonates in a period of three months, and 32 hypercubes in total are available. For the training of 3D-CNN model, data augmentation methods, such as rotation, height shifting, width shifting, and shearing were applied to hyperspectral data. A number of 32 hypercubes taken from neonates in NICU were augmented to 160 hypercubes. Spectral signatures were examined and 51 bands in the range of 700-850 nm with distinctive features were used for the classification. The spectral dimension was reduced by applying Principal Component Analysis (PCA) to all hypercubes. In addition, it is aimed to obtain both spectral and spatial features with the 3D-CNN. For increasing the classification efficiency, ROI extraction was made and four datasets were created in different spatial dimensions. These datasets contain 160, 640, 1440, and 5760 hypercubes, respectively. ResultsThe best result was achieved by using 5760 hypercubes of 25x25x51. As a result of the classification of the hypercubes, accuracy 98.00%, sensitivity 97.22%, and specificity 98.78% were obtained. It was determined how many PCs used to achieve the best result. Further, the proposed 3D-CNN model is compared to 2D-CNN model to evaluate the performance of the study. ConclusionIt was aimed to evaluate the health status of neonates fastly by using HSI and 3D-CNN for the first time. The obtained results are an indication that HSI and 3D-CNN are very effective for the evaluation of unhealthy and healthy neonates.

Fast hyperspectral imager driven by a low-cost and compact galvo-mirror

Imaging spectrometer is a critical device for remote sensing in visible and infrared band. Nowadays, it has become a powerful instrument to obtain three-dimensional hyperspectral cube (two-dimensional spatial information and one-dimensional spectral information) of targets for high precision object recognition. However, we often cannot meet the requirements of high spatial resolution and high spectral resolution at the same time. So there is still a trade-off between detection speed and data accuracy (including spatial and spectral resolution). In this paper, we developed a fast and high spectral resolution imaging spectrometer to obtain the spectral-spatial information with the help of a low-cost and compact galvo-mirror. The spectrometer has a wavelength resolution of about 10 nm at visible-near infrared band (400 nm–800 nm), and it can measure two-dimensional spatial images of various biological samples (including corals, dragon fruits and human hands) at various wavelengths repeatedly. This spectrometer can facilitate collection of a large number of spectral image data during half cycle of galvo-mirror scanning. The theoretical acquisition speed of hyperspectral cube can reach more than 1 MHz. However, at this acquisition speed, the signal-to-noise ratio of the camera is poor, and it is unable to obtain high-quality signals. We believe that with the improvement of camera performance, our prototype can obtain hyperspectral data at a higher sampling speed.

Deep CNNs as a method to classify rotating objects based on monostatic RCS

Radar systems emit a time-varying signal and measure the response of a radar-reflecting surface. In the case of narrowband, monostatic radar signal domain, all spatial information is projected into a radar cross-section (RCS) scalar. The authors address the challenging problem of determining shape class using monostatic RCS estimates collected as a time series from a rotating object tumbling with unknown motion parameters under detectability limitations and signal noise. Previous shape classification methods have relied on image-like synthetic aperture radar or multistatic (multiview) radar configurations with known geometry. Convolutional neural networks (CNNs) have revolutionised learning tasks in the computer vision domain by leveraging images and video rich with high-resolution two-dimensional (2D) or 3D spatial information. They show that a feed-forward CNN can be trained to successfully classify object shape using only noisy monostatic RCS signals with unknown motion. They construct datasets containing over 100,000 simulated RCS signals belonging to different shape classes. They introduce deep neural network architectures that produce 2% classification error on testing data. They also introduce a refinement network that transforms simulated signals to appear more realistic and improve training utility. The results are a pioneering step toward the recognition of more complex targets using narrowband, monostatic radar.

Three-dimensional imaging flow cytometry through light-sheet fluorescence microscopy.

Flow cytometry is the tool of choice for high-speed acquisition and analysis of large cell populations, with the tradeoff of lacking intracellular spatial information. Although in the last decades flow cytometry systems that can actually acquire two-dimensional spatial information were developed, some of the limitations remained though, namely constrains related to sample size and lack of depth or dynamic information. The combination of fluidics and light-sheet illumination has the potential to address these limitations. By having cells travelling with the flowing sheath one can, in a controlled fashion, force them at constant speed through the light-sheet enabling the synchronized acquisition of several optical sections, that is, three-dimensional imaging. This approach has already been used for imaging cellular spheroids, plankton, and zebra-fish embryos. In this review, we discuss the known solutions and standing challenges of performing three-dimensional high-throughput imaging of multicellular biological models using fluidics, while retaining cell and organelle-level resolution. © 2017 International Society for Advancement of Cytometry.

Comparison of hadron shower data in the PAMELA experiment with Geant 4 simulations

The sampling imaging electromagnetic calorimeter of ≈ 16.3 radiation lengths and ≈ 0.6 nuclear interaction length designed and constructed by the PAMELA collaboration as a part of the large magnetic spectrometer PAMELA. Calorimeter consists of 44 single-sided silicon sensor planes interleaved with 22 plates of tungsten absorber (thickness of each tungsten layer 0.26 cm). Silicon planes are composed of a 3 × 3 matrix of silicon detectors, each segmented into 32 read-out strips with a pitch of 2.4 mm. The orientation of the strips of two consecutive layers is orthogonal and therefore provides two-dimensional spatial information. Due to the high granularity, the development of hadronic showers can be study with a good precision.In this work a Monte Carlo simulations (based on Geant4) performed using different available models, and including detector and physical effects, compared with the experimental data obtained on the near Earth orbit. Response of the PAMELA calorimeter to hadronic showers investigated including total energy release in calorimeter and transverse shower profile characteristics.

Detection Method for Bean Cotyledon Locations under Vinyl Mulch Using Multiple Infrared Sensors

Purpose: Pulse crop damage due to wild birds is a serious problem, to the extent that the rate of damage during the period of time between seeding and the stage of cotyledon reaches 45.4% on average. This study investigated a method of fundamentally blocking birds from eating crops by conducting vinyl mulching after seeding and identifying the growing locations for beans to perform punching. Methods: Infrared (IR) sensors that could measure the temperature without contact were used to recognize the locations of soybean cotyledons below vinyl mulch. To expand the measurable range, 10 IR sensors were arranged in a linear array. A sliding mechanical device was used to reconstruct the two-dimensional spatial variance information of targets. Spatial interpolation was applied to the two-dimensional temperature distribution information measured in real time to improve the resolution of the bean coleoptile locations. The temperature distributions above the vinyl mulch for five species of soybeans over a period of six days from the appearance of the cotyledon stage were analyzed. Results: During the experimental period, cases where bean cotyledons did and did not come into contact with the bottom of the vinyl mulch were both observed, and depended on the degree of growth of the bean cotyledons. Although the locations of bean cotyledons could be estimated through temperature distribution analyses in cases where they came into contact with the bottom of the vinyl mulch, this estimation showed somewhat large errors according to the time that had passed after the cotyledon stage. The detection results were similar for similar types of crops. Thus, this method could be applied to crops with similar growth patterns. According to the results of 360 experiments that were conducted (five species of bean <TEX>${\times}$</TEX> six days <TEX>${\times}$</TEX> four speed levels <TEX>${\times}$</TEX> three repetitions), the location detection performance had an accuracy of 36.9%, and the range of location errors was 0-4.9 cm (RMSE = 3.1 cm). During a period of 3-5 days after the cotyledon stage, the location detection performance had an accuracy of 59% (RMSE = 3.9 cm). Conclusions: In the present study, to fundamentally solve the problem of damage to beans from birds in the early stage after seeding, a working method was proposed in which punching is carried out after seeding, thereby breaking away from the existing method in which seeding is carried out after punching. Methods for the accurate detection of soybean growing locations were studied to allow punching to promote the continuous growth of soybeans that had reached the cotyledon stage. Through experiments using multiple IR sensors and a sliding mechanical device, it was found that the locations of the crop could be partially identified 3-5 days after reaching the cotyledon stage regardless of the kind of pulse crop. It can be concluded that additional studies of robust detection methods considering environmental factors and factors for crop growth are necessary.

Alignment error analysis of the snapshot imaging polarimeter.

A snapshot imaging polarimeter (SIP) system is able to reconstruct two-dimensional spatial polarization information through a single interferogram. In this system, the alignment errors of the half-wave plate (HWP) and the analyzer have a predominant impact on the accuracies of reconstructed complete Stokes parameters. A theoretical model for analyzing the alignment errors in the SIP system is presented in this paper. Based on this model, the accuracy of the reconstructed Stokes parameters has been evaluated by using different incident states of polarization. An optimum thickness of the Savart plate for alleviating the perturbation introduced by the alignment error of the HWP is found by using the condition number of the system measurement matrix as an objective function in a minimization procedure. The result shows that when the thickness of a Savart plate is 23 mm, corresponding to the condition number 2.06, the precision of the SIP system can reach to 0.21% at 1° alignment tolerance of the HWP.

Hard Exclusive Meson Production at COMPASS

The concept of Generalized Parton Distributions (GPDs) combines two-dimensional spatial information given by form factors, with longitudinal momentum information from Parton Distribution Functions. GPDs provide comprehensive description of the nucleon structure involving a wealth of new information. For instance, according to Ji’s sum rule, the GPDs [Formula: see text] and [Formula: see text] enable access to the total angular momenta of quarks, antiquarks and gluons. While [Formula: see text] can be approached using measurements of electroproduction cross sections, asymmetry measurements in hard exclusive meson production off transversely polarized targets can help to constrain the GPD [Formula: see text] and chiral-odd GPDs. In 2007 and 2010 the COMPASS experiment at CERN collected data by scattering a [Formula: see text] muon beam off a transversely polarized NH3 target. Exclusive vector-meson production [Formula: see text] with a [Formula: see text] or [Formula: see text] meson in the final state is studied and five single-spin and three double-spin azimuthal asymmetries are measured.

Mobile Video-to-Audio Transducer and Motion Detection for Sensory Substitution

Visuo-auditory sensory substitution systems are augmented reality devices that translate a video stream into an audio stream in order to help the blind in daily tasks requiring visuo-spatial information. In this work, we present both a new mobile device and a transcoding method specifically designed to sonify moving objects. Frame differencing is used to extract spatial features from the video stream and two-dimensional spatial information is converted into audio cues using pitch, interaural time difference and interaural level difference. Using numerical methods, we attempt to reconstruct visuo-spatial information based on audio signals generated from various video stimuli. We show that despite a contrasted visual background and a highly lossy encoding method, the information in the audio signal is sufficient to allow object localization, object trajectory evaluation, object approach detection, and spatial separation of multiple objects. We also show that this type of audio signal can be interpreted by human users by asking ten subjects to discriminate trajectories based on generated audio signals.

Karhunen-Loève transform for compressive sampling hyperspectral images

Compressed sensing (CS) is a new jointly sampling and compression technology for remote sensing. In hyperspectral imaging, a typical CS method encodes the two-dimensional (2-D) spatial information of each spectral band or encodes the third spectral information simultaneously. However, encoding the spatial information is much easier than encoding the spectral information. Therefore, it is crucial to make use of the spectral information to improve the compression rate on 2-D CS data. We propose to encode the third spectral information with an adaptive Karhunen–Loeve transform. With a mathematical proof, we show that interspectral correlations are preserved among 2-D randomly encoded spatial information. This property means that one can compress 2-D CS data effectively with a Karhunen–Loeve transform. Experiments demonstrate that the proposed method can better reconstruct both spectral curves and spatial images than traditional compression methods at the bit rates 0 to 1.

Deblurring and Sparse Unmixing of Hyperspectral Images Using Multiple Point Spread Functions

This paper is concerned with deblurring and spectral analysis of ground-based astronomical images of space objects. A numerical approach is provided for deblurring and sparse unmixing of ground-based hyperspectral images (HSIs) of objects taken through atmospheric turbulence. Hyperspectral imaging systems capture a three-dimensional (3D) datacube (tensor) containing two-dimensional (2D) spatial information and one-dimensional (1D) spectral information at each spatial location. Pixel intensities vary with wavelength bands, providing a spectral trace of intensity values and generating a spatial map of spectral variation (spectral signatures of materials). The deblurring and spectral unmixing problem is quite challenging since the point spread function (PSF) depends on the imaging system as well as the seeing conditions and is wavelength varying. We show how to efficiently construct an optimal Kronecker product--based preconditioner, and provide numerical methods for estimating the multiple PSFs using spectral data from an isolated (guide) star for joint deblurring and sparse unmixing of HSI datasets in order to spectrally analyze the image objects. The methods are illustrated with numerical experiments on a commonly used test example, a simulated HSI of the Hubble Space Telescope satellite.

Effective Feature Extraction and Data Reduction in Remote Sensing Using Hyperspectral Imaging [Applications Corner

With numerous and contiguous spectral bands acquired from visible light (400?1,000 nm) to (near) infrared (1,000?1,700 nm and over), hyperspectral imaging (HSI) can potentially identify different objects by detecting minor changes in temperature, moisture, and chemical content. As a result, HSI has been widely applied in a number of application areas, including remote sensing [1]. HSI data contains two-dimensional (2-D) spatial and one-dimensional spectral information, and naturally forms a three-dimensional (3-D) hypercube with a high spectral resolution in nanometers that enables robust discrimination of ground features. However, new challenges arise in dealing with extremely large data sets. For a hypercube with relatively small spatial dimension of 600 ? 400 pixels at 16 bits-per-band-per-pixel, the data volume becomes 120 MB for 250 spectral bands. In some cases, this large data volume can be linearly increased when multiple hypercubes are acquired across time to monitor system dynamics in consecutive time instants. When the ratio between the feature dimension (spectral bands) and the number of data samples (in vector-based pixels) is vastly different, high-dimensional data suffers from the well-known curse of dimensionality. For feature extraction and dimensionality reduction, principal components analysis (PCA) is widely used in HSI [2], where the number of extracted components is significantly reduced compared to the original feature dimension, i.e., the number of spectral bands. For effective analysis of large-scale data in HSI, conventional PCA faces three main challenges: