Single Two-dimensional Image Research Articles

Mass concentration of plastic particles from two-dimensional images

Imaging techniques play a crucial role in characterizing environmental plastics. However, most reported findings rely on two-dimensional projections of particles resting on flat surfaces. This limitation makes it challenging to accurately determine mass concentration, which is essential for deriving toxicologically relevant exposure data. The primary issue arises from the loss of information regarding particle height or thickness. This study aims to evaluate the assumptions necessary to compensate this loss of information. To achieve this, we used a set of environmental plastic particles, mesoplastics and microplastics, from marine campaigns, and precisely measured their three spatial dimensions and mass. Our study demonstrated the feasibility of estimating the mass of plastic particles through two-dimensional images. However, for enhanced accuracy, additional information derived from the dataset of particles under examination is necessary. Specifically, estimating the mass of platelike particles requires information about their height. Similarly, calculating the volume for elongated shapes as cylinders, should be limited to particles with the same width and height and for which their length can be precisely determined, even if the image depicts twisted forms. In conclusion, while obtaining mass concentration from single two-dimensional images enables reasonable estimations, achieving the precision needed for exposure data requires acquiring additional information from the sample and carefully considering the shape of each individual particle.

Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches.

The field of computer vision has been focusing on achieving accurate three-dimensional (3D) object representations from a single two-dimensional (2D) image through deep artificial neural networks. Recent advancements in 3D shape reconstruction techniques that combine structured light and deep learning show promise in acquiring high-quality geometric information about object surfaces. This paper introduces a new single-shot 3D shape reconstruction method that uses a nonlinear fringe transformation approach through both supervised and unsupervised learning networks. In this method, a deep learning network learns to convert a grayscale fringe input into multiple phase-shifted fringe outputs with different frequencies, which act as an intermediate result for the subsequent 3D reconstruction process using the structured-light fringe projection profilometry technique. Experiments have been conducted to validate the practicality and robustness of the proposed technique. The experimental results demonstrate that the unsupervised learning approach using a deep convolutional generative adversarial network (DCGAN) is superior to the supervised learning approach using UNet in image-to-image generation. The proposed technique's ability to accurately reconstruct 3D shapes of objects using only a single fringe image opens up vast opportunities for its application across diverse real-world scenarios.

Three-dimensional reconstruction of granular porous media based on deep generative models.

Reconstruction of microstructure in granular porous media, which can be viewed as granular assemblies, is crucial for studying their characteristics and physical properties in various fields concerned with the behavior of such media, including petroleum geology and computational materials science. In spite of the fact that many existing studies have investigated grain reconstruction, most of them treat grains as simplified individuals for discrete reconstruction, which cannot replicate the complex geometrical shapes and natural interactions between grains. In this work, a hybrid generative model based on a deep-learning algorithm is proposed for high-quality three-dimensional (3D) microstructure reconstruction of granular porous media from a single two-dimensional (2D) slice image. The method extracts 2D prior information from the given image and generates the grain set as a whole. Both a self-attention module and effective pattern loss are introduced in a bid to enhance the reconstruction ability of the model. Samples with grains of varied geometrical shapes are utilized for the validation of our method, and experimental results demonstrate that our proposed approach can accurately reproduce the complex morphology and spatial distribution of grains without any artificiality. Furthermore, once the model training is complete, rapid end-to-end generation of diverse 3D realizations from a single 2D image can be achieved.

Single exposure passive three-dimensional information reconstruction based on an ordinary imaging system

Existing three-dimensional (3D) imaging technologies have issues such as requiring active illumination, multiple exposures, or coding modulation. We propose a passive single 3D imaging method based on an ordinary imaging system. Using the point spread function of the imaging system to realize the non-coding measurement on the target, the full-focus images and depth information of the 3D target can be extracted from a single two-dimensional (2D) image through the compressed sensing algorithm. Simulation and experiments show that this approach can complete passive 3D imaging based on an ordinary imaging system without any coding operations. This method can achieve millimeter-level vertical resolution under single exposure conditions and has the potential for real-time dynamic 3D imaging. It improves the efficiency of 3D information detection, reduces the complexity of the imaging system, and may be of considerable value to the field of computer vision and other related applications.

3D reconstruction of unlimited-size real-world porous media by combining a BicycleGAN-based multimodal dictionary and super-dimension reconstruction

Three-dimensional reconstruction from a single two-dimensional image is important in the field of numerical simulation of porous media (including sandstones, composite, and alloys). Owing to limitations of computer storage space, the algorithm itself, and GPU memory, traditional algorithms and increasingly popular deep-learning algorithms cannot reconstruct complete real-size core images. To solve this problem, we propose a generative adversarial network dictionary-based super-dimension (GANDB-SD) algorithm that uses a BicycleGAN model to establish eight multimodal mappings of small-size structures. These mapping relations are used as the mapping dictionary when the super-dimension reconstruction framework is used to reconstruct the real core images without size limitation. Based on this algorithm, we further propose a three-dimensional co-occurrence matrix function as the loss function for the first time and propose a diversity preservation strategy to solve the pattern repetition problem during reconstruction. We performed an experiment to reconstruct a 5123 image, in which the morphological parameters and reconstruction time of different algorithms were compared. Results show that the algorithm has advantages in reconstruction accuracy and time. We further reconstructed 10003 real cores, and the statistical and morphological similarities between the reconstructed and target structures proved that the proposed algorithm is competent in the task of reconstructing real-world cores with unlimited size. We remark that our approach opens the door to a new methodology that incorporating machine learning/deep learning techniques into conventional super-dimension methods to address 3D reconstruction of unlimited-size real-world porous media issues in a wide range of related fields.

Preliminary exploration of geometrical regularized SAR tomography

Synthetic Aperture Radar Tomography (TomoSAR) is a popular technique in the SAR imaging community due to its ability to solve the problem of layover and obtain a three-dimensional model of observed scene. However, current TomoSAR imaging methods rely on estimating the reflectivity distribution in the elevation direction pixel by pixel based on multi-baseline observations. This approach requires tens or even hundreds of repeated-track observations to achieve satisfactory results, which limits its applicability in time-sensitive applications. With the knowledge that humans can infer three-dimensional structures from a single two-dimensional SAR image, we propose a geometrical regularized TomoSAR imaging approach that leverages three-dimensional clues. The approach first extracts geometrical structures, which are then used to estimate the expected scatterer distribution using the initial reconstructed point cloud. This reduces uncertainty in the scatterer’s position and allows for an adaptive restriction of the search range based on different metrics, such as signal-to-noise level. As a result, the proposed method yields better performance with fewer observations and lower signal-to-noise levels. Simulation results demonstrate the superior performance of our proposed method over classical TomoSAR imaging methods in several aspects. Additionally, experiments with our own multi-baseline systems show that the proposed method outperforms traditional approaches with respect to 3D entropy, and outliers are significantly restricted to the estimated geometrical structures, which confirms the effectiveness of our proposed method.

Reconstruction of 3D Random Media from 2D Images: Generative Adversarial Learning Approach

This paper presents an algorithm for stochastic reconstruction of three-dimensional material microstructure from its single two-dimensional cross-sectional image, based on the neural network operating on a principle of generative adversarial learning. The novelty of the proposed algorithm is in introduction of the reconstruction error, which is invariant to translational and rotational transformations and increases stability of the neural-network training and quality of generated structures. It is shown that a use of variational autoencoder helps to extract useful features from a cross-sectional image and provide additional information to a generator for accurate structure reconstruction. A set of 3D microstructures with corresponding 2D slice from each of them is required for model training. The model was trained and tested on sets of binary microstructures of porous materials with open-cell and closed-cell internal morphology. The obtained results for statistical evaluation of material microstructure demonstrate the effectiveness of the proposed algorithm.

Perspective Transformer and MobileNets-Based 3D Lane Detection from Single 2D Image

Three-dimensional (3D) lane detection is widely used in image understanding, image analysis, 3D scene reconstruction, and autonomous driving. Recently, various methods for 3D lane detection from single two-dimensional (2D) images have been proposed to address inaccurate lane layouts in scenarios (e.g., uphill, downhill, and bumps). Many previous studies struggled in solving complex cases involving realistic datasets. In addition, these methods have low accuracy and high computational resource requirements. To solve these problems, we put forward a high-quality method to predict 3D lanes from a single 2D image captured by conventional cameras, which is also cost effective. The proposed method comprises the following three stages. First, a MobileNet model that requires low computational resources was employed to generate multiscale front-view features from a single RGB image. Then, a perspective transformer calculated bird’s eye view (BEV) features from the front-view features. Finally, two convolutional neural networks were used for predicting the 2D and 3D coordinates and respective lane types. The results of the high-reliability experiments verified that our method achieves fast convergence and provides high-quality 3D lanes from single 2D images. Moreover, the proposed method requires no exceptional computational resources, thereby reducing its implementation costs.

Comparison of accuracy and reproducibility of colposcopic impression based on a single image versus a two-minute time series of colposcopic images

ObjectiveColposcopy is an important part of cervical screening/management programs. Colposcopic appearance is often classified, for teaching and telemedicine, based on static images that do not reveal the dynamics of acetowhitening. We compared the accuracy and reproducibility of colposcopic impression based on a single image at one minute after application of acetic acid versus a time-series of 17 sequential images over two minutes. MethodsApproximately 5000 colposcopic examinations conducted with the DYSIS colposcopic system were divided into 10 random sets, each assigned to a separate expert colposcopist. Colposcopists first classified single two-dimensional images at one minute and then a time-series of 17 sequential images as ‘normal,’ ‘indeterminate,’ ‘high grade,’ or ‘cancer’. Ratings were compared to histologic diagnoses. Additionally, 5 colposcopists reviewed a subset of 200 single images and 200 time series to estimate intra- and inter-rater reliability. ResultsOf 4640 patients with adequate images, only 24.4% were correctly categorized by single image visual assessment (11% of 64 cancers; 31% of 605 CIN3; 22.4% of 558 CIN2; 23.9% of 3412 < CIN2). Individual colposcopist accuracy was low; Youden indices (sensitivity plus specificity minus one) ranged from 0.07 to 0.24. Use of the time-series increased the proportion of images classified as normal, regardless of histology. Intra-rater reliability was substantial (weighted kappa = 0.64); inter-rater reliability was fair ( weighted kappa = 0.26). ConclusionSubstantial variation exists in visual assessment of colposcopic images, even when a 17-image time series showing the two-minute process of acetowhitening is presented. We are currently evaluating whether deep-learning image evaluation can assist classification.

Cascaded Progressive Generative Adversarial Networks for Reconstructing Three-Dimensional Grayscale Core Images From a Single Two-Dimensional Image

It is very challenging to accurately understand and characterize the internal structure of three-dimensional (3D) rock masses using geological monitoring and conventional laboratory measures. One important method for obtaining 3D core images involves reconstructing their 3D structure from two-dimensional (2D) core images. However, traditional 2D–3D reconstruction methods are mostly designed for binary core images, rather than grayscale images. Furthermore, the reconstruction structure cannot reflect the gray level distribution of the core. Here, by combining the dimension promotion theory in super-dimension (SD) reconstruction and framework of deep learning, we propose a novel convolutional neural network framework, the cascaded progressive generative adversarial network (CPGAN), to reconstruct 3D grayscale core images. Within this network, we propose a loss function based on the gray level distribution and pattern distribution to maintain the texture information of the reconstructed structure. Simultaneously, by adopting SD dimension promotion theory, we set the input and output of every single node of the CPGAN network to be deep gray-padding structures of equivalent size. Through the cascade of every single node network, we thus ensured continuity and variability between the reconstruction layers. In addition, we used 3D convolution to determine the spatial characteristics of the core. The reconstructed 3D results showed that the gray level information in the 2D image were accurately reflected in the 3D space. This proposed method can help us to understand and analyze various parameter characteristics in cores.

Arbitrary image reinflation: A deep learning technique for recovering 3D photoproduct distributions from a single 2D projection.

Many charged particle imaging measurements rely on the inverse Abel transform (or related methods) to reconstruct three-dimensional (3D) photoproduct distributions from a single two-dimensional (2D) projection image. This technique allows for both energy- and angle-resolved information to be recorded in a relatively inexpensive experimental setup, and its use is now widespread within the field of photochemical dynamics. There are restrictions, however, as cylindrical symmetry constraints on the overall form of the distribution mean that it can only be used with a limited range of laser polarization geometries. The more general problem of reconstructing arbitrary 3D distributions from a single 2D projection remains open. Here, we demonstrate how artificial neural networks can be used as a replacement for the inverse Abel transform and-more importantly-how they can be used to directly "reinflate" 2D projections into their original 3D distributions, even in cases where no cylindrical symmetry is present. This is subject to the simulation of appropriate training data based on known analytical expressions describing the general functional form of the overall anisotropy. Using both simulated and real experimental data, we show how our arbitrary image reinflation (AIR) neural network can be utilized for a range of different examples, potentially offering a simple and flexible alternative to more expensive and complicated 3D imaging techniques.

A hybrid of statistical and conditional generative adversarial neural network approaches for reconstruction of 3D porous media (ST-CGAN)

A coupled statistical and conditional generative adversarial neural network is used for 3D reconstruction of both homogeneous and heterogeneous porous media from a single two-dimensional image. A statistical approach feeds the deep network with conditional data, and then the reconstruction is trained on a deep generative network. The conditional nature of the generative model helps in network stability and convergence which has been optimized through a gradient-descent-based optimization method. Moreover, this coupled approach allows the reconstruction of heterogeneous samples, a critical and serious challenge in conventional reconstruction methods. The main contribution of this work is to develop an adaptable framework that can efficiently reconstitute heterogeneous porous media using the power of conditional generative adversarial networks. The reconstruction time is accelerated approximately 1000-fold compared to traditional statistical reconstruction methods. Various matching criteria in both morphological and physical characteristics are used to evaluate the model performance. To validate the approach, the reconstructed realizations have been compared to the models generated by a conventional 3D GAN along with a well-known statistical method. The results confirm that the proposed approach is a reliable framework for extracting information from a single 2D image to reconstruct 3D microstructures.

HNet: Single-shot 3D shape reconstruction using structured light and h-shaped global guidance network

Retrieving three-dimensional (3D) shape information from a single two-dimensional (2D) image has recently gained enormous attention in a variety of fields. In spite of recent advancements in algorithms and hardware developments, the easy-to-use characteristics and the accuracy of the 3D shape reconstruction are always of great interest. This paper presents a robust 3D shape reconstruction technique that integrates structured-light 3D imaging scheme with deep convolutional neural network (CNN) learning. The structured-light patterns facilitate the featuring process while the CNN modeling surpasses the complexity of the traditional 3D shape reconstructions. In the supervised learning pipeline, the input is either a single fringe-pattern or a single speckle-pattern image, and the output is its corresponding high-accuracy 3D shape label. Unlike the well-received autoencoder-based CNN model, a global guidance network path with multi-scale feature fusion is introduced into the CNN model to improve the accuracy of the 3D shape reconstruction. Experimental evaluations have been conducted to demonstrate the validity and robustness of the proposed technique, which provides a promising tool for ever-increasing scientific research and engineering applications.

Permeability prediction for porous sandstone using digital twin modeling technology and Lattice Boltzmann method

The permeability of porous rock is a crucial indicator evaluating its ability to allow liquids to pass through it. However, challenges are always encountered in measurement and prediction of permeability, including the complicated process of laboratory experiments and the refined 3D numerical reconstruction. In order to improve the efficiency and accuracy of permeability prediction, the digital twin modeling technology is proposed to generate homologous and heterogeneous digital sandstone models based on a single two-dimensional image in this paper. Comparison of the CT reconstruction with the digital twin of sandstone indicates that they are very similar in complementary solid-phase morphology of porous media, global parameters (total porosity, fractal dimension, tortuosity), and local parameters (porosity of sequence layer, pore and throat characteristics, coordination number). In addition, the permeability of CT reconstruction and ten digital twins are analyzed by using the Lattice Boltzmann Method. The results show that the permeability of digital twins is very stable and close to the CT reconstruction at three orthogonal directions. The findings in this study may provide an accurate, stable, rapid and economic method for predicting the permeability of porous media.

Visual Perception: How Better Imaging Can Make Things Worse

Summary Visual search is critical to daily life and to socially important tasks — from cancer screening to airport security. New research shows how a technological advancement can interact with the human visual system to improve search for one type of target while making matters worse for another. Part of the problem is that we are surprisingly bad at knowing where we have looked.

Two-Dimensional Image-Based Screening Tool for Infants with Positional Cranial Deformities: A Machine Learning Approach.

Positional cranial deformities are relatively common conditions, characterized by asymmetry and changes in skull shape. Although three-dimensional (3D) scanning is the gold standard for diagnosing such deformities, it requires expensive laser scanners and skilled maneuvering. We therefore developed an inexpensive, fast, and convenient screening method to classify cranial deformities in infants, based on single two-dimensional vertex cranial images. In total, 174 measurements from 80 subjects were recorded. Our screening software performs image processing and machine learning-based estimation related to the deformity indices of the cranial ratio (CR) and cranial vault asymmetry index (CVAI) to determine the severity levels of brachycephaly and plagiocephaly. For performance evaluations, the estimated CR and CVAI values were compared to the reference data obtained using a 3D cranial scanner. The CR and CVAI correlation coefficients obtained via support vector regression were 0.85 and 0.89, respectively. When the trained model was evaluated using the unseen test data for the three CR and three CVAI classes, an 86.7% classification accuracy of the proposed method was obtained for both brachycephaly and plagiocephaly. The results showed that our method for screening cranial deformities in infants could aid clinical evaluations and parental monitoring of the progression of deformities at home.

An auditory scaling method for reverb synthesis from a single two-dimensional image

An auditory scaling method for reverb synthesis from a single two-dimensional image

An end-to-end three-dimensional reconstruction framework of porous media from a single two-dimensional image based on deep learning

Stochastically reconstructing a three-dimensional (3D) structure of porous media from a given two-dimensional (2D) image is an outstanding problem. For such problem, despite that the big progress has been made on reconstruction methods such as optimization-based and multi-point statistics-based algorithms, however, the reconstruction of topologically complex or non-stationary materials is still not well addressed. Besides, the reconstruction efficiency is another remarkable challenge, and a 1283 reconstruction using these methods generally requires several hours. In this paper, to overcome these problems, we propose a general end-to-end deep learning-based 3D reconstruction framework. Specially, the mapping (function) between a 2D slice and its 3D structure is first learned by a neural network. Then, the 3D reconstruction of a new 2D image using this mapping is instantaneous. For a 1283 reconstruction, our method only requires 0.2s, thus achieving a 3.6×104 speedup factor compared with the classical methods. Besides, to yield diverse 3D structures for the same 2D input, a Gaussian noise is introduced into the network. Our approach is tested on two statistically isotropic materials and a non-stationary porous material, and evaluated in terms of both visual and quantitative comparisons. Experimental results indicate that the proposed method is accurate, fast, and stable. The proposed framework also enables that theoretically an arbitrary number of constraints can be incorporated to further improve the reconstruction accuracy.

Deep learning speeds up video compressive sensing from days to minutes

Researchers can recover a video from a single two-dimensional image within tens of milliseconds (or sub-seconds), thanks to deep learning.

Automatic Diagnosis Based on Spatial Information Fusion Feature for Intracranial Aneurysm.

Timely and accurate auxiliary diagnosis of intracranial aneurysm can help radiologist make treatment plans quickly, saving lives and cutting costs at the same time. At present, Digital Subtraction Angiography (DSA) is the gold standard for the diagnosis of intracranial aneurysm, but as radiologists interpret those imaging sequences frame by frame, misdiagnosis might occur. The utilization of computer-aided diagnosis (CAD) can ease the burdens of radiologists and improve the detection accuracy of aneurysms. In this article, a deep learning method is applied to detect the intracranial aneurysm in 3D Rotational Angiography (3D-RA) based on a spatial information fusion (SIF) method, and instead of a 3D vascular model, 2D image sequences are used. Given the intracranial aneurysm and vascular overlap having similar feature in the most time, rather than focusing on distinguishing them in one frame, the morphological differences between frames are considered as major feature. In the training data, consecutive frames of every imaging time series are extracted and concatenated in a specific way, so that the spatial contextual information could be embedded into a single two-dimensional image. This method enables the time series with obvious correlation between frames be directly trained on 2D convolutional neural network (CNN), instead of 3D-CNN with huge computational cost. Finally, we got an accuracy of 98.89%, with sensitivity and specificity of 99.38% and 98.19%, respectively, which proves the feasibility and availability of the SIF feature.