Real SEM Images Research Articles

Improving the Neural Segmentation of Blurry Serial SEM Images by Blind Deblurring.

Serial scanning electron microscopy (sSEM) has recently been developed to reconstruct complex largescale neural connectomes, through learning-based instance segmentation. However, blurry images are inevitable amid prolonged automated data acquisition due to imprecision in autofocusing and autostigmation, which impose a great challenge to accurate segmentation of the massive sSEM image data. Recently, learning-based methods, such as adversarial learning and supervised learning, have been proven to be effective for blind EM image deblurring. However, in practice, these methods suffer from the limited training dataset and the underrepresentation of high-resolution decoded features. Here, we propose a semisupervised learning guided progressive decoding network (SGPN) to exploit unlabeled blurry images for training and progressively enrich high-resolution feature representation. The proposed method outperforms the latest deblurring models on real SEM images with much less ground truth input. The improvement of the PSNR and SSIM is 1.04 dB and 0.086, respectively. We then trained segmentation models with deblurred datasets and demonstrated significant improvement in segmentation accuracy. The A-rand (Bogovic et al. 2013) decreased by 0.119 and 0.026, respectively, for 2D and 3D segmentation.

Spatial Image Resolution Assessment by Fourier Analysis (SIRAF).

Determining spatial resolution from images is crucial when optimizing focus, determining smallest resolvable object, and assessing size measurement uncertainties. However, no standard algorithm exists to measure resolution from electron microscopy (EM) images, though several have been proposed, where most require user decisions. We present the Spatial Image Resolution Assessment by Fourier analysis (SIRAF) algorithm that uses fast Fourier transform analysis to estimate resolution directly from a single image without user inputs. The method is derived from the underlying assumption that objects display intensity transitions, resembling a step function blurred by a Gaussian point spread function. This hypothesis is tested and verified on simulated EM images with known resolution. To identify potential pitfalls, the algorithm is also tested on simulated images with a variety of settings, and on real SEM images acquired at different magnification and defocus settings. Finally, the versatility of the method is investigated by assessing resolution in images from several microscopy techniques. It is concluded that the algorithm can assess resolution from a large selection of image types, thereby providing a measure of this fundamental image parameter. It may also improve autofocus methods and guide the optimization of magnification settings when balancing spatial resolution and field of view.

Mask defect detection with hybrid deep learning network

Background: Deep learning is a very fast-growing field in the area of artificial intelligence with remarkable results in recent years. Many works in the lithography and photomask field have shown progress of technology in this application area and the potential to improve lithography by the automation of processes. Despite this progress, the use of machine learning techniques in the field of mask repair still seems to be at the beginning. Aim: We show that deep learning-based methods can successfully be applied to mask repair applications. Approach: The presented system is a hybrid and modular approach based on a combination of several deep learning networks and analytical methods, enabling the detection of mask pattern defects and the determination of the exact defect shapes from SEM images. In the current version, the system is trained for line/space patterns and contact patterns with typical defect types. The modularity allows for the extensibility to new use cases. The issue of an insufficient amount of training data is addressed using purely computer-generated simplified SEM data in combination with a specific network architecture. Results: The very good functionality and defect detection accuracy of the system are demonstrated with a set of real SEM images with line/space patterns and contact patterns with numerous defects. In particular, a 100% true defect detection rate could be obtained. Conclusions: The presented machine learning approach demonstrates the successful defect identification, location, and shape determination from real mask SEM images.

Deep learning for scanning electron microscopy: Synthetic data for the nanoparticles detection

Deep learning algorithms are one of most rapid developing fields into the modern computation technologies. One of the bottlenecks into the implementation of such advaced algorithms is their requirement for a large amount of manually-labelled data for training. For the general-purpose tasks, such as general purpose image classification/detection the huge images datasets are already labelled and collected. For more subject specific tasks (such as electron microscopy images treatment), no labelled data available. Here I demonstrate that a deep learning network can be successfully trained for nanoparticles detection using semi-synthetic data. The real SEM images were used as a textures for rendered nanoparticles at the surface. Training of RetinaNet architecture using transfer learning can be helpful for the large-scale particle distribution analysis. Beyond such applications, the presented approach might be applicable to other tasks, such as image segmentation.

Towards Automated Electron Microscopy Image Segmentation for Nanoparticles of Complex Shape by Convolutional Neural Networks

In this contribution different ways are explored with the aim to generate suitable training data for ‘non-ideal’ samples using various approaches, e.g., computer-generated images or unsupervised learning algorithms such as generative adversarial networks (GANs). We used these data to train simple CNNs to produce segmentation masks of SEM images and tested the trained networks on real SEM images of complex nanoparticle samples. The novel use of CNN for the automated analysis of the size of nanoparticles of complex shape and with a high degree of agglomeration has proved to be a promising tool for the evaluation of particle size distribution on a large number of constituent particles. Further development and validation of the preliminary model, respectively larger training and validation data sets are necessary.

Tilted beam scanning electron microscopy, 3-D metrology for microelectronics industry

In the microelectronics industry, most of the dimensional metrology relies on critical dimension (CD) estimation. These measurements are mainly performed by critical dimension scanning electron microscopy, because it is a very fast, mainly nondestructive method and enables direct measurements on wafers. To measure CDs, the distance is estimated between the edges of the observed pattern on an SEM image. As the CD becomes smaller and smaller, the needs for more reliable metrology techniques emerge. In order to obtain more meaningful and reproducible CD measurements regardless of the pattern type (line, space, contact, hole, etc.), one needs to perform a CD measurement at a known and constant height due to a methodology that determines the topographic shape of the pattern from SEM images. An SEM capable of bending the electron beam (up to 12 deg in our case) allows images to be caught at different angles, giving access to more information. From the analysis of such images, pattern height and sidewall angles can be determined using geometric considerations. Understanding interaction between three-dimensional (3-D) shapes, pattern materials, and the electron beam becomes essential to correlate topography information. A preliminary work based on Monte–Carlo simulations was conducted using JMONSEL, a software developed by the National Institute of Standards and Technology. With this analysis, it is possible to determine theoretical trends for different topographies and beam tilt conditions. Due to the effects highlighted by simulations, the processing of the tilted beam SEM images will be presented, as well as the method used to create a mathematical model allowing topographic reconstruction from these images. Finally some reconstruction using this model will be shown and compared to reference measurements. The overall flow used to process images is presented. First, images are transformed into grayscale profiles. After a smoothing procedure, positional descriptors are computed for specific profile derivatives values. Then, from these descriptors coming from two images of the same pattern taken at different tilt angles, we use a low-complexity linear model in order to obtain the geometrical parameters of the structure. This model is created and initially calibrated using JMONSEL simulations and then recalibrated on real silicon patterns. We demonstrate that the use of real SEM images coming from real silicon patterns with our model leads to results that are coherent with conventional 3-D measurements techniques taken as reference. Moreover, we are able to make reliable reconstructions on patterns of various heights with a single calibrated model. Our batch of experiment shows a three-sigma standard deviation of 10 nm on the estimated height for heights ranging from 50 nm to more than 200 nm. Based on simulations, we are able to reconstruct the corner rounding (CR) from SEM images. However, because our wafer has no CR variability, measurements still need to be assessed on real wafer.

Average fiber diameter measurement in Scanning Electron Microscopy images based on Gabor filtering and Hough transform

In this paper, an algorithm is proposed for determining micro/nano fiber diameter from Scanning Electron Microscopy images. The proposed algorithm is simple to apply and design, which makes it practical algorithm. In this algorithm, first, robust key points on the fibers' skeleton are selected using Hough transform. Then with the help of Gabor filter bank, co-oriented fibers are detected in different scales. Fiber diameter is determined from the weighted average of the distance of these key points from the fiber edges. The contrast level of Gabor filters output plays an important role to determine the weights. The proposed algorithm is validated using both of digital synthetic images of gray lines on a black background and real SEM images. The results of these experiments show that the proposed algorithm has a high precision in determining the average diameter of fiber web in the sense of having a lower mean absolute error.

Peridynamic micromechanical prediction of nonlocal damage initiation and propagation in DP steels based on real microstructure

Dual phase steels are the complex heterogeneous materials that are widely used in various automobile industries. It is important to develop numerical methods to enhance the capability of the damage mechanism prediction. Dual phase steels consist of soft ferrite and hard martensite phases and the grain boundaries play the main role as the important location with possible occurrence of damage in these materials. In this study, a microstructure-based approach using representative volume elements is utilized to evaluate the damage initiation and propagation. The material is modeled at the microscopic scale using real microstructural SEM images. A two-dimensional model is developed and the damage mechanisms of dual phase steels are simulated using a bond based peridynamic microstructural model. According to the peridynamic simulation, no mesh refinement is necessary, as damage is inherently characterized by bond breakages and the obtained damage patterns are compared with experimental SEM images. Finally, a sensitivity analysis is conducted to understand the effect of peridynamic discretization, real microstructure, horizon size and interface behavior on the damage pattern of the proposed model under tension static loading. It is shown that the obtained damage initiation and propagation patterns are in good agreement with the experimental results.

Shape‐from‐shading and simulation of SEM images using surface slope and curvature

AbstractWe have developed an alternative approach to Monte Carlo simulation for simulating SEM images, which is several orders of magnitude faster to compute, simulating an image in a fraction of a second. This approach also enables one to solve the inverse problem of reconstructing surface topography from SEM images within minutes to hours. Our method is limited to surfaces that are height fields and that are composed of only a single material. Also, we assume there are no charging affects and that the shading of the surface is rotation invariant (no detector asymmetry).We represent the input surface to our simulator as a height image with the same resolution as the desired SEM image. An SEM image is simulated from the height image by first convolving it with each of a set of filter kernels. Each of these filter kernels extracts local information about slope and curvature at different scales, and the resulting filtered images are combined to produce the simulated image.We fit our model to examples of height images and corresponding SEM images that are either generated synthetically using the Monte Carlo method or acquired from an actual SEM. In the case of real SEM images, the corresponding height image must be either known from design data or measured independently (e.g. using an AFM). Copyright © 2005 John Wiley & Sons, Ltd.