Classification Of Microstructures Research Articles

Overview: Machine Learning for Segmentation and Classification of Complex Steel Microstructures

The foundation of materials science and engineering is the establishment of process–microstructure–property links, which in turn form the basis for materials and process development and optimization. At the heart of this is the characterization and quantification of the material’s microstructure. To date, microstructure quantification has traditionally involved a human deciding what to measure and included labor-intensive manual evaluation. Recent advancements in artificial intelligence (AI) and machine learning (ML) offer exciting new approaches to microstructural quantification, especially classification and semantic segmentation. This promises many benefits, most notably objective, reproducible, and automated analysis, but also quantification of complex microstructures that has not been possible with prior approaches. This review provides an overview of ML applications for microstructure analysis, using complex steel microstructures as examples. Special emphasis is placed on the quantity, quality, and variance of training data, as well as where the ground truth needed for ML comes from, which is usually not sufficiently discussed in the literature. In this context, correlative microscopy plays a key role, as it enables a comprehensive and scale-bridging characterization of complex microstructures, which is necessary to provide an objective and well-founded ground truth and ultimately to implement ML-based approaches.

Enhancing machine learning classification of microstructures: A workflow study on joining image data and metadata in CNN

AbstractIn view of the paradigm shift toward data-driven research in materials science and engineering, handling large amounts of data becomes increasingly important. The application of FAIR (findable, accessible, interoperable, reusable) data principles emphasizes the importance of metadata describing datasets. We propose a novel data processing and machine learning (ML) pipeline to extract metadata from micrograph image files, then combine image data and their metadata for microstructure classification with a deep learning approach compared to a classic ML approach. The ML model attained excellent performances with and without metadata and bears potential for performance improvement of further use cases within the community. Graphical abstract

Generic dual-phase classification models through deep learning semantic segmentation method and image gray-level optimization

Two generic deep learning models for automatic classification of dual-phase microstructures were constructed through the semantic segmentation method of DeepLab v3+, based on a small data including 6 SEM images of the U-2Nb alloys with dual-phase microstructure and the corresponding ground truth. One model is suitable for directly classifying images with different gray-level ranges and has a good average classifying accuracy. Although the other model is only suitable for a specific gray-level range, but it cost less for training and can realize a higher classifying accuracy than the former model combined with optimal gray-level adjustment for images. The two DL models were applied to classify dual-phase microstructures of different morphologies and materials, including the isothermal cooling and continuous cooling microstructures of U-2Nb alloys, and the dual-phase steel microstructure, the results of which manifest that both the two models have high classification accuracy and excellent general applicability.

Automated characterization and classification of 3D microstructures: an application to 3D deformation twin networks in titanium

The advent of techniques enabling three-dimensional (3D) analysis of objects, defects, and fields has been key to discoveries and paradigm shifts in molecular biology, astrophysics, medicine, quantum physics, etc. In materials science, the 3D nature of materials microstructures remains largely hidden; leading to a fragmented understanding of microstructure-property linkages. Current tools cannot characterize large volumes of 3D microstructures at fine resolution. To this end, this study introduces a graph-theory-based framework to automatically extract 3D microstructures and statistics of electron-backscatter diffraction datasets. Further, leveraging network science, the study introduces a new approach to classify and compare microstructures; the keystone to materials taxonomy. The significance of this tool is demonstrated by studying deformation twin structures in Titanium. The study reveals extraordinarily complex and tortuous twin networks never observed via traditional two-dimensional analysis. This changes our perception of the ability of metals to withstand severe microstructure changes without failing.

Effectiveness of Machine-Learning and Deep-Learning Strategies for the Classification of Heat Treatments Applied to Low-Carbon Steels Based on Microstructural Analysis

This work aims to compare the effectiveness of different machine-learning techniques for the image classification of steel microstructures. For this, we use a set of samples of hypoeutectoid steels subjected to three heat treatments: annealing, quenching and quenching with tempering. Logically, the samples contain the typical constituents expected, and these are different for each treatment. Images are obtained by optical microscopy at 400× magnification and from different low-carbon steels to generate the data with some heterogeneity. Learning models are created with an image dataset for classification into three classes based on the respective heat treatments. Likewise, we develop two kinds of models by using, on the one hand, classical machine-learning methods based on the “bag of features” technique and, on the other hand, convolutional neural networks (CNN) with a transfer-learning approach by using GoogLeNet and ResNet50. We demonstrate the superiority of deep-learning techniques (CNN) over classical machine-learning methods.

Image-driven deep-learning-enabled automatic microstructural recognition

The deep learning (DL) method consisting of a convolutional neural network (CNN) was employed to automate the task of microstructural recognition and classification to identify dendritic characteristics in metallic microstructures. Dendrites are an important feature that decide the mechanical properties of an alloy; further, the dendritic arm spacing is critical in ascertaining the values of strength and ductility. The current work was divided into two tasks – namely, classification of microstructures into dendritic and non-dendritic microstructures (task 1) and further classification of the dendritic microstructures into longitudinal and transverse cross-sectional views (task 2). The data set comprised micrographs from experimental and online sources covering a broad range of alloy compositions, micrograph magnifications and orientations. The tasks were achieved by employing a four-layered CNN to yield an accuracy of 97.17 ± 0.64% for task 1 and 87.86 ± 1.07% for task 2 independently. The employment of the DL model for classification of microstructures circumvents the feature extraction step while ensuring high accuracy. This work reduces dependency on skilled and experienced researchers and expedites the material development cycle.

Automated, high-accuracy classification of textured microstructures using a convolutional neural network

Crystallographic texture is an important descriptor of material properties but requires time-intensive electron backscatter diffraction (EBSD) for identifying grain orientations. While some metrics such as grain size or grain aspect ratio can distinguish textured microstructures from untextured microstructures after significant grain growth, such morphological differences are not always visually observable. This paper explores the use of deep learning to classify experimentally measured textured microstructures without knowledge of crystallographic orientation. A deep convolutional neural network is used to extract high-order morphological features from binary images to distinguish textured microstructures from untextured microstructures. The convolutional neural network results are compared with a statistical Kolmogorov–Smirnov tests with traditional morphological metrics for describing microstructures. Results show that the convolutional neural network achieves a significantly improved classification accuracy, particularly at early stages of grain growth, highlighting the capability of deep learning to identify the subtle morphological patterns resulting from texture. The results demonstrate the potential of a convolutional neural network as a tool for reliable and automated microstructure classification with minimal preprocessing.

Polarization clustering of biological structures with Mueller matrix parameters.

Mueller matrix imaging polarimetry (MMIP) is a promising technique for the characterization of biological tissues, including the classification of microstructures in pathological diagnosis. To expand the parameter space of Mueller matrix parameters, we propose new vector parameters (VPs) according to the Mueller matrix polar decomposition method. We measure invasive bladder cancer (IBC) with extensive necrosis and high-grade ductal carcinoma in situ (DCIS) with MMIP, and the regions of cancer cells and fibrotic stroma are classified with the VPs. Then the proposed and existing VPs are mapped on the Poincaré sphere with 3D visualization, and an indicator of spatial feature is defined based on the minimum enclosing sphere to evaluate the classification capability of the VPs. For both IBC and DCIS, the results show that the proposed VPs exhibit evident contrast between the regions of cancer cells and fibrotic stroma. This study broadens the fundamental Mueller matrix parameters and helps to improve the characterization ability of the MMIP technique.

Classification of Microstructures of Al–Si Casting Alloy in Different Cooling Rates with Machine Learning Technique

The classification of the microstructures of Al–Si–Mg casting alloy in different cooling rates was accomplished by using our originally developed methods and machine learning techniques. The mechanical properties of the samples were slightly increased with the increase of cooling rates. The microstructures of the samples were similar because of the approximate cooling rates. High classification rates of about 80% to 90% were obtained using the software with machine learning techniques developed by us. The classification rate change with the number of images in training data was tested and a suitable number of images for training in the machine learning process was found.

A generic high-throughput microstructure classification and quantification method for regular SEM images of complex steel microstructures combining EBSD labeling and deep learning

• A novel method is proposed for complex microstructure classification and qualification combining deep learning and EBSD. • Trained models can be extended to new DP and Q&P steels not associated with the model development. • The good robustness is well elaborated by its applications to SEM images with different qualities and magnifications. • Present method shows significant advantages in both accuracy and efficiency compared to binary method and EBSD experiment. We present an electron backscattered diffraction (EBSD)-trained deep learning (DL) method integrating traditional material characterization informatics and artificial intelligence for a more accurate classification and quantification of complex microstructures using only regular scanning electron microscope (SEM) images. In this method, EBSD analysis is applied to produce accurate ground truth data for guiding the DL model training. An U-Net architecture is used to establish the correlation between SEM input images and EBSD ground truth data using only small experimental datasets. The proposed method is successfully applied to two engineering steels with complex microstructures, i.e., a dual-phase (DP) steel and a quenching and partitioning (Q&P) steel, to segment different phases and quantify phase content and grain size. Alternatively, once properly trained the method can also produce quasi-EBSD maps by inputting regular SEM images. The good generality of the trained models is demonstrated by using DP and Q&P steels not associated with the model training. Finally, the method is applied to SEM images with various states, i.e., different imaging modes, image qualities and magnifications, demonstrating its good robustness and strong application ability. Furthermore, the visualization of feature maps during the segmenting process is utilised to explain the mechanism of this method's good performance.

ALGORITHMIC STABILITY OF DEEP LEARNING NEURAL NETWORKS IN RECOGNIZING THE MICROSTRUCTURE OF MATERIALS

The division of data for training a neural network into training and test data in various proportions to each other is investigated. The question is raised about how the quality of data distribution and their correct annotation can affect the final result of constructing a neural network model. The paper investigates the algorithmic stability of training a deep neural network in problems of recognition of the microstructure of materials. The study of the stability of the learning process makes it possible to estimate the performance of a neural network model on incomplete data distorted by up to 10%. Purpose. Research of the stability of the learning process of a neural network in the classification of microstructures of functional materials. Materials and methods. Artificial neural network is the main instrument on the basis of which produced the study. Different subtypes of deep convolutional networks are used such as VGG and ResNet. Neural networks are trained using an improved backpropagation method. The studied model is the frozen state of the neural network after a certain number of learning epochs. The amount of data excluded from the study was randomly distributed for each class in five different distributions. Results. Investigated neural network learning process. Results of experiments conducted computing training with gradual decrease in the number of input data. Distortions of calculation results when changing data with a step of 2 percent are investigated. The percentage of deviation was revealed, equal to 10, at which the trained neural network model loses its stability. Conclusion. The results obtained mean that with an established quantitative or qualitative deviation in the training or test set, the results obtained by training the network can hardly be trusted. Although the results of this study are applicable to a particular case, i.e., microstructure recognition problems using ResNet-152, the authors propose a simpler technique for studying the stability of deep learning neural networks based on the analysis of a test, not a training set.

Machine Learning for Microstructure Classification: How to Assign the Ground Truth in the Most Objective Way

Abstract This article demonstrates the application of machine learning to the classification and segmentation of bainitic microstructures and compares three approaches for assigning the ground truth: correlative microscopy using EBSD as an additional information source, comparison with images obtained from specially produced reference samples, and unsupervised learning.

Microstructure representation learning using Siamese networks

Obtaining a good statistical representation of material microstructures is crucial for establishing robust process–structure–property linkages and machine learning techniques can bridge this gap. One major difficulty in leveraging recent advances in deep learning for this purpose is the scarcity of good quality data with enough metadata. In machine learning, similarity metric learning using Siamese networks has been used to deal with sparse data. Inspired by this, the authors propose a Siamese architecture to learn microstructure representations. The authors show that analysis tasks such as the classification of microstructures can be done more efficiently in the learned representation space.

Classification of steel microstructures using Modified Alternate Local Ternary Pattern

Steel is the most preferred and highly demanded material on the Earth due to its recyclable properties. It has high durability and strength which makes it an important material for modern construction. Based on the carbon content, plain carbon steel is categorized into low, medium and high carbon steel. Usually, microstructural classification is done manually by the metallurgists. Automation can be brought in with the help of digital image processing techniques. In this article, a novel texture descriptor named Modified Alternate Local Ternary Pattern (MALTP) has been introduced for automatic classification of steel microstructures. The database consists of 300 optical microstructural images of three steel grades (low, medium and high carbon steel) under 100 X magnification. In the proposed method, the texture features extracted by MALTP are used for microstructural classification with two fold (k = 2) cross validation. Classification is performed using machine learning techniques such as K-Nearest Neighbors (K-NN) and Support Vector Machine (SVM). The performance of SVM in microstructural classification with respect to different kernels like linear, polynomial and radial basis function are analyzed. Out of these machine learning techniques, SVM with radial basis function kernel provides the highest accuracy of 98.89%.

Machine Learning-Based Classification of Dislocation Microstructures

Dislocations -- the carrier of plastic deformation -- are responsible for a wide range of mechanical properties of metals or semiconductors. Those line-like objects tend to form complex networks that are very difficult to characterize or to link to macroscopic properties on the specimen scale. In this work a machine learning based approach for classification of coarse-grained dislocation microstructures in terms of different dislocation density field variables is used. The performance of the learning algorithm combined with domain knowledge from the underlying physics helps to shed light on the interplay between coarse-graining voxel size and the set of suitable or even required density variables for a faithful microstructure characterization.

Autonomous Interpretation of the Microstructure of Steels and Special Alloys

The main objective of presented research is an attempt of application of techniques taken from a dynamically developing field of image analysis based on Artificial Intelligence, particularly on Deep Learning, in classification of steel microstructures. Our research focused on developing and implementation of Deep Convolutional Neural Networks (DCNN) for classification of different types of steel microstructure photographs received from the light microscopy at the TU Bergakademie, Freiberg. First, brief presentation of the idea of the system based on DCNN is given. Next, the results of tests of developed classification system on 8 different types (classes) of microstructure of the following different steel grades: C15, C45, C60, C80, V33, X70 and carbide free steel. The DCNN based classification systems require numerous training data and the system accuracy strongly depend on the size of these data. Therefore, created data set of numerous micrograph images of different types of microstructure (33283 photographs) gave the opportunity to develop high precision classification systems and segmentation routines, reaching the accuracy of 99.8%. Presented results confirm, that DCNN can be a useful tool in microstructure classification.

An automated morphological classification of ferrite-martensite dual-phase microstructures

In this study, a simple classification scheme based on image processing is proposed to categorize optical microstructures of dual-phase steels with varying morphology (i.e., shape, size and distribution) of ferrite-martensite phases. Dual-phase microstructures with varying morphologies have been developed by employing different heat treatment schedules; namely, intercritical annealing (IA), intermediate quenching (IQ) and step quenching (SQ) using a low carbon micro-alloyed steel. The images of optical microstructures of the developed steels have been first segmented using Otsu thresholding technique and then, the ratio of the black (martensite) and white (ferrite) run-lengths for the phases have been calculated. The mean and median values of these ratios have been considered further, and it has been demonstrated that the mean values are more appropriate than the median values for the classification purpose. The mean and standard deviation of the former parameter for IA, IQ and SQ microstructures are 0.679 ± 0.040, 1.540 ± 0.090 and 0.989 ± 0.119, respectively. The simple classification technique is found to be advantageous from different aspects.

PARAMETERIZATION OF MICROSTRUCTURES IMAGES OF SPHEROIDAL GRAPHITE CAST IRON ON THE BASIS OF THE GRAPHITE DISTRIBUTION DENSITY FUNCTION OF THE INCLUSIONS SIZE

The possibilities of parameterization of microstructures such as SHG2, SHG4, SHG6, SHG10, SHG12, SHgd15, SHgd25, SHgd45, SHgd90, SHgd180, SHgd360 according to GOST (State All-union standard) 3443–87 based on the microstructures of cast iron with different diameter spheroidal graphite are shown. It is proposed to use the function of the density distribution of graphite inclusions in size for the classification of cast iron microstructures with spheroidal graphite. It is shown that density function of graphite inclusions can be used as a criterion of classification of microstructures of cast iron with spheroidal graphite, that offers new opportunities for the quantitative analysis of relations «microstructure-property» for cast iron with spheroidal graphite.

Quantification and classification of microstructures in ternary eutectic alloys using 2-point spatial correlations and principal component analyses

Eutectic solidification gives rise to a number of distinct microstructure patterns that might include lamella, rods and labyrinths in binary alloys. However, as the number of phases and components increases, the number of possible patterns that might be obtained during bulk solidification also become larger. While the morphological attributes of binary eutectic solidification have been fairly well understood, the same is not true for ternary and higher multicomponent alloys. In this paper, we study and quantify microstructures in ternary alloys as a function of two essential parameters, namely, the volume fraction of the solid phases and the surface energies of the interfaces (in particular the solid–liquid interfaces). For the selected ensemble of microstructures, quantification and classification were carried out using a recently developed data-driven (objective) approach based on principal component analyses of 2-point correlations. It is demonstrated that the method is capable of analyzing and quantifying the similarity/difference measures between the elements of the selected ensemble of microstructures.

The Classification of Microstructures and Access Structures

The Classification of Microstructures and Access Structures