Visual Surface Inspection Research Articles

Stochastic geometry models for texture synthesis of machined metallic surfaces: sandblasting and milling

Training defect detection algorithms for visual surface inspection systems requires a large and representative set of training data. Often there is not enough real data available which additionally cannot cover the variety of possible defects. Synthetic data generated by a synthetic visual surface inspection environment can overcome this problem. Therefore, a digital twin of the object is needed, whose micro-scale surface topography is modeled by texture synthesis models. We develop stochastic texture models for sandblasted and milled surfaces based on topography measurements of such surfaces. As the surface patterns differ significantly, we use separate modeling approaches for the two cases. Sandblasted surfaces are modeled by a combination of data-based texture synthesis methods that rely entirely on the measurements. In contrast, the model for milled surfaces is procedural and includes all process-related parameters known from the machine settings.

Progressive refinement imaging with depth-assisted disparity correction

In recent years, the increasing on-board compute power of mobile camera devices gave rise to a class of digitization algorithms that dynamically fuse a stream of camera observations into a progressively updated scene representation. Previous algorithms either obtain general 3D surface representations, often exploiting range maps from a depth camera, such as, Kinect Fusion, etc.; or they reconstruct planar (or distant spherical, respectively) 2D images with respect to a single (perspective or orthographic) reference view, such as, panoramic stitching or aerial mapping. Our work sets out to combine aspects of both, reconstructing a 2.5-D representation (color and depth) as seen from a fixed viewpoint, at spatially variable resolution. Inspired by previous work on “progressive refinement imaging”, we propose a hierarchical representation that enables progressive refinement of both colors and depths by ingesting RGB-D images from a handheld depth camera that is carried through the scene. We evaluate our system by comparing it against state-of-the-art methods in 2D progressive refinement and 3D scene reconstruction, using high-detail indoor and outdoor data sets comprising medium to large disparities. As we will show, the restriction to 2.5-D from a fixed viewpoint affords added robustness (particularly against self-localization drift, as well as backprojection errors near silhouettes), increased geometric and photometric fidelity, as well as greatly improved storage efficiency, compared to more general 3D reconstructions. We envision that our representation will enable scene exploration with realistic parallax from within a constrained range of vantage points, including stereo pair generation, visual surface inspection, or scene presentation within a fixed VR viewing volume.

Procedural Defect Modeling for Virtual Surface Inspection Environments.

Development of automated visual surface inspection systems heavily depends on the availability of defected product samples. Both inspection hardware configuration and training of defect detection models require diversified, representative and precisely annotated data. Reliable training data of sufficient size is frequently challenging to obtain. Using virtual environments, it is possible to simulate defected products which would serve both for configuration of acquisition hardware as well as for generation of required datasets. In this work, we present parameterized models for adaptable simulation of geometrical defects, based on procedural methods. Presented models are suitable for creating defected products in virtual surface inspection planning environments. As such, they enable inspection planning experts to assess defect visibility for various configurations of acquisition hardware. Finally, the presented method enables pixel-precise annotations alongside image synthesis for the creation of training-ready datasets.

An Automatic Visual Inspection of Oil Tanks Exterior Surface Using Unmanned Aerial Vehicle with Image Processing and Cascading Fuzzy Logic Algorithms

This paper presents an automatic visual inspection of exterior surface defects of oil tanks using unmanned aerial vehicles (UAVs) and image processing with two cascading fuzzy logic algorithms. Corrosion is one of the defects that has a serious effect on the safety of the surface of oil and gas tanks. At present, human inspection, and climbing robots inspection are the dominant approach for rust detection in oil and gas tanks. However, there are many shortcomings to this approach, such as taking longer, high cost, and covering less surface area inspection of the tank. The purpose of this research is to detect the rust in oil tanks by localizing visual inspection technology using UAVs, as well as to develop algorithms to distinguish between defects and noise. The study focuses on two basic aspects of oil tank inspection through the images captured by the UAV, namely, the detection of defects and the distinction between defects and noise. For the former, an image processing algorithm was developed to improve or remove noise, adjust the brightness of the captured image, and extract features to identify defects in oil tanks. Meanwhile, for the latter aspect, a cascading fuzzy logic algorithm and threshold algorithm were developed to distinguish between defects and noise levels and reduce their impact through three stages of processing: The first stage of fuzzy logic aims to distinguish between defects and low noise generated by the appearance of objects on the surface of the tank, such as trees or stairs, and reduce their impact. The second stage aims to distinguish between defects and medium noise generated by shadows or the presence of small objects on the surface of the tank and reduce their impact. The third stage of the thresholding algorithm aims to distinguish between defects and high noise generated by sedimentation on the surface of the tank and reduce its impact. The samples were classified based on the output of the third stage of the threshold process into defective or non-defective samples. The proposed algorithms were tested on 180 samples and the results show its superiority in the inspection and detection of defects with an accuracy of 83%.

Application of magnesium alloy sacrificial anode for restraining chloride ingress into mortar

This study aims to restrain the chloride ingress into mortar by magnesium alloy sacrificial anode system. Cube reinforced mortar specimens with embedded magnesium alloy anode meshes were prepared. The output electrochemical parameters of magnesium alloy mesh, the chloride profiles and corrosion electrochemical performance of steel in mortars immersed in chloride-contaminated solutions were detected. Besides, the visual inspection of steel surface after immersion was performed. The results indicate that the magnesium alloy mesh can restrain effectively chloride ingress into mortar. With increasing water-cement ratio and chloride content, the galvanic potential and current density between the steel and magnesium alloy mesh increase. The chloride diffusion coefficient of mortar decreases after the application of magnesium alloy sacrificial anode and the decreased magnitude rises with increasing water-cement ratio, chloride concentration and decreasing steel cover thickness. The mitigation of steel corrosion and morphology of the steel surface after immersion further demonstrate the good effect of magnesium alloy sacrificial anode on reducing chloride ingress into mortar.

Defect segmentation for multi-illumination quality control systems

Thanks to recent advancements in image processing and deep learning techniques, visual surface inspection in production lines has become an automated process as long as all the defects are visible in a single or a few images. However, it is often necessary to inspect parts under many different illumination conditions to capture all the defects. Training deep networks to perform this task requires large quantities of annotated data, which are rarely available and cumbersome to obtain. To alleviate this problem, we devised an original augmentation approach that, given a small image collection, generates rotated versions of the images while preserving illumination effects, something that random rotations cannot do. We introduce three real multi-illumination datasets, on which we demonstrate the effectiveness of our illumination preserving rotation approach. Training deep neural architectures with our approach delivers a performance increase of up to 51% in terms of AuPRC score over using standard rotations to perform data augmentation.

Mortality evolution in Algeria: What can we learn about data quality?

Mortality in Algeria has declined significantly since the country declared its independence in 1962. This trend has been accompanied by improvements in data quality and changes in estimation methodology, both of which are scarcely documented, and may distort the natural evolution of mortality as reported in official statistics. In this paper, our aim is to detect these methodological and data quality changes by means of the visual inspection of mortality surfaces, which represent the evolution of mortality rates, mortality improvement rates and the male-female mortality ratio over age and time. Data quality problems are clearly visible during the 1977–1982 period. The quality of mortality data has improved after 1983, and even further since the population census of 1998, which coincided with the end of the civil war. Additional inexplicable patterns have also been detected, such as a changing mortality age pattern during the period before 1983, and a changing pattern of excess female mortality at reproductive ages, which suddenly appears in 1983 and disappears in 1992.

Bio-waste influence on air lime mortar performance corrosion – optimization using the surface response method

PurposeThe purpose of this paper is to characterize the properties lightweight green air lime and marble waste mixtures, relating microstructural and chemical properties with physical development of the material, an effort has been made to simulate the structure of the different mortar reinforced by two main layers plants.Design/methodology/approachThis paper presents an experimental design of response surface methodology, a model which predicts the mechanical strength and evaluate the effectiveness of bio-waste as a corrosion inhibitor to resist the steel corrosion in air lime mortars as a function of the proportion of the constituents of a new air lime mortar based on a combination of different percentages of marble waste (MRW), air lime and deferent type, layers of natural fiber reinforcement. Luffa sponge gourd and oakum hemp fiber residues capabilities in civil engineering are evaluated by combining numerical and experimental approaches for repair mortar based on air lime and marble waste. Several electrochemical techniques, mechanical strength tests and visual inspection of steel surface were performed.FindingsThe results revealed good mechanical strength and corrosion protection properties of air lime mortar containing the fiber naturel. These green wastes are considered economically feasible, as well having possessing good performance efficiency in protecting rebar reinforcement. These results were confirmed via polarization curves and electrochemical impedance spectroscopy measurements.Originality/valueThe prepared green air lime mortar provided good corrosion protection to the rebar. The significance of this study is to encourage the usage of solid white marble waste to prepare biomass-based repair mortar with good mechanical and anti-corrosion properties on the long term is still a big challenge.

A Novel Decomposition Model for Visual Rail Surface Inspection

Rail surface inspection plays a pivotal role in large-scale railway construction and development. However, accurately identifying possible defects involving a large variety of visual appearances and their dynamic illuminations remains challenging. In this paper, we fully explore and use the essential attributes of our defect structure data and the inherent temporal and spatial characteristics of the track to establish a general theoretical framework for practical applications. As such, our framework can overcome the bottleneck associated with machine vision inspection technology in complex rail environments. In particular, we consider a differential regular term for background rather than a traditional low-rank constraint to ensure that the model can tolerate dynamic background changes without losing sensitivity when detecting defects. To better capture the compactness and completeness of a defect, we introduce a tree-shaped hierarchical structure of sparse induction norms to encode the spatial structure of the defect area. The proposed model is evaluated with respect to two newly released Type-I/II rail surfaces discrete defects (RSDD) data sets and a practical rail line. Qualitative and quantitative evaluations show that the decomposition model can handle the dynamics of the track surface well and that the model can be used for structural detection of the defect area.

Learning to Recognize Irregular Features on Leather Surfaces

As part of industrial quality control in the leather industry, it is important to identify the abnormal features in wet-blue leather samples. Manual inspection of leather samples is the current norm in industrial settings. To comply with the current industrial standards that advocate large-scale automation, visual inspection based leather processing is imperative. Visual inspection of irregular surfaces is a challenging problem as the characteristics of the abnormalities can take a variety of shape and color variations. The aim of this work is to automatically categorize leather images into normal or abnormal by visual analysis of the surfaces. To achieve this aim, a deep learning based approach is devised that learns to recognize regular and irregular leather surfaces and categorize leather images on its basis. To this end, we propose an ensemble of multiple convolutional neural networks for classifying leather images. The proposed ensemble network exhibited competitive performance obtaining 92.68% test accuracy on our own curated leather images dataset.

Development and evaluation of relationships between surface condition rating and objective pavement condition parameters

ABSTRACT Highway authorities make efforts to relate visual ratings with directly measured pavement condition data to reduce or remove the necessity of manual pavement condition surveys that involve subjectivity and potential safety risk of assessors. This research develops a set of relationships between pavement surface condition rating and objective pavement condition parameters to assist road asset managers in triggering periodic bituminous resurfacing programs at the network level. These condition parameters include cracking (% area affected), rutting (mm), texture loss (% of left wheel path texture), and roughness (m/km). In the literature, deterministic and probabilistic modelling approaches are used to predict visual surface inspection rating (SIR) from directly measured pavement distresses. The Factorial ANOVA results that are typically used have inferred that cracking and rutting interact with each other significantly for the asphalt surfaced road network. However, the percentages of variation explained by the linear regression models that predict SIR from cracking/rutting are low (24–31%). Alternatively, developed ordinal logistic models for predicting the probability of a road section being in any particular surface condition, with any quantified cracking/rutting data, prove to be statistically better with overall success rates of 46% and 51% for sprayed sealed and asphalt surfaced network, respectively.

A Deep Extractor for Visual Rail Surface Inspection

Rail surface inspection using a visual inspection system is an important task in the maintenance of railway networks. Recent inspection approaches have attempted to harness low-level features to identify visual defects. These methods' main issues are the limitations of low-level features and prior information about rail surfaces that show a wide variety of visual appearances on dynamic backgrounds. To overcome these problems, we propose a deep extractor (DE) that combines the strengths of fully convolutional networks and conditional random fields (CRFs), so that the deep network can learn the abstract features needed for the specified inspection task. Specifically, motivated by fully convolutional networks, a bilateral fully convolutional network containing two branches is proposed here: one encode-decode branch focuses semantic meaning; the other decode-encode branch encodes information on tiny defects. The two branches are aggregated to obtain a high-level feature map. Furthermore, the mean-field inference of a dense CRF with Gaussian pairwise potentials formulated as a recurrent neural network (CRF-RNN) is introduced to contribute to smoothing constraints and obtaining a fine-grained inspection result. By doing so, it can avoid offline post-processing and further lead the whole deep architecture to achieve end-to-end deep learning. Compared with classical approaches, our approach outperforms state-of-art approaches in analyzing two publicly available datasets.

Feature-Driven Viewpoint Placement for Model-Based Surface Inspection

The goal of visual surface inspection is to analyze an object’s surface and detect defects by looking at it from different angles. Developments over the past years have made it possible to partially automate this process. Inspection systems use robots to move cameras and obtain pictures that are evaluated by image processing algorithms. Setting up these systems or adapting them to new models is primarily done manually. A key challenge is to define camera viewpoints from which the images are taken. The number of viewpoints should be as low as possible while still guaranteeing an inspection of the desired quality. System engineers define and evaluate configurations that are improved based on a time-consuming trial-and-error process leading to a sufficient, but not necessarily optimal, configuration. With the availability of 3D surface models defined by triangular meshes, this step can be done virtually. This paper presents a new scalable approach to determine a small number of well-placed camera viewpoints for optical surface inspection planning. The initial model is approximated by B-spline surfaces. A set of geometric feature functionals is defined and used for an adaptive, non-uniform surface sampling that is sparse in geometrically low-complexity areas and dense in regions of higher complexity. The presented approach is applicable to solid objects with a given 3D surface model. It makes camera viewpoint generation independent of the resolution of the triangle mesh, and it improves previous results considering number of viewpoints and their relevance.

Storage stability characterization and improvement of SBS and crumb rubber composite modified asphalt

The styrene–butadiene-styrene (SBS) and crumb rubber (CR) composite modified asphalt (SBS/CRMA) possesses favourable mechanical performance but poor storage stability. The purposes of this study are aiming to investigate the changes in storage properties of SBS/CRMA and propose a corresponding improvement approach in the laboratory and further provide a storage guidance for the field application. The Fluorescence Microscope and the Fourier Transform Infrared Spectroscopy were applied to investigate the phase distribution state of SBS. While the phase distribution state of CR was investigated by visual inspection of fractural surfaces. The softening point, rutting factor and Zero Shear Viscosity (ZSV) were used to evaluate the storage properties. It was concluded that the CR sank and the obvious separation occurred within 6 h. The SBS floated up and the evident separation happened within 12 h. In terms of performance, the separation was most distinct in 12 h. Additionally, The effects of Sulphur and Naphthenic oil on storage stability were evaluated. The verification of the feasibility was conducted in a field project. It was verified that the binder prepared in the construction site possessed better high-temperature performance and storage stability due to different production processes and tank truck storage. The little difference in the high-temperature performance of SBS/CRMA indicated the feasibility of the storage performance optimisation method.

Strength grading of hardwoods using transversal ultrasound

Detection of local wood inhomogeneities is important for accurate strength and stiffness prediction. In hardwood specimens, visual characteristics (e.g. knots or fibre deviation) are difficult to detect, either with a visual surface inspection or by the machine. Transversal ultrasound scan (TUS) is a non-destructive evaluation method with high potential for hardwoods. The method relies on differences in ultrasound wave propagation in perpendicular to the grain direction. The aim of this study is to estimate and analyse the capabilities of TUS for defect detection in hardwoods and prediction of mechanical property values. In the current paper, the TUS was applied to the hardwood species European ash (Fraxinus excelsior L.), Norway maple (Acer platanoides L.) and sycamore maple (Acer pseudoplatanus L.). In total, 16 boards of both specimens were completely scanned perpendicular to the grain using a laboratory scanner with dry-coupled transducers. The measurements were processed to 2D scan images of the boards, and image processing routines were applied to further feature extraction, defect detection and grading criteria calculation. In addition, as a reference for each board, all relevant visual characteristics and mechanical properties from the tensile test were measured. Using the TUS global fibre deviation, the size and the position of the knots can be detected. Knottiness correlates to the strength properties similarly or even better compared to the manual knottiness measurement. Between the global fibre angle measured using TUS and measured on the failure pattern, no correlation could be found. The ultrasound modulus of elasticity perpendicular to the grain does not show any meaningful correlation to the elastic properties parallel to the grain. In overall, TUS shows high potential for the strength grading of hardwoods.

Failure analysis and design optimization of alternator shaft used in rail coaches

The intension of this work is to conduct a failure analysis and optimize the design of the shaft in a 4.5KW alternator used in rail coaches. The shafts in alternator of rail coaches are continuously subjected to sudden and repeated loads. The alternator shaft is made from carbon steel rods supplied by SAIL (Steel Authority of India Limited) and is machined as per the design. This shaft is subjected to stress reversals and found to have failed in several occasions during the operation. Visual inspection of fractured surface showed cracks initiated from the stepped surface. The breakage occurs at the specific region where the stress concentration is high. The stress concentration is also due to the tool marking seen in the machined surface of the fillet region of the shaft. Under the different test conditions, the various problems related to the finish and design of the shaft was observed. According to the observation, failure and design problems of the shaft will be analyzed using finite element method based on which an alternate design is suggested. In this analysis, we consider only bending load on two dimensional axi-symmetrical model of the shaft.

Combined Use of 3D Metric Survey and GPR for the Diagnosis of the Trapezophoros with Two Griffins Attacking a Doe of Ascoli Satriano (Foggia, Italy)

Protection and conservation are not feasible without a deeper knowledge of the cultural heritage of the subject of intervention. A careful analysis of documentation, a detailed visual inspection of surfaces, and a systematic diagnosis of the inner properties are the basis for planning preventive strategies of restoration. Digital noninvasive techniques represent an advantageous tool with useful outputs to estimate the state of health of cultural heritage in order to minimize the progress of degradation. This paper deals with a 3D metric survey through photogrammetry and ground-penetrating radar (GPR) tests applied to the study of the trapezophoros with two griffins attacking a doe of Ascoli Satriano, a masterpiece of ancient art that needs to be protected. This work provided information on both visible and hidden defects, such as numerous cracks that affect the sculpture.

Influence of Die Surface Topography and Lubrication on the Product Quality during Indirect Extrusion of Copper-Clad Aluminum Rods

Copper-clad aluminum rods are usually fabricated using hydrostatic extrusion, since during direct and indirect extrusion fracture of the copper sleeve is difficult to avoid. In this study, different die surface topographies and lubrication conditions were applied to improve the material flow during indirect extrusion of copper-clad aluminum rods. Thus, conic dies with different roughness (polished and sandblasted) and surfaces shapes (fine and coarse grooves) were tested. Additionally, the effects of a wax-graphite-based lubricant as well as a graphite-like carbon (GLC) coating of the die conic surfaces were investigated. The composite billets were made of aluminum EN AW-1080A cores and copper CW004A sleeves with an equivalent copper cross section of 0.24 of the total billet cross section. For all trials, an extrusion ratio of 14.8:1 and a conic die angle of 2α = 90° were chosen. Non-isothermal extrusion trials were carried out using a container at 330 °C and billet and tools at room temperature to reduce the flow stress ratio σCu/σAl. The extruded composite rods’ integrity, surface quality, interface integrity, and equivalent copper cross section were analyzed. In addition, a visual inspection of the sleeve-die contact surface was performed. The results showed that the GLC coating proved to be unsuitable due to a lack of lubrication, which causes accumulated sleeve fractures and longitudinal grooves on the extruded rods. The best results were achieved with the combination of the sandblasted die surface and the wax-graphite-based lubricant, observing a uniform material flow without sleeve fractures.

Extended Depth-of-Field of a Miniature Optical Endoscope Using Wavefront Coding

Endoscopic imaging is an indispensable tool for visual inspection of surfaces in hard-to-access places. The conventional optical endoscope only enables imaging the regions that lie close to the focal plane, and therefore improving the depth of field (DOF) of an endoscopic system has recently attracted much attention. Here, we propose and demonstrate a simple, cost-effective, and easily available industrial endoscopic system, which is composed of a rigid tube and group of optical lenses. The outer diameter of the designed endoscopic modality presented in this article is 3 mm. By employing the wavefront coding technique, the DOF of newly-designed endoscope was extended to 2.7 mm in the proof-of-concept experiment, which is an increase of almost 10 times compared to the initial counterpart.

A comparative study of two indirect methods to monitor surface integrity of ground components

One of the major challenges in grinding is to conciliate the material removal rate with components free from damages. Due to the poor thermal conductivity of conventional grinding wheels, most of the heat generated during grinding is transferred to the workpiece surface and subsurface, which can cause thermal damages and impair the performance in service of machined components. Hence, it is very important to monitor the grinding process to ensure the quality of the machined parts. Thus, this work presents an innovative study comparing two indirect monitoring methods in monitoring surface integrity of steel during grinding: the acoustic emission technique and the electromechanical impedance technique. Worktable speed and radial depth of cut were used as input parameters. Visual inspection and scanning electron microscopy images of ground surfaces as well as microhardness were the output variables used to assess surface integrity and to establish a relationship with the acoustic emission and electromechanical impedance techniques. Since the acoustic emission signals are non-stationary in nature, these signals were analyzed in the time–frequency domain by applying the short-time Fourier transform and the continuous wavelet transform. The root mean square deviation index was extracted as feature from the acoustic emission and electromechanical impedance signals. Results showed that both techniques presented similar results. The root mean square deviation index showed a good correlation with alterations in surface integrity under the conditions investigated.