Previous Inspection Research Articles

Multi-scale GAN with transformer for surface defect inspection of IC metal packages

Integrated circuit plays an important role in the information technology industry. Surface defect inspection of IC packages is an essential process in the IC packaging manufacturing. Here, an automatic optical inspection system is proposed based on semi-supervised deep learning for surface defect inspection of IC metal packages. Different from previous inspection methods, we propose an entirely multi-scale inspection framework to implement the evaluation of defects at multiple scales. To well capture the intrinsic patterns of qualified samples at multiple scales, an entirely multi-scale GAN with transformer is elaborately designed, which incorporates several novel modules. Specifically, multi-scale CNN encoder with a novel feature extraction scheme and a cross-scale feature fusion module is designed to sufficiently extract the features from the IC metal package image. Different from previous GAN models, a Swin Transformer decoder is designed to strengthen the modeling ability of the proposed GAN model. Also, several novel multi-scale inspection schemes, including multi-scale weight mask, multi-scale adaptive thresholding and multi-scale image patch-based defect evaluation, are proposed to suppress the reconstruction errors, highlight the potential defects and further evaluate them at multiple scales, respectively. Experimental results demonstrate the effectiveness and feasibility of the proposed multi-scale inspection framework, which achieves an excellent inspection performance of 0.70% error rate, 0.57% omission rate, 99.3% accuracy, 99.8% precision, 99.3% recall and 0.996 F-score with a reasonable speed of 70.9 FPS and is superior to the state-of-the-art semi-supervised deep learning inspection methods.

Correlative tomography and authentication features of a shrunken head (tsantsa).

Museum personnel and the general public have become quite familiar with the presence of shrunken heads in museum collections, but the procedures to authenticate the history and origin of these unique cultural items are not yet reliable. These shrunken heads, called tsantsas, are meant to be the cultural material remains of ceremonies conducted by the Shuar and Achuar Peoples of South America. This project seeks to integrate the use of micro-computed tomography (micro-CT) scanning with methods used in previous studies (clinical computed tomography (CT) and visual inspections) to examine authentication procedures of shrunken heads (tsantsas) held in contemporary museum collections. We use a correlative tomographic approach using several scans at successively higher resolutions to determine whether a tsantsa was created from human remains, and if so, what key features can best contribute to its authentication. Conclusively, our correlative tomographic approaches provide new insights into the determination process of whether a tsantsa was created from real human remains or not. Also, this study questions whether the previously conceptualized dichotomy of ceremonial or commercial might be better thought of as a continuum of practice. Investigating and redefining the examination and authentication procedures of tsantsas is crucial for future ethical curation, management, and repatriation efforts of this unique cultural material of the Shuar and Achuar Peoples.

Recurrent neural network-based model for estimating the life condition of a dry gas pipeline

The use of expansive pipeline networks guarantees domestic and industrial users for accessing a continuous flow of valuable liquids and gases. These pipeline systems were considered the most economical and safest pipeline of transport for oil and gas and are of great strategic importance. The risks during operating conditions need to be controlled to handle the pipeline safely and smoothly in the complex globalized economy. Accordingly, life prediction is an essential issue in the pipeline network of the maintenance systems. Recently proposed data-driven and statistical approaches for pipeline's life prediction continue to rely on previous information to create health indicators and define thresholds, which is inefficient in the significant data era. Thus, a recurrent neural network (RNN)-based method for predicting the life of equipment with corrosion dimension classes exposed to critical condition monitoring is proposed in this research. The RNN model uses the multiple condition monitoring observations at the current and previous inspection points as its inputs and the pipe life condition and corrosion type as its outputs. Further, to improve the predictability of the proposed model, a validation mechanism is introduced during the training process. The proposed RNN method is validated using the data from oil and gas fields that have been collected in real-time. A critical sensitivity analysis with missing parameters is performed to evaluate the model's effectiveness in forecasting the life situation when inputs are absent. A comparative study is also carried out between the proposed RNN method and an adapted version of the reported method. The results show the advantage of the proposed method in achieving an actual life expectancy which can reduce the annual maintenance costs and help take necessary actions for better protection and safety of a pipeline.

Hybrid-Learning-Based Operational Visual Quality Inspection for Edge-Computing-Enabled IoT System

Deep learning-enhanced Internet of Things (IoT) plays a pivot role in advancing the transformation toward smart manufacturing, and an essential component in many smart manufacturing IoT systems is the quality inspection. However, challenges, such as expensive data labeling, innumerable types of defects, and high costs for iterative optimization, hinder the industrial applicability of previous visual surface quality inspection methods. In this article, we present an edge-computing-enabled IoT system based on an innovative hybrid learning method for visual surface quality inspection using only few labeled data and minimum iterative optimization efforts. Our hybrid learning method first employs a deep neural network to synthesize global representations of real-world industrial images, which are subsequently analyzed via an unsupervised clustering algorithm for anomaly detection. Besides, enhancement strategies, such as fine-tuning and data augmentation, are proposed to improve the robustness against the noisy data set and support low-cost inference in multiple edge devices for manufacturing operation. On a holdout data set collected from real-world factories, our method achieves classification accuracies between 90% and 98%, outperforming the benchmark method by 7%–12%. Moreover, this hybrid learning method demonstrates the effectiveness in detecting new types of surface defects and achieves test recalls between 86% and 97%, outperforming the benchmark method by 11%–34%.

Factors affecting effectiveness of food control inspections in food production establishments in Finland

Inspections are an important tool for food control. However, there is a lack of knowledge about how inspection history, interval between inspections, and pre-announcement of the inspection affect compliance with food safety legislation in food production establishments. We used register-based meat, fish, and dairy establishment food control inspection data (5550 inspections from 757 establishments) from Finland in 2016–2019 to study compliance in relation to these factors. Hypothesis was that there is an association between inspection grades and (1) inspection history, (2) interval, and (3) pre-announcement. Results indicate that minor non-compliances that do not impair food safety often precede future more severe non-compliances (5.3% non-compliance rate if full compliance at previous inspection compared to 16.8% non-compliance rate if minor non-compliances at previous inspection [Fisher’s exact test, p < 0.0001]), and that longer inspection intervals are associated with a decrease in inspection grades (5.9% difference in inspection intervals for inspection with full compliance compared to inspections with severe non-compliances [Generalized estimating equations, p = 0.02]). In addition, pre-announcement of the inspection affects the inspection grades, severe non-compliances were 2.4 times more common at unannounced inspections compared to pre-announced inspections. To conclude, there is an association between inspection grades and inspection history, inspection interval, and inspection announcement.

Lessons learned from the investigation and management of an outbreak of Shigella flexneri associated with a restaurant in London, 2019–2020

BackgroundFoodborne outbreaks of Shigella flexneri infection are uncommon in the UK. In November 2019, the United Kingdom Health Security Agency investigated an outbreak of S. flexneri associated with a fast-food restaurant in London. MethodsEpidemiological investigations included case ascertainment and interviewing suspected cases using enhanced surveillance questionnaires. Whole-genome sequencing (WGS) was used for characterisation of human isolates. Environmental investigations included a review of food safety processes at the implicated restaurant, administration of exposure questionnaires and stool sampling of staff. ResultsBetween November 2019 and February 2020, 17 cases were confirmed as part of the outbreak by WGS in London. Among these, 15 were linked to the implicated restaurant. A review of the food safety processes at the restaurant was satisfactory. Despite initial suboptimal coverage of stool screening of staff, all staff members working at the restaurant during the sampling period were screened and an asymptomatic food handler tested positive for S.flexneri with the outbreak WGS profile. The individual underwent microbiological clearance, and no further cases were reported. It was not possible to confirm the direction of transmission for the community cases or the staff member. ConclusionWe report an outbreak of S. flexneri in a fast-food restaurant in London with previous inspection ratings indicating good compliance with food safety and hygiene standards. WGS was crucial in identifying cases linked to the outbreak. This outbreak highlights the importance of prompt testing of food handlers in outbreaks suspected to be associated with food businesses.

GAP ANAGAP ANALYSIS ON RISK BASED INSPECTION (RBI) IMPLEMENTATION AT PT. XYZLYSIS ON RISK BASED INSPECTION (RBI) IMPLEMENTATION AT PT. XYZ

Risk Based Inspection (RBI) has been implemented mainly in oil and gas industry to manage the risk of aging facilities. The RBI plan has also been introduced for new facilities and become part of the design requirements, making it the right time for PT. XYZ to improve their RBI implementation to support the Facility Risk Integrity Management System. This is a semi-quantitative study on primary data collected through focus group discussion using the RBI evaluation parameters based on API 580, API 581, and field observation and secondary data from previous RBI reports, maintenance program, and inspection program. Finally, an evaluation was conducted to assess whether the RBI management strategy has been integrated to the safety and asset management; inspection, testing, and monitoring; and operation strategies. This gap analysis aims to evaluate the effectiveness of the ongoing RBI implementation at PT. XYZ in general while specifically identify the part of RBI and related Asset Integrity Management already implemented and those that still need further improvement. Results show an overall score of 328 of 470, showing a good implementation of RBI. The largest gaps identified are RBI on specific equipment (score=3.0), documented RBI management policy and strategy (score=3.8), risk target and risk acceptable level (score=4.0), and specific damage mechanism components (score=5.3).

What Lies Behind Successful Regulation? A Qualitative Evaluation of Pilot Implementation of Kenya's Health Facility Inspection Reforms.

Health facility regulation in low- and middle-income countries (LMICs) is generally weak, with potentially serious consequences for safety and quality. Innovative regulatory reforms were piloted in three Kenyan counties including: a Joint Health Inspection Checklist (JHIC) synthesizing requirements across multiple regulatory agencies; increased inspection frequency; allocating facilities to compliance categories which determined warnings, sanctions and/or time to re-inspection; and public display of regulatory results. The reforms substantially increased inspection scores compared with control facilities. We developed lessons for future regulatory policy from this pilot by identifying key factors that facilitated or hindered its implementation. We conducted a qualitative study to understand views and experiences of actors involved in the one-year pilot. We interviewed 77 purposively selected staff from the national, county and facility levels. Data were analyzed using the framework approach, identifying facilitating/hindering factors at the facility, inspection system, and health system levels. The joint health inspections (JHIs) were generally viewed as fair, objective and transparent, which enhanced their perceived legitimacy. Interactions with inspectors were described as friendly and supportive, in contrast to the punitive culture of previous inspections when bribery had been common. Inspector training and use of an electronic checklist were strongly praised. However, practical challenges with transport, route planning and budgets highlighted the critical nature of strong logistical management. The effectiveness of inspection in improving compliance was hampered by limitations in related systems, particularly facility licensing, enforcement of closures and, in the public sector, control of funds. However, an inclusive reform development process had led to high buy-in across regulatory agencies which was key to the system's success. Effective facility inspection involves more than "hardware" such as checklists, protocols and training. Cultural, relational and institutional "software" are also crucial for legitimacy, feasibility of implementation and enforceability, and should be carefully integrated into regulatory reforms.

Audit Forum: Lessons From Previous FDA Observations – Drugs

Reviewing previous FDA Inspection Findings (FDA-483s) and Warning Letters can provide useful information in preparing for audits and regulatory inspections at your facility. While it can be difficult to find specific FDA-483s on the Internet, there is data published by FDA of the specific types of drug, device, and biologics inspections observations for each year. This data is provided in Excel spreadsheets by FDA. It can be useful to re-organize the information into more useful formats for analysis. By year, there is a table of the number of observations in each major category.

Context-aware sequence labeling for condition information extraction from historical bridge inspection reports

Effective upkeep of aging infrastructure systems with limited funding and resources calls for efficient bridge management systems. Although data-driven models have been extensively studied in the last decade for extracting knowledge from past experience to guide future maintenance decision making, their performance and usefulness have been limited by the level of detail and accuracy of currently available bridge condition databases. This paper leverages an untapped resource for bridge condition data and proposes a new method to extract condition information from it at a high level of detail. To that end, a natural language processing approach was developed to formalize structural condition knowledge by formulating a sequence labeling task and modeling inspection narratives as a combination of words representing defects, their severity and location, while accounting for the context of each word. The proposed framework employs a deep-learning-based approach and incorporates context-aware components including a bi-directional Long Short Term Memory (LSTM) neural network architecture and a Conditional Random Field (CRF) classifier to account for the context of words when assigning labels. A dependency-based word embedding model was also used to represent the raw text while incorporating both semantic and contextual information. The sequence labeling model was trained using bridge inspection reports collected from the Virginia Department of Transportation bridge inspection database and achieved an F1 score of 94.12% during testing. The proposed model also demonstrated improvements compared with baseline sequence labeling models, and was further used to demonstrate the capability of detecting condition changes with respect to previous inspection records. Results of this study show that the proposed method can be used to extract and create a condition information database that can further assist in developing data-driven bridge management and condition forecasting models, as well as automated bridge inspection systems.

A two-stage optimization approach for inspection plan formulation of comprehensive inspection train: The China case

The regular and dynamic inspections of high-speed railways (HSR) are carried out by comprehensive inspection train (CIT). Generally, the network of HSR (e.g., the HSR network in China) has some obvious characteristics such as complex network structure, high traffic density and strict maintenance restrictions of CIT, which make the formulation of CIT inspection plan extremely hard. However, formulating and optimizing the CIT inspection plan is one of the key issues to ensure the safety of HSR operations and to improve the efficiency of testing equipment. In this paper, the original model is proposed for the multiple CITs inspection plan formulation problem (MCIT-IPFP) firstly, and it is proved that the problem belongs to the NP-hard class of computational complexity by reduction. By decomposing the problem into two stages, one is used to divide the HSR network evenly for each CIT, and the other one is used to generate the path, the timetable, and maintenance base assignment for each CIT under requirements of reducing the total time cost and the difficulty of solving. In order to improve the efficiency of calculation, a Constructive Heuristic Algorithm with feedback regulation based on the greedy algorithm is designed to generate the task division plan. With the purpose of quickly formulating the inspection plans with low time cost for each CIT with Euler circuit characteristics, a simulated annealing algorithm with efficient neighborhood Euler circuit generation strategy is proposed. Finally, a case study is conducted to test the efficiency of the algorithm by taking the China’s HSR network as the background. Compared with the previous manual inspection plan, the utilization rate of CITs will increase by 30%-50% when using the models and algorithms proposed in this paper.

АНАЛИЗ ПАССАЖИРОПОТОКОВ НА МАРШРУТАХ АВТОБУСА Г. КУРСКА

Results of inspection and the analysis of passenger traffics on routes of the bus of Kursk are presented, the main shortcomings of the existing route network are revealed. The analysis of change of daily volume of transportations of passengers made on the basis of data of it and the previous inspections of passenger traffics and also distribution of total power of a passenger traffic on hours of day is provided. Results of development of rational route bus network of Kursk which is almost completely deprived of the shortcomings inherent in the existing route network are presented.

Dispersion curve regression – assisted wideband local wavenumber analysis for characterizing three-dimensional (3D) profile of hidden corrosion damage

Corrosion as common damage in civil, petrochemical, nuclear, and aerospace structures affects the integrity and safety of the structures and may lead to catastrophic failures. This paper presents dispersion curve regression-assisted local wavenumber analysis method, which can analyze the time–space wavefield containing wideband information of wave-damage interaction and further extract the structural information carried by such wavefield for characterizing hidden corrosion damage in an isotropic plate. To acquire the time–space wavefield, a noncontact scanning laser Doppler vibrometer is used. In our analysis method, the acquired time–space wavefield is firstly processed to generate a local wavenumber function. The derived wavenumber function is further analyzed with a dispersion curve regression step, which searches in a set of theoretical frequency-wavenumber dispersion curves for different plate thicknesses and identifies the optimal plate thickness whose theoretical curve best matches the frequency-wavenumber relation contained in the local wavenumber function. By this means, a 3D profile including both in plane and thickness dimensions of the structure can be constructed for corrosion visualization and quantification. The experimental study demonstrates that our method can quantify the profile of hidden uniform corrosion in metal plates in terms of its in-plane shape and size as well as its out-of-plane depth in the subwavelength scale. Moreover, compared to previous Lamb wave-based corrosion inspection methods, our technique allows for noncontact 3D characterization of hidden corrosion from a far distance to the structure. We believe this work will inspire new 3D damage quantification methods that are based on wavefield analysis as well as enable potential applications for the quantification of hidden corrosion in civil, petrochemical, nuclear, and aerospace structures.

An automatic and accurate method for tool wear inspection using grayscale image probability algorithm based on bayesian inference

Accurate, rapid and automated tool wear inspection is critical to manufacturing quality, cost and efficiency in smart manufacturing systems. However, manual inspection of tool wear is still a common industrial practice which is inefficient, prone to human errors and not suitable for digitized manufacturing. Previously reported automatic tool wear inspection methods were inaccurate because they only used the remaining worn boundary (i.e., the partial-absence original tool boundary) to approximate tool wear. The authors discovered the association principle between the change law of the cutting edge grayscale and the relative position of the original and worn boundary, which was used to establish the probability functions to accurately reconstruct the curved original tool boundary via Bayesian Inference. The experiment results reported in this paper proved higher efficiency and accuracy than previous automatic tool wear inspection methods.

Immersive Work Zone Inspection Training using Virtual Reality

Can virtual reality tools be used to train engineers that inspect work zones? In this paper, we share the findings of a research project that developed an interactive and immersive training platform using virtual reality to train state department of transportation (DOT) staff that inspect work zones for compliance. Virtual reality offers an immersive platform that closely replicates the actual experience of an inspector driving through a work zone, but in a safer, cheaper, and quicker way than field visits. The current training practice involves reviewing temporary traffic control procedures, and reports and pictures from previous inspections. The developed platform consists of a learning module and an immersive module. The learning module is founded on the historical knowledge gained by DOT staff from inspections dating back at least 5 years. This knowledge incorporated representative inspection reports from previous years from all DOT districts including photographs of deficiencies. The synthesized knowledge was converted into a concise easy-to-consume format for training. The immersive module places the trainee in a vehicle moving through a work zone, thus providing a realistic experience to the engineer before inspecting a real work zone. The research team developed and tested two immersive scenarios of a freeway work zone. The training platform was tested by 34 individuals that worked for the Missouri Department of Transportation. An overwhelming majority (97%) agreed that virtual reality offered a realistic and effective way to train inspectors.

The parking warden tour problem

AbstractIn this work we introduce the parking warden tour problem (PWTP), a new arc routing problem arising in irregular parking detection to model the maximization of the fines collected by a team of parking wardens. The peculiarity of the PWTP is that the arc revenue depends on the time elapsed from the previous inspection of a warden, since the revenue on a road link slumps to zero if a warden has just visited the link. For this new arc routing problem and a variant of it, we propose two mixed integer linear program formulations, a simple but effective matheuristic based on them and a bounding procedure. All the solution approaches have been tested and compared on both real instances generated on a road network area of Milano, through a survey and on real‐like random instances.

Unexpected deposits in the anesthetic circuit: a possible cause of PEEP/Pmax valve malfunction.

PEEP is regulated by the internal PEEP/maximum peak inspiratory pressure limit (Pmax) valve. Malfunctioning of the PEEP/Pmax valve can result in the creation of unintentional or unstable PEEP, and a reduction of inspired tidal volume. Some of our Dräger Fabius® anesthesia machines were noted to exhibit changes in expiratory waveforms and unstable PEEP during general anesthesia. We considered that the cause was associated with PEEP/Pmax valve malfunction, and then investigated the problems in collaboration with the manufacturer. Seven of the 22 Dräger Fabius® anesthesia workstations at our department exhibited problems with their PEEP/Pmax valves. We replaced the PEEP membrane and sealing washers in these seven anesthesia machines, and the problems were temporarily resolved. After a short interval, however, one of the seven machines began to show a similar phenomenon. We then asked the manufacturer to overhaul the PEEP/Pmax valve and the entire breathing circuit of the machine. On close investigation, we found that the valve components and the internal surface of the breathing circuit were contaminated with unexpected deposits. The build-up of deposits occurred within a year after the previous regular inspection. Our troubleshooting process determined the issue with the PEEP/Pmax valve, which could go unnoticed because the valve is encased inside the breathing circuit, and requires disassembly for close inspection. Our findings should raise awareness regarding the importance of the preventive maintenance cycle as a safety precaution to keep the anesthetic circuit free of unexpected contamination.

Advanced vacuolation indicates propagation of various salmonid alphavirus type 2 isolates in Acholeplasma-infected BF-2 cells.

During previous routine inspections of bluegill fry (BF-2) and rainbow trout gonad (RTG-2) cells incubated with organ samples from asymptomatic Arctic char Salvelinus alpinus, brook trout Salvelinus fontinalis, and rainbow trout Oncorhynchus mykiss, a distinctive, reproducible cytopathic effect (CPE) appeared. The striking CPE, involving progressive vacuolation turning into slowly proceeding pyknotic degeneration, was originally attributed exclusively to enhanced growth of Acholeplasma sp. However, at a recent re-examination of re-infected BF-2 cells using electron microscopy (EM), conventional PCR, and quantitative PCR (qPCR), a virus was also detected. Two days post inoculation (dpi), EM revealed characteristic virions inside cytoplasmic vacuoles and next to bacteria outside the cells. The nucleotide sequences of the viral nsP3 gene fragment obtained from supernatants of infected cells were 100% identical and representative for salmonid alphavirus type 2 (SAV 2). The 16S RNA gene (16S rDNA) fragment sequences of the Mollicutes-specific PCR product obtained from SAV-infected as well as virus-free BF-2 control cells were identical with Acholeplasma laidlawii. In addition, qPCR results indicated enhanced propagation of virus and bacteria increasing with vacuolation between 5 and 8 dpi. Advanced vacuolation can be regarded as a CPE of both SAV and A. laidlawii, suggesting a viral impact on the bacterial infection that turns a latent intracellular stage into an apparent degenerative condition.

Probabilistic inspection planning of offshore welds subject to the transition from protected to corrosive environment

In this paper, a novel method is proposed to optimize inspection plans for fatigue accounting for the situations where corrosion-free or protected environments are changed to a corrosive environment, denoted as the Transitional Environmental Protection (TEP) process. Probabilistic fracture mechanics and S-N curve approaches are calibrated to simulate crack propagations and planning of inspections and repairs of offshore welds. The method presented is relatively simple and conservative: The transition between protected and corrosive environments is addressed by shifting the S-N curve parameters during the damage calculations, performing crack size recalibration and modifying the crack growth material parameters. These concepts are incorporated in an algorithm, which is applicable to new designs or existing structures where no data from previous inspections is available. An example illustrates the potential of the algorithm.

Reliability of in-situ diagnosis in external wall renders

This study focuses on the reliability of in-situ diagnosis performed on external rendered façades. This analysis was based on a set of parameters drawn from different visual observation, in-situ and laboratory testing techniques. The results were collected based on the previous inspections of 98 rendered façades of buildings and individual testing walls with different components, ages and degradation types. For that purpose, a reliability indicator was proposed and applied for better results interpretation of multiple testing techniques. In each case study, the parameters with higher reliability are defined. The final reliability indicator was established based on the average of three partial indicators, namely a reliability indicator for the established base criteria’ uncertainty in laboratory testing (reference reliability indicator); a reliability indicator for the assessment techniques’ uncertainty (verification method reliability indicator); and a reliability indicator for the results’ uncertainty (number of the tests and their accuracy; reliability indicator of results obtained in-service). The reference parameters for each testing technique were synthesised. The relevant aspects, such as uncertainty associated with the reference parameter and type of verification method (quantitative, qualitative, and intervals), were defined. The analysis of the relationships between the parameters (exemplified for mechanical properties) and their correlation coefficients were assessed. Selection matrices were proposed that qualitatively identify the set of parameters that contribute to indirectly analyse the characteristics and performance relevant to the behaviour of the renders applied on façades.