Unforeseen Failures Research Articles

Towards robust visual understanding: A paradigm shift in computer vision from recognition to reasoning

AbstractModels that learn from data are widely and rapidly being deployed today for real‐world use, but they suffer from unforeseen failures that limit their reliability. These failures often have several causes such as distribution shift; adversarial attacks; calibration errors; scarcity of data and/or ground‐truth labels; noisy, corrupted, or partial data; and limitations of evaluation metrics. But many failures also occur because many modern AI tasks require reasoning beyond pattern matching and such reasoning abilities are difficult to formulate as data‐based input–output function fitting. The reliability problem has become increasingly important under the new paradigm of semantic “multimodal” learning. In this article, I will discuss findings from our work to provide avenues for the development of robust and reliable computer vision systems, particularly by leveraging the interactions between vision and language. This article expands upon the invited talk at AAAI 2024 and covers three thematic areas: robustness of visual recognition systems, open‐domain reliability for visual reasoning, and challenges and opportunities associated with generative models in vision.

Dual-Path Neural Network based on Deep Features and Transfer Learning for Bearing Fault Diagnosis

The safe operation of mechanical equipment is a basic requirement in the industrial manufacturing process. As an important component of mechanical equipment, rolling bearings are prone to unforeseen failures due to long-term operation under complex conditions. The occurrence of failures, no matter how big or small, will cause economic losses. Therefore, reliable diagnosis of rolling bearings is crucial. Aiming at the problem of insufficient feature recognition of convolutional neural networks under strong noise background, a deep feature extraction network integrating multiscale convolutional neural network (MSCNN) and bidirectional gated recurrent unit (BiGRU) is proposed. MSCNN and BiGRU are used to extract multiscale features and temporal features from noisy vibration signals respectively, and different weights are assigned to the fused features through the attention mechanism module to achieve important feature selection. Furthermore, in order to solve the problem of different feature distributions between the source domain and the target domain under variable working conditions, transfer learning is introduced in the proposed deep feature extraction network. The difference in feature distribution between the source domain and the target domain is measured by a multi-level distance formula, and the measurement result is added to the loss function. The back propagation of the loss is used to achieve the alignment of feature distribution between the source domain and the target domain. Finally, the model uses the SoftMax function as a classifier for rolling bearing fault diagnosis. Experimental comparison and analysis show that the proposed model has good migration ability and achieves a higher fault identification accuracy.

Multilayer restoration in IP-Optical networks by adjustable robust optimization and deep reinforcement learning

Today, IP-Optical networks apply IP restoration as the default strategy to recover IP traffic from optical failures. This strategy has been preferred over optical restoration as it circumvents the lengthy delays involved in the reconfiguration of the optical layer. Although fast, IP restoration requires the overprovisioning of costly capacity to cope with optical failures. The advent of software-defined optical networking enables a changeover towards more efficient methods that integrate IP-Optical restoration. These methods should not only restore traffic from failures considered in the planning phase, but they should also efficiently restore traffic from unforeseen failures. This paper studies this problem by investigating optimization algorithms for capacity planning and multilayer restoration based on the theory of adjustable robust optimization (ARO). The approach performs offline optimization of the capacities of IP links as well as the routing and capacities of IP tunnels in both failure-free mode of operation and in a foreseen set of optical failures. Besides, the approach optimizes an affine policy that is applied online to recover IP traffic from unforeseen failures, thereby providing robustness to optical failures not regarded in the planning phase. To overcome the limitations of the affine policy, an alternative robust algorithm is formulated based on deep reinforcement learning (DRL) and graph neural networks (GNNs). By training a DRL-GNN agent, the performance of the restoration process is improved by further minimizing the traffic losses when unforeseen optical failures occur. Results in selected scenarios show that the algorithms outperform IP restoration in terms of capacity requirements, while minimizing the traffic losses in the case of failures. Moreover, the DRL-GNN method significantly improves the ARO-based affine algorithm, which shows the capability of learning the complex relationship between the capacity impairments caused by optical failures and the routing strategy required to restore IP traffic.

Novel Reference Method for the Characterization of PD Measuring Systems Using HFCT Sensors.

During their lifespan, high-voltage (HV) electrical systems are subjected to operating conditions in which electrical, mechanical, thermal and environmental-related stresses occur. These conditions over time lead to unforeseen failures caused by various types of defects. For this reason, there are several technologies for measuring and monitoring the electrical systems, with the aim of minimizing the number of faults. The early detection of defects, preferably in their incipient state, will enable the necessary corrective actions to be taken in order to avoid unforeseen failures. These failures generally lead to human risks and material damage, lack of power supply and significant economic losses. An efficient maintenance technique for the early detection of defects consists of the supervision of the dielectrics status in the installations by means of on-line partial discharge (PD) measurement. Nowadays, there are numerous systems in the market for the measurement of PD in HV installations. The most efficient with a reasonable cost will be those that offer greater security guarantees and the best positioned in the market. Currently, technology developers and users of PD measuring systems face difficulties related to the lack of reference procedures for their complete characterization and to the technical and economic drawback of performing the characterization tests on site or in laboratory installations. To deal with the previous difficulties, in this paper a novel method for the complete and standardized characterization of PD measuring systems is presented. The applicability of this method is mainly adapted for the characterization of systems operating in on-line applications using high-frequency current transformer (HFCT) sensors. For the appropriate application of the method, an associated and necessary scale modular test platform is used. In the test platform, the real on-site measuring conditions of an HV insulated distribution line are simulated in a controlled way. Practical characterizations, showing the convenience and advantages of applying the method using the modular test platform, are also presented.

Inhomogeneous Strain Behaviors of the High Strength Pipeline Girth Weld under Longitudinal Loading.

Unforeseen failures in girth welds present a significant challenge for the pipeline industry. This study utilizes 3D Digital Image Correlation (DIC) assisted cross-weld tensile testing to analyze the strain response of high-strength thick-walled pipelines, providing essential insights into the strain migration and fracture mechanisms specific to girth welds. The results reveal that the welding process significantly affects the mechanical distribution within the girth weld. The tested Shielded Metal Arc Welded (SMAW-ed) pipe exhibited undermatched girth welds due to high heat input, while Gas Metal Arc Welding (GMAW) introduced a narrower weld and Heat-Affected Zone (HAZ) with higher hardness than the base metal, indicative of overmatched girth welds. Strain migration, resulting from a combination of metallurgical heterogeneous materials and geometrical reinforcement strengthening, progressed from the softer HAZ to the base metal in the SMAW-ed sample with reinforcement, ultimately leading to fracture in the base metal. In contrast, the GMAW-ed sample shows no strain migration. Reinforcement significantly improves the tensile strength of girth welds and effectively prevents failure in the weld region. Sufficient reinforcement is crucial for minimizing the risk of failure in critical areas such as the weld metal and HAZ, particularly in SMAW-ed pipes.

Resource Allocation in an Uncertain Environment: Application to Snowplowing Operations in Utah

We consider a two-stage planning problem where a fleet of snowplow trucks is divided among a set of independent regions, each of which then designs routes for efficient snow removal. The central authority wishes to allocate trucks to improve service quality across the regions. Stochasticity is introduced by uncertain weather conditions and unforeseen failures of snowplow trucks. We study two versions of this problem. The first aims to minimize the maximum turnaround time (across all regions) that can be sustained with a user-specified probability. The second seeks to minimize the total expected workload that has not been completed within a user-specified time frame. We develop algorithms that solve these problems effectively and demonstrate their practical value through a case application to snowplowing operations in Utah, obtaining solutions that significantly outperform the allocation currently used in practice. Funding: Financial support from the Utah Department of Transportation [Grant 218138]; the National Science Foundation [Grant CMMI-2112758]; and the Mountain-Plains Consortium [Grant 637] is gratefully acknowledged. Supplemental Material: The e-companion is available at https://doi.org/10.1287/trsc.2023.0024 .

Towards Robust Visual Understanding: from Recognition to Reasoning

Models that learn from data are widely and rapidly being deployed today for real-world use, but they suffer from unforeseen failures due to distribution shift, adversarial attacks, noise and corruption, and data scarcity. But many failures also occur because many modern AI tasks require reasoning beyond pattern matching -- and such reasoning abilities are difficult to formulate as data-based input-output function fitting. The reliability problem has become increasingly important under the new paradigm of semantic ``multimodal'' learning. My research provides avenues to develop robust and reliable computer vision systems, particularly by leveraging the interactions between vision and language. In this AAAI New Faculty highlights talk, I will cover three thematic areas of my research, ranging from robustness in computer vision, open-domain reliability in visual reasoning, and challenges and opportunities in evaluation of generative models. Readers are encouraged to refer to my website (www.tejasgokhale.com) for more details and updates from my lab's activities towards the goal of robust visual understanding.

Identification and analysis of stochastic deception attacks on cyber–physical systems

Cyber–Physical Systems (CPSs) refer to control systems which are composed of sensors, actuators, computers and network components. These systems are vulnerable to unforeseen failures and external malicious attacks. In this paper, we analyze the stability of CPSs under stochastic deception attacks. To this end: (i) we propose a statistical Intrusion Detection System (IDS) for detection of deception attacks in CPSs; (ii) identify the place of such attacks by taking advantage of a novel cryptographic adversarial model; (iii) based on the real-time data and characterizing the deception attacks by IDS, we analyze the effect of both adaptive and blind/random types of deception attacks on the stability of CPSs. We do this through Markov chain modeling and subsequently extract the sufficient stability conditions. (iv) as an extra effort, introduce a new adaptive and intelligent deception attack. We validate our findings by illustrative examples at the end of the paper. Our results show that proposed IDS can detect deception attacks with low false positive and negative rates in real time. The results also confirm the validity of the theoretically-predicted stability conditions.

Preliminary Nose Landing Gear Digital Twin for Damage Detection

An increase in aircraft availability and readiness is one of the most desired characteristics of aircraft fleets. Unforeseen failures cause additional expenses and are particularly critical when thinking about combat jets and Unmanned Aerial Vehicles (UAVs). For instance, these systems are used under extreme conditions, and there can be situations where standard maintenance procedures are impractical or unfeasible. Thus, it is important to develop a Health and Usage Monitoring System (HUMS) that relies on diagnostic and prognostic algorithms to minimise maintenance downtime, improve safety and availability, and reduce maintenance costs. In particular, within the realm of aircraft structures, landing gear emerges as one of the most intricate systems, comprising several elements, such as actuators, shock absorbers, and structural components. Therefore, this work aims to develop a preliminary digital twin of a nose landing gear and implement diagnostic algorithms within the framework of the Health and Usage Monitoring System (HUMS). In this context, a digital twin can be used to build a database of signals acquired under healthy and faulty conditions on which damage detection algorithms can be implemented and tested. In particular, two algorithms have been implemented: the first is based on the Root-Mean-Square Error (RMSE), while the second relies on the Mahalanobis distance (MD). The algorithms were tested for three nose landing gear subsystems, namely, the steering system, the retraction/extraction system, and the oleo-pneumatic shock absorber. A comparison is made between the two algorithms using the ROC curve and accuracy, assuming equal weight for missed detections and false alarms. The algorithm that uses the Mahalanobis distance demonstrated superior performance, with a lower false alarm rate and higher accuracy compared to the other algorithm.

Uncertainty quantification in multiaxial fatigue life prediction using Bayesian neural networks

In practical engineering scenarios, unforeseen failures may result in significant costs and risks. Assessing the uncertainty related to multiaxial fatigue life prediction is crucial for engineering applications. However, establishing a reliable multiaxial fatigue life prediction model remains challenging due to uncertainties in physical models, material properties, and measurement data. This paper proposes an effective Bayesian Neural Network (BNN) model for quantifying the uncertainty in multiaxial fatigue life prediction. Fatigue parameters in the multiaxial fatigue life prediction model are used as inputs for the BNN. The BNN method utilizes No-U-turn Sampler (NUTS) and Automatic Differentiation Variational Inference (ADVI) to address the inference problem in the model and perform life predictions for unknown non-proportional loading paths and different materials. Validation using experimental data from six different materials confirms the effectiveness of the BNN model. Experiments show that the performance of BNN models is beneficial in most cases compared to classical ML models, assessed through different error standards, statistical tests, Taylor diagrams, uncertainty analysis, and scatter index analysis. The proposed method provides a clear understanding of the uncertainty in multiaxial fatigue life prediction, offering a promising approach for simulating fatigue life under multiaxial loading in engineering applications.

Optimal geometry of the powered roof support’s operation

Monitoring the working parameters of powered roof support is an area for improvement in hard coal mining. The phenomena occurring during the operation generate many risks from difficult geological and mining conditions, leading to undesirable events. In addition, improper use of machinery and equipment results in a high accident rate in mining. Thus, monitoring the operation of machines in mining reduces accidents and losses resulting from stops and prevents unforeseen failures caused by operational and external factors. The paper presents the research results on the optimal geometry of the powered roof support operation in the mining wall. The research included the powered roof support’s essential elements’ operation. Sensors constituting the measuring system were installed on these elements. The measurements made by the sensors made it possible to determine the working height at a given stage of the section’s operation. The research was carried out in three sections, which were part of the powered roof support. The measurements were taken during actual changes occurring in the coal mining process.

VISCOSITY MONITORING AND ANALYSIS FOR ROTATING MACHINERIES

Lubricants used in machines degrade with time. This may be caused by load, contamination and operating environment. Proper monitoring and analysis of the state of the lubricant is required to keep the machine in good condition and to minimize the cost of maintenance. This research is concerned with the application of lubrication standards to ensure proper operation of rotating equipment. The methods outlined include monitoring, analysis and comparison with lubricant performance standards. The method when rightly applied can signal wear, level of contamination and chemistry of the lubricant. Viscosity monitoring and analysis is useful to determine the state of the lubricant and hence the health of the machine, which is useful in taking maintenance decisions. With proper monitoring and analysis of the lubricant in rotating equipment, safety is ensured in addition to reduction of unforeseen failures and the cost of maintenance of the machines.

Designing Isolation Valve System to Prevent Unexpected Water Quality Incident

Designing an effective isolation valve system (IVS) is vital to enhance resilience against unforeseen failures in water systems. During isolation, the system’s hydraulics undergo changes, potentially causing alterations in flow direction and velocity, leading to the dislodgement of accumulated materials and triggering unexpected water quality incidents. This study presents a novel IVS design approach by integrating the consideration of flow direction change (FDC) as an additional constraint within conventional reliability-based models. Two optimization models, Optimization I and Optimization II, prioritize reliability, with the latter also factoring in valve installation cost as a multi-objective function. Performance evaluation metrics, such as the Hydraulic Geodesic Index (HGI), Modified Resilience Index (MRI), and robustness index, were employed for a comprehensive analysis. The results indicated more than 40 instances of FDC in the traditional design, challenging the conventional notion that a higher number of valves inherently reduces risk. The superiority of the proposed model persisted for the single reservoir network in Optimization II. However, for networks with multiple reservoirs, the traditional design outperformed the proposed model, particularly in terms of cost. Nevertheless, when comparing designs with similar reliability, the proposed model showcased a superior performance, despite its higher associated cost. Notably, the proposed approach exhibits potential cost-effectiveness, considering the potential economic losses attributable to water quality incidents. In summary, the implementation of this methodology can effectively manage both water quality and quantity, enabling the identification of vulnerable pipes within the network for sustainable management.

A framework for GPR‐based water leakage detection by integrating hydromechanical modelling into electromagnetic modelling

AbstractTimely and accurate detection of water pipe leakage is critical to preventing the loss of freshwater and predicting potential hazards induced by the change in underground water conditions, thereby developing mitigation strategies to improve the resilience of pipeline infrastructure. Ground‐penetrating radar (GPR) has been widely applied to investigating ground conditions and detecting pipe leakage. However, due to uncertainties of complex underground environments and time‐lapse change, a proper interpretation of GPR data has been a challenging task. This paper aims to leverage hydromechanical (HM) modelling to predict electromagnetic (EM) responses of water leakage detection in diverse leakage cases. A high‐fidelity 3D digital model of an actual pipeline network, hosting pipes with various sizes and materials, was reconstructed to represent the complex geometry and various mediums. The interoperability between the digital model and the numerical models utilized in HM and EM simulations was improved to better capture the irregular pipelines. Based on Kriging interpolation and the volumetric complex refractive index model, a linking technique was employed to replicate material heterogeneity caused by water intrusion. Thus, a framework was developed to accommodate the interoperability among digital modelling, HM modelling and finite‐difference time‐domain forward modelling. Moreover, sensitivity studies were conducted to evaluate the influences of different time stages, leak positions and pipe types on GPR responses. In GPR B‐scans, the presence of hyperbolic motion and horizontal reflections serve as indicators to estimate the location and scale of water leakage. Specifically, a downward‐shifting hyperbola indicates that the pipeline is submerged by leaked water, whereas the emergence of horizontal reflection is linked to the wetting front of saturated areas. The developed framework can be expanded for complicated applications, such as unknown locations and unforeseen failure modes of pipelines. It will increase the efficiency and accuracy of traditional interpretations of GPR‐based water leakage detection and thus enable automated interpretations by data‐driven methods.

Application of structural methods to assess the reliability of ship’s mechanical systems

At present, reliability indicators are of great importance at the design stages, during operation and in cases of extending the technical life. This is one of the reasons for choosing the research topic on the application of structural methods for evaluating the reliability of ship mechanical systems. The operational stage of the life cycle of ship’s mechanical systems is considered in the paper. Durability criteria such as service life, operating time, frequency of repairs are chosen as reliability indicators. In turn, work on the repair, maintenance or replacement of ship’s mechanical systems elements is carried out in accordance with the assigned regulations or in the case of an unforeseen failure. The comparison of event models of changes in the operating conditions of ship’s mechanical systems is carried out. To estimate the remaining resource in the form of graphs, events are shown by service life and operating hours, distributed by years. In order to improve the existing methods for assessing reliability, a set of elements of the average values of the residual resource and residual output is considered. The approach proposed by the authors for assessing both element by element and the entire ship mechanical system as a whole allows us to consider the residual resource from the moment of monitoring its technical condition to the transition to the limit state, to plan technical measures and prevent possible malfunctions. Three different time slices models corresponding to planned repairs and maintenance of systems, replacement of system elements and unforeseen accidents are considered in the study. The difference in the amplitudes of the calculated indicators of the ship’s mechanical systems reliability has shown that the closer the options between the maximum and minimum values are to the average, the more accurately the statistical indicators characterize this pattern. However, amplitude deviations can be used to apply fault risk assessment. This line of research seems to be important for ships in the Arctic navigation area, which are characterized by increased requirements for ship survivability.

Preventive Maintenance Analysis Based on Mean Time Between Failure (MTBF) and Mean Time to Repair (MTTR)

A company's ability to meet its goals depends heavily on the smooth operation of its production process. To ensure that the production machinery operates smoothly and without any breakdowns, it is essential to implement effective maintenance that addresses any issues related to the equipment. This research aims to identify the root causes of equipment failure in the soda and dolomite bucket elevator by analyzing the Mean Time Between Failure (MTBF), Mean Time to Repair (MTTR), and conducting a qualitative evaluation using Failure Mode Effects Analysis (FMEA). The analysis reveals that the company performs preventive maintenance every 3 days, which is more frequent than the optimal schedule of once every 33 days. This suggests that the company may be addressing unforeseen failures. The FMEA analysis indicates that the highest Risk Priority Number is associated with suboptimal maintenance of the Staff. To address this issue, the company can use Radio-Frequency Identification (RFID) technology.

EXPLORING THE ENTREPRENEURIAL INTENTION OF GENERATION Y PEOPLE IN THAILAND

The main purpose of this study is to analyse the factors affecting and exploring entrepreneurial intention of generation Y people in Thailand with particular attention to demographics, personal planned behaviours, and contextual factors, those factors will be the key tools that may encourage entrepreneurship. This study based on interview the person who born in 1980 to 1997 called generation Yers. The results showed they have a high expectation of being a self-employee rather that working in a well-known organisation and be able to understand the idea how these young entrepreneurs identify business opportunities and overcome challenges through self-attitudes toward unforeseen failure in business. Yers believe that they are the huge wave of movement to challenge and develop a globalisation throughout creativity and innovation. This study also shown the result of relationship among demographics, personal planned behaviours, and other contextual details that are the main key tools were the determinants of Thai entrepreneurial intention.

Digital Twin-enabled and Knowledge-driven decision support for tunnel electromechanical equipment maintenance

Urban tunnel infrastructure plays critical roles in sustaining the wellbeing of a society. The operation of tunnels relies on a diverse range of tunnel electromechanical equipment (TEE), such as ventilation, drainage, and the lighting system. However, effectively and proactively maintaining TEE to prevent unforeseen failures using limited resources remains an unresolved challenge. The utilization of digital twin technology, which combines Building Information Modeling (BIM), Internet of Things (IoT) and Semantic Web technologies, offers a knowledge-rich environment for the development of improved maintenance strategies. This study proposes a digital twin-enabled and knowledge-driven decision support method for proactive TEE maintenance. Initially, a digital twin conceptual framework for proactive TEE maintenance is presented, which integrates a knowledge-driven approach to support decision making. Subsequently, a controlled vocabulary-based method is developed to extract and update maintenance knowledge. Finally, a novel combinatory reasoner, including a rule selection algorithm and an inference algorithm, is devised to automatically generate maintenance schemes. Based on the proposed method, a decision support tool was developed and applied to Wenyi Road Tunnel in Hangzhou, China. The results demonstrated the effectiveness of the method, which can continuously update maintenance knowledge and assist fault detection and TEE state prediction with the combinatory reasoner. Moreover, the inference efficiency achieved by our method surpasses that of traditional approaches by approximately 162 ms.

An integrated resilience assessment methodology for emergency response systems based on multi-stage STAMP and dynamic Bayesian networks

The interactions of external disruptions and technical-human-organizational factors in emergency operations are usually observed. Resilience assessment of emergency systems can improve emergency response capability and system functional recovery. The increasing complexity and coupling of factors in emergency response systems need to be investigated from a system resilience perspective. In this paper, we propose to integrate a multi-stage System-Theoretic Accident Model and Processes (STAMP) with a dynamic Bayesian network (DBN) for the resilience assessment of emergency response systems. In the proposed methodology, emergency response systems are viewed as multi-step emergency operations for STAMP to analyze the hierarchical control and feedback structures. The output of multi-stage STAMP in controllers, actuators, sensors, and controlled processes is applied to develop a DBN for resilience assessment. For known external shocks (e.g., natural disasters), the effects of external shocks on the system are decomposed into subsystems or components. System degradation and recovery models are established. Regarding unknown external disruption (e.g., unforeseen failure modes), degeneration and recovery are temporally integrated into the analysis of system functionality. System performance is evaluated through the combination of socio-technical factors and external disasters. Eventually, the resilience of emergency response systems is obtained from the performance curves. The results demonstrate that the proposed model can evaluate system resilience after the system suffers from external disasters.

A Novel Fine-Tuning Model Based on Transfer Learning for Future Capacity Prediction of Lithium-Ion Batteries

Future capacity prediction of lithium-ion batteries is a highly researched topic in the field of battery management systems, owing to the gradual degradation of battery capacity over time due to various factors such as chemical changes within the battery, usage patterns, and operating conditions. The accurate prediction of battery capacity can aid in optimizing its usage, extending its lifespan, and mitigating the risk of unforeseen failures. In this paper, we proposed a novel fine-tuning model based on a deep learning model with a transfer learning approach comprising of two key components: offline training and online prediction. Model weights and prediction parameters were transferred from offline training using source data to the online prediction stage. The transferred Bi-directional Long Short-Term Memory with an Attention Mechanism model weights and prediction parameters were utilized to fine-tune the model by partial target data in the online prediction phase. Three battery batches with different charging policy were used to evaluate the proposed approach’s robustness, reliability, usability, and accuracy for the three charging policy batteries’ real-world data. The experiment results show that the proposed method’s efficacy improved, with an increase in the cycle number of the starting point, exhibiting a linear relationship with the starting point. The proposed method yields relative error values of 8.70%, 6.38%, 9.52%, 7.58%, 1.94%, and 2.29%, respectively, for the six target batteries in online prediction. Thus, the proposed method is effective in predicting the future capacity of lithium-ion batteries and holds potential for use in predictive maintenance applications.