Novelty Detection Research Articles

Conformal novelty detection for multiple metabolic networks.

Graphical representations are useful to model complex data in general and biological interactions in particular. Our main motivation is the comparison of metabolic networks in the wider context of developing noninvasive accurate diagnostic tools. However, comparison and classification of graphs is still extremely challenging, although a number of highly efficient methods such as graph neural networks were developed in the recent decade. Important aspects are still lacking in graph classification: interpretability and guarantees on classification quality, i.e., control of the risk level or false discovery rate control. In our contribution, we introduce a statistically sound approach to control the false discovery rate in a classification task for graphs in a semi-supervised setting. Our procedure identifies novelties in a dataset, where a graph is considered to be a novelty when its topology is significantly different from those in the reference class. It is noteworthy that the procedure is a conformal prediction approach, which does not make any distributional assumptions on the data and that can be seen as a wrapper around traditional machine learning models, so that it takes full advantage of existing methods. The performance of the proposed method is assessed on several standard benchmarks. It is also adapted and applied to the difficult task of classifying metabolic networks, where each graph is a representation of all metabolic reactions of a bacterium and to real task from a cancer data repository. Our approach efficiently controls - in highly complex data - the false discovery rate, while maximizing the true discovery rate to get the most reasonable predictive performance. This contribution is focused on confident classification of complex data, what can be further used to explore complex human pathologies and their mechanisms.

Learning to Detect Novel Species with SAM in the Wild

AbstractThis paper tackles the limitation of a closed-world object detection model that was trained on one species. The expectation for this model is that it will not generalize well to recognize the instances of new species if they were present in the incoming data stream. We propose a novel object detection framework for this open-world setting that is suitable for applications that monitor wildlife, ocean life, livestock, plant phenotype and crops that typically feature one species in the image. Our method leverages labeled samples from one species in combination with a novelty detection method and Segment Anything Model, a vision foundation model, to (1) identify the presence of new species in unlabeled images, (2) localize their instances, and (3) retrain the initial model with the localized novel class instances. The resulting integrated system assimilates and learns from unlabeled samples of the new classes while not “forgetting” the original species the model was trained on. We demonstrate our findings on two different domains, (1) wildlife detection and (2) plant detection. Our method achieves an AP of 56.2 (for 4 novel species) to 61.6 (for 1 novel species) for wildlife domain, without relying on any ground truth data in the background.

Novelty detection for long-term diagnostic data with Gaussian and non-Gaussian disturbances using a support vector machine

Abstract In condition monitoring lack of properly balanced data sets with faulty and healthy cases makes proper condition recognition very challenging. In many cases, one may have good condition data only as the machine is unique and there is no other example. This issue is addressed by proposing a support vector machine (SVM) for novelty detection applied to health index (HI) data. In this scheme, the moving window approach has been utilised in which the simple statistical parameterisation of the data is carried out. Then the model in the multidimensional (mD) space is constructed whose shape is defined by an estimated hypersphere border. If the data lies inside the border then it can be used to re-train the model. Whereas if it is outside the border then it cannot be recognized as a healthy case. The size of the mD hypersphere (for m=2) describes the location of the good-condition data cloud as a potential feature. If the size of the data cloud is growing, it means more dispersion of the data. The efficiency of the method is tested on simulated and well-known real data sets having Gaussian and non-Gaussian disturbances.

Open-world continual learning: Unifying novelty detection and continual learning

As AI agents are increasingly used in the real open world with unknowns or novelties, they need the ability to (1) recognize objects that (a) they have learned before and (b) detect items that they have never seen or learned, and (2) learn the new items incrementally to become more and more knowledgeable and powerful. (1) is called novelty detection or out-of-distribution (OOD) detection and (2) is called class incremental learning (CIL), which is a setting of continual learning (CL). In existing research, OOD detection and CIL are regarded as two completely different problems. This paper first provides a theoretical proof that good OOD detection for each task within the set of learned tasks (called closed-world OOD detection) is necessary for successful CIL. We show this by decomposing CIL into two sub-problems: within-task prediction (WP) and task-id prediction (TP), and proving that TP is correlated with closed-world OOD detection. The key theoretical result is that regardless of whether WP and OOD detection (or TP) are defined explicitly or implicitly by a CIL algorithm, good WP and good closed-world OOD detection are necessary and sufficient conditions for good CIL, which unifies novelty or OOD detection and continual learning (CIL, in particular). We call this traditional CIL the closed-world CIL as it does not detect future OOD data in the open world. The paper then proves that the theory can be generalized or extended to open-world CIL, which is the proposed open-world continual learning, that can perform CIL in the open world and detect future or open-world OOD data. Based on the theoretical results, new CIL methods are also designed, which outperform strong baselines in CIL accuracy and in continual OOD detection by a large margin.

The hippocampal CA2 region discriminates social threat from social safety.

The dorsal cornu ammonis 2 (dCA2) region of the hippocampus enables the discrimination of novel from familiar conspecifics. However, the neural bases for more complex social-spatial episodic memories are unknown. Here we report that the spatial and social contents of an aversive social experience require distinct hippocampal regions. While dorsal CA1 (dCA1) pyramidal neurons mediate the memory of an aversive location, dCA2 pyramidal neurons enable the discrimination of threat-associated (CS+) from safety-associated (CS-) conspecifics in both female and male mice. Silencing dCA2 during encoding or recall trials disrupted social fear discrimination memory, resulting in fear responses toward both the CS+ and CS- mice. Calcium imaging revealed that the aversive experience strengthened and stabilized dCA2 representations of both the CS+ and CS- mice, with the incorporation of an abstract representation of social valence into representations of social identity. Thus, dCA2 contributes to both social novelty detection and the adaptive discrimination of threat-associated from safety-associated individuals during an aversive social episodic experience.

Early maturation of neural auditory novelty detection − Typical development with no major effects of dyslexia risk or music intervention

ObjectiveTo determine the early development of novelty detection and the effect of familial dyslexia risk and infant music intervention on this development. MethodsIn the longitudinal DyslexiaBaby study, we investigated the maturation of novelty-P3 and late-discriminative negativity (LDN) event-related potentials to novel sounds at birth (N = 177) and at the ages of 6 (N = 83) and 28 months (N = 131). ResultsNovelty-P3 was elicited at all ages, whereas LDN was elicited at 6 and 28 months. Novelty-P3 amplitude was largest at 6 months, and its latency decreased with age. LDN amplitude decreased and latency increased between 6 to 28 months. Dyslexia risk or intervention had no effects, apart from a longer LDN latency in the high-risk than no-risk group. ConclusionsAlready neonates respond to novel environmental sounds, indicating prerequisites for detecting potentially relevant events at birth. Maturation influences neural novelty detection. SignificanceNovelty detection is crucial for perceiving important events, but its early development has been scarcely studied. We found, with a large sample, that neonates detect novel events, and showed the developmental pattern of its neural signature. The results serve as a reference for studies on typical and atypical novelty-detection development in infancy when behavioral testing is challenging.

Economical hybrid novelty detection leveraging global aleatoric semantic uncertainty for enhanced MRI-based ACL tear diagnosis

This study presents an innovative hybrid deep learning (DL) framework that reformulates the sagittal MRI-based anterior cruciate ligament (ACL) tear classification task as a novelty detection problem to tackle class imbalance. We introduce a highly discriminative novelty score, which leverages the aleatoric semantic uncertainty as this is modeled in the class scores outputted by the YOLOv5-nano object detection (OD) model. To account for tissue continuity, we propose using the global scores (probability vector) when the model is applied to the entire sagittal sequence. The second module of the proposed pipeline constitutes the MINIROCKET timeseries classification model for determining whether a knee has an ACL tear. To better evaluate the generalization capabilities of our approach, we also carry out cross-database testing involving two public databases (KneeMRI and MRNet) and a validation-only database from University General Hospital of Larissa, Greece. Our method consistently outperformed (p-value<0.05) the state-of-the-art (SOTA) approaches on the KneeMRI dataset and achieved better accuracy and sensitivity on the MRNet dataset. It also generalized remarkably good, especially when the model had been trained on KneeMRI. The presented framework generated at least 2.1 times less carbon emissions and consumed at least 2.6 times less energy, when compared with SOTA. The integration of aleatoric semantic uncertainty-based scores into a novelty detection framework, when combined with the use of lightweight OD and timeseries classification models, have the potential to revolutionize the MRI-based injury detection by setting a new precedent in diagnostic precision, speed and environmental sustainability. Our resource-efficient framework offers potential for widespread application.

Integrating deep learning architectures for enhanced biomedical relation extraction: a pipeline approach.

Biomedical relation extraction from scientific publications is a key task in biomedical natural language processing (NLP) and can facilitate the creation of large knowledge bases, enable more efficient knowledge discovery, and accelerate evidence synthesis. In this paper, building upon our previous effort in the BioCreative VIII BioRED Track, we propose an enhanced end-to-end pipeline approach for biomedical relation extraction (RE) and novelty detection (ND) that effectively leverages existing datasets and integrates state-of-the-art deep learning methods. Our pipeline consists of four tasks performed sequentially: named entity recognition (NER), entity linking (EL), RE, and ND. We trained models using the BioRED benchmark corpus that was the basis of the shared task. We explored several methods for each task and combinations thereof: for NER, we compared a BERT-based sequence labeling model that uses the BIO scheme with a span classification model. For EL, we trained a convolutional neural network model for diseases and chemicals and used an existing tool, PubTator 3.0, for mapping other entity types. For RE and ND, we adapted the BERT-based, sentence-bound PURE model to bidirectional and document-level extraction. We also performed extensive hyperparameter tuning to improve model performance. We obtained our best performance using BERT-based models for NER, RE, and ND, and the hybrid approach for EL. Our enhanced and optimized pipeline showed substantial improvement compared to our shared task submission, NER: 93.53 (+3.09), EL: 83.87 (+9.73), RE: 46.18 (+15.67), and ND: 38.86 (+14.9). While the performances of the NER and EL models are reasonably high, RE and ND tasks remain challenging at the document level. Further enhancements to the dataset could enable more accurate and useful models for practical use. We provide our models and code at https://github.com/janinaj/e2eBioMedRE/. Database URL: https://github.com/janinaj/e2eBioMedRE/.

Sex differences in distributed error-related neural activation in problem-drinking young adults

BackgroundDetecting and responding to errors is central to goal-directed behavior and cognitive control and is thought to be supported by a network of structures that includes the anterior cingulate cortex and anterior insula. Sex differences in the maturational timing of cognitive control systems create differential periods of vulnerability for psychiatric conditions, such as substance use disorders. MethodsWe examined sex differences in error-related activation across an array of distributed brain regions during a Go/No-Go task in young adults with problem alcohol use (N=69; 34 females; M=19.4 years). Regions of interest previously linked to error-related activation, including anterior cingulate cortex, insula, and frontoparietal structures, were selected in a term-based meta-analysis. Individual differences in their responses to false alarm (FA) inhibitory errors relative to “go” trials (FA>GO) and correct rejections (FA>CR) were indexed using multivariate summary measures derived from principal components analysis. ResultsFA>GO and FA>CR activation both revealed a first component that explained the majority of the variance across error-associated regions and displayed the strongest loadings on salience network structures. Compared to females, males exhibited significantly higher levels of the FA>GO component but not the FA>CR component. ConclusionsMales exhibit greater salience network activation in response to inhibitory errors, which could be attributed to sex differences in error-monitoring processes or to other functions (e.g., novelty detection). The findings are relevant for the further characterization of sex differences in cognitive control and may have implications for understanding individual differences in those at risk for substance use or other cognitive control disorders.

Identifying fine-scale archaeological features using KH-9 HEXAGON mapping and panoramic camera images: evidence from Liangzhu Ancient City

ABSTRACT Historical fine-scale information of archaeological landscapes is crucial in archaeological investigations. However, documenting such information using satellite sensor data prior to 2000 remains a daunting challenge. Images from the declassified archives of KH-9 HEXAGON (KH-9) cameras, including the panoramic camera system (PCS) and mapping camera system (MCS), offer fine-scale information about archaeological sites. However, noise, contrast distortion and the availability of only a single panchromatic band can limit their potential, particularly for identifying features in subtropical climates within heterogeneous landscape types. This paper focuses on developing a novel multifaceted analytical framework with two components: image pre-processing and feature identification. The image pre-processing component is divided into two steps. First, a trained stationary wavelet transform (SWT) based on the normalized sill (NS) is developed to not only de-noises the image, but also preserve its original image characteristics. Then, the contrast of the de-noised images is optimized by the multi-resolution Top-hat (MTH) using multi-scale information. In the feature identification component, the MCS image is analysed using spatial colour composite write function memory (SCCWFM) and spatial novelty detection (SND). An ultra-fine spatial three-dimensional colour composite (UFSTCC) image and ultra-fine spatial digital surface model (UFSDSM) are produced to aid interpretation of the KH-9 PCS images. The proposed processing pipelines were tested on KH-9 MCS and PCS images of the World Heritage site at Liangzhu Ancient City (LAC) in China, which is characterized by a subtropical climate and a heterogeneous landscape types. The proposed pre-processing pipeline improved considerably the appearance of these images across the LAC landscape while maintaining the original image information. The developed digital analytical approaches for KH-9 PCS and MCS images facilitated straightforward identification of archaeological features in the LAC. The proposed framework has the potential to increase exploitation of the available KH-9 images in archaeological applications.

Prototyping a Secure and Usable User Authentication Mechanism for Mobile Passenger ID Devices for Land/Sea Border Control.

As the number of European Union (EU) visitors grows, implementing novel border control solutions, such as mobile devices for passenger identification for land and sea border control, becomes paramount to ensure the convenience and safety of passengers and officers. However, these devices, handling sensitive personal data, become attractive targets for malicious actors seeking to misuse or steal such data. Therefore, to increase the level of security of such devices without interrupting border control activities, robust user authentication mechanisms are essential. Toward this direction, we propose a risk-based adaptive user authentication mechanism for mobile passenger identification devices for land and sea border control, aiming to enhance device security without hindering usability. In this work, we present a comprehensive assessment of novelty and outlier detection algorithms and discern OneClassSVM, Local Outlier Factor (LOF), and Bayesian_GaussianMixtureModel (B_GMM) novelty detection algorithms as the most effective ones for risk estimation in the proposed mechanism. Furthermore, in this work, we develop the proposed risk-based adaptive user authentication mechanism as an application on a Raspberry Pi 4 Model B device (i.e., playing the role of the mobile device for passenger identification), where we evaluate the detection performance of the three best performing novelty detection algorithms (i.e., OneClassSVM, LOF, and B_GMM), with B_GMM surpassing the others in performance when deployed on the Raspberry Pi 4 device. Finally, we evaluate the risk estimation overhead of the proposed mechanism when the best performing B_GMM novelty detection algorithm is used for risk estimation, indicating efficient operation with minimal additional latency.

Classification and Novelty Detection of Tampered ICs Using ResCav

This paper investigates the capabilities of the resonant cavity system (ResCav) for detecting tampered integrated circuits (ICs) within supply chains. Prior research showcased ResCav’s ability to discern minor circuit variations, this study focuses on enhancing supervised classification results and introduces a one-class support vector machine (SVM) approach with a modified radial basis function kernel for novelty detection. Through finer hyperparameter tuning, the system achieves improved classification accuracy, demonstrating its potential to identify nuanced alterations with even higher precision and recall rates. Additionally, the application of a one-class SVM enables the detection of tampered ICs without reliance on labeled datasets, expanding utility in scenarios where access is limited to golden ICs. These advancements in ResCav’s capabilities signify progress in failure prevention methodologies, offering an efficient and non-destructive solution crucial for safeguarding against counterfeit and non-conforming components infiltrating critical supply chains.

Unsupervised Anomaly Detection via Nonlinear Manifold Learning

Abstract Anomalies are samples that significantly deviate from the rest of the data and their detection plays a major role in building machine learning models that can be reliably used in applications such as data-driven design and novelty detection. The majority of existing anomaly detection methods either are exclusively developed for (semi) supervised settings, or provide poor performance in unsupervised applications where there are no training data with labeled anomalous samples. To bridge this research gap, we introduce a robust, efficient, and interpretable methodology based on nonlinear manifold learning to detect anomalies in unsupervised settings. The essence of our approach is to learn a low-dimensional and interpretable latent representation (aka manifold) for all the data points such that normal samples are automatically clustered together and hence can be easily and robustly identified. We learn this low-dimensional manifold by designing a learning algorithm that leverages either a latent map Gaussian process (LMGP) or a deep autoencoder (AE). Our LMGP-based approach, in particular, provides a probabilistic perspective on the learning task and is ideal for high-dimensional applications with scarce data. We demonstrate the superior performance of our approach over existing technologies via multiple analytic examples and real-world datasets.

Multi-modal data novelty detection with adversarial autoencoders

Novelty detection is usually defined as the identification of new or abnormal objects (outliers) from the normal ones (inliers), which has wide potential applications including instrument fault, credit card theft warning, and disease diagnosis in the real world. Most of the existing researches focus on the novelty detection for single-modal data, which may fail to provide an accurate and reliable decision because the single-modal data sometimes cannot fully reflect the actual condition of the problem. However, research on the novelty detection from multiple sources is limited. This study developed an end-to-end deep learning architecture of novelty detection for multi-modal data based on adversarial autoencoders (AAE), which requires no input data of the novelty class during the training process. The proposed model consists of three deep networks, which are trained to compete with each other, collaborate to understand the underlying concept in the normal class, and thus classify the testing samples. Among the three networks, two of them work as the novelty detectors, and the other one called the generator will support the two novelty detectors by enhancing the inliers and distorting the outliers. In addition, a pseudo novelty mechanism is developed to expand the groups of adversarial training with the aim to greatly improve the precision and robustness of the whole model. Then the proposed model is employed for the novelty detection based on the multi-modal data that is composed of images, audios, videos, and texts, etc. The experimental results on the PKU FG-XMedia dataset and the MSR-VTT dataset reveal that our proposed method learns the normal class effectively and is superior to the baseline and state-of-the-art methods.

Enabling Seamless Connectivity: Networking Innovations in Wireless Sensor Networks for Industrial Application.

The wide-ranging applications of the Internet of Things (IoT) show that it has the potential to revolutionise industry, improve daily life, and overcome global challenges. This study aims to evaluate the performance scalability of mature industrial wireless sensor networks (IWSNs). A new classification approach for IoT in the industrial sector is proposed based on multiple factors and we introduce the integration of 6LoWPAN (IPv6 over low-power wireless personal area networks), message queuing telemetry transport for sensor networks (MQTT-SN), and ContikiMAC protocols for sensor nodes in an industrial IoT system to improve energy-efficient connectivity. The Contiki COOJA WSN simulator was applied to model and simulate the performance of the protocols in two static and moving scenarios and evaluate the proposed novelty detection system (NDS) for network intrusions in order to identify certain events in real time for realistic dataset analysis. The simulation results show that our method is an essential measure in determining the number of transmissions required to achieve a certain reliability target in an IWSNs. Despite the growing demand for low-power operation, deterministic communication, and end-to-end reliability, our methodology of an innovative sensor design using selective surface activation induced by laser (SSAIL) technology was developed and deployed in the FTMC premises to demonstrate its long-term functionality and reliability. The proposed framework was experimentally validated and tested through simulations to demonstrate the applicability and suitability of the proposed approach. The energy efficiency in the optimised WSN was increased by 50%, battery life was extended by 350%, duplicated packets were reduced by 80%, data collisions were reduced by 80%, and it was shown that the proposed methodology and tools could be used effectively in the development of telemetry node networks in new industrial projects in order to detect events and breaches in IoT networks accurately. The energy consumption of the developed sensor nodes was measured. Overall, this study performed a comprehensive assessment of the challenges of industrial processes, such as the reliability and stability of telemetry channels, the energy efficiency of autonomous nodes, and the minimisation of duplicate information transmission in IWSNs.

Combination of generic novelty detection and supervised classification pipelines for industrial condition monitoring

Abstract Machine learning in industrial condition monitoring is currently a rapidly developing field of research, to improve the efficiency and reliability of industrial processes. Many of the used algorithms are supervised methods, which can learn and recognize hidden patterns in the data. However, training data is required to learn these patterns, which can only be generated to a limited extent in an industrial environment due to the high costs involved. Furthermore, it is impossible to represent all possible events in the training data. In contrast, unsupervised or semi-supervised methods can be used to detect new conditions or events. However, these usually do not allow diagnosis or quantification of a fault condition, which is why their usefulness for modern maintenance strategies is limited. Consequently, a robust condition monitoring system should combine the functionality of both approaches. This paper presents a methodology for the combination of supervised classification and semi-supervised novelty detection to build an expandable and adaptable condition monitoring by transferring recurring novelties as new conditions to the supervised classification. A superordinate algorithm is proposed to achieve a stepwise extension of the supervised model based on new conditions detected by novelty detection. With this approach, a condition monitoring system can at first be based on “normal” data of a new machine or process by adding failures or novel conditions step-by-step. Furthermore, the supervised methods can be used to help the corresponding staff identify unknown conditions by analyzing the features selected by the supervised classification. The general workflow is demonstrated for condition monitoring of the pneumatic drive system of a welding gun.

Dissecting the contribution of human chromosome 21 syntenic regions to recognition memory processes in adult and aged mouse models of Down syndrome.

Trisomy of human chromosome 21 (Hsa21) results in a constellation of features known as Down syndrome (DS), the most common genetic form of intellectual disability. Hsa21 is orthologous to three regions in the mouse genome on mouse chromosome 16 (Mmu16), Mmu17 and Mmu10. We investigated genotype-phenotype relationships by assessing the contribution of these three regions to memory function and age-dependent cognitive decline, using three mouse models of DS, Dp1Tyb, Dp(17)3Yey, Dp(10)2Yey, that carry an extra copy of the Hsa21-orthologues on Mmu16, Mmu17 and Mmu10, respectively. Prior research on cognitive function in DS mouse models has largely focused on models with an extra copy of the Mmu16 region and relatively little is known about the effects of increased copy number on Mmu17 and Mmu10 on cognition and how this interacts with the effects of aging. As aging is is a critical contributor to cognitive and psychiatric changes in DS, we hypothesised that ageing would differentially impact memory function in Dp1Tyb, Dp(17)3Yey, and Dp(10)2Yey, models of DS. Young (12-13 months and old (18-20 months mice Dp1Tyb, Dp(17)3Yey and Dp(10)2Yey mice were tested on a battery of object recognition memory test that assessed object novelty detection, novel location detection and associative object-in place memory. Following behavioral testing, hippocampal and frontal cortical tissue was analysed for expression of glutamatergic receptor proteins using standard immunoblot techniques. Young (12-13 months and old (18-20 months mice Dp1Tyb, Dp(17)3Yey and Dp(10)2Yey mice were tested on a battery of object recognition memory test that assessed object novelty detection, novel location detection and associative object-in place memory. Following behavioral testing, hippocampal and frontal cortical tissue was analysed for expression of glutamatergic receptor proteins using standard immunoblot techniques. Our results show that distinct Hsa21-orthologous regions contribute differentially to cognitive dysfunction in DS mouse models and that aging interacts with triplication of Hsa21-orthologous genes on Mmu10.

Scalable solution for agricultural soil organic carbon measurements using laser-induced breakdown spectroscopy

Effective verification of soil organic carbon (SOC) improvement interventions through soil carbon sequestration (SCS) requires robust methodologies to measure, report, and verify changes in soil carbon (C) levels. Furthermore, soil C must be monitored over time to ensure that sequestered C is not being re-emitted, thus ensuring the permanence of C removals. The traditional methods for soil C measurement are time-consuming, labor-intensive, and energy-intensive, increasing analysis costs. In this article, we verify the use of a commercially available laser-induced breakdown spectroscopy analyzer, the LaserAg-Quantum, coupled with the recursive feature addition, the gradient-boosted decision trees regression model, and the novelty detection model to predict C in soils. The developed method shows promising performance with an average limit of quantification of 0.75% of C and a precision of 4.10%. Accuracy metrics, including R2, mean absolute error, and root mean square error, yielded values of 0.81, 0.27%, and 0.37% for the validation dataset. Additionally, around 10% of validation samples after the novelty detection model exhibited relative error greater than 30%. Finally, our findings demonstrate the potential of the LaserAg-Quantum process to support measuring SOC in agricultural soils on a large scale.

Mitigation of unmeasured environmental and operational variability in long-term bridge health monitoring by unsupervised statistical learning

Environmental and operational variations present a significant challenge in the long-term health monitoring of bridge structures due to their potential to cause serious confounding influences and wrong decisions. These influences may lead to false alarms of damage during normal operation of a civil structure or, conversely, mask actual damage, resulting in a failure to correctly signal a warning. Consequently, a critical task in structural health monitoring, particularly for bridges, is to mitigate the negative impact of environmental variability. An initial approach is to measure the most influential environmental and operational factors and then apply statistical tools to eliminate their effects. However, this approach may not be effective under all conditions. Therefore, the primary objective of this paper is to propose an innovative unsupervised statistical learning method that addresses the challenges posed by environmental and operational variability by considering unmeasured factors. The proposed method involves three steps: data clustering, unsupervised feature selection, and novelty detection. The initial step utilizes k-means clustering to categorize features (modal frequencies) into five types of clusters, reflecting the influence of five environmental and operational factors on bridge structures; that is, temperature, humidity/rain/snow, wind speed/direction, traffic, and damage. These clustered features are then processed through an unsupervised feature selection algorithm based on reconstruction independent component analysis to extract new features. These reduced features are subsequently employed in a novelty detection model, developed using the Mahalanobis-squared distance, to analyze the environmental and operational effects. Real dynamic data of a steel arch bridge is utilized to verify the proposed method. Results demonstrate that this method can effectively handle the demanding issue stemming from unmeasured environmental and operational variability conditions.

A Hybrid Approach for Damage Detection and Localization on a Thin Metallic Plate

This paper illustrates a hybrid method for identifying structural damage that includes concepts from Physics-Based approaches within a Data-Driven framework. The lessons learned from the use of damage identification methods based on the analysis of modal curvature allows us to develop effective feature engineering in the development of Machine Learning (ML) algorithms. Moreover, the training of the algorithms exploits data provided from a population of damage cases generated by Finite Element (FE) analysis. The above data-analysis pipeline is applied to the identification of damage in a rectangular metallic plate for which measured modal data are available from a dedicated experimental campaign in CNR-INM laboratory using accelerometers. The damage is modelled as a stiffness reduction over a small area being also representative of material thinning due to corrosion. To have meaningful samples of the training database, the modal convergence of the FE model to the real structure is guaranteed by a structural optimization process. Experimentally identified noise, representative of real-life applications, is then added to the FE results before algorithm training. Damage existence and position are determined by a Novelty Detection approach and a Regression Neural Network, respectively. First, damage is identified on new test cases generated by the same FE procedure used for training. Secondly, the trained algorithms are applied to the experimental dataset. Sensitivity analysis on several parameters (number of samples for training, damage severity and noise levels) is numerically carried out to understand the applicability limits of the present methodology.