Convolutional Autoencoder Research Articles

Unsupervised Feature Extraction From Raw Data for Gesture Recognition With Wearable Ultralow-Power Ultrasound.

Wearable ultrasound (US) is a novel sensing approach that shows promise in multiple application domains, and specifically in hand gesture recognition (HGR). In fact, US enables to collect information from deep musculoskeletal structures at high spatiotemporal resolution and high signal-to-noise ratio, making it a perfect candidate to complement surface electromyography for improved accuracy performance and on-the-edge classification. However, existing wearable solutions for US-based gesture recognition are not sufficiently low power for continuous, long-term operation. On top of that, practical hardware limitations of wearable US devices (limited power budget, reduced wireless throughput, and restricted computational power) set the need for the compressed size of models for feature extraction and classification. To overcome these limitations, this article presents a novel end-to-end approach for feature extraction from raw musculoskeletal US data suited for edge computing, coupled with an armband for HGR based on a truly wearable (12 cm2, 9 g), ultralow-power (ULP) (16 mW) US probe. The proposed approach uses a 1-D convolutional autoencoder (CAE) to compress raw US data by 20× while preserving the main amplitude features of the envelope signal. The latent features of the autoencoder are used to train an XGBoost classifier for HGR on datasets collected with a custom US armband, considering armband removal/repositioning in between sessions. Our approach achieves a classification accuracy of 96%. Furthermore, the proposed unsupervised feature extraction approach offers generalization capabilities for intersubject use, as demonstrated by testing the pretrained encoder on a different subject and conducting posttraining analysis, revealing that the operations performed by the encoder are subject-independent. The autoencoder is also quantized to 8-bit integers and deployed on a ULP wearable US probe along with the XGBoost classifier, allowing for a gesture recognition rate ≥ 25 Hz and leading to 21% lower power consumption [at 30 frames/s (FPS)] compared to the conventional approach (raw data transmission and remote processing).

Convolutional Autoencoders for Signal Reconstruction and their Application to Damage Signature Extraction

The utilization of Machine Learning (ML) techniques for Structural Health Monitoring (SHM) has increased in the last years. Using ML techniques for signal reconstruction has been explored in the literature of this field. In this work, a database of Ultrasonic Guided Waves (UGW), propagated through a thermoplastic composite plate, and generated and acquired with piezoelectric transducers (PZT), has been preprocessed, organized, and analyzed to reconstruct signals by using Convolutional Autoencoders (cAE) neural networks. This special type of autoencoders (AE) is based on the utilization of convolutional stages, which are formed by deep-learning layers that are especially useful for their utilization on discretized time-series signals, due to their ability to convolve different sections of the signal, thus extracting different features which are automatically learned by the network. The cAE, which have a bottleneck section in their middle part (between the encoder and the decoder sections), have the ability to reduce the dimensionality of the input data, thus allowing the extraction of a minimum number of relevant features of the signals. Summarizing, this paper presents a neural network designed as a cAE which is able to reconstruct data from a reduced number of features, reaching a correlation value between the real and the artificial data higher than 98%.

Anomaly Detection of Sensor Arrays of Underwater Methane Remote Sensing by Explainable Sparse Spatio-Temporal Transformer

The increasing discovery of underwater methane leakage underscores the importance of monitoring methane emissions for environmental protection. Underwater remote sensing of methane leakage is critical and meaningful to protect the environment. The construction of sensor arrays is recognized as the most effective technique to increase the accuracy and sensitivity of underwater remote sensing of methane leakage. With the aim of improving the reliability of underwater methane remote-sensing sensor arrays, in this work, a deep learning method, specifically an explainable sparse spatio-temporal transformer, is proposed for detecting the failures of the underwater methane remote-sensing sensor arrays. The data input into the explainable sparse block could decrease the time complexity and the computational complexity (O (n)). Spatio-temporal features are extracted on various time scales by a spatio-temporal block automatically. In order to implement the data-driven early warning system, the data-driven warning return mechanism contains a warning threshold that is associated with physically disturbing information. Results show that the explainable sparse spatio-temporal transformer improves the performance of the underwater methane remote-sensing sensor array. A balanced F score (F1 score) of the model is put forward, and the anomaly accuracy is 0.92, which is superior to other reconstructed models such as convolutional_autoencoder (CAE) (0.81) and long-short term memory_autoencoder (LSTM-AE) (0.66).

CRE-TSCAE: A Novel Classification Model Based on Stacked Convolutional Autoencoder for Dual-Target RSVP-BCI Tasks.

The Rapid Serial Visual Presentation (RSVP) paradigm facilitates target identification in a rapid picture stream, which is applied extensively in military target surveillance and police monitoring. Most researchers concentrate on the single target RSVP-BCI whereas the study of dual-target is scarcely conducted, limiting RSVP application considerably. This paper proposed a novel classification model named Common Representation Extraction-Targeted Stacked Convolutional Autoencoder (CRE-TSCAE) to detect two targets with one nontarget in RSVP tasks. CRE generated a common representation for each target class to reduce variability from different trials of the same class and distinguish the difference between two targets better. TSCAE aimed to control uncertainty in the training process while requiring less target training data. The model learned a compact and discriminative feature through the training from several learning tasks so as to distinguish each class effectively. It was validated on the World Robot Contest 2021 and 2022 ERP datasets. Experimental results showed that CRE-TSCAE outperformed the state-of-the-art RSVP decoding algorithms and the Average ACC was 71.25%, improving 6.5% at least over the rest. It demonstrated that CRE-TSCAE showed a strong ability to extract discriminative latent features in detecting the differences among two targets with nontarget, which guaranteed increased classification accuracy. CRE-TSCAE provided an innovative and effective classification model for dual-target RSVP-BCI tasks and some insights into the neurophysiological distinction between different targets.

Meta-Path Semantic and Global-Local Representation Learning Enhanced Graph Convolutional Model for Disease-Related miRNA Prediction.

Dysregulation of miRNAs is closely related to the progression of various diseases, so identifying disease-related miRNAs is crucial. Most recently proposed methods are based on graph reasoning, while they did not completely exploit the topological structure composed of the higher-order neighbor nodes and the global and local features of miRNA and disease nodes. We proposed a prediction method, MDAP, to learn semantic features of miRNA and disease nodes based on various meta-paths, as well as node features from the entire heterogeneous network perspective, and node pair attributes. Firstly, for both the miRNA and disease nodes, node category-wise meta-paths were constructed to integrate the similarity and association connection relationships. Each target node has its specific neighbor nodes for each meta-path, and the neighbors of longer meta-paths constitute its higher-order neighbor topological structure. Secondly, we constructed a meta-path specific graph convolutional network module to integrate the features of higher-order neighbors and their topology, and then learned the semantic representations of nodes. Thirdly, for the entire miRNA-disease heterogeneous network, a global-aware graph convolutional autoencoder was built to learn the network-view feature representations of nodes. We also designed semantic-level and representation-level attentions to obtain informative semantic features and node representations. Finally, the strategy based on the parallel convolutional-deconvolutional neural networks were designed to enhance the local feature learning for a pair of miRNA and disease nodes. The experiment results showed that MDAP outperformed other state-of-the-art methods, and the ablation experiments demonstrated the effectiveness of MDAP's major innovations. MDAP's ability in discovering potential disease-related miRNAs was further analyzed by the case studies over three diseases.

A knowledge-data integration framework for rolling element bearing RUL prediction across its life cycle

Prediction of Remaining Useful Life (RUL) for Rolling Element Bearings (REB) has attracted widespread attention from academia and industry. However, there are still several bottlenecks, including the effective utilization of multi-sensor data, the interpretability of prediction models, and the prediction across the entire life cycle, which limit prediction accuracy. In view of that, we propose a knowledge-based explainable life-cycle RUL prediction framework. First, considering the feature fusion of fast-changing signals, the Pearson correlation coefficient matrix and feature transformation objective function are incorporated to an Improved Graph Convolutional Autoencoder. Furthermore, to integrate the multi-source signals, a Cascaded Multi-head Self-attention Autoencoder with Characteristic Guidance is proposed to construct health indicators. Then, the whole life cycle of REB is divided into different stages based on the Continuous Gradient Recognition with Outlier Detection. With the development of Measurement-based Correction Life Formula and Bidirectional Recursive Gated Dual Attention Unit, accurate life-cycle RUL prediction is achieved. Data from self-designed test rig and PHM 2012 Prognostic challenge datasets are analyzed with the proposed framework and five existing prediction models. Compared with the strongest prediction model among the five, the proposed framework demonstrates significant improvements. For the data from self-designed test rig, there is a 1.66 % enhancement in Corrected Cumulative Relative Accuracy (CCRA) and a 49.00 % improvement in Coefficient of Determination (R2). For the PHM 2012 datasets, there is a 4.04 % increase in CCRA and a 120.72 % boost in R2.

Identifying Climate Patterns Using Clustering Autoencoder Techniques

Abstract The complexity of growing spatiotemporal resolution of climate simulations produces a variety of climate patterns under different projection scenarios. This paper proposes a new data-driven climate classification workflow via an unsupervised deep learning technique that can dimensionally reduce the vast volume of spatiotemporal numerical climate projection data into a compact representation. We aim to identify distinct zones that capture multiple climate variables as well as their future changes under different climate change scenarios. Our approach leverages convolutional autoencoders combined with k-means clustering (standard autoencoder) and online clustering based on the Sinkhorn–Knopp algorithm (clustering autoencoder) across the conterminous United States (CONUS) to capture unique climate patterns in a data-driven fashion from the Geophysical Fluid Dynamics Laboratory Earth System Model with GOLD component (GFDL-ESM2G). The developed approach compresses 70 years of GFDL-ESM2G simulation at 0.125° spatial resolution across the CONUS under multiple warming scenarios to a lower-dimensional space by a factor of 660 000 and then tested on 150 years of GFDL-ESM2G simulation data. The results show that five climate clusters capture physically reasonable and spatially stable climatological patterns matched to known climate classes defined by human experts. Results also show that using a clustering autoencoder can reduce the computational time for clustering by up to 9.2 times when compared to using a standard autoencoder. Our five unique climate patterns resulting from the deep learning–based clustering of the lower-dimensional space thereby enable us to provide insights on hydrometeorology and its spatial heterogeneity across the conterminous United States immediately without downloading large climate datasets. Significance Statement This paper presents a data-driven climate classification approach using unsupervised deep learning to dimensionally reduce climate model outputs and to identify distinct climate regions for their future changes. Our approach compresses climate information for 70 years of Geophysical Fluid Dynamics Laboratory Earth System Model data across the conterminous United States (CONUS) at 0.125° spatial resolution. The results reveal that five climate clusters capture reasonable and stable climatological patterns matched to known climate patterns. The embedded clustering process in deep learning provides ×9.2 times faster execution than the k-means clustering technique. These results give us insight about climate spatial patterns and heterogeneity of hydrological patterns across the conterminous United States without downloading large climate datasets.

ScGAAC: A graph attention autoencoder for clustering single-cell RNA-sequencing data

Single-cell RNA-sequencing (scRNA-seq) enables the investigation of intricate mechanisms governing cell heterogeneity and diversity. Clustering analysis remains a pivotal tool in scRNA-seq for discerning cell types. However, persistent challenges arise from noise, high dimensionality, and dropout in single-cell data. Despite the proliferation of scRNA-seq clustering methods, these often focus on extracting representations from individual cell expression data, neglecting potential intercellular relationships. To overcome this limitation, we introduce scGAAC, a novel clustering method based on an attention-based graph convolutional autoencoder. By leveraging structural information between cells through a graph attention autoencoder, scGAAC uncovers latent relationships while extracting representation information from single-cell gene expression patterns. An attention fusion module amalgamates the learned features of the graph attention autoencoder and the autoencoder through attention weights. Ultimately, a self-supervised learning policy guides model optimization. scGAAC, a hypothesis-free framework, performs better on four real scRNA-seq datasets than most state-of-the-art methods. The scGAAC implementation is publicly available on Github at: https://github.com/labiip/scGAAC.

A deep learning method for locating fetal heart rate decelerations during labour using crowd-sourced data

Monitoring the heart rate of a fetus is the only method for continuously monitoring fetal well-being during labour. Decelerations in the fetal heart rate, usually corresponding with contractions, provide information regarding the ability of the fetus to cope with a reduced oxygen supply caused by contractions. For this reason, visual and computer-based assessments of fetal well-being depend upon the correct identification of decelerations. Agreement between clinical professionals when identifying these features is poor. Automated methods offer a more consistent alternative, but must perform at a level comparable to a trained human observer. Collecting data to train these systems can be difficult due to time restraints on experts in fetal medicine. In this study, we designed a website to crowd-source data from clinical professionals at their convenience. This data was combined with that from other publicly available sources to produce a dataset that was used to train a convolutional auto-encoder to locate decelerations. Crowd-sourcing was found to be an effective method for collecting data that was suitable to successfully train a deep learning model to locate decelerations. The model trained using this data provided a comparable level of accuracy to users of the website and other models used for the same purpose.

Identification of Distorted Fingerprints Using Wavelet Method and Convolutional Neural Network (CNN)

Biometrics offers a valuable tool for disaster victim identification, particularly through fingerprints. However, distorted or damaged fingerprints pose a significant challenge for recognition. This study explores the potential of Wavelet and Convolutional Neural Network (CNN) techniques to enhance the accuracy of distorted fingerprint recognition. Wavelet transform addresses the non-stationary nature of images and reduces detected noise. Convolutional Autoencoder, a CNN component, generates simplified feature representations from input images and attempts to reconstruct them. Utilizing 500 fingerprint samples, the testing results demonstrate accuracy variations ranging from 11% to 59.2%. Image reconstruction achieved 7.16% to 12.47% accuracy, while fingerprint matching attained accuracies between 92.71% and 93.96%. Averaging across all damage levels, the overall accuracy reached 37.65%, with average fingerprint reconstruction at 9.31% and average matching accuracy at 93.03%. The successful reconstruction and matching of distorted fingerprints within a certain range of damage using Wavelet and Convolutional Neural Network highlights the promising potential of these techniques for improved fingerprint identification in forensic and security contexts.

Classification of compressed full-waveform airborne lidar data

ABSTRACT Airborne laser scanners produce 3D data that can be used for a range of applications, such as urban planning, facility monitoring, flood mapping, and forest management. Additional information on the surveyed area can be obtained from the backscattered waveforms recorded by modern light detection and ranging (lidar) sensors. However, the high-dimensional representation of full-waveform data has hindered progress in its use due to difficulties in processing and storage. This paper develops a quantized convolutional autoencoder network to compress lidar waveform data into a condensed feature representation, resulting in a compression rate of up to 20:1. This, together with height information, is fed into a U-net convolutional neural network that achieves an accuracy of 93.7% on six classes.

Convolutional Autoencoder for Reconstruction of Historical Document Images: Ancient Manuscript Babad Lombok

The Babad Lombok is an ancient literary or manuscripts document that generally contains stories about the origins of the people of Lombok. This document is written on a lontar leaf, which in the past was used to write manuscripts, letters, and documents. At present, the Babad Lombok document can be seen in the form of photos or scans, so it can be viewed without having to go to a museum or cultural heritage site where the document is usually exhibited. However, because this document is an ancient artifact that has been around for hundreds of years, it has naturally experienced fading in the original document or its scanned versions. This makes the text inside less clear. This paper proposes to automatically reconstruct/repair the Babad Lombok document using a neural network. The type of neural network used is an Autoencoder or Convolutional Autoencoder (CAE). The CAE model is built sequentially and trained using original images of Babad Lombok as its training data and manually corrected images of Babad Lombok as the target or ground truth data. In the process, the two types of data are iteratively cropped to a size of 64x64 along the original size of the Babad Lombok image. This process results in input and target data for the CAE training process in this research, each consisting of 48,288 images. Testing the trained autoencoder model shows that the Babad images have been successfully repaired, making the text quality clearer before reconstruction. Ultimately, the proposed CAE has achieved training and validation accuracies of 89.09% and 94.57%, with corresponding loss values of 0.0418 and 0.0226.

A new indirect transfer fault diagnosis method based on feature separation

The existing fault diagnosis methods based on deep transfer learning achieve domain adaptation by matching shared features between the source and target domains. However, there are different private characteristics between the source domain and the target domain, and traditional cross-domain fault diagnosis algorithms ignore the impact of private attributes in the target domain on classification tasks. Meanwhile, the existence of private features causes a negative transfer problem for domain adaptation methods in domains with large aligned domain gaps. In practical applications, indirect transfer between the source domain and target domain can reduce the limitations of inherent attributes and enhance the guidance of private features in the target domain for classification tasks, which can enrich the transfer of fault knowledge and improve the effectiveness of cross-domain fault diagnosis. Therefore, an indirect transfer fault diagnosis method based on feature separation (ITFS) is proposed. Firstly, an attention-enhanced feature separation framework is proposed, which constructs a set of joint attention convolutional autoencoders (CAEs) to separate features from two domains, avoiding the influence of domain shift in private features. Secondly, to better adapt the two domains, an indirect domain adaptation method based on Gaussian distribution is proposed, which combines Gaussian indirect transfer learning with pseudo-labeling enhancement to reduce the domain gap in the two domains. Six transfer tasks on the PU datasets demonstrate that compared to MCD, the average accuracy of the proposed method is improved by 14.09 %, and it is superior to other popular existing methods. Experiments on three publicly available datasets have shown that this method outperforms other popular existing methods.

Discrimination of multiple sclerosis using scanning laser ophthalmoscopy images with autoencoder-based feature extraction

ObjectiveOptical coherence tomography (OCT) investigations have revealed that the thickness of inner retinal layers becomes decreased in multiple sclerosis (MS) patients, compared to healthy control (HC) individuals. To date, a number of studies have applied machine learning to OCT thickness measurements, aiming to enable accurate and automated diagnosis of the disease. However, there have much less emphasis on other less common retinal imaging modalities, like infrared scanning laser ophthalmoscopy (IR-SLO), for classifying MS. IR-SLO uses laser light to capture high-resolution fundus images, often performed in conjunction with OCT to lock B-scans at a fixed position. MethodsWe incorporated two independent datasets of IR-SLO images from the Isfahan and Johns Hopkins centers, consisting of 164 MS and 150 HC images. A subject-wise data splitting approach was employed to ensure that there was no leakage between training and test datasets. Several state-of-the-art convolutional neural networks (CNNs), including VGG-16, VGG-19, ResNet-50, and InceptionV3, and a CNN with a custom architecture were employed. In the next step, we designed a convolutional autoencoder (CAE) to extract semantic features subsequently given as inputs to four conventional ML classifiers, including support vector machine (SVM), k-nearest neighbor (K-NN), random forest (RF), and multi-layer perceptron (MLP). ResultsThe custom CNN (85 % accuracy, 85 % sensitivity, 87 % specificity, 93 % area under the receiver operating characteristics [AUROC], and 94 % area under the precision-recall curve [AUPRC]) outperformed state-of-the-art models (84 % accuracy, 83 % sensitivity, 87 % specificity, 92 % AUROC, and 94 % AUPRC); however, utilizing a combination of the CAE and MLP yields even superior results (88 % accuracy, 86 % sensitivity, 91 % specificity, 94 % AUROC, and 95 % AUPRC). ConclusionsWe utilized IR-SLO images to differentiate between MS and HC eyes, with promising results achieved using a combination of CAE and MLP. Future multi-center studies involving more heterogenous data are necessary to assess the feasibility of integrating IR-SLO images into routine clinical practice.

Koopman dynamic-oriented deep learning for invariant subspace identification and full-state prediction of complex systems

One strategy for predicting the state of nonlinear dynamical systems (typically of high dimensionality) is global linearization, such as utilizing the Koopman analysis model to transform the system state into an invariant subspace that evolves linearly. A critical challenge in the Koopman model is designing or deriving observation functions, typically nonlinear, to linearize the dynamical systems. To address the challenge, this study proposes a model called SKCAE (Skip-connected Koopman Convolutional AutoEncoder) that combines guidance of physics and data to learn observation functions accurately and efficiently. The novelties of SKCAE are twofold: (1) a coordinate-transforming main network for the construction of nonlinear observation functions, enabling the explicit identification of the low-dimensional dynamics that dominate the system's dynamical evolution, and (2) a Koopman dynamics-oriented subnetwork for quantitatively interpreting the system's intrinsic mechanisms from the frequency perspective and efficiently predicting the system state. Four numerical cases with discrete/continuous spectra on Eulerian/Lagrangrian descriptions are studied, i.e., fixed-point attractor, Duffing oscillator, fluid flow past a cylinder, and channel turbulence. Results demonstrate that SKCAE achieves significant improvement (up to a hundred times) in accuracy compared to conventional models and possesses remarkable capability in handling scenarios with data noises and/or loss, owing to its intrinsic advantage in retaining a broad range of frequency components of a dynamical system.

A Denoising Convolutional Autoencoder for SNR Enhancement in Chemical Exchange Saturation Transfer imaging: (DCAE-CEST).

To develop a SNR enhancement method for chemical exchange saturation transfer (CEST) imaging using a denoising convolutional autoencoder (DCAE), and compare its performance with state-of-the-art denoising methods. The DCAE-CEST model encompasses an encoder and a decoder network. The encoder learns features from the input CEST Z-spectrum via a series of 1D convolutions, nonlinearity applications and pooling. Subsequently, the decoder reconstructs an output denoised Z-spectrum using a series of up-sampling and convolution layers. The DCAE-CEST model underwent multistage training in an environment constrained by Kullback-Leibler divergence, while ensuring data adaptability through context learning using Principal Component Analysis processed Z-spectrum as a reference. The model was trained using simulated Z-spectra, and its performance was evaluated using both simulated data and in-vivo data from an animal tumor model. Maps of amide proton transfer (APT) and nuclear Overhauser enhancement (NOE) effects were quantified using the multiple-pool Lorentzian fit, along with an apparent exchange-dependent relaxation metric. In digital phantom experiments, the DCAE-CEST method exhibited superior performance, surpassing existing denoising techniques, as indicated by the peak SNR and Structural Similarity Index. Additionally, in vivo data further confirms the effectiveness of the DCAE-CEST in denoising the APT and NOE maps when compared to other methods. While no significant difference was observed in APT between tumors and normal tissues, there was a significant difference in NOE, consistent with previous findings. The DCAE-CEST can learn the most important features of the CEST Z-spectrum and provide the most effective denoising solution compared to other methods.

Deep-learning-based prediction of glaucoma conversion in normotensive glaucoma suspects

Background/aimsTo assess the performance of deep-learning (DL) models for prediction of conversion to normal-tension glaucoma (NTG) in normotensive glaucoma suspect (GS) patients.MethodsDatasets of 12 458 GS eyes were reviewed. Two...

Addressing data limitations in seizure prediction through transfer learning

According to the literature, seizure prediction models should be developed following a patient-specific approach. However, seizures are usually very rare events, meaning the number of events that may be used to optimise seizure prediction approaches is limited. To overcome such constraint, we analysed the possibility of using data from patients from an external database to improve patient-specific seizure prediction models. We present seizure prediction models trained using a transfer learning procedure. We trained a deep convolutional autoencoder using electroencephalogram data from 41 patients collected from the EPILEPSIAE database. Then, a bidirectional long short-term memory and a classifier layers were added on the top of the encoder part and were optimised for 24 patients from the Universitätsklinikum Freiburg individually. The encoder was used as a feature extraction module. Therefore, its weights were not changed during the patient-specific training. Experimental results showed that seizure prediction models optimised using pretrained weights present about four times fewer false alarms while maintaining the same ability to predict seizures and achieved more 13% validated patients. Therefore, results evidenced that the optimisation using transfer learning was more stable and faster, saving computational resources. In summary, adopting transfer learning for seizure prediction models represents a significant advancement. It addresses the data limitation seen in the seizure prediction field and offers more efficient and stable training, conserving computational resources. Additionally, despite the compact size, transfer learning allows to easily share data knowledge due to fewer ethical restrictions and lower storage requirements. The convolutional autoencoder developed in this study will be shared with the scientific community, promoting further research.

Self-Adaptive Server Anomaly Detection Using Ensemble Meta-Reinforcement Learning

As the user’s behavior changes at any time with cloud computing and network services, abnormal server resource utilization traffic will lead to severe service crashes and system downtime. The traditional single anomaly detection model cannot handle the rapid failure prediction ahead. Therefore, this study proposed ensemble learning combined with model-agnostic meta-reinforcement learning called ensemble meta-reinforcement learning (EMRL) to implement self-adaptive server anomaly detection rapidly and precisely, according to the time series of server resource utilization. The proposed ensemble approach combines hidden Markov model (HMM), variational autoencoder (VAE), temporal convolutional autoencoder (TCN-AE), and bidirectional long short-term memory (BLSTM). The EMRL algorithm trains this combination with several tasks to learn the implicit representation of various anomalous traffic, where each task executes trust region policy optimization (TRPO) to quickly adapt the time-varying data distribution and make rapid decisions precisely for an agent response. As a result, our proposed approach can improve the precision of anomaly prediction by 2.4 times and reduce the model deployment speed by 5.8 times on average because a meta-learner can immediately be applied to new tasks.

Fluid viscosity prediction leveraging computer vision and robot interaction

Accurately determining fluid viscosity is crucial for various industrial and scientific applications. Traditional methods of viscosity measurement, though reliable, often require manual intervention and cannot easily adapt to real-time monitoring. With advancements in machine learning and computer vision, this work explores the feasibility of predicting fluid viscosity by analyzing fluid oscillations captured in video data. The pipeline employs a 3D convolutional autoencoder pretrained in a self-supervised manner to extract and learn features from semantic segmentation masks of oscillating fluids. Then, the latent representations of the input data, produced from the pretrained autoencoder, are processed with a distinct inference head to infer either the fluid category (classification) or the fluid viscosity (regression) in a time-resolved manner. When the latent representations generated by the pre-trained autoencoder are used for classification, the system achieves a 97.1% accuracy across a total of 4140 test datapoints. Similarly, for regression tasks, employing an additional fully-connected network as a regression head allows the pipeline to achieve a mean absolute error of 0.258 cP over 4416 test datapoints. This study represents an innovative contribution to both fluid characterization and the evolving landscape of Artificial Intelligence, demonstrating the potential of deep learning in achieving near real-time viscosity estimation and addressing practical challenges in fluid dynamics through the analysis of video data capturing oscillating fluid dynamics.