Sequence Research Articles

Cloud-based machine learning algorithms for anomalies detection

Gradient boosting machines harnesses the inherent capabilities of decision trees and meticulously corrects their errors in a sequential fashion, culminating in remarkably precise predictions. Word2Vec, a prominent word embedding technique, occupies a pivotal role in natural language processing (NLP) tasks. Its proficiency lies in capturing intricate semantic relationships among words, thereby facilitating applications such as sentiment analysis, document classification, and machine translation to discern subtle nuances present in textual data. Bayesian networks introduce probabilistic modeling capabilities, predominantly in contexts marked by uncertainty. Their versatile applications encompass risk assessment, fault diagnosis, and recommendation systems. Gated recurrent units (GRU), a variant of recurrent neural networks, emerges as a formidable asset in modeling sequential data. Both training and testing are crucial to the success of an intrusion detection system (IDS). During the training phase, several models are created, each of which can recognize typical from anomalous patterns within a given dataset. To acquire passwords and credit card details, "phishing" usually entails impersonating a trusted company. Predictions of student performance on academic tasks are improved by hyper parameter optimization of the gradient boosting regression tree using the grid search approach.

Experiences with mixed-mode surveys in times of COVID-19 at Statistics Netherlands

Statistics Netherlands’ social surveys are based on a sequential mixed-mode data collection approach using web, telephone, and face-to-face interviewing. This article illustrates how Statistics Netherlands addressed the sudden, unforeseen loss of face-to-face interviews in social surveys amidst the COVID-19 pandemic. At the beginning of the pandemic, survey processes were immediately adjusted in several ways to mitigate the negative effects of respondent attrition. Where possible, sampled people initially assigned to face-to-face interviewing were motivated to respond through web or telephone to minimize the loss of response. At the same time regression analysis and simulation were conducted to obtain quantitative insight into the effects of losing face-to-face responses in the sequential mixed-mode designs. Furthermore, alternative model-based estimation procedures based on structural time series models were implemented to compensate for the bias that is a result of the loss of face-to-face responses. These initiatives are illustrated with applications to the Dutch Labor Force Survey, the Housing Survey, and the Health Survey.

Variance-Reduced Stochastic Optimization for Efficient Inference of Hidden Markov Models

Hidden Markov models (HMMs) are popular models to identify a finite number of latent states from sequential data. However, fitting them to large datasets can be computationally demanding because most likelihood maximization techniques require iterating through the entire underlying dataset for every parameter update. We propose a novel optimization algorithm that updates the parameters of an HMM without iterating through the entire dataset. Namely, we combine a partial E step with variance-reduced stochastic optimization within the M step. We prove the algorithm converges under certain regularity conditions. We test our algorithm empirically using a simulation study as well as a case study of kinematic data collected using suction-cup attached biologgers from eight northern resident killer whales (Orcinus orca) off the western coast of Canada. In both, our algorithm converges in fewer epochs, with less computation time, and to regions of higher likelihood compared to standard numerical optimization techniques. Our algorithm allows practitioners to fit complicated HMMs to large time-series datasets more efficiently than existing baselines. Supplemental materials are available online.

All-optical matching systems based on the squarer for QPSK and 8PSK optical signals

Photonic firewall which can conduct the data identification and intrusion detection directly at the optical layer is a high-speed and low-energy network security equipment. For the photonic firewall, the all-optical matching system is the core part. This paper focuses on designing all-optical matching systems for high-order phase modulated optical signals. Based on the squarer, two all-optical matching systems are designed for Quadrature Phase Shift Keying (QPSK) and 8 Phase Shift Keying (8PSK) optical signals respectively. The systems mainly consist of squarers, multiplexers, AND gates and regenerators. The squarer is implemented based on the four-wave mixing (FWM) effect of highly nonlinear fiber (HNLF). The theoretical analysis and simulation verification are conducted for the proposed system. The performances of feasibility and noise impact are evaluated for the proposed system with the data rate of 100 Gbaud. The results show that the system can correctly identify and localize the target sequence with the length of 4 and 8 symbols in the data sequence with the length of 16 symbols. The impact of noise is evaluated by the extinction ratio (ER) and contrast ratio (CR). The ER of two all-optical matching systems reaches 0 dB when the optical signal-to-noise ratio (OSNR) of input optical signal is about 5.59 dB and 9.24 dB respectively. The CR of two all-optical matching systems reaches 0 dB when the OSNR of input optical signal is about 4 dB and 8.62 dB respectively.

Linear matrix genetic programming as a tool for data-driven black-box control-oriented modeling in conditions of limited access to training data

In the paper, a new evolutionary technique called Linear Matrix Genetic Programming (LMGP) is proposed. It is a matrix extension of Linear Genetic Programming and its application is data-driven black-box control-oriented modeling in conditions of limited access to training data. In LMGP, the model is in the form of an evolutionarily-shaped program which is a sequence of matrix operations. Since the program has a hidden state, running it for a sequence of input data has a similar effect to using well-known recurrent neural networks such as Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU). To verify the effectiveness of the LMGP, it was compared with different types of neural networks. The task of all the compared techniques was to reproduce the behavior of a nonlinear model of an underwater vehicle. The results of the comparative tests are reported in the paper and they show that the LMGP can quickly find an effective and very simple solution to the given problem. Moreover, a detailed comparison of models, generated by LMGP and LSTM/GRU, revealed that the former are up to four times more accurate than the latter in reproducing vehicle behavior.

Identifying stationary microbial interaction networks based on irregularly spaced longitudinal 16S rRNA gene sequencing data

IntroductionThe microbial interactions within the human microbiome are complex, and few methods are available to identify these interactions within a longitudinal microbial abundance framework. Existing methods typically impose restrictive constraints, such as requiring long sequences and equal spacing, on the data format which in many cases are violated.MethodsTo identify microbial interaction networks (MINs) with general longitudinal data settings, we propose a stationary Gaussian graphical model (SGGM) based on 16S rRNA gene sequencing data. In the SGGM, data can be arbitrarily spaced, and there are no restrictions on the length of data sequences from a single subject. Based on the SGGM, EM -type algorithms are devised to compute the L1-penalized maximum likelihood estimate of MINs. The algorithms employ the classical graphical LASSO algorithm as the building block and can be implemented efficiently. ResultsExtensive simulation studies show that the proposed algorithms can significantly outperform the conventional algorithms if the correlations among the longitudinal data are reasonably high. When the assumptions in the SGGM areviolated, e.g., zero inflation or data from heterogeneous microbial communities, the proposed algorithms still demonstrate robustness and perform better than the other existing algorithms. The algorithms are applied to a 16S rRNA gene sequencing data set from patients with cystic fibrosis. The results demonstrate strong evidence of an association between the MINs and the phylogenetic tree, indicating that the genetically related taxa tend to have more/stronger interactions. These results strengthen the existing findings in literature. DiscussionThe proposed algorithms can potentially be used to explore the network structure in genome, metabolome etc. as well.

Canopy light transmittance-based crop growth monitoring and leaf area index estimation in a garlic field

ABSTRACT A smart agricultural system is necessary for monitoring crop growth and stress conditions using near-ground-based remote sensing techniques. Crop growth can be estimated using several standard crop growth parameters. However, obtaining timed sequential data for observing leaf area index (LAI) is challenging, and normalized difference vegetation index (NDVI) estimation in crop fields requires the installation of sensors on frames structure that are taller than the crop. Canopy light transmittance (CLT) indicates the degree of decrease in the amount of light passing through some material. It was conventionally used to understand canopy radiative transfer. This study examined the viability of CLT as a novel crop parameter for monitoring crop growth conditions. The CLT, LAI, and NDVI of a garlic field were recorded for five years. The correlation between daily CLT and LAI was higher than that between NDVI and LAI. Thus, CLT has the potential to sequentially estimate crop LAI values, particularly for capturing the temporal patterns of LAI. In addition, like NDVI, CLT showed sensitivity in representing the canopy structure and the amount of biomass because CLT is conceptually related to the sky gap fraction. Thus, CLT has the potential to serve as a novel growth parameter for continuous crop growth monitoring.

Weld Defect Downscaling Recognition for Phased Array Ultrasonic Data Based on Semantic Segmentation

Phased array ultrasonic testing (PAUT) is one of the most advanced ultrasonic inspection methods, providing inspection data in image sequences with increased resolution and coverage, consequently enhancing inspection efficiency. Due to the inherently abstract nature of ultrasound images, the analysis of PAUT weld data continues to rely on the expertise of experienced inspectors, particularly in the context of shipbuilding welds, which exhibit characteristics such as arbitrary lengths, varying thicknesses, and various weld methods, particularly when associated with hybrid laser-arc welding involving higher caps and narrower fill widths. This paper applies a semantic segmentation architecture to PAUT data for hybrid laser-arc welding to segment defects automatically. A downscaling PAUT image dataset incorporating PAUT imaging principles with scan processes is established. The proposed architecture adopts an encoder-decoder architecture and segments defects downscaling, enabling direct localization and sizing defects in long PAUT data. Meanwhile, it handles PAUT image sequences of arbitrary length, eliminating the need for horizontal scale normalization while preserving the integrity of defect shapes. The proposed architecture was validated in the dataset to demonstrate the superiority of the proposed method in locating and sizing defects in PAUT weld data, offering a promising automated solution to a challenge in ultrasonic inspection methodologies.

Iterative Ensemble Smoothers for Data Assimilation in Coupled Nonlinear Multiscale Models

Abstract This paper identifies and explains particular differences and properties of adjoint-free iterative ensemble methods initially developed for parameter estimation in petroleum models. The aim is to demonstrate the methods’ potential for sequential data assimilation in coupled and multiscale unstable dynamical systems. For this study, we have introduced a new nonlinear and coupled multiscale model based on two Kuramoto–Sivashinsky equations operating on different scales where a coupling term relaxes the two model variables toward each other. This model provides a convenient testbed for studying data assimilation in highly nonlinear and coupled multiscale systems. We show that the model coupling leads to cross covariance between the two models’ variables, allowing for a combined update of both models. The measurements of one model’s variable will also influence the other and contribute to a more consistent estimate. Second, the new model allows us to examine the properties of iterative ensemble smoothers and assimilation updates over finite-length assimilation windows. We discuss the impact of varying the assimilation windows’ length relative to the model’s predictability time scale. Furthermore, we show that iterative ensemble smoothers significantly improve the solution’s accuracy compared to the standard ensemble Kalman filter update. Results and discussion provide an enhanced understanding of the ensemble methods’ potential implementation and use in operational weather- and climate-prediction systems.

AI Assisted Deceptive Content Analysis System Using Bi-LSTM

Abstract: In an age of widespread misinformation, creating accurate machine learning algorithms is vital for ensuring the integrity of information sharing. This project aims to develop a machine learning model capable of distinguishing between fake and true news articles. [1]The model utilizes a Bidirectional Long Short-Term Memory (Bi-LSTM) neural network architecture. Exploratory data analysis techniques are employed to gain in-sights into the characteristics and distributions within the dataset. Data visualization techniques aid in understanding patterns and relationships within the dataset. Additionally, unigram analysis is conducted to extract meaningful features from the text data. The datasets are then prepared for model training, involving preprocessing steps such as tokenization and vectorization. Fi-nally, the Bi-LSTM model is constructed, leveraging its ability to capture long-range dependencies in sequential data. The model is trained on the prepared datasets, optimized using appropri-ate techniques, and evaluated using metrics such as accuracy, precision, recall, and F1-score. The Bi-LSTM architecture offers the advantage of capturing long-range dependencies in sequential data, thereby enhancing the model’s ability to discern nuanced patterns in news articles. The primary objective of this project is to develop a robust and accurate system for automatically detecting fake news, thus playing a pivotal role in enhancing the dissemination of reliable information in the digital age.

Sign Language Recognition for Real-time Communication

Abstract: Sign language is an essential communication tool for India's Deaf and Hard of Hearing people. This study introduces a novel approach for recognising and synthesising Indian Sign Language (ISL) using Long Short-Term Memory (LSTM) networks. LSTM, a kind of recurrent neural network (RNN), has demonstrated promising performance in sequential data processing. In this study, we leverage LSTM to develop a robust ISL recognition system, which can accurately interpret sign gestures in real-time. Additionally, we employ LSTM-based models for ISL synthesis, enabling the conversion of spoken language into sign language for improved inclusivity and accessibility. We evaluate the proposed approach on a diverse dataset of ISL signs, achieving high recognition accuracy and natural sign synthesis. The integration of LSTM in ISL technology holds significant potential for breaking down communication barriers and improving the quality of life for India's deaf and hard of hearing people

LipNet: End-to-End Lipreading

Lipreading is the task of decoding text from the movement of a speaker’s mouth. This research presents the development of an advanced end-to-end lipreading system. Leveraging deep learning architectures and multimodal fusion techniques, the proposed system interprets spoken language solely from visual cues, such as lip movements. Through meticulous data collection, annotation, preprocessing, model development, and evaluation, diverse datasets encompassing various speakers, accents, languages, and environmental conditions are curated to ensure robustness and generalization. Conventional methods divided the task into two phases: prediction and designing or learning visual characteristics. Most deep lipreading methods are trainable from end to end. In the past, lipreading has been tackled using tedious and sometimes unsatisfactory techniques that break down speech into smaller units like phonemes or visemes. But these methods often fail when faced with real-world problems, such contextual factors, accents, and differences in speech patterns. Nevertheless, current research on end-to-end trained models only carries out word classification; sentence-level sequence prediction is not included. LipNet is an end-to-end trained model that uses spatiotemporal convolutions, a recurrent network, and the connectionist temporal classification loss to translate a variable-length sequence of video frames to text. LipNet breaks from this traditional paradigm by using an all-encompassing, end-to-end approach supported by deep learning algorithms, Convolutional neural networks (CNNs) and recurrent neural networks (RNNs), which are skilled at processing sequential data and extracting high-level representations, are fundamental to LipNet's architecture.LipNet achieves 95.2% accuracy in sentence-level on the GRID corpus, overlapped speaker split task, outperforming experienced human lipreaders and the previous 86.4% word-level state-of-the-art accuracy.The results underscore the transformative potential of the lipreading system in real-world applications, particularly in domains such as assistive technology and human-computer interaction, where it can significantly improve communication accessibility and inclusivity for individuals with hearing impairments.

Deep learning algorithm-enabled sediment characterization techniques to determination of water saturation for tight gas carbonate reservoirs in Bohai Bay Basin, China

Understanding water saturation levels in tight gas carbonate reservoirs is vital for optimizing hydrocarbon production and mitigating challenges such as reduced permeability due to water saturation (Sw) and pore throat blockages, given its critical role in managing capillary pressure in water drive mechanisms reservoirs. Traditional sediment characterization methods such as core analysis, are often costly, invasive, and lack comprehensive spatial information. In recent years, several classical machine learning models have been developed to address these shortcomings. Traditional machine learning methods utilized in reservoir characterization encounter various challenges, including the ability to capture intricate relationships, potential overfitting, and handling extensive, multi-dimensional datasets. Moreover, these methods often face difficulties in dealing with temporal dependencies and subtle patterns within geological formations, particularly evident in heterogeneous carbonate reservoirs. Consequently, despite technological advancements, enhancing the reliability, interpretability, and applicability of predictive models remains imperative for effectively characterizing tight gas carbonate reservoirs. This study employs a novel data-driven strategy to prediction of water saturation in tight gas reservoir powered by three recurrent neural network type deep/shallow learning algorithms—Gated Recurrent Unit (GRU), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), Support Vector Machine (SVM), K-nearest neighbor (KNN) and Decision tree (DT)—customized to accurately forecast sequential sedimentary structure data. These models, optimized using Adam's optimizer algorithm, demonstrated impressive performance in predicting water saturation levels using conventional petrophysical data. Particularly, the GRU model stood out, achieving remarkable accuracy (an R-squared value of 0.9973) with minimal errors (RMSE of 0.0198) compared to LSTM, RNN, SVM, KNN and, DT algorithms, thus showcasing its proficiency in processing extensive datasets and effectively identifying patterns. By achieving unprecedented accuracy levels, this study not only enhances the understanding of sediment properties and fluid saturation dynamics but also offers practical implications for reservoir management and hydrocarbon exploration in complex geological settings. These insights pave the way for more reliable and efficient decision-making processes, thereby advancing the forefront of reservoir engineering and petroleum geoscience.

Evolutionary Deep Learning for Sequential Data Processing in Music Education

Evolutionary Deep Learning for Sequential Data Processing in Music Education

Structure-aware machine learning strategies for antimicrobial peptide discovery

Machine learning models are revolutionizing our approaches to discovering and designing bioactive peptides. These models often need protein structure awareness, as they heavily rely on sequential data. The models excel at identifying sequences of a particular biological nature or activity, but they frequently fail to comprehend their intricate mechanism(s) of action. To solve two problems at once, we studied the mechanisms of action and structural landscape of antimicrobial peptides as (i) membrane-disrupting peptides, (ii) membrane-penetrating peptides, and (iii) protein-binding peptides. By analyzing critical features such as dipeptides and physicochemical descriptors, we developed models with high accuracy (86–88%) in predicting these categories. However, our initial models (1.0 and 2.0) exhibited a bias towards α-helical and coiled structures, influencing predictions. To address this structural bias, we implemented subset selection and data reduction strategies. The former gave three structure-specific models for peptides likely to fold into α-helices (models 1.1 and 2.1), coils (1.3 and 2.3), or mixed structures (1.4 and 2.4). The latter depleted over-represented structures, leading to structure-agnostic predictors 1.5 and 2.5. Additionally, our research highlights the sensitivity of important features to different structure classes across models.

Molecular characterization, clinical value, and cancer–immune interactions of genes related to disulfidptosis and ferroptosis in colorectal cancer

BackgroundThis research strived to construct a new signature utilizing disulfidptosis-related ferroptosis (SRF) genes to anticipate response to immunotherapy, prognosis, and drug sensitivity in individuals with colorectal cancer (CRC).MethodsThe data for RNA sequencing as well as corresponding clinical information of individuals with CRC, were extracted from The Cancer Genome Atlas (TCGA) dataset. SRF were constructed with the help of the random forest (RF), least absolute shrinkage and selection operator (LASSO), and stepwise regression algorithms. To validate the SRF model, we applied it to an external cohort, GSE38832. Prognosis, immunotherapy response, drug sensitivity, molecular functions of genes, and somatic mutations of genes were compared across the high- and low-risk groups (categories). Following this, all statistical analyses were conducted with the aid of the R (version 4.23) software and various packages of the Cytoscape (version 3.8.0) tool.ResultsSRF was developed based on five genes (ATG7, USP7, MMD, PLIN4, and THDC2). Both univariate and multivariate Cox regression analyses established SRF as an independent, prognosis-related risk factor. Individuals from the high-risk category had a more unfavorable prognosis, elevated tumor mutational burden (TMB), and significant immunosuppressive status. Hence, they might have better outcomes post-immunotherapy and might benefit from the administration of pazopanib, lapatinib, and sunitinib.ConclusionIn conclusion, SRF can act as a new biomarker for prognosis assessment. Moreover, it is also a good predictor of drug sensitivity and immunotherapy response in CRC but should undergo optimization before implementation in clinical settings.

Abnormal Traffic Detection System Based on Feature Fusion and Sparse Transformer

This paper presents a feature fusion and sparse transformer-based anomalous traffic detection system (FSTDS). FSTDS utilizes a feature fusion network to encode the traffic data sequences and extracting features, fusing them into coding vectors through shallow and deep convolutional networks, followed by deep coding using a sparse transformer to capture the complex relationships between network flows; finally, a multilayer perceptron is used to classify the traffic and achieve anomaly traffic detection. The feature fusion network of FSTDS improves feature extraction from small sample data, the deep encoder enhances the understanding of complex traffic patterns, and the sparse transformer reduces the computational and storage overhead and improves the scalability of the model. Experiments demonstrate that the number of FSTDS parameters is reduced by up to nearly half compared to the baseline, and the success rate of anomalous flow detection is close to 100%.

Partial ordered Wasserstein distance for sequential data

Measuring the distance between data sequences is a challenging problem, especially in the presence of outliers and local distortions. Existing measures typically align the two sequences before calculating their distance based on the difference between the corresponding elements. However, those alignments are not flexible enough to accommodate local distortions and severe effects of outliers. In this article, we propose a novel distance, termed as Partial Ordered Wasserstein (POW), which is flexible to align two sequences and robust w.r.t outliers. We further analyze some properties of the proposed distance, and show that POW enables a simple way to automatically and adaptively select the amount of transported mass, so as to accommodate outliers. Two different applications of POW are then studied: time-series classification and multi-step localization. Finally, we conduct extensive experiments on widely available public datasets to evaluate the performance of the proposed distances. Experimental results, obtained via a thorough experimental protocol, show the performance superiority of POW over several existing distance measures. Our Python source code is available on https://github.com/TungDP/Partial-Ordered-Wasserstein-Distance

Generative design of compounds with desired potency from target protein sequences using a multimodal biochemical language model

Deep learning models adapted from natural language processing offer new opportunities for the prediction of active compounds via machine translation of sequential molecular data representations. For example, chemical language models are often derived for compound string transformation. Moreover, given the principal versatility of language models for translating different types of textual representations, off-the-beaten-path design tasks might be explored. In this work, we have investigated generative design of active compounds with desired potency from target sequence embeddings, representing a rather provoking prediction task. Therefore, a dual-component conditional language model was designed for learning from multimodal data. It comprised a protein language model component for generating target sequence embeddings and a conditional transformer for predicting new active compounds with desired potency. To this end, the designated “biochemical” language model was trained to learn mappings of combined protein sequence and compound potency value embeddings to corresponding compounds, fine-tuned on individual activity classes not encountered during model derivation, and evaluated on compound test sets that were structurally distinct from training sets. The biochemical language model correctly reproduced known compounds with different potency for all activity classes, providing proof-of-concept for the approach. Furthermore, the conditional model consistently reproduced larger numbers of known compounds as well as more potent compounds than an unconditional model, revealing a substantial effect of potency conditioning. The biochemical language model also generated structurally diverse candidate compounds departing from both fine-tuning and test compounds. Overall, generative compound design based on potency value-conditioned target sequence embeddings yielded promising results, rendering the approach attractive for further exploration and practical applications.Scientific contributionThe approach introduced herein combines protein language model and chemical language model components, representing an advanced architecture, and is the first methodology for predicting compounds with desired potency from conditioned protein sequence data.

Task offloading with enhanced Deep Q-Networks for efficient industrial intelligent video analysis in edge–cloud collaboration

With the rapid development of 5G and IoT technologies, industrial video surveillance systems face increasingly significant challenges in processing large-scale data. Traditional cloud-centric video analysis methods often lead to processing delays and inefficiencies in resource allocation due to centralized data processing. In this context, the emergence of edge computing, especially in edge–cloud collaborative intelligent video analysis, offers an innovative solution. Within this architecture, task offloading strategies focus on intelligent computation task distribution to minimize latency and optimize resource utilization. However, existing methods frequently overlook key application-specific metrics in video analysis, such as accuracy and timeliness. This study introduces a novel Edge–Cloud Collaborative Industrial Video Intelligent Analysis Architecture (ECIVA) based on the GRU-Enhanced Deep Q-Network (GEDQN). This architecture integrates the strengths of Gated Recurrent Units (GRU) for processing sequential data with the efficiency of Deep Q-Networks in optimizing task-offloading decisions. By incorporating a token bucket mechanism, the architecture optimally manages the task offloading rate within the constraints of limited network bandwidth, thus ensuring high accuracy and real-time responsiveness in video analysis. Empirical evaluation with offshore drilling platform video data shows that the architecture achieves 87.5% mean Average Precision (mAP) at a 0.5 IoU threshold and increases system throughput by 39.2% over traditional cloud-centric methods.