Hybrid machine learning based scale bridging framework for permeability prediction of fibrous structures

New approaches in molecular structure prediction

Structured prediction is the problem of learning a function that maps structured inputs to structured outputs. Prototypical examples of structured prediction include part-of-speech tagging and semantic segmentation of images. Inspired by the recent successes of search-based structured prediction, we introduce a new framework for structured prediction called HC-Search. Given a structured input, the framework uses a search procedure guided by a learned heuristic H to uncover high quality candidate outputs and then employs a separate learned cost function C to select a final prediction among those outputs. The overall loss of this prediction architecture decomposes into the loss due to H not leading to high quality outputs, and the loss due to C not selecting the best among the generated outputs. Guided by this decomposition, we minimize the overall loss in a greedy stage-wise manner by first training H to quickly uncover high quality outputs via imitation learning, and then training C to correctly rank the outputs generated via H according to their true losses. Importantly, this training procedure is sensitive to the particular loss function of interest and the time-bound allowed for predictions. Experiments on several benchmark domains show that our approach significantly outperforms several state-of-the-art methods.

This thesis focuses on Hierarchical Machine Translation (HMT). HMT allows to model long distance phenomena and reordering at a cost of higher complexity of the decoder. With this thesis we reduce the complexity of HMT. In the first part we propose a series of alternative Cube Pruning (CP) algorithms that leverage on more aggressive pruning and less memory usage. Then we propose a linear time CP that solves exactly a relaxation of the decoding problem. All these algorithms can substitute the standard CP algorithm in any of its applications. In the second part of the thesis we present a novel Structured Prediction approach to HMT. The proposed model builds the structures incrementally, by choosing a single action at each step, and pruning all incompatible alternatives to that action. This approach allows translations to be constructed in an undirectional manner, thus not being constrained by the bottomup ordering of CKYlike algorithms.

BackgroundRecently developed methods of protein contact prediction, a crucially important step for protein structure prediction, depend heavily on deep neural networks (DNNs) and multiple sequence alignments (MSAs) of target proteins. Protein sequences are accumulating to an increasing degree such that abundant sequences to construct an MSA of a target protein are readily obtainable. Nevertheless, many cases present different ends of the number of sequences that can be included in an MSA used for contact prediction. The abundant sequences might degrade prediction results, but opportunities remain for a limited number of sequences to construct an MSA. To resolve these persistent issues, we strove to develop a novel framework using DNNs in an end-to-end manner for contact prediction.ResultsWe developed neural network models to improve precision of both deep and shallow MSAs. Results show that higher prediction accuracy was achieved by assigning weights to sequences in a deep MSA. Moreover, for shallow MSAs, adding a few sequential features was useful to increase the prediction accuracy of long-range contacts in our model. Based on these models, we expanded our model to a multi-task model to achieve higher accuracy by incorporating predictions of secondary structures and solvent-accessible surface areas. Moreover, we demonstrated that ensemble averaging of our models can raise accuracy. Using past CASP target protein domains, we tested our models and demonstrated that our final model is superior to or equivalent to existing meta-predictors.ConclusionsThe end-to-end learning framework we built can use information derived from either deep or shallow MSAs for contact prediction. Recently, an increasing number of protein sequences have become accessible, including metagenomic sequences, which might degrade contact prediction results. Under such circumstances, our model can provide a means to reduce noise automatically. According to results of tertiary structure prediction based on contacts and secondary structures predicted by our model, more accurate three-dimensional models of a target protein are obtainable than those from existing ECA methods, starting from its MSA. DeepECA is available from https://github.com/tomiilab/DeepECA.

A novel machine learning framework for impact force prediction of foam-filled multi-layer lattice composite structures

Recent advances in machine learning and self-supervised deep language modeling have made it possible to accurately predict protein structural properties. Most existing models and pretraining methods leverage evolutionary information in multiple sequence alignments (MSAs) to obtain very promising results in protein structural property prediction. However, methods which make use of MSAs are computationally intensive and time consuming, and these methods cannot be applied to proteins which lack sequence homologs. Here, we present LMProtein, a fast and accurate framework for predicting protein structural properties such as protein secondary structure, backbone dihedral angles, fluorescence landscape and stability landscape only using protein primary sequence. By combining the unsupervised pretrained language model ESM-2 with a convolutional neural network, long short-term memory neural networks and multilayer perceptron, LMProtein achieves better performances than recent MSA-based models and single-sequence-based models. The accuracy of the eight-state secondary structures (SS8) prediction is approximately 74%, the mean absolute error of dihedral angle prediction is 19° and 29° for Phi and Psi, respectively, and Spearman's ρ between the experimental and predicted values of fluorescence and stability is 0.69 and 0.79, respectively. We believe that our framework has broad potential for predicting protein structural characteristics, providing important opportunities to accelerate the progress of protein engineering and drug target identification.

ASTRO-FOLD: A Combinatorial and Global Optimization Framework for Ab Initio Prediction of Three-Dimensional Structures of Proteins from the Amino Acid Sequence

Structured prediction makes coherent decisions as structured objects to present the interrelations of these predicted variables. They have been widely used in many areas, such as bioinformatics, computer vision, speech recognition, and natural language processing. Machine Learning with reduced supervision aims to leverage the laborious and error-prone annotation effects and benefit the low-resource languages. In this dissertation we study structured prediction with reduced supervision for two sets of problems, sequence labeling and dependency parsing, both of which are representatives of structured prediction problems in NLP. We investigate three different approaches. The first approach is learning with modular architecture by task decomposition. By decomposing the labels into location sub-label and type sub-label, we designed neural modules to tackle these sub-labels respectively, with an additional module to infuse the information. The experiments on the benchmark datasets show the modular architecture outperforms existing models and can make use of partially labeled data together with fully labeled data to improve on the performance of using fully labeled data alone.The second approach builds the neural CRF autoencoder (NCRFAE) model that combines a discriminative component and a generative component for semi-supervised sequence labeling. The model has a unified structure of shared parameters, using different loss functions for labeled and unlabeled data. We developed a variant of the EM algorithm for optimizing the model with tractable inference. The experiments on several languages in the POS tagging task show the model outperforms existing systems in both supervised and semi-supervised setup.The third approach builds two models for semi-supervised dependency parsing, namely local autoencoding parser (LAP) and global autoencoding parser (GAP). LAP assumes the chain-structured sentence has a latent representation and uses this representation to construct the dependency tree, while GAP treats the dependency tree itself as a latent variable. Both models have unified structures for sentence with and without annotated parse tree. The experiments on several languages show both parsers can use unlabeled sentences to improve on the performance with labeled sentences alone, and LAP is faster while GAP outperforms existing models.

Using machine learning, especially deep learning, to facilitate biological research is a fascinating research direction. However, in addition to the standard classi cation or regression problems, whose outputs are simple vectors or scalars, in bioinformatics, we often need to predict more complex structured targets, such as 2D images and 3D molecular structures. The above complex prediction tasks are referred to as structured prediction. Structured prediction is more complicated than the traditional classi cation but has much broader applications, especially in bioinformatics, considering the fact that most of the original bioinformatics problems have complex output objects. Due to the properties of those structured prediction problems, such as having problem-speci c constraints and dependency within the labeling space, the straightforward application of existing deep learning models on the problems can lead to unsatisfactory results. In this dissertation, we argue that the following two ideas can help resolve a wide range of structured prediction problems in bioinformatics. Firstly, we can combine deep learning with other classic algorithms, such as probabilistic graphical models, which model the problem structure explicitly. Secondly, we can design and train problem-speci c deep learning architectures or methods by considering the structured labeling space and problem constraints, either explicitly or implicitly. We demonstrate our ideas with six projects from four bioinformatics sub elds, including sequencing analysis, structure prediction, function annotation, and network analysis. The structured outputs cover 1D electrical signals, 2D images, 3D structures, hierarchical labeling, and heterogeneous networks. With the help of the above ideas, all of our methods can achieve state-of-the-art performance on the corresponding problems. The success of these projects motivates us to extend our work towards other more challenging but important problems, such as health-care problems, which can directly bene t people's health and wellness. We thus conclude this thesis by discussing such future works, and the potential challenges and opportunities.

An explainable non-invasive hybrid machine learning framework for accurate prediction of thyroid-stimulating hormone levels.

An integrated framework of surface accuracy prediction for clearance-affected extendible support structures with dimensional deviations and elastic deformations

Digital twin-driven fatigue life prediction framework of mechanical structures using a power density theory: Application to off-road vehicle front axle housing

Integrated data-driven modeling and experimental optimization of granular hydrogel matrices

A hybrid machine learning and natural language processing model for early detection of acute coronary syndrome

Wearable biosensors, coupled with predictive analytics, are transforming real-time health monitoring by providing continuous, personalized insights into physiological metrics. This framework leverages wearable technology and advanced machine learning algorithms to process biometric data, enabling early detection of health anomalies and proactive intervention. Through the integration of biosensors monitoring parameters such as heart rate, blood pressure, glucose levels, and respiratory rate, the predictive framework can identify patterns indicative of potential health risks. Machine learning algorithms analyze real-time data streams, facilitating precise predictions and personalized feedback to users and healthcare providers. This approach not only enhances patient engagement and preventive care but also supports the management of chronic conditions by allowing continuous tracking outside clinical settings. Despite its potential, challenges such as data privacy, battery life limitations, and the accuracy of sensor data must be addressed. This research explores the design, functionality, and implications of a predictive health monitoring framework, examining the role of wearable biosensors in enabling proactive healthcare solutions.