Probabilistic Features Research Articles

The epistemic and the deontic preface paradox

Abstract This paper generalizes the preface paradox beyond the conjunctive aggregation of beliefs and constructs an analogous paradox for deontic reasoning. The analysis of the deontic case suggests a systematic restriction of intuitive rules for reasoning with obligations. This proposal can be transferred to the epistemic case: It avoids the preface and the lottery paradox and saves one of the two directions of the Lockean Thesis (i.e. high credence is sufficient, but not necessary for rational belief). The resulting account compares favorably to competing proposals; in particular, we can formulate the rules of correct doxastic reasoning without reference to probabilistic features of the involved propositions.

Possum: identification and interpretation of potassium ion inhibitors using probabilistic feature vectors

Possum: identification and interpretation of potassium ion inhibitors using probabilistic feature vectors

Multi-path long-term vessel trajectories forecasting with probabilistic feature fusion for problem shifting

This paper presents a deep auto-encoder model and a phased framework approach to predict the next 12 h of vessel trajectories using 1 to 3 h of Automatic Identification System data as input. The strategy involves fusing spatiotemporal features from AIS messages with probabilistic features engineered from historical AIS data to reduce forecasting uncertainty. The probabilistic features have an F1-Score of approximately 85% and 75% for the vessel route and destination prediction, respectively. Under such circumstances, we achieved an R2 Score of over 98% with different layer structures and varying feature combinations; the high R2 Score is a natural outcome of the well-defined shipping lanes in the study region. However, our proposal stands out among competing approaches as it demonstrates the capability of complex decision-making during turnings and route selection. Furthermore, we have shown that our model achieves more accurate forecasting with average and median errors of 11km and 6km, respectively, a 25% improvement from the current state-of-the-art approaches. The resulting model from this proposal is deployed as part of a broader Decision Support System to safeguard whales by preventing the risk of vessel-whale collisions under the smartWhales initiative and acting on the Gulf of St. Lawrence in Atlantic Canada.

Sparse Bayesian learning using TMB (Template Model Builder)

Sparse Bayesian Learning, and more specifically the Relevance Vector Machine (RVM), can be used in supervised learning for both classification and regression problems. Such methods are particularly useful when applied to big data in order to find a sparse (in weight space) representation of the model. This paper demonstrates that the Template Model Builder (TMB) is an accurate and flexible computational framework for implementation of sparse Bayesian learning methods.The user of TMB is only required to specify the joint likelihood of the weights and the data, while the Laplace approximation of the marginal likelihood is automatically evaluated to numerical precision. This approximation is in turn used to estimate hyperparameters by maximum marginal likelihood. In order to reduce the computational cost of the Laplace approximation we introduce the notion of an “active set” of weights, and we devise an algorithm for dynamically updating this set until convergence, similar to what is done in other RVM type methods. We implement two different methods using TMB; the RVM and the Probabilistic Feature Selection and Classification Vector Machine method, where the latter also performs feature selection. Experiments based on benchmark data show that our TMB implementation performs comparable to that of the original implementation, but at a lower implementation cost. TMB can also calculate model and prediction uncertainty, by including estimation uncertainty from both latent variables and the hyperparameters. In conclusion, we find that TMB is a flexible tool that facilitates implementation and prototyping of sparse Bayesian methods.

Stack-HDAC3i: A high-precision identification of HDAC3 inhibitors by exploiting a stacked ensemble-learning framework

Epigenetics involves reversible modifications in gene expression without altering the genetic code itself. Among these modifications, histone deacetylases (HDACs) play a key role by removing acetyl groups from lysine residues on histones. Overexpression of HDACs is linked to the proliferation and survival of tumor cells. To combat this, HDAC inhibitors (HDACi) are commonly used in cancer treatments. However, pan-HDAC inhibition can lead to numerous side effects. Therefore, isoform-selective HDAC inhibitors, such as HDAC3i, could be advantageous for treating various medical conditions while minimizing off-target effects. To date, computational approaches that use only the SMILES notation without any experimental evidence have become increasingly popular and necessary for the initial discovery of novel potential therapeutic drugs. In this study, we develop an innovative and high-precision stacked-ensemble framework, called Stack-HDAC3i, which can directly identify HDAC3i using only the SMILES notation. Using an up-to-date benchmark dataset, we first employed both molecular descriptors and Mol2Vec embeddings to generate feature representations that cover multi-view information embedded in HDAC3i, such as structural and contextual information. Subsequently, these feature representations were used to train baseline models using nine popular ML algorithms. Finally, the probabilistic features derived from the selected baseline models were fused to construct the final stacked model. Both cross-validation and independent tests showed that Stack-HDAC3i is a high-accuracy prediction model with great generalization ability for identifying HDAC3i. Furthermore, in the independent test, Stack-HDAC3i achieved an accuracy of 0.926 and Matthew’s correlation coefficient of 0.850, which are 0.44–6.11% and 0.83–11.90% higher than its constituent baseline models, respectively.

Physics-informed probabilistic slow feature analysis

This paper presents a novel approach called physics-informed probabilistic slow feature analysis. The probabilistic slow feature analysis method has been employed to extract slowly varying latent patterns from high-dimensional measured data. The extracted slow features have proven effective in industrial applications such as soft sensing and process monitoring. However, industrial processes come with various physical constraints that must be taken into account, such as energy requirements, equipment limitations, and safety considerations. The conventional black-box nature of the slow feature model often leads to physically inconsistent or unacceptable results. To address this issue, we propose integrating physics principles into the probabilistic slow feature model, ensuring that the extracted features adhere to physics laws. Our formulation incorporates two types of physical constraints: linear algebraic equality and inequality constraints. Through an industrial case study, we demonstrate the effectiveness of our methodology, showcasing the advantages of incorporating physics in feature extraction. These advantages include improved interpretability, reduced data dimensionality, and enhanced generalization performance.

A novel semi-supervised robust learning framework for dynamic generative latent variable models and its application to industrial virtual metrology

Among a variety of virtual metrology models, dynamic generative latent variable models (DGLVMs) have proven to be an effective tool, owing to outstanding advantages in dealing with complicated data characteristics including temporal correlations, variable colinearities, high dimensionality, and process and measurement uncertainties. Presently, the vast majority of DGLVMs are developed upon the assumption that data are Gaussian-distributed, which is however, non-robust against outlying data and leads the DGLVMs to compromised performance, as the Gaussian distribution is susceptible to outliers that would skew the estimated parameters of DGLVMs. In view of this, this paper proposes a semi-supervised robust learning (SsRL) framework for the DGLVMs by designing an ad-hoc fully robust model structure that handles the outlying data by means of the heavy-tail properties of Student’s t distribution. In addition, the SsRL-DGLVM framework is initiated as a concrete model using probabilistic slow feature analysis (denoted RoSsPSFA) for which a rigorous training algorithm without resorting to any approximations is developed. The performance of the RoSsPSFA is thoroughly evaluated by both numerical and real-world industrial cases, showing the RoSsPSFA’s better immunity against outlying data and higher generalization accuracy.

Probabilistic quantile multiple fourier feature network for lake temperature forecasting: incorporating pinball loss for uncertainty estimation

Probabilistic quantile multiple fourier feature network for lake temperature forecasting: incorporating pinball loss for uncertainty estimation

Assessment of Soccer Teaching Ability Based on Deep Learning Algorithm

Soccer teaching involves imparting fundamental skills, tactics, and strategies related to the sport of soccer. Coaches and instructors focus on teaching players proper techniques for dribbling, passing, shooting, and defending, while also emphasizing teamwork, sportsmanship, and game awareness. Soccer teaching sessions typically include a combination of drills, scrimmages, and tactical discussions tailored to the age and skill level of the players. This paper proposed an effective soccer assessment technique for teaching ability with the Automated Probabilistic Deep Learning (APDL) model. The proposed APDL model comprises an automated model for the assessment of student performance. The proposed APDL model processes the input soccer images with the pre-processing of the features. With APDL model uses the probabilistic computation features for the computation of the variables in the soccer data. With the extraction of the features, the maximum likelihood is computed for the classification with the deep learning model features. The APDL model implements the classification-based deep learning model features with the examination of soccer teaching, instruction, development of skill, and players. Simulation results demonstrated that prediction with the APDL model estimates the probability of prediction as 0.91 with an estimated uncertainty value of 0.08. In the case of teaching and coaching assessment, the uncertainty is stated as 0.08 for both with the prediction assessment score of 0.92. The classification accuracy of the proposed APDL model is achieved as 0.95 with the precision value of 0.97. The findings of this research contribute to the advancement of coaching methodologies in soccer, providing coaches and educators with valuable insights into their teaching effectiveness and areas for improvement. Additionally, the automated nature of the APDL model offers scalability and efficiency in assessing coaching performance, paving the way for enhanced coaching practices and player development in soccer.

Generation of Graphic Design Color Schemes Based on CMYK Color Model and Corrosion Algorithms

Graphic design colour schemes play a fundamental role in shaping the visual identity and impact of a design. This paper proposed an efficient Feature Extraction Madhami Fuzzy Clustering Probabilistic Corrosion (FEMFcPC) for the graphic design. The proposed FEMFcPC model comprises the CMYK color scheme for the estimation of the features in the images. With the estimation of the FEMFcPC model corrosion algorithm is implemented for the computation of the graphic design features. Once the features are estimated the related features for the graphic design are designed with the mamdhami fuzzy clustering model. The estimated features are clustered with the CMYK color schemes. The proposed FEMFcPC model estimates the patterns and relationships among graphic designs to provide valuable insights for design categorization and recommendation systems. A dataset comprising various graphic design examples with their corresponding CMYK color schemes is utilized for analysis. Through the application of the FEMFcPC algorithm, clustering results are obtained, revealing distinct grouping patterns and similarities among the designs. The simulation results stated that the proposed FEMFcPC model achieves a significant color scheme for the graphic design. Through the clustering process, the proposed FEMFcPC model with the values of the probabilistic features for the graphic design. The Key findings include the identification of dominant clusters, even distribution of designs across multiple clusters, detection of outliers, and assessment of cluster density. These insights offer significant implications for design categorization, recommendation systems, and targeted marketing strategies within the graphic design domain. The study contributes to advancing our understanding of graphic design analysis and provides a foundation for future research in this field.

M3S-ALG: Improved and robust prediction of allergenicity of chemical compounds by using a novel multi-step stacking strategy

A wide variety of chemicals cannot be introduced to the marketplace because of their high allergenicity. Therefore, it is fundamentally crucial to assess the allergenic potential of chemicals before introducing them into clinical therapeutics. However, assessing the allergenicity of chemical compounds experimentally is time-consuming and costly. To tackle this challenge, we propose M3S-ALG, a novel multi-step stacking strategy (M3S) for rapid and accurate identification of the allergenicity of chemical compounds by using only the SMILES notation. The proposed M3S method involves three steps, as follows. First, ten different balanced datasets were constructed using an under-sampling approach. Second, for each balanced dataset, 144 base-classifiers were trained and optimized to generate the prediction scores of allergenic chemical compounds considered as new probabilistic features. Third, we selected the important probabilistic features and employed them to construct the final stacked model (M3S-ALG). Experimental results show that M3S-ALG outperforms conventional ensemble strategies and its constituent base-classifiers on both the training and independent test datasets. This indicates the effectiveness and robustness of our proposed strategy in identifying the allergenicity of chemical compounds. In addition, M3S-ALG exhibited excellent prediction performance compared to existing methods on the independent test dataset, achieving a balanced accuracy of 0.877, MCC of 0.712, and AUC of 0.931. Finally, we developed a user-friendly online web server at https://pmlabqsar.pythonanywhere.com/M3SALG. This new approach is anticipated to facilitate the drug discovery and development community for the large-scale identification of chemical compounds with no allergenic properties.

A robust segmentation of retinal fluids from OCT images using MCFAR-net

This paper presents a novel architecture to detect Macular Edema (ME) in Optical Coherence Tomography (OCT) images that occurs due to high fluid accumulation between the retinal layers of the eye. These excessive fluids have swollen the macular region and may result in visual impairments. To alleviate this problem in its early stage, the proposed model, Multiscale Context Enhancive Aggregation & Refinement Network (MCFAR-Net) is implemented to detect these fluids enabling the prevention of visual loss. This MCFAR-Net is incorporated with two Context Feature Enhancive Modules (CFEM). The upper module is initially trained using contrastive loss for extracting the most pertinent multiscale feature maps allowing to segment thick fluid regions from OCT images. Further, the lower module elevates the feature maps to segment the minute fluid regions using the outputs of upper CFEM along with input image. Finally, the output fluid probabilistic feature maps of both paths of the two modules are stacked together and fed as input to Feature Synthesizer (FS) Module. This module improves the true positive rate of the proposed algorithm and segments the fluid region more accurately. The performance of the proposed model is trained and evaluated using publicly available datasets like RETOUCH, OPTIMA, and DUKE datasets. This model outperformed existing state-of the-art algorithms by attaining an average dice coefficient of 95.63 % when tested on the RETOUCH dataset. The performance of proposed MCFAR-Net reduced the misclassification errors enabling to identify ME more precisely in its early stage allowing the expert doctor to provide immediate treatment to the patients.

AMP-RNNpro: a two-stage approach for identification of antimicrobials using probabilistic features

Antimicrobials are molecules that prevent the formation of microorganisms such as bacteria, viruses, fungi, and parasites. The necessity to detect antimicrobial peptides (AMPs) using machine learning and deep learning arises from the need for efficiency to accelerate the discovery of AMPs, and contribute to developing effective antimicrobial therapies, especially in the face of increasing antibiotic resistance. This study introduced AMP-RNNpro based on Recurrent Neural Network (RNN), an innovative model for detecting AMPs, which was designed with eight feature encoding methods that are selected according to four criteria: amino acid compositional, grouped amino acid compositional, autocorrelation, and pseudo-amino acid compositional to represent the protein sequences for efficient identification of AMPs. In our framework, two-stage predictions have been conducted. Initially, this study analyzed 33 models on these feature extractions. Then, we selected the best six models from these models using rigorous performance metrics. In the second stage, probabilistic features have been generated from the selected six models in each feature encoding and they are aggregated to be fed into our final meta-model called AMP-RNNpro. This study also introduced 20 features with SHAP, which are crucial in the drug development fields, where we discover AAC, ASDC, and CKSAAGP features are highly impactful for detection and drug discovery. Our proposed framework, AMP-RNNpro excels in the identification of novel Amps with 97.15% accuracy, 96.48% sensitivity, and 97.87% specificity. We built a user-friendly website for demonstrating the accurate prediction of AMPs based on the proposed approach which can be accessed at http://13.126.159.30/.

Dependence on Tail Copula

In real-world scenarios, we encounter non-exchangeable dependence structures. Our primary focus is on identifying and quantifying non-exchangeability in the tails of joint distributions. The findings and methodologies presented in this study are particularly valuable for modeling bivariate dependence, especially in fields where understanding dependence patterns in the tails is crucial, such as quantitative finance, quantitative risk management, and econometrics. To grasp the intricate relationship between the strength of dependence and various types of margins, we explore three fundamental tail behavior patterns for univariate margins. Capitalizing on the probabilistic features of tail non-exchangeability structures, we introduce graphical techniques and statistical tests designed for analyzing data that may manifest non-exchangeability in the joint tail. The effectiveness of the proposed approaches is illustrated through a simulation study and a practical example.

The Role of Attention in Category Representation.

Numerous studies have found that selective attention affects category learning. However, previous research did not distinguish between the contribution of focusing and filtering components of selective attention. This study addresses this issue by examining how components of selective attention affect category representation. Participants first learned a rule-plus-similarity category structure, and then were presented with category priming followed by categorization and recognition tests. Additionally, to evaluate the involvement of focusing and filtering, we fit models with different attentional mechanisms to the data. In Experiment 1, participants received rule-based category training, with specific emphasis on a single deterministic feature (D feature). Experiment 2 added a recognition test to examine participants' memory for features. Both experiments indicated that participants categorized items based solely on the D feature, showed greater memory for the D feature, were primed exclusively by the D feature without interference from probabilistic features (P features), and were better fit by models with focusing and at least one type of filtering mechanism. The results indicated that selective attention distorted category representation by highlighting the D feature and attenuating P features. To examine whether the distorted representation was specific to rule-based training, Experiment 3 introduced training, emphasizing all features. Under such training, participants were no longer primed by the D feature, they remembered all features well, and they were better fit by the model assuming only focusing but no filtering process. The results coupled with modeling provide novel evidence that while both focusing and filtering contribute to category representation, filtering can also result in representational distortion.

Predicting the Activity of Unidentified Chemicals in Complementary Bioassays from the HRMS Data to Pinpoint Potential Endocrine Disruptors.

The majority of chemicals detected via nontarget liquid chromatography high-resolution mass spectrometry (HRMS) in environmental samples remain unidentified, challenging the capability of existing machine learning models to pinpoint potential endocrine disruptors (EDs). Here, we predict the activity of unidentified chemicals across 12 bioassays related to EDs within the Tox21 10K dataset. Single- and multi-output models, utilizing various machine learning algorithms and molecular fingerprint features as an input, were trained for this purpose. To evaluate the models under near real-world conditions, Monte Carlo sampling was implemented for the first time. This technique enables the use of probabilistic fingerprint features derived from the experimental HRMS data with SIRIUS+CSI:FingerID as an input for models trained on true binary fingerprint features. Depending on the bioassay, the lowest false-positive rate at 90% recall ranged from 0.251 (sr.mmp, mitochondrial membrane potential) to 0.824 (nr.ar, androgen receptor), which is consistent with the trends observed in the models' performances submitted for the Tox21 Data Challenge. These findings underscore the informativeness of fingerprint features that can be compiled from HRMS in predicting the endocrine-disrupting activity. Moreover, an in-depth SHapley Additive exPlanations analysis unveiled the models' ability to pinpoint structural patterns linked to the modes of action of active chemicals. Despite the superior performance of the single-output models compared to that of the multi-output models, the latter's potential cannot be disregarded for similar tasks in the field of in silico toxicology. This study presents a significant advancement in identifying potentially toxic chemicals within complex mixtures without unambiguous identification and effectively reducing the workload for postprocessing by up to 75% in nontarget HRMS.

Unifying Multi-Modal Uncertainty Modeling and Semantic Alignment for Text-to-Image Person Re-identification

Text-to-Image person re-identification (TI-ReID) aims to retrieve the images of target identity according to the given textual description. The existing methods in TI-ReID focus on aligning the visual and textual modalities through contrastive feature alignment or reconstructive masked language modeling (MLM). However, these methods parameterize the image/text instances as deterministic embeddings and do not explicitly consider the inherent uncertainty in pedestrian images and their textual descriptions, leading to limited image-text relationship expression and semantic alignment. To address the above problem, in this paper, we propose a novel method that unifies multi-modal uncertainty modeling and semantic alignment for TI-ReID. Specifically, we model the image and textual feature vectors of pedestrian as Gaussian distributions, where the multi-granularity uncertainty of the distribution is estimated by incorporating batch-level and identity-level feature variances for each modality. The multi-modal uncertainty modeling acts as a feature augmentation and provides richer image-text semantic relationship. Then we present a bi-directional cross-modal circle loss to more effectively align the probabilistic features between image and text in a self-paced manner. To further promote more comprehensive image-text semantic alignment, we design a task that complements the masked language modeling, focusing on the cross-modality semantic recovery of global masked token after cross-modal interaction. Extensive experiments conducted on three TI-ReID datasets highlight the effectiveness and superiority of our method over state-of-the-arts.

Heart failure survival prediction using novel transfer learning based probabilistic features.

Heart failure is a complex cardiovascular condition characterized by the heart's inability to pump blood effectively, leading to a cascade of physiological changes. Predicting survival in heart failure patients is crucial for optimizing patient care and resource allocation. This research aims to develop a robust survival prediction model for heart failure patients using advanced machine learning techniques. We analyzed data from 299 hospitalized heart failure patients, addressing the issue of imbalanced data with the Synthetic Minority Oversampling (SMOTE) method. Additionally, we proposed a novel transfer learning-based feature engineering approach that generates a new probabilistic feature set from patient data using ensemble trees. Nine fine-tuned machine learning models are built and compared to evaluate performance in patient survival prediction. Our novel transfer learning mechanism applied to the random forest model outperformed other models and state-of-the-art studies, achieving a remarkable accuracy of 0.975. All models underwent evaluation using 10-fold cross-validation and tuning through hyperparameter optimization. The findings of this study have the potential to advance the field of cardiovascular medicine by providing more accurate and personalized prognostic assessments for individuals with heart failure.

A novel stacking-based predictor for accurate prediction of antimicrobial peptides

Antimicrobial peptides (AMPs) are gaining acceptance and support as a chief antibiotic substitute since they boost human immunity. They retain a wide range of actions and have a low risk of developing resistance, which are critical properties to the pharmaceutical industry for drug discovery. Antibiotic sensitivity, however, is an issue that affects people all around the world and has the potential to one day lead to an epidemic. As cutting-edge therapeutic agents, AMPs are also expected to cure microbial infections. In order to produce tolerable drugs, it is crucial to understand the significance of the basic architecture of AMPs. Traditional laboratory methods are expensive and time-consuming for AMPs testing and detection. Currently, bioinformatics techniques are being successfully applied to the detection of AMPs. In this study, we have developed a novel STacking-based ensemble learning framework for AntiMicrobial Peptide (STAMP) prediction. First, we constructed 84 different baseline models by using 12 different feature encoding schemes and 7 popular machine learning algorithms. Second, these baseline models were trained and employed to create a new probabilistic feature vector. Finally, based on the feature selection strategy, we determined the optimal probabilistic feature vector, which was further utilized for the construction of our stacked model. Resultantly, the STAMP predictor achieved excellent performance during cross-validation with an accuracy and Matthew’s correlation coefficient of 0.930 and 0.860, respectively. The corresponding metrics during the independent test were 0.710 and 0.464, respectively. Overall, STAMP achieved a more accurate and stable performance than the baseline models and significantly outperformed the existing predictors, demonstrating the effectiveness of our proposed hybrid framework. Furthermore, STAMP is expected to assist community-wide efforts in identifying AMPs and will contribute to the development of novel therapeutic methods and drug-design for immunity. Communicated by Ramaswamy H. Sarma

Unsupervised deep learning approach for structural anomaly detection using probabilistic features

Civil structures may deteriorate during their service life due to degradation or damage imposed by natural hazards such as earthquakes, wind, and impact. Structural performance anomaly detection is essential to provide an early warning of structural degradation limit states in order to prevent potential catastrophic failure. Data-driven machine learning approaches have been widely used for this, due to their capability in capturing features sensitive to damage-induced anomalies from structural health monitoring (SHM) data, assuming that such data are available. Although machine learning models have been used, many are challenged by the vast operational and environmental variability that can corrupt SHM data and by (typically) strongly correlated information from different sensors in the SHM data. This paper proposes an unsupervised deep learning approach for the detection of structural anomaly based on a deep convolutional variational autoencoder (DCVAE) for feature extraction coupled with support vector data description (SVDD) for anomaly detection. The proposed DCVAE-SVDD method has several appealing strengths. First, the variational latent encoding is used to capture the features of monitoring data through a probability distribution. The integration of the Kullback–Leibler divergence in the loss function provides accurate estimation of the probability distributions. Second, the DCVAE designed with convolutional and deconvolutional operations utilizes the correlation among multisensor data to avoid loss of correlation features and achieve better performance in feature extraction. Third, the SVDD is utilized to create a minimum-volume hypersphere that contains the anomaly-sensitive statistical features of the state. The hypersphere accurately separates anomaly-sensitive statistical features of reference states of structure from the anomalous ones. A computational frame model and a laboratory grandstand model are used to evaluate the performance of the proposed method for detecting structural anomaly. The results demonstrate the superiority of the proposed DCVAE-SVDD in detection accuracy over the other commonly used structural anomaly detection methods (deep autoencoder combined with SVDD autoregressive model with one-class support vector machine, and principal component analysis).