Incorporating Domain Knowledge Research Articles

Depression Detection Using Digital Traces on Social Media: A Knowledge-aware Deep Learning Approach

ABSTRACT Depression is a pressing yet underdiagnosed issue in health management. Because depressed patients share their symptoms, life events, and treatments on digital platforms, information systems (IS) scholars resort to user-generated digital traces for depression detection. While they facilitate innovative information technology (IT) approaches to alleviate the social and economic burden of depression, most studies lack effective means to incorporate domain knowledge in depression detection systems or suffer from feature extraction difficulties. Following the design science research in IS, we propose a Deep-Knowledge-Aware Depression Detection system to detect social media users at risk of depression and explain the detection factors. We deploy extensive empirical analyses to evaluate our designed IT artifact, which shows domain knowledge greatly improves performance. Our work has significant implications for IS research in knowledge-aware machine learning, digital traces utilization, and generalizable design principles. Practically, the early detection and factor explanation from our IT artifact can assist depression management and enable large-scale assessment of the population’s mental health.

Data-Driven Classification and Logging Prediction of Mudrock Lithofacies Using Machine Learning: Shale Oil Reservoirs in the Eocene Shahejie Formation, Bonan Sag, Bohai Bay Basin, Eastern China

As the world’s energy demand continues to expand, shale oil has a substantial influence on the global energy reserves. The third submember of the Mbr 3 of the Shahejie Fm, characterized by complicated mudrock lithofacies, is one of the significant shale oil enrichment intervals of the Bohai Bay Basin. The classification and identification of lithofacies are key to shale oil exploration and development. However, the efficiency and reliability of lithofacies identification results can be compromised by qualitative classification resulting from an incomplete workflow. To address this issue, a comprehensive technical workflow for mudrock lithofacies classification and logging prediction was designed based on machine learning. Principal component analysis (PCA) and hierarchical cluster analysis (HCA) were conducted to realize the automatic classification of lithofacies, which can classify according to the internal relationship of the data without the disturbance of human factors and provide an accurate lithofacies result in a much shorter time. The PCA and HCA results showed that the third submember can be split into five lithofacies: massive argillaceous limestone lithofacies (MAL), laminated calcareous claystone lithofacies (LCC), intermittent lamellar argillaceous limestone lithofacies (ILAL), continuous lamellar argillaceous limestone lithofacies (CLAL), and laminated mixed shale lithofacies (LMS). Then, random forest (RF) was performed to establish the identification model for each of the lithofacies and the obtained model is optimized by grid search (GS) and K-fold cross validation (KCV), which could then be used to predict the lithofacies of the non-coring section, and the three validation methods showed that the accuracy of the GS–KCV–RF model were all above 93%. It is possible to further enhance the performance of the models by resampling, incorporating domain knowledge, and utilizing the mechanism of attention. Our method solves the problems of the subjective and time-consuming manual interpretation of lithofacies classification and the insufficient generalization ability of machine-learning methods in the previous works on lithofacies prediction research, and the accuracy of the model for mudrocks lithofacies prediction is also greatly improved. The lithofacies machine-learning workflow introduced in this study has the potential to be applied in the Bohai Bay Basin and comparable reservoirs to enhance exploration efficiency and reduce economic costs.

Training neural networks with classification rules for incorporating domain knowledge

Neural networks are capable of learning complex concepts and tasks, given abundant training data. In real-world applications where data collection can be difficult, integrating domain knowledge into the model can reduce the burden on data requirements and allow human experts greater control over model decisions. This paper focuses on incorporating conditional statements for tabular data as classification rules, which have a simple structure and are easy to construct. We introduce a general rule loss constraint to guide neural network training in a model agnostic manner, and propose confidence learning to automatically weigh the contribution of multiple candidate rules. Experimental evaluation with three real-world datasets shows that the rule loss can substantially increase model performance, particularly when training data is limited.

Fault detection and diagnosis of electric bus air conditioning systems incorporating domain knowledge and probabilistic artificial intelligence

The air conditioning systems in electric city buses usually operate in rapidly changing ambient conditions and are more likely to suffer from mechanical faults. Although many fault detection and diagnosis (FDD) methods have been developed for building air conditioning systems, they are difficult to be applied to bus air conditioners since its operation is highly dynamic and fault-free data are usually unavailable. Therefore, this paper proposes an FDD method for electric bus air conditioners to tackle the above issues. First, the method identifies faults in an unsupervised manner by comparing selected features among a group of peer systems. Then, considering the features are influenced by the operating conditions, Gaussian process regression (GPR) models are established to find the relationships between each feature and its influential parameters. The probabilistic nature of the GPR is used to differentiate predictions with large uncertainty, which are then excluded from FDD. In this way, robustness of the method is evidently improved. Finally, fault indexes are defined to detect and diagnose mechanical faults. We applied the method to a group of air conditioners in a city bus fleet. Results showed that it can effectively identify refrigerant undercharge and indoor and outdoor fan problems with low false positive/genitive rates. Also, the method is highly robust and not sensitive to the faulty systems in the bus fleet.

Edge computing-oriented smart agricultural supply chain mechanism with auction and fuzzy neural networks

Powered by data-driven technologies, precision agriculture offers immense productivity and sustainability benefits. However, fragmentation across farmlands necessitates distributed transparent automation. We developed an edge computing framework complemented by auction mechanisms and fuzzy optimizers that connect various supply chain stages. Specifically, edge computing offers powerful capabilities that enable real-time monitoring and data-driven decision-making in smart agriculture. We propose an edge computing framework tailored to agricultural needs to ensure sustainability through a renewable solar energy supply. Although the edge computing framework manages real-time crop monitoring and data collection, market-based mechanisms, such as auctions and fuzzy optimization models, support decision-making for smooth agricultural supply chain operations. We formulated invisible auction mechanisms that hide actual bid values and regulate information flows, combined with machine learning techniques for robust predictive analytics. While rule-based fuzzy systems encode domain expertise in agricultural decision-making, adaptable training algorithms help optimize model parameters from the data. A two-phase hybrid learning approach is formulated. Fuzzy optimization models were formulated using domain expertise for three key supply chain decision problems. Auction markets discover optimal crop demand–supply balancing and pricing signals. Fuzzy systems incorporate domain knowledge into interpretable crop-advisory models. An integrated evaluation of 50 farms over five crop cycles demonstrated the high performance of the proposed edge computing-oriented auction-based fuzzy neural network model compared with benchmarks.

Coarse-Grained Crystal Graph Neural Networks for Reticular Materials Design.

Reticular materials, including metal-organic frameworks and covalent organic frameworks, combine the relative ease of synthesis and an impressive range of applications in various fields from gas storage to biomedicine. Diverse properties arise from the variation of building units─metal centers and organic linkers─in almost infinite chemical space. Such variation substantially complicates the experimental design and promotes the use of computational methods. In particular, the most successful artificial intelligence algorithms for predicting the properties of reticular materials are atomic-level graph neural networks, which optionally incorporate domain knowledge. Nonetheless, the data-driven inverse design involving these models suffers from the incorporation of irrelevant and redundant features such as a full atomistic graph and network topology. In this study, we propose a new way of representing materials, aiming to overcome the limitations of existing methods; the message passing is performed on a coarse-grained crystal graph that comprises molecular building units. To highlight the merits of our approach, we assessed the predictive performance and energy efficiency of neural networks built on different materials representations, including composition-based and crystal-structure-aware models. Coarse-grained crystal graph neural networks showed decent accuracy at low computational costs, making them a valuable alternative to omnipresent atomic-level algorithms. Moreover, the presented models can be successfully integrated into an inverse materials design pipeline as estimators of the objective function. Overall, the coarse-grained crystal graph framework is aimed at challenging the prevailing atom-centric perspective on reticular materials design.

The effect of incorporating domain knowledge with deep learning in identifying benign and malignant gastric whitish lesions: A retrospective study.

Early whitish gastric neoplasms can be easily misdiagnosed; differential diagnosis of gastric whitish lesions remains a challenge. We aim to build a deep learning (DL) model to diagnose whitish gastric neoplasms and explore the effect of adding domain knowledge in model construction. We collected 4558 images from two institutions to train and test models. We first developed two sole DL models (1 and 2) using supervised and semi-supervised algorithms. Then we selected diagnosis-related features through literature research and developed feature-extraction models to determine features including boundary, surface, roundness, depression, and location. Then predictions of the five feature-extraction models and sole DL model were combined and inputted into seven machine-learning (ML) based fitting-diagnosis models. The optimal model was selected as ENDOANGEL-WD (whitish-diagnosis) and compared with endoscopists. Sole DL 2 had higher sensitivity (83.12% vs 68.67%, Bonferroni adjusted P=0.024) than sole DL 1. Adding domain knowledge, the decision tree performed best among the seven ML models, achieving higher specificity than DL 1 (84.38% vs 72.27%, Bonferroni adjusted P<0.05) and higher accuracy than DL 2 (80.47%, Bonferroni adjusted P<0.001) and was selected as ENDOANGEL-WD. ENDOANGEL-WD showed better accuracy compared with 10 endoscopists (75.70%, P<0.001). We developed a novel system ENDOANGEL-WD combining domain knowledge and traditional DL to detect gastric whitish neoplasms. Adding domain knowledge improved the performance of traditional DL, which provided a novel solution for establishing diagnostic models for other rare diseases potentially.

The Rise of Hypothesis-Driven Artificial Intelligence in Oncology.

Cancer is a complex disease involving the deregulation of intricate cellular systems beyond genetic aberrations and, as such, requires sophisticated computational approaches and high-dimensional data for optimal interpretation. While conventional artificial intelligence (AI) models excel in many prediction tasks, they often lack interpretability and are blind to the scientific hypotheses generated by researchers to enable cancer discoveries. Here we propose that hypothesis-driven AI, a new emerging class of AI algorithm, is an innovative approach to uncovering the complex etiology of cancer from big omics data. This review exemplifies how hypothesis-driven AI is different from conventional AI by citing its application in various areas of oncology including tumor classification, patient stratification, cancer gene discovery, drug response prediction, and tumor spatial organization. Our aim is to stress the feasibility of incorporating domain knowledge and scientific hypotheses to craft the design of new AI algorithms. We showcase the power of hypothesis-driven AI in making novel cancer discoveries that can be overlooked by conventional AI methods. Since hypothesis-driven AI is still in its infancy, open questions such as how to better incorporate new knowledge and biological perspectives to ameliorate bias and improve interpretability in the design of AI algorithms still need to be addressed. In conclusion, hypothesis-driven AI holds great promise in the discovery of new mechanistic and functional insights that explain the complexity of cancer etiology and potentially chart a new roadmap to improve treatment regimens for individual patients.

Developing an Appendicectomy Surgical Pathway Ontology (ASPO).

This study proposes an ontology model to represent the surgical pathways for appendicectomy, incorporating domain knowledge extracted from literature and electronic records in an Australian health district. The ontology comprised 108 concepts and 81 object properties. The model is useful for continuous quality improvement initiatives, i.e., process mining to understand what happens in the hospital versus what is required by the policy.

Automated site planning using CAIN-GAN model

Automated site planning, powered by deep generative methods, excels in creating solutions responsive to exiting city structures but often overlooks user-specific design scenarios, leading to less performative solutions across varied urban contexts. Overcoming this challenge requires integrating domain knowledge and nuances of the built environment to enhance context-awareness in automated site planning. This study therefore proposes the context-aware site planning generative adversarial networks (CAIN-GAN) framework. In the case study of New York City (NYC), CAIN-GAN demonstrates its capability to not only synthesize visually realistic and semantically reasonable design solutions, but also evaluate their performance in urban sustainability for informed decision-making. This context-aware, learning-based, data-driven, and user-guided generation process signifies a pivotal advancement in more performative and tailored design solutions. Future studies will focus on refining the CAIN-GAN framework to accommodate diverse user-centric design needs and enhance human-machine interaction in urban development.

Review of Prediction of Stress Corrosion Cracking in Gas Pipelines Using Machine Learning

Pipeline integrity and safety depend on the detection and prediction of stress corrosion cracking (SCC) and other defects. In oil and gas pipeline systems, a variety of corrosion-monitoring techniques are used. The observed data exhibit characteristics of nonlinearity, multidimensionality, and noise. Hence, data-driven modeling techniques have been widely utilized. To accomplish intelligent corrosion prediction and enhance corrosion control, machine learning (ML)-based approaches have been developed. Some published papers related to SCC have discussed ML techniques and their applications, but none of the works has shown the real ability of ML to detect or predict SCC in energy pipelines, though fewer researchers have tested their models to prove them under controlled environments in laboratories, which is completely different from real work environments in the field. Looking at the current research status, the authors believe that there is a need to explore the best technologies and modeling approaches and to identify clear gaps; a critical review is, therefore, required. The objective of this study is to assess the current status of machine learning’s applications in SCC detection, identify current research gaps, and indicate future directions from a scientific research and application point of view. This review will highlight the limitations and challenges of employing machine learning for SCC prediction and also discuss the importance of incorporating domain knowledge and expert inputs to enhance the accuracy and reliability of predictions. Finally, a framework is proposed to demonstrate the process of the application of ML to condition assessments of energy pipelines.

Bayesian-Based Causal Structure Inference With a Domain Knowledge Prior for Stable and Interpretable Soft Sensing.

Due to the high-stakes nature of industrial processes, there is an immediate and pressing need on soft sensors for stability and interpretability. In this regard, causality-inspired modeling aims to learn causal features corresponding to the direct causes of quality variables, exhibiting great potential in terms of both stability and interpretability. However, most existing causality-inspired methods overlook temporal modeling and domain knowledge integration, which hinders their real-world application in industrial soft sensing. To this end, this article proposes a novel causality-inspired stable long short-term memory (Stable-LSTM), which leverages Bayesian-based causal structure inference and incorporates domain knowledge as a prior to enhance the performance stability and physical interpretability of soft sensors. After extracting temporal features via long short-term memory (LSTM), a Bayesian-based causal structure inference approach is developed by leveraging variational inference to learn the underlying hidden causal structure within the industrial processes. Through a hidden explanation of domain knowledge, a prior distribution is placed on the hidden causal structure, which will greatly enhance the physical interpretability and facilitate the exploration for true causality. Moreover, we also introduce a global sample reweighting strategy to remove spurious correlations and reveal causal effects between time series hidden features and quality variables. Finally, the performance stability and physical interpretability of the proposed Stable-LSTM are verified using a three-phase flow facility and a m-phenylenediamine distillation process. The results show that the Stable-LSTM achieves the highest soft sensing accuracy under distribution shift, and the inferred causal structure exhibits the greatest consistency with the domain knowledge, when compared with the seven existing methods.

A novel hypoglycemia alarm framework for type 2 diabetes with high glycemic variability.

In patients with type 2 diabetes (T2D), accurate prediction of hypoglycemic events is crucial for maintaining glycemic control and reducing their frequency. However, individuals with high blood glucose variability experience significant fluctuations over time, posing a challenge for early warning models that rely on static features. This article proposes a novel hypoglycemia early alarm framework based on dynamic feature selection. The framework incorporates domain knowledge and introduces multi-scale blood glucose features, including predicted values, essential for early warnings. To address the complexity of the feature matrix, a dynamic feature selection mechanism (Relief-SVM-RFE) is designed to effectively eliminate redundancy. Furthermore, the framework employs online updates for the random forest model, enhancing the learning of more relevant features. The effectiveness of the framework was evaluated using a clinical dataset. For T2D patients with a high coefficient of variation (CV), the framework achieved a sensitivity of 81.15% and specificity of 98.14%, accurately predicting most hypoglycemic events. Notably, the proposed method outperformed other existing approaches. These results indicate the feasibility of anticipating hypoglycemic events in T2D patients with high CV using this innovative framework.

State of charge estimation for lithium-ion batteries based on battery model and data-driven fusion method

The voltage plateau phase of LiFeO4 (LFP) batteries presents a formidable challenge when estimating the state of charge (SOC). Although numerous data-driven methods have been developed to address this issue, they often depend heavily on high-quality training data. Furthermore, these methods are typically regarded as “black-box” models, lacking interpretability. To overcome these challenges, a novel approach that integrates both model-based and data-driven techniques for battery SOC estimation is proposed in this paper. The proposed method is grounded in the battery model and complemented by the data-driven model, thus enhancing interpretability by incorporating domain knowledge. To reduce computational complexity, the Rint model is used for rough SOC estimation, with the eXtreme Gradient Boosting model (XGBoost) used for residual learning. The parameters of the battery model serve as input features for the XGBoost model. The effectiveness of the proposed method is empirically validated under different dynamic testing profiles. Experimental results demonstrate that integration battery domain knowledge into data-driven approaches not only enhances method interpretability but also significantly improves the SOC estimation accuracy for LFP batteries. Moreover, it exhibits robustness and exceptional generalization performance under unseen dynamic conditions, yielding root mean square errors of less than 1 % and maximum errors of less than 2 %.

Lithofacies identification from well-logging curves via integrating prior knowledge into deep learning

Lithofacies is a key parameter in reservoir characterization. With advances in machine learning, many researchers have attempted to predict lithofacies from well-log curves by using a machine-learning algorithm. However, existing models are built purely on data, which do not provide interpretability. In addition, lithofacies distribution is highly imbalanced. We incorporate domain knowledge into a gated recurrent unit network to force the model to learn from the data and knowledge. The domain knowledge that we use is expressed as first-order logic rules and is incorporated into the machine-learning pipeline through additional loss terms. Specifically, these rules are: (1) if the density is smaller than or equal to [Formula: see text], then the lithofacies is coal; (2) if the density is larger than or equal to [Formula: see text] or the neutron porosity is smaller than or equal to [Formula: see text], then the lithofacies is anhydrite; and (3) if the gamma-ray value is larger than or equal to [Formula: see text], then the lithofacies is shale. Here, [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] are the parameters that are learned by the model. By applying this domain knowledge, we aim to elucidate why the model predicts lithofacies as coal, anhydrite, or shale and reduce the effect of imbalanced data on the model’s performance. We evaluate the method on a data set from the North Sea, and the machine-learning pipeline with domain knowledge embedded is slightly superior compared with the baseline model that does not consider domain knowledge. One drawback of the method is that the domain knowledge that we provide only works for coal, anhydrite, and shale, which is incomplete. In future work, we will attempt to develop more rules that work for other types of lithofacies.

Visual Features for Improving Endoscopic Bleeding Detection Using Convolutional Neural Networks.

The presented paper investigates the problem of endoscopic bleeding detection in endoscopic videos in the form of a binary image classification task. A set of definitions of high-level visual features of endoscopic bleeding is introduced, which incorporates domain knowledge from the field. The high-level features are coupled with respective feature descriptors, enabling automatic capture of the features using image processing methods. Each of the proposed feature descriptors outputs a feature activation map in the form of a grayscale image. Acquired feature maps can be appended in a straightforward way to the original color channels of the input image and passed to the input of a convolutional neural network during the training and inference steps. An experimental evaluation is conducted to compare the classification ROC AUC of feature-extended convolutional neural network models with baseline models using regular color image inputs. The advantage of feature-extended models is demonstrated for the Resnet and VGG convolutional neural network architectures.

Domain-knowledge-aided airborne ground moving targets tracking

This paper investigates the problem of traffic surveillance using an unmanned aerial vehicle (UAV) and proposes a domain-knowledge-aided airborne ground moving target tracking algorithm. To improve the accuracy of multiple target tracking, the proposed algorithm incorporates domain knowledge into the joint probabilistic data association (JPDA) filter as state constraints. The domain knowledge considered in this paper includes both road information extracted from a given map and local traffic regulations. Conventional track update method is modified to enhance the capability of recognition of temporarily track loss. A variable structure multiple model (VS-MM) method is developed to assign the road segment to a given target. The main contribution of this paper is that we establish a comprehensive framework for target tracking with the aid of domain knowledge and analyze the effect of diverse constraints on target tracking performance, while the limitations involve only 2-D target behaviors are considered, and the sophisticated interactions between targets are neglected. The effectiveness of proposed algorithm is demonstrated through extensive numerical simulations.

TrueDeep: A systematic approach of crack detection with less data

Supervised and semi-supervised semantic segmentation algorithms require significant amount of annotated data to achieve a good performance. In many situations, the data is either not available or the annotation is expensive. The objective of this work is to show that by incorporating domain knowledge along with deep learning architectures, we can achieve similar performance with less data. We have used publicly available crack segmentation datasets and shown that selecting the input images using knowledge can significantly boost the performance of deep-learning based architectures. Our proposed approaches have many fold advantages such as low annotation and training cost, and less energy consumption. We have measured the performance of our algorithm quantitatively in terms of mean intersection over union (mIoU) and F-score. Our representative model, developed with 23% of the overall data; surpasses the baseline model (>1.5% F-score gain) on the test data while matching the benchmark performance and, with a suitable augmentation strategy, outperforms the baseline model (11.5% average F-score gain) and the benchmark (>8.75% average F-score gain) over the selected blind datasets.

Asteria-Pro: Enhancing Deep Learning-based Binary Code Similarity Detection by Incorporating Domain Knowledge

Widespread code reuse allows vulnerabilities to proliferate among a vast variety of firmware. There is an urgent need to detect these vulnerable codes effectively and efficiently. By measuring code similarities, AI-based binary code similarity detection is applied to detecting vulnerable code at scale. Existing studies have proposed various function features to capture the commonality for similarity detection. Nevertheless, the significant code syntactic variability induced by the diversity of IoT hardware architectures diminishes the accuracy of binary code similarity detection. In our earlier study and the tool Asteria , we adopted a Tree-LSTM network to summarize function semantics as function commonality, and the evaluation result indicates an advanced performance. However, it still has utility concerns due to excessive time costs and inadequate precision while searching for large-scale firmware bugs. To this end, we propose a novel deep learning-enhancement architecture by incorporating domain knowledge-based pre-filtration and re-ranking modules, and we develop a prototype named Asteria-Pro based on Asteria . The pre-filtration module eliminates dissimilar functions, thus reducing the subsequent deep learning-model calculations. The re-ranking module boosts the rankings of vulnerable functions among candidates generated by the deep learning model. Our evaluation indicates that the pre-filtration module cuts the calculation time by 96.9%, and the re-ranking module improves MRR and Recall by 23.71% and 36.4%, respectively. By incorporating these modules, Asteria-Pro outperforms existing state-of-the-art approaches in the bug search task by a significant margin. Furthermore, our evaluation shows that embedding baseline methods with pre-filtration and re-ranking modules significantly improves their precision. We conduct a large-scale real-world firmware bug search, and Asteria-Pro manages to detect 1,482 vulnerable functions with a high precision 91.65%.

Improving the out-of-sample generalization ability of data-driven chiller performance models using physics-guided neural network

Modeling of the chiller performance is essential for the implementation of optimal energy-efficient control strategies in a heating, ventilation, and air conditioning (HVAC) system. Though classical data-driven chiller performance models are widely adopted in the industry, they generally suffer from poor out-of-sample generalization abilities, which refers to the model's capability to extrapolate for new data outside the range of the training dataset. In practice, however, the available chiller operation data for model development are often insufficient or collected from a few limited operating conditions, such that extrapolation is unavoidable after the model is applied for control purposes. To deal with this issue, this paper proposed a physics-guided neural network (PGNN) to model the energy performance of chillers. By adopting a new neural network architecture, modifying the loss function, and adding limited out-of-sample data, the PGNN incorporates domain knowledge into the data-driven model to achieve better out-of-sample generalization performance. Meanwhile, the convexity and monotonicity between the dependent and independent variables in the PGNN are properly addressed. The proposed PGNN is applied to model the chiller serving a high-rise building, and results show that PGNN performs much better in extrapolation than classical models and the multi-layer perceptron model. The research demonstrated the usefulness and effectiveness of the PGNN in modeling HVAC equipment.