Macro-averaged F1-score Research Articles

Abstract 4259: Identification of anti-neoplastic therapy given before initial visit at a referral center using natural language processing applied to medical oncology initial consultation notes

Abstract Anticancer therapy changes tumor physiology and genomics, making it a key variable in cancer studies. Although antineoplastics given at a single institution may be available in research-ready format, treatment at external institutions prior to receiving care at academic medical centers, common among patients at these centers, is often only described in free-text clinical notes, necessitating manual curation for downstream analysis. To overcome this bottleneck, we trained and validated natural language processing (NLP) models using initial consult notes to identify whether patients had received treatment at external institutions and studied the impact of these putative treatments on tumor genomics. Training data were derived from the AACR Project GENIE Biopharma Collaborative (BPC) for 2,663 patients at Memorial Sloan Kettering (MSK) across four cancer types. For each patient, we selected initial visits with medical and radiation oncologists based on an a priori note prioritization scheme and determined “ground-truth” prior external medications based on manually curated BPC administration records, whitelisting MSK-given medications. We trained logistic regression and clinical longformer models to identify external treatment receipt and evaluated model performance with 5-fold cross-validation. The clinical longformer model performed best across evaluation metrics, with an average area under the receiver operating characteristic curve of 0.972, macro-averaged precision/recall of 0.854/0.902 and macro-averaged F1 score of 0.876. Re-review of discrepant cases suggested that 75% of “false positives” may be due to curation error. We used our model to infer treatment status in a pan-cancer cohort with tumor genomic profiling using our institutional sequencing platform. Out of 48,447 patients, 11,900 were predicted to have received external treatment. Patients with putative external treatment had higher alteration frequencies in resistance-related genes than untreated patients and comparable to known pre-treated patients, including ESR1 in patients with breast cancer, AR in patients with prostate cancer, and EGFR T790M in patients with EGFR-mutated non-small cell lung cancer. Patients with putative external treatments, similar to known pre-treated patients, had shorter survival compared to treatment-naïve patients of the same cancer type. NLP can abstract external treatment status from clinical notes. When applied at scale, our model could help mitigate confounding variables and identify relationships between clinicogenomic variables and anticancer therapy. Citation Format: Thinh N. Tran, Karl B. Pichotta, Si-Yang Liu, Christopher Fong, Anisha Luthra, Brooke Mastrogiacomo, Steven Maron, Deborah Schrag, Sohrab P. Shah, Pedram Razavi, Bob T. Li, Gregory J. Riely, Nikolaus Schultz, Justin Jee. Identification of anti-neoplastic therapy given before initial visit at a referral center using natural language processing applied to medical oncology initial consultation notes. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4259.

Zero-shot stance detection via multi-perspective contrastive learning with unlabeled data

Stance detection is to distinguish whether the text’s author supports, opposes, or maintains a neutral stance towards a given target. In most real-world scenarios, stance detection needs to work in a zero-shot manner, i.e., predicting stances for unseen targets without labeled data. One critical challenge of zero-shot stance detection is the absence of contextual information on the targets. Current works mostly concentrate on introducing external knowledge to supplement information about targets, but the noisy schema-linking process hinders their performance in practice. To combat this issue, we argue that previous studies have ignored the extensive target-related information inhabited in the unlabeled data during the training phase, and propose a simple yet efficient Multi-Perspective Contrastive Learning Framework for zero-shot stance detection. Our framework is capable of leveraging information not only from labeled data but also from extensive unlabeled data. To this end, we design target-oriented contrastive learning and label-oriented contrastive learning to capture more comprehensive target representation and more distinguishable stance features. We conduct extensive experiments on three widely adopted datasets (from 4870 to 33,090 instances), namely SemEval-2016, WT-WT, and VAST. Our framework achieves 53.6%, 77.1%, and 72.4% macro-average F1 scores on these three datasets, showing 2.71% and 0.25% improvements over state-of-the-art baselines on the SemEval-2016 and WT-WT datasets and comparable results on the more challenging VAST dataset.

Transformer-based structuring of free-text radiology report databases

ObjectivesTo provide insights for on-site development of transformer-based structuring of free-text report databases by investigating different labeling and pre-training strategies.MethodsA total of 93,368 German chest X-ray reports from 20,912 intensive care unit (ICU) patients were included. Two labeling strategies were investigated to tag six findings of the attending radiologist. First, a system based on human-defined rules was applied for annotation of all reports (termed “silver labels”). Second, 18,000 reports were manually annotated in 197 h (termed “gold labels”) of which 10% were used for testing. An on-site pre-trained model (Tmlm) using masked-language modeling (MLM) was compared to a public, medically pre-trained model (Tmed). Both models were fine-tuned on silver labels only, gold labels only, and first with silver and then gold labels (hybrid training) for text classification, using varying numbers (N: 500, 1000, 2000, 3500, 7000, 14,580) of gold labels. Macro-averaged F1-scores (MAF1) in percent were calculated with 95% confidence intervals (CI).ResultsTmlm,gold (95.5 [94.5–96.3]) showed significantly higher MAF1 than Tmed,silver (75.0 [73.4–76.5]) and Tmlm,silver (75.2 [73.6–76.7]), but not significantly higher MAF1 than Tmed,gold (94.7 [93.6–95.6]), Tmed,hybrid (94.9 [93.9–95.8]), and Tmlm,hybrid (95.2 [94.3–96.0]). When using 7000 or less gold-labeled reports, Tmlm,gold (N: 7000, 94.7 [93.5–95.7]) showed significantly higher MAF1 than Tmed,gold (N: 7000, 91.5 [90.0–92.8]). With at least 2000 gold-labeled reports, utilizing silver labels did not lead to significant improvement of Tmlm,hybrid (N: 2000, 91.8 [90.4–93.2]) over Tmlm,gold (N: 2000, 91.4 [89.9–92.8]).ConclusionsCustom pre-training of transformers and fine-tuning on manual annotations promises to be an efficient strategy to unlock report databases for data-driven medicine.Key Points• On-site development of natural language processing methods that retrospectively unlock free-text databases of radiology clinics for data-driven medicine is of great interest.• For clinics seeking to develop methods on-site for retrospective structuring of a report database of a certain department, it remains unclear which of previously proposed strategies for labeling reports and pre-training models is the most appropriate in context of, e.g., available annotator time.• Using a custom pre-trained transformer model, along with a little annotation effort, promises to be an efficient way to retrospectively structure radiological databases, even if not millions of reports are available for pre-training.

LexID: The Metadata and Semantic Knowledge Graph Construction of Indonesian Legal Document

The Legal Fiction principle stipulates that the government needs to ensure the public availability of all of their legal documents. Unfortunately, the text-based search services they provide cannot return satisfactory answers in retrieval scenarios requiring proper representation of relationships between various legal documents. A key problem here is the lack of explicit representation of such relationships behind the employed retrieval engines. We aim to address this problem by proposing LexID knowledge graph (KG) that provides an explicit knowledge representation for Indonesian legal domain usable for such retrieval purposes. The KG contains both legal metadata information and semantic content of the legal clauses of the legal document's articles, modeled using formal vocabulary from the LexID ontology also presented in this paper. The KG is constructed from thousands of Indonesian legal documents. Since the procedure of writing a legal document regulated by the government is clear and detailed, we use a rule-based approach to construct our KG. At the end, we describe several use cases of the KG to address different retrieval needs. In Addition, we evaluated the quality of our KG by measuring its ability to answer questions and got that LexID can answer questions with the macro average F1 score is about 0.91.

Revealing the Boundaries of Selected Gastro-Intestinal (GI) Organs by Implementing CNNs in Endoscopic Capsule Images.

The detection of where an organ starts and where it ends is achievable and, since this information can be delivered in real time, it could be quite important for several reasons. For one, by having the practical knowledge of the Wireless Endoscopic Capsule (WEC) transition through an organ's domain, we are able to align and control the endoscopic operation with any other possible protocol, i.e., delivering some form of treatment on the spot. Another is having greater anatomical topography information per session, therefore treating the individual in detail (not "in general"). Even the fact that by gathering more accurate information for a patient by merely implementing clever software procedures is a task worth exploiting, since the problems we have to overcome in real-time processing of the capsule findings (i.e., wireless transfer of images to another unit that will apply the necessary real time computations) are still challenging. This study proposes a computer-aided detection (CAD) tool, a CNN algorithm deployed to run on field programmable gate array (FPGA), able to automatically track the capsule transitions through the entrance (gate) of esophagus, stomach, small intestine and colon, in real time. The input data are the wireless transmitted image shots of the capsule's camera (while the endoscopy capsule is operating). We developed and evaluated three distinct multiclass classification CNNs, trained on the same dataset of total 5520 images extracted by 99 capsule videos (total 1380 frames from each organ of interest). The proposed CNNs differ in size and number of convolution filters. The confusion matrix is obtained by training each classifier and evaluating the trained model on an independent test dataset comprising 496 images extracted by 39 capsule videos, 124 from each GI organ. The test dataset was further evaluated by one endoscopist, and his findings were compared with CNN-based results. The statistically significant of predictions between the four classes of each model and the comparison between the three distinct models is evaluated by calculating the p-values and chi-square test for multi class. The comparison between the three models is carried out by calculating the macro average F1 score and Mattheus correlation coefficient (MCC). The quality of the best CNN model is estimated by calculations of sensitivity and specificity. Our experimental results of independent validation demonstrate that the best of our developed models addressed this topological problem by exhibiting an overall sensitivity (96.55%) and specificity of (94.73%) in the esophagus, (81.08% sensitivity and 96.55% specificity) in the stomach, (89.65% sensitivity and 97.89% specificity) in the small intestine and (100% sensitivity and 98.94% specificity) in the colon. The average macro accuracy is 95.56%, the average macro sensitivity is 91.82%.

Combining molecular and cell painting image data for mechanism of action prediction

The mechanism of action (MoA) of a compound describes the biological interaction through which it produces a pharmacological effect. Multiple data sources can be used for the purpose of predicting MoA, including compound structural information, and various assays, such as those based on cell morphology, transcriptomics and metabolomics. In the present study we explored the benefits and potential additive/synergistic effects of combining structural information, in the form of Morgan fingerprints, and morphological information, in the form of five-channel Cell Painting image data. For a set of 10 well represented MoA classes, we compared the performance of deep learning models trained on the two datasets separately versus a model trained on both datasets simultaneously. On a held-out test set we obtained a macro-averaged F1 score of 0.58 when training on only the structural data, 0.81 when training on only the image data, and 0.92 when training on both together. Thus indicating clear additive/synergistic effects and highlighting the benefit of integrating multiple data sources for MoA prediction.

RECA: Related Tables Enhanced Column Semantic Type Annotation Framework

Understanding the semantics of tabular data is of great importance in various downstream applications, such as schema matching, data cleaning, and data integration. Column semantic type annotation is a critical task in the semantic understanding of tabular data. Despite the fact that various approaches have been proposed, they are challenged by the difficulties of handling wide tables and incorporating complex inter-table context information. Failure to handle wide tables limits the usage of column type annotation approaches, while failure to incorporate inter-table context harms the annotation quality. Existing methods either completely ignore these problems or propose ad-hoc solutions. In this paper, we propose Related tables Enhanced Column semantic type Annotation framework (RECA), which incorporates inter-table context information by finding and aligning schema-similar and topic-relevant tables based on a novel named entity schema. The design of RECA can naturally handle wide tables and incorporate useful inter-table context information to enhance the annotation quality. We conduct extensive experiments on two web table datasets to comprehensively evaluate the performance of RECA. Our results show that RECA achieves support-weighted F1 scores of 0.853 and 0.937 with macro average F1 scores of 0.674 and 0.783 on the two datasets respectively, which outperform the state-of-the-art methods.

Spanish Corpora of tweets about COVID-19 vaccination for automatic stance detection

The paper presents new annotated corpora for performing stance detection on Spanish Twitter data, most notably Health-related tweets. The objectives of this research are threefold: (1) to develop a manually annotated benchmark corpus for emotion recognition taking into account different variants of Spanish in social posts; (2) to evaluate the efficiency of semi-supervised models for extending such corpus with unlabelled posts; and (3) to describe such short text corpora via specialised topic modelling.A corpus of 2,801 tweets about COVID-19 vaccination was annotated by three native speakers to be in favour (904), against (674) or neither (1,223) with a 0.725 Fleiss’ kappa score. Results show that the self-training method with SVM base estimator can alleviate annotation work while ensuring high model performance. The self-training model outperformed the other approaches and produced a corpus of 11,204 tweets with a macro averaged f1 score of 0.94. The combination of sentence-level deep learning embeddings and density-based clustering was applied to explore the contents of both corpora. Topic quality was measured in terms of the trustworthiness and the validation index.

Automatic machine learning-based classification of mandibular third molar impaction status

ObjectiveThe goal of this study was to evaluate the performance of automatic machine learning (ML)-based classification of the impacted status of the mandibular third molar. MethodsThe dataset consisted of 1864 mandibular third molar images, and the impaction pattern for each mandibular third molar was annotated based on the Pell and Gregory classification and the Winter classification. To improve performance, data augmentation techniques were applied, including rotation, flip, and pseudo-images that mimic prostheses, and ML was performed using the VGG16 convolutional neural network. ResultsIn the Pell and Gregory Class classification, the performance of the model trained on the augmented dataset exhibited good classification performance in three metrics, obtaining accuracy of 0.8609, macro-average F1-score of 0.7624, and an area under the receiver operating characteristic curve (AUC) of macro-average receiver operating characteristic (ROC) of 0.9334. For Pell and Gregory Position classification, the model obtained an accuracy value of 0.8432, a macro-average F1-score of 0.8156, and an AUC of macro-average ROC value of 0.9395. For Winter classification, the model obtained an accuracy value of 0.7959, a macro-average F1-score of 0.6423, and an AUC of macro-average ROC value of 0.9549. ConclusionsThe constructed ML classification model for mandibular third molar impaction status demonstrated good performance when data augmentation was applied.

Estimating building energy efficiency from street view imagery, aerial imagery, and land surface temperature data

Current methods to determine the energy efficiency of buildings generally require on-site visits of certified energy auditors which makes the process slow, costly, and geographically incomplete. To accelerate the identification of promising retrofit targets on a large scale, we propose to estimate building energy efficiency from widely available and remotely sensed data sources only, namely street view, aerial view, footprint, and satellite-borne land surface temperature (LST) data. After collecting data for almost 40,000 buildings in the United Kingdom, we combine these data sources by training multiple end-to-end deep learning models with the objective to classify buildings as energy efficient (EU rating A–D) or inefficient (EU rating E–G). After evaluating the trained models quantitatively as well as qualitatively, we extend our analysis by studying the predictive power of each data source in an ablation study. We find that the end-to-end deep learning model trained on all four data sources achieves a macro-averaged F1 score of 64.64% and outperforms models trained on building energy consumption data by 9.78%. As industry experts use building energy consumption data to approximate a building’s energy efficiency prior to a potential on-site inspection, this work shows the potential and complementary nature of remotely sensed data in predicting energy efficiency and opens up new opportunities for future work to integrate additional data sources.

Accuracy Analysis of the End-to-End Extraction of Related Named Entities from Russian Drug Review Texts by Modern Approaches Validated on English Biomedical Corpora

An extraction of significant information from Internet sources is an important task of pharmacovigilance due to the need for post-clinical drugs monitoring. This research considers the task of end-to-end recognition of pharmaceutically significant named entities and their relations in texts in natural language. The meaning of “end-to-end” is that both of the tasks are performed within a single process on the “raw” text without annotation. The study is based on the current version of the Russian Drug Review Corpus—a dataset of 3800 review texts from the Russian segment of the Internet. Currently, this is the only corpus in the Russian language appropriate for research of the mentioned type. We estimated the accuracy of the recognition of the pharmaceutically significant entities and their relations in two approaches based on neural-network language models. The first core approach is to sequentially solve tasks of named-entities recognition and relation extraction (the sequential approach). The second one solves both tasks simultaneously with a single neural network (the joint approach). The study includes a comparison of both approaches, along with the hyperparameters selection to maximize resulting accuracy. It is shown that both approaches solve the target task at the same level of accuracy: 52–53% macro-averaged F1-score, which is the current level of accuracy for “end-to-end” tasks on the Russian language. Additionally, the paper presents the results for English open datasets ADE and DDI based on the joint approach, and hyperparameter selection for the modern domain-specific language models. The result is that the achieved accuracies of 84.2% (ADE) and 73.3% (DDI) are comparable or better than other published results for the datasets.

Automatic detection of spongiosis associated with oral lichenoid lesions using machine learning

ObjectiveThe purpose of this study was to evaluate whether it is possible to automatically detect spongiosis on whole-slide images (WSIs) derived from patients with either oral lichenoid lesions (OLLs) or oral lichen planus, aimed at reducing pathologist’s diagnostic time and effort. MethodsPatches from WSIs were used to construct a detection model using the VGG16 convolutional neural network. We conducted a one-step method to directly detect spongiosis and a two-step method to detect spongiosis within the epithelium after detection. To compare the impact of imbalanced data, we performed training with and without data augmentation (DA) and random dawn sampling (RD). ResultsFor spongiosis detection, the two-step model with DA and RD obtained an accuracy of 0.97032, a macro-average F1-score of 0.97020, and an area under the receiver operating characteristic curve (AUC) macro-average receiver operating characteristic (ROC) of 0.99513. For the one-step model with DA and RD, the model obtained an accuracy of 0.97388, a macro-average F1-score of 0.97364, and an AUC macro-average ROC of 0.99738. Compared to the oral pathologists marking of WSIs, we noted that the artificial intelligence (AI) model often over diagnosed. ConclusionsThe constructed machine learning classification model for the detection of spongiosis demonstrated good performance, and it is appropriate that oral pathologists use the AI model to suggest results as a reference to reduce the time and effort required for diagnosis.

VulEye: A Novel Graph Neural Network Vulnerability Detection Approach for PHP Application

Following advances in machine learning and deep learning processing, cyber security experts are committed to creating deep intelligent approaches for automatically detecting software vulnerabilities. Nowadays, many practices are for C and C++ programs, and methods rarely target PHP application. Moreover, many of these methods use LSTM (Long Short-Term Memory) but not GNN (Graph Neural Networks) to learn the token dependencies within the source code through different transformations. That may lose a lot of semantic information in terms of code representation. This article presents a novel Graph Neural Network vulnerability detection approach, VulEye, for PHP applications. VulEye can assist security researchers in finding vulnerabilities in PHP projects quickly. VulEye first constructs the PDG (Program Dependence Graph) of the PHP source code, slices PDG with sensitive functions contained in the source code into sub-graphs called SDG (Sub-Dependence Graph), and then makes SDG the model input to train with a Graph Neural Network model which contains three stack units with a GCN layer, Top-k pooling layer, and attention layer, and finally uses MLP (Multi-Layer Perceptron) and softmax as a classifier to predict if the SDG is vulnerable. We evaluated VulEye on the PHP vulnerability test suite in Software Assurance Reference Dataset. The experiment reports show that the best macro-average F1 score of the VulEye reached 99% in the binary classification task and 95% in the multi-classes classification task. VulEye achieved the best result compared with the existing open-source vulnerability detection implements and other state-of-art deep learning models. Moreover, VulEye can also locate the precise area of the flaw, since our SDG contains code slices closely related to vulnerabilities with a key triggering sensitive/sink function.

Handling severe data imbalance in chest X-Ray image classification with transfer learning using SwAV self-supervised pre-training

Ever since the COVID-19 outbreak, numerous researchers have attempted to train accurate Deep Learning (DL) models, especially Convolutional Neural Networks (CNN), to assist medical personnel in diagnosing COVID-19 infections from Chest X-Ray (CXR) images. However, data imbalance and small dataset sizes have been an issue in training DL models for medical image classification tasks. On the other hand, most researchers focused on complex novel methods instead and few explored this problem. In this research, we demonstrated how Self-Supervised Learning (SSL) can assist DL models during pre-training, and Transfer Learning (TL) can be used in training the models, which can produce models that are more robust to data imbalance. The Swapping Assignment between Views (SwAV) algorithm in particular has been known to be outstanding in enhancing the accuracy of CNN models for classification tasks after TL. By training a ResNet-50 model pre-trained using SwAV on a severely imbalanced CXR dataset, the model managed to greatly outperform its counterpart pre-trained in a standard supervised manner. The SwAV-TL ResNet-50 model attained 0.952 AUROC with 0.821 macro-averaged F1 score when trained on the imbalanced dataset. Hence, it was proven that TL using models pre-trained through SwAV can achieve better accuracy even when the dataset is severely imbalanced, which is usually the case in medical image datasets.

Feasibility Study of the Prediction of Radiologist's Instructions with the Bi-LSTM Model Trained with Descriptions of MR Imaging Order-statement

Magnetic resonance (MR) images provide essential diagnostic information; however, it is also a very burdensome examination for patients. At our hospital, radiologists make imaging instructions for all MR examination orders, but this is a time-consuming task. If a natural language processing model can predict the imaging instructions, it will be possible to reduce the burden on radiologists and the instruction quality can be assured. The purpose of this study was to investigate the feasibility of using natural language processing to predict MR imaging instructions with the aim of assisting radiologists. Considering the uniqueness of the MR imaging protocols at each facility and the particularity of the test order text, we considered that the use of large datasets and pre-training models would be unsuitable. We focused on LSTM, which has been used for natural language processing, and built a 4-layer bi-LSTM model in combination with our own morphological preprocessing to predict MR imaging instructions. The proposed method achieved macro-average precision, recall, and F1-score of 70.9%, 65.4%, and 66.6%, respectively. Compared to the previous studies, the proposed method achieved satisfactory performance in the natural language analysis task for Japanese. It is considered that the proposed method improved the prediction accuracy of the minority class through direct and indirect effects of vocabulary reduction, optimization, and similarity learning. It is suggested that the proposed method is effective and that the prediction of MR imaging instructions using natural language analysis in combination with the proposed method is feasible.

Neutron-gamma events discrimination under complex circumstances using ResNet

On scintillator detectors, neutron and gamma rays could generate signals. These signals are generated internally and cannot be shielded, and there are various complex circumstances during data collection, such as piled-up pulse at high count rates and overflowed pulse caused by detector range limits. It is challenging to screen these complex circumstances using conventional methods like charge comparison. Conventional methods have low accuracy and lack self adaptability, to address these problem, a neural network based on residual connection structure (ResNet) was proposed. On the basis of the collected real signal, the pulse waveforms under complex circumstances were simulated. After integrating these waveforms into a dataset, they were trained and validated by ResNet and compared with two neural network algorithms MLP and CNN. The false predictions number of ResNet is 60% and 73% lower than that of CNN and MLP. The macro average F1 score of ResNet was 0.9956, which was significantly higher than 0.9885 of CNN and 0.9855 of MLP. And in the ROC and AUC, ResNet is still the best performing method, furthermore the improvement of ResNet to CNN is higher than that of CNN to MLP. These results indicated that Proposed ResNet is more suitable for neutron-gamma events discrimination in complex situations.

Unsupervised anomaly detection in unbalanced time series data from screw driving processes using k-means clustering

Since bolted joints are ubiquitous in manufacturing, their effective and reliable quality assurance is particularly important. Most tightening processes rely on statistical methods to detect faulty screw connections already during assembly. In this paper, we address the detection of faulty tightening processes using a clustering based approach from the field of Unsupervised Machine Learning. In particular, we deploy the k-Means algorithm on a real-world dataset from the automotive industry. The model uses Dynamic Time Warping to determine the similarity between the normal and abnormal tightening processes, treating each one as an independent temporal sequence. This approach offers three advantages compared to existing supervised methods: 1.) time series with different lengths can be utilized without extensive preprocessing steps, 2.) errors never seen before can be detected using the unsupervised approach, and 3.) extensive manual efforts to generate labels are no longer necessary. To evaluate the approach, it is applied in a scenario where actual class labels are available. This allows evaluating the clustering results using traditional classification scores. The approach manages to achieve an accuracy of up to 88.89% and a macro-average F1-score of up to 63.65%.

A predictive decision support system for coronavirus disease 2019 response management and medical logistic planning.

Coronavirus disease 2019 demonstrated the inconsistencies in adequately responding to biological threats on a global scale due to a lack of powerful tools for assessing various factors in the formation of the epidemic situation and its forecasting. Decision support systems have a role in overcoming the challenges in health monitoring systems in light of current or future epidemic outbreaks. This paper focuses on some applied examples of logistic planning, a key service of the Earth Cognitive System for Coronavirus Disease 2019 project, here presented, evidencing the added value of artificial intelligence algorithms towards predictive hypotheses in tackling health emergencies. Earth Cognitive System for Coronavirus Disease 2019 is a decision support system designed to support healthcare institutions in monitoring, management and forecasting activities through artificial intelligence, social media analytics, geospatial analysis and satellite imaging. The monitoring, management and prediction of medical equipment logistic needs rely on machine learning to predict the regional risk classification colour codes, the emergency rooms attendances, and the forecast of regional medical supplies, synergically enhancing geospatial and temporal dimensions. The overall performance of the regional risk colour code classifier yielded a high value of the macro-average F1-score (0.82) and an accuracy of 85%. The prediction of the emergency rooms attendances for the Lazio region yielded a very low root mean square error (<11 patients) and a high positive correlation with the actual values for the major hospitals of the Lazio region which admit about 90% of the region's patients. The prediction of the medicinal purchases for the regions of Lazio and Piemonte has yielded a low root mean squared percentage error of 16%. Accurate forecasting of the evolution of new cases and drug utilisation enables the resulting excess demand throughout the supply chain to be managed more effectively. Forecasting during a pandemic becomes essential for effective government decision-making, managing supply chain resources, and for informing tough policy decisions.

DeepEmotionNet: Emotion mining for corporate performance analysis and prediction

Since previous studies in cognitive psychology show that individuals’ affective states can help analyze and predict their future behaviors, researchers have explored emotion mining for predicting online activities, firm profitability, and so on. Existing emotion mining methods are divided into two categories: feature-based approaches that rely on handcrafted annotations and deep learning-based methods that thrive on computational resources and big data. However, neither category can effectively detect emotional expressions captured in text (e.g., social media postings). In addition, the utilization of these methods in downstream explanatory and predictive applications is also rare. To fill the aforementioned research gaps, we develop a novel deep learning-based emotion detector named DeepEmotionNet that can simultaneously leverage contextual, syntactic, semantic, and document-level features and lexicon-based linguistic knowledge to bootstrap the overall emotion detection performance. Based on three emotion detection benchmark corpora, our experimental results confirm that DeepEmotionNet outperforms state-of-the-art baseline methods by 4.9% to 29.8% in macro-averaged F-score. For the downstream application of DeepEmotionNet to a real-world financial application, our econometric analysis highlights that top executives’ emotions of fear and anger embedded in their social media postings are significantly associated with corporate financial performance. Furthermore, these two emotions can significantly improve the predictive power of corporate financial performance when compared to sentiments. To the best of our knowledge, this is the first study to develop a deep learning-based emotion detection method and successfully apply it to enhance corporate performance prediction.

FF-MR: A DoH-Encrypted DNS Covert Channel Detection Method Based on Feature Fusion

In this paper, in order to accurately detect Domain Name System (DNS) covert channels based on DNS over HTTPS (DoH) encryption and to solve the problems of weak single-feature differentiation and poor performance in the existing detection methods, we have designed a DoH-encrypted DNS covert channel detection method based on features fusion, called FF-MR. FF-MR is based on a Multi-Head Attention and Residual Neural Network. It fuses session statistical features with multi-channel session byte sequence features. Some important features that play a key role in the detection task are screened out of the fused features through the calculation of the Multi-Head Attention mechanism. Finally, a Multi-Layer Perceptron (MLP) is used to detect encrypted DNS covert channels. By considering both global and focused features, the main idea of FF-MR is that the degree of correlation between each feature and all other features is expressed as an attention weight. Thus, features are re-represented as the result of the weighted fusion of all features using the Multi-Head Attention mechanism. Focusing on certain important features according to the distribution of attention weights improves the detection performance. While detecting the traffic in encrypted DNS covert channels, FF-MR can also accurately identify encrypted traffic generated by the three DNS covert channel tools. Experiments on the CIRA-CIC-DoHBrw-2020 dataset show that the macro-averaging recall and precision of the FF-MR method reach 99.73% and 99.72%, respectively, and the macro-averaging F1-Score reached 0.9978, which is up to 4.56% higher than the existing methods compared in the paper. FF-MR achieves at most an 11.32% improvement in macro-averaging F1-Score in identifying three encrypted DNS covert channels, indicating that FF-MR has a strong ability to detect and identify DoH-encrypted DNS covert channels.