Deep Learning-based Detection System Research Articles

Automating shareable cyber threat intelligence production for closed source software vulnerabilities: a deep learning based detection system

AbstractSoftware can be vulnerable to various types of interference. The production of cyber threat intelligence for closed source software requires significant effort, experience, and many manual steps. The objective of this study is to automate the process of producing cyber threat intelligence, focusing on closed source software vulnerabilities. To achieve our goal, we have developed a system called cti-for-css. Deep learning algorithms were used for detection. To simplify data representation and reduce pre-processing workload, the study proposes the function-as-sentence approach. The MLP, OneDNN, LSTM, and Bi-LSTM algorithms were trained using this approach with the SOSP and NDSS18 binary datasets, and their results were compared. The aforementioned datasets contain buffer error vulnerabilities (CWE-119) and resource management error vulnerabilities (CWE-399). Our results are as successful as the studies in the literature. The system achieved the best performance using Bi-LSTM, with F1 score of 82.4%. Additionally, AUC score of 93.0% was acquired, which is the best in the literature. The study concluded by producing cyber threat intelligence using closed source software. Shareable intelligence was produced in an average of 0.1 s, excluding the detection process. Each record, which was represented using our approach, was classified in under 0.32 s on average.

Automating shareable cyber threat intelligence production for closed source software vulnerabilities: a deep learning based detection system

AbstractSoftware can be vulnerable to various types of interference. The production of cyber threat intelligence for closed source software requires significant effort, experience, and many manual steps. The objective of this study is to automate the process of producing cyber threat intelligence, focusing on closed source software vulnerabilities. To achieve our goal, we have developed a system called cti-for-css. Deep learning algorithms were used for detection. To simplify data representation and reduce pre-processing workload, the study proposes the function-as-sentence approach. The MLP, OneDNN, LSTM, and Bi-LSTM algorithms were trained using this approach with the SOSP and NDSS18 binary datasets, and their results were compared. The aforementioned datasets contain buffer error vulnerabilities (CWE-119) and resource management error vulnerabilities (CWE-399). Our results are as successful as the studies in the literature. The system achieved the best performance using Bi-LSTM, with F1 score of 82.4%. Additionally, AUC score of 93.0% was acquired, which is the best in the literature. The study concluded by producing cyber threat intelligence using closed source software. Shareable intelligence was produced in an average of 0.1 s, excluding the detection process. Each record, which was represented using our approach, was classified in under 0.32 s on average.

Enhancing false negative and positive rates for efficient insider threat detection

Insider threats on information security can become a burden for organizations. However, outsider attacks have received more attention compared to insider attacks. Many researchers studied insider threats and proposed various approaches (such as signature based, machine learning based, and deep learning based) to alleviate this type of threats. In this work, we present a novel insider threat detection system based on a deep learning network of Long Short Term Memory (LSTM). The developed detection system aims to analyze and mitigate the negative effect of insiders by differentiating benign activities from malicious ones. The detection system utilizes sentiment analysis to classify the users’ activities and gray encoding to maintain temporal behavior between activities (especially correlated activities). This allows us to reform a dataset in which each row represents a variable length sample to train a deep learning based detection system. Different data representations, such as binary encoding (BE), real-valued data, without encoding (WE), were used to test the effectiveness of gray encoding in maintaining the temporal relationships between activities. The proposed detection techniques were evaluated using log files from CERT r4.2 insiders’ dataset that represent activities of employees for eighteen working months. The evaluation results have shown enhanced false positive of 0.29%, false negative of 2.47% and an AUC value of 97%.

Multi-Center Validation Study of Automated Classification of Pathological Slowing in Adult Scalp Electroencephalograms Via Frequency Features.

Pathological slowing in the electroencephalogram (EEG) is widely investigated for the diagnosis of neurological disorders. Currently, the gold standard for slowing detection is the visual inspection of the EEG by experts, which is time-consuming and subjective. To address those issues, we propose three automated approaches to detect slowing in EEG: Threshold-based Detection System (TDS), Shallow Learning-based Detection System (SLDS), and Deep Learning-based Detection System (DLDS). These systems are evaluated on channel-, segment-, and EEG-level. The three systems perform prediction via detecting slowing at individual channels, and those detections are arranged in histograms for detection of slowing at the segment- and EEG-level. We evaluate the systems through Leave-One-Subject-Out (LOSO) cross-validation (CV) and Leave-One-Institution-Out (LOIO) CV on four datasets from the US, Singapore, and India. The DLDS achieved the best overall results: LOIO CV mean balanced accuracy (BAC) of 71.9%, 75.5%, and 82.0% at channel-, segment- and EEG-level, and LOSO CV mean BAC of 73.6%, 77.2%, and 81.8% at channel-, segment-, and EEG-level. The channel- and segment-level performance is comparable to the intra-rater agreement (IRA) of an expert of 72.4% and 82%. The DLDS can process a 30 min EEG in 4 s and can be deployed to assist clinicians in interpreting EEGs.

Added Value of Deep Learning-based Detection System for Multiple Major Findings on Chest Radiographs: A Randomized Crossover Study.

Background Previous studies assessing the effects of computer-aided detection on observer performance in the reading of chest radiographs used a sequential reading design that may have biased the results because of reading order or recall bias. Purpose To compare observer performance in detecting and localizing major abnormal findings including nodules, consolidation, interstitial opacity, pleural effusion, and pneumothorax on chest radiographs without versus with deep learning-based detection (DLD) system assistance in a randomized crossover design. Materials and Methods This study included retrospectively collected normal and abnormal chest radiographs between January 2016 and December 2017 (https://cris.nih.go.kr/; registration no. KCT0004147). The radiographs were randomized into two groups, and six observers, including thoracic radiologists, interpreted each radiograph without and with use of a commercially available DLD system by using a crossover design with a washout period. Jackknife alternative free-response receiver operating characteristic (JAFROC) figure of merit (FOM), area under the receiver operating characteristic curve (AUC), sensitivity, specificity, false-positive findings per image, and reading times of observers with and without the DLD system were compared by using McNemar and paired t tests. Results A total of 114 normal (mean patient age ± standard deviation, 51 years ± 11; 58 men) and 114 abnormal (mean patient age, 60 years ± 15; 75 men) chest radiographs were evaluated. The radiographs were randomized to two groups: group A (n = 114) and group B (n = 114). Use of the DLD system improved the observers' JAFROC FOM (from 0.90 to 0.95, P = .002), AUC (from 0.93 to 0.98, P = .002), per-lesion sensitivity (from 83% [822 of 990 lesions] to 89.1% [882 of 990 lesions], P = .009), per-image sensitivity (from 80% [548 of 684 radiographs] to 89% [608 of 684 radiographs], P = .009), and specificity (from 89.3% [611 of 684 radiographs] to 96.6% [661 of 684 radiographs], P = .01) and reduced the reading time (from 10-65 seconds to 6-27 seconds, P < .001). The DLD system alone outperformed the pooled observers (JAFROC FOM: 0.96 vs 0.90, respectively, P = .007; AUC: 0.98 vs 0.93, P = .003). Conclusion Observers including thoracic radiologists showed improved performance in the detection and localization of major abnormal findings on chest radiographs and reduced reading time with use of a deep learning-based detection system. © RSNA, 2021 Online supplemental material is available for this article.

Deep learning-based detection system for multiclass lesions on chest radiographs: comparison with observer readings.

To investigate the feasibility of a deep learning-based detection (DLD) system for multiclass lesions on chest radiograph, in comparison with observers. A total of 15,809 chest radiographs were collected from two tertiary hospitals (7204 normal and 8605 abnormal with nodule/mass, interstitial opacity, pleural effusion, or pneumothorax). Except for the test set (100 normal and 100 abnormal (nodule/mass, 70; interstitial opacity, 10; pleural effusion, 10; pneumothorax, 10)), radiographs were used to develop a DLD system for detecting multiclass lesions. The diagnostic performance of the developed model and that of nine observers with varying experiences were evaluated and compared using area under the receiver operating characteristic curve (AUROC), on a per-image basis, and jackknife alternative free-response receiver operating characteristic figure of merit (FOM) on a per-lesion basis. The false-positive fraction was also calculated. Compared with the group-averaged observations, the DLD system demonstrated significantly higher performances on image-wise normal/abnormal classification and lesion-wise detection with pattern classification (AUROC, 0.985 vs. 0.958; p = 0.001; FOM, 0.962 vs. 0.886; p < 0.001). In lesion-wise detection, the DLD system outperformed all nine observers. In the subgroup analysis, the DLD system exhibited consistently better performance for both nodule/mass (FOM, 0.913 vs. 0.847; p < 0.001) and the other three abnormal classes (FOM, 0.995 vs. 0.843; p < 0.001). The false-positive fraction of all abnormalities was 0.11 for the DLD system and 0.19 for the observers. The DLD system showed the potential for detection of lesions and pattern classification on chest radiographs, performing normal/abnormal classifications and achieving high diagnostic performance. • The DLD system was feasible for detection with pattern classification of multiclass lesions on chest radiograph. • The DLD system had high performance of image-wise classification as normal or abnormal chest radiographs (AUROC, 0.985) and showed especially high specificity (99.0%). • In lesion-wise detection of multiclass lesions, the DLD system outperformed all 9 observers (FOM, 0.962 vs. 0.886; p < 0.001).