Terms Of F-score Research Articles

Developing an Advanced Software Requirements Classification Model Using BERT: An Empirical Evaluation Study on Newly Generated Turkish Data

Requirements Engineering (RE) is an important step in the whole software development lifecycle. The problem in RE is to determine the class of the software requirements as functional (FR) and non-functional (NFR). Proper and early identification of these requirements is vital for the entire development cycle. On the other hand, manual identification of these classes is a timewaster, and it needs to be automated. Methodically, machine learning (ML) approaches are applied to address this problem. In this study, twenty ML algorithms, such as Naïve Bayes, Rotation Forests, Convolutional Neural Networks, and transformers such as BERT, were used to predict FR and NFR. Any ML algorithm requires a dataset for training. For this goal, we generated a unique Turkish dataset having collected the requirements from real-world software projects with 4600 samples. The generated Turkish dataset was used to assess the performance of the three groups of ML algorithms in terms of F-score and related statistical metrics. In particular, out of 20 ML algorithms, BERTurk was found to be the most successful algorithm for discriminating FR and NFR in terms of a 95% F-score metric. From the FR and NFR identification problem point of view, transformer algorithms show significantly better performances.

Enhanced Signal Area Estimation in Radio-Communication Spectrograms Based on Morphological Image Processing

The concept of signal area (SA), defined as the rectangular time–frequency region in a spectrogram where a signal is detected, plays an important role in spectrum usage measurements. The need for signal area estimation (SAE) is justified by its role in the process of allocating white space spectrum to secondary users in dynamic spectrum access systems as well as in other interesting applications such as compliance verification and enforcement of spectrum regulations, signal interception, and network planning and optimisation. Existing SAE methods are far from perfect and therefore new solutions capable to provide more accurate estimations are thus required. In this study, a novel approach based on image processing techniques is explored. Concretely, the feasibility of using morphological operations (MOs) is explored to examine its usefulness in the context of SAE. By means of extensive simulations, the performance of different MOs (erosion, dilation, opening, and closing) in the context of SAE is investigated under various configurations, including different shapes and sizes of the structuring element (SE), when used both as standalone SAE methods and in combination with other SAE methods from the literature. Based on the obtained results, an MO-based SAE method is formulated based on the optimum MO and SE configuration for each specific SNR regime, which can improve substantially the performance of other proposed SAE methods when used as a pre- or postprocessing technique. Concretely, the accuracy improvement in terms of F1 score is up to 40% in the low-SNR regime while achieving a perfect accuracy of 100% in the high-SNR regime. This is achieved without having a noticeable impact on the associated computational cost (and even reducing it by up to 15% at high SNR). The performance improvement is thus particularly significant in the low-SNR regime, where most methods’ performances are limited, and as a result the proposed SAE approach can extend the operational SNR range of the existing SAE methods.

Model-Informed Dosing Regimen of Ticagrelor in Chinese Patients With Acute Coronary Syndrome.

The exposure to ticagrelor (BRILINTA) is higher in the East Asian population compared with the White population, thus, East Asians have an increased risk of bleeding. We developed a population pharmacokinetic (PopPK) model of ticagrelor based on a randomized 3 × 3 crossover study in healthy subjects. The area under the concentration-time curve (AUC) of Chinese patients with acute coronary syndrome was simulated based on this model. Following this, eight machine learning (ML) methods were used to construct bleeding risk models. Variables included in the final bleeding risk model were age, hypertension, body weight, AUC, drinking status, calcium channel blockers, antidiabetic medications, β-blockers, peripheral vascular disease, diabetes, transient ischemic attack, sex, and proton pump inhibitor. In terms of F1 scores and area under the curve of receiver operating characteristic curve (ROC-AUC), the Random Forest model performed best among all models, with an F1 score of 0.73 and ROC-AUC of 0.81. Moreover, the PopPK model and ML algorithm were used to bridge the real-world data to build a bleeding risk prediction model based on drug exposure and clinical information. Using this model, a ticagrelor regimen that is associated with a lower risk of bleeding in individuals can be obtained. This model should be further validated prospectively in clinical settings.

Use of Affective Visual Information for Summarization of Human-Centric Videos

The increasing volume of user-generated human-centric video content and its applications, such as video retrieval and browsing, require compact representations addressed by the video summarization literature. Current supervised studies formulate video summarization as a sequence-to-sequence learning problem, and the existing solutions often neglect the surge of the human-centric view, which inherently contains affective content. In this study, we investigate the affective-information enriched supervised video summarization task for human-centric videos. First, we train a visual input-driven state-of-the-art continuous emotion recognition model (CER-NET) on the RECOLA dataset to estimate activation and valence attributes. Then, we integrate the estimated emotional attributes and their high-level embeddings from the CER-NET with the visual information to define the proposed affective video summarization (AVSUM) architectures. In addition, we investigate the use of attention to improve the AVSUM architectures and propose two new architectures based on temporal attention (TA-AVSUM) and spatial attention (SA-AVSUM). We conduct video summarization experiments on the TvSum and COGNIMUSE datasets. The proposed temporal attention-based TA-AVSUM architecture attains competitive video summarization performances with strong improvements for the human-centric videos compared to the state-of-the-art in terms of F-score, self-defined face recall, and rank correlation metrics.

Pre-implementation Method Name Prediction for Object-oriented Programming

Method naming is a challenging development task in object-oriented programming. In recent years, several research efforts have been undertaken to provide automated tool support for assisting developers in this task. In general, literature approaches assume the availability of method implementation to infer its name. Methods, however, are usually named before their implementations. In this work, we fill the gap in the literature about method name prediction by developing an approach that predicts the names of all methods to be implemented within a class. Our work considers the class name as the input: The overall intuition is that classes with semantically similar names tend to provide similar functionalities, and hence similar method names. We first conduct a large-scale empirical analysis on 258K+ classes from real-world projects to validate our hypotheses. Then, we propose a hybrid big code-driven approach, Mario , to predict method names based on the class name: We combine a deep learning model with heuristics summarized from code analysis. Extensive experiments on 22K+ classes yielded promising results: compared to the state-of-the-art code2seq model (which leverages method implementation data), our approach achieves comparable results in terms of F-score at token-level prediction; our approach, additionally, outperforms code2seq in prediction at the name level. We further show that our approach significantly outperforms several other baselines.

Cross-Attention-Guided Feature Alignment Network for Road Crack Detection

Road crack detection is one of the important issues in the field of traffic safety and urban planning. Currently, road damage varies in type and scale, and often has different sizes and depths, making the detection task more challenging. To address this problem, we propose a Cross-Attention-guided Feature Alignment Network (CAFANet) for extracting and integrating multi-scale features of road damage. Firstly, we use a dual-branch visual encoder model with the same structure but different patch sizes (one large patch and one small patch) to extract multi-level damage features. We utilize a Cross-Layer Interaction (CLI) module to establish interaction between the corresponding layers of the two branches, combining their unique feature extraction capability and contextual understanding. Secondly, we employ a Feature Alignment Block (FAB) to align the features from different levels or branches in terms of semantics and spatial aspects, which significantly improves the CAFANet’s perception of the damage regions, reduces background interference, and achieves more precise detection and segmentation of damage. Finally, we adopt multi-layer convolutional segmentation heads to obtain high-resolution feature maps. To validate the effectiveness of our approach, we conduct experiments on the public CRACK500 dataset and compare it with other mainstream methods. Experimental results demonstrate that CAFANet achieves excellent performance in road crack detection tasks, which exhibits significant improvements in terms of F1 score and accuracy, with an F1 score of 73.22% and an accuracy of 96.78%.

Streamflow classification by employing various machine learning models for peninsular Malaysia

Due to excessive streamflow (SF), Peninsular Malaysia has historically experienced floods and droughts. Forecasting streamflow to mitigate municipal and environmental damage is therefore crucial. Streamflow prediction has been extensively demonstrated in the literature to estimate the continuous values of streamflow level. Prediction of continuous values of streamflow is not necessary in several applications and at the same time it is very challenging task because of uncertainty. A streamflow category prediction is more advantageous for addressing the uncertainty in numerical point forecasting, considering that its predictions are linked to a propensity to belong to the pre-defined classes. Here, we formulate streamflow prediction as a time series classification with discrete ranges of values, each representing a class to classify streamflow into five or ten, respectively, using machine learning approaches in various rivers in Malaysia. The findings reveal that several models, specifically LSTM, outperform others in predicting the following n-time steps of streamflow because LSTM is able to learn the mapping between streamflow time series of 2 or 3 days ahead more than support vector machine (SVM) and gradient boosting (GB). LSTM produces higher F1 score in various rivers (by 5% in Johor, 2% in Kelantan and Melaka and Selangor, 4% in Perlis) in 2 days ahead scenario. Furthermore, the ensemble stacking of the SVM and GB achieves high performance in terms of F1 score and quadratic weighted kappa. Ensemble stacking gives 3% higher F1 score in Perak river compared to SVM and gradient boosting.

Comparing Selective Masking Methods for Depression Detection in Social Media

Abstract Identifying those at risk for depression is a crucial issue and social media provides an excellent platform for examining the linguistic patterns of depressed individuals. A significant challenge in depression classification problems is ensuring that prediction models are not overly dependent on topic keywords (i.e., depression keywords) such that it fails to predict when such keywords are unavailable. One promising approach is masking—that is, by selectively masking various words and asking the model to predict the masked words, the model is forced to learn the inherent language patterns of depression. This study evaluates seven masking techniques. Moreover, predicting the masked words during the pre-training or fine-tuning phase was also examined. Last, six class imbalanced ratios were compared to determine the robustness of masked words selection methods. Key findings demonstrate that selective masking outperforms random masking in terms of F1-score. The most accurate and robust models are identified. Our research also indicates that reconstructing the masked words during the pre-training phase is more advantageous than during the fine-tuning phase. Further discussion and implications are discussed. This is the first study to comprehensively compare masked words selection methods, which has broad implications for the field of depression classification and general NLP. Our code can be found at: https://github.com/chanapapan/Depression-Detection.

Performance Evaluation of Machine Learning Nave Bayes Algorithms for Network Traffic Classification

Network Traffic Classification (NTC) is an important field for different network statistics like management, malware detection and other paramount constraints. Artificial Intelligence (AI) including Machine Learning (ML) and Deep Learning (DL), on the other hand, plays a very important field nowadays due to its significant capabilities with an extremely different fields and for complex problems. ML, specifically, provides tools in the most important network fields like traffic management, security, etc. Performance evaluation is a very important aspect of any system. This research paper provides a method for NTC using ML Nave Bayes (NB) algorithm in terms of Bernoulli, Multinominal and Gaussian for classifying captured network traffic in two different datasets and perform a performance evaluation and a comparison among these algorithms. The first dataset is a VPN-nonVPN (ISCXVPN2016) dataset while the second is a packet-captured regular Wi-Fi traffic flow dataset for video browsing on the web. Results were comparable in terms of f1-score, accuracy and processing time. Bernoulli NB provides average 93.05% accuracy with 742 ms, Multinominal NB provides average 98.78% accuracy with 78.3 ms processing time and finally, Gaussian NB provides average 69.14% with 46.85 ms processing time.

Lymphocyte detection for cancer analysis using a novel fusion block based channel boosted CNN

Tumor-infiltrating lymphocytes, specialized immune cells, are considered an important biomarker in cancer analysis. Automated lymphocyte detection is challenging due to its heterogeneous morphology, variable distribution, and presence of artifacts. In this work, we propose a novel Boosted Channels Fusion-based CNN “BCF-Lym-Detector” for lymphocyte detection in multiple cancer histology images. The proposed network initially selects candidate lymphocytic regions at the tissue level and then detects lymphocytes at the cellular level. The proposed “BCF-Lym-Detector” generates diverse boosted channels by utilizing the feature learning capability of different CNN architectures. In this connection, a new adaptive fusion block is developed to combine and select the most relevant lymphocyte-specific features from the generated enriched feature space. Multi-level feature learning is used to retain lymphocytic spatial information and detect lymphocytes with variable appearances. The assessment of the proposed “BCF-Lym-Detector” show substantial improvement in terms of F-score (0.93 and 0.84 on LYSTO and NuClick, respectively), which suggests that the diverse feature extraction and dynamic feature selection enhanced the feature learning capacity of the proposed network. Moreover, the proposed technique’s generalization on unseen test sets with a good recall (0.75) and F-score (0.73) shows its potential use for pathologists’ assistance.

Machine learning models for online detection of wear and friction behaviour of biomedical graded stainless steel 316L under lubricating conditions

Particularly in sectors where mechanisation is increasing, there has been persistent effort to maximise the use of existing assets. Since maintenance management is accountable for the accessibility of assets, it stands to acquire prominence in this setting. One of the most common methods for keeping equipment in good working order is predictive maintenance with machine learning methods. Failures can be spotted before they cause any downtime or extra expenses, and with this aim, the present work deals with the online detection of wear and friction characteristics of stainless steel 316L under lubricating conditions with machine learning models. Wear rate and friction forces were taken into account as reaction parameters, and biomedical-graded stainless steel 316L was chosen as the work material. With more testing, the J48 method’s accuracy improves to 100% in low wear conditions and 99.27% in heavy wear situations. In addition, the graphic showed the accuracy values for several models. The J48 model is the most precise amongst all others, with a value of 100% (minimum wear) and an average of 98.92% (higher wear). Amongst all the models tested under varying machining conditions, the J48’s 98.92% (low wear) and 98.92% (high wear) recall scores stand out as very impressive (higher wear). In terms of F1-score, J48 performs better than any competing model at 99.45% (low wear) and 98.92% (higher wear). As a result, the J48 improves the model’s overall performance.

FNReq-Net: A hybrid computational framework for functional and non-functional requirements classification

Requirements classification is a key component of software development life cycle. It enhances our understanding about project requirements, which in turn enables us to effectively identify and mitigate risks that could lead to project failure. Existing requirements classification predictors do not utilize feature selection methods competence in their predictive pipelines and lack in performance. To empower the process of automatic requirements classification, contributions of this paper are manifold. Firstly, it explores the potential of 7 filter-based feature selection techniques and 11 traditional machine learning classifiers. Secondly, for the first time it investigates combined potential of traditional feature selection and 9 diverse types of deep learning predictors. Thirdly, it presents a hybrid computational predictor namely FNReq-Net that reaps combine benefits of traditional feature selection and a novel deep learning predictor based on attention mechanism. Over two public benchmark datasets, large-scale experimental results reveal feature selection not only improves predictive performance of traditional machine learning predictors, but it also improves performance of deep learning predictors. The proposed FNReq-Net predictor outperforms state-of-the-art functional and non-functional requirements classification predictors by 4% and 1% in terms of F1-score over Promise and Promise-exp datasets, respectively.

A freshwater algae classification system based on machine learning with StyleGAN2-ADA augmentation for limited and imbalanced datasets

Automated algae classification using machine learning is a more efficient and effective solution compared to manual classification, which can be tedious and time-consuming. However, the practical application of such a classification approach is restricted by the scarcity of labeled freshwater algae datasets, especially for rarer algae. To overcome these challenges, this study proposes to generate artificial algal images with StyleGAN2-ADA and use both the generated and real images to train machine-learning-driven algae classification models. This approach significantly enhances the performance of classification models, particularly in their ability to identify rare algae. Overall, the proposed approach improves the F1-score of lightweight MobileNetV3 classification models covering all 20 freshwater algae covered in this research from 88.4% to 96.2%, while for the models that cover only the rarer algae, the experiments show an improvement from 80% to 96.5% in terms of F1-score. The results show that the approach enables the trained algae classification systems to effectively cover algae with limited image data.

Prediction Model of Word EERIE Based on BP Neural Network

This paper uses BP neural network algorithm to determine the difficulty level of the word EERIE by constructing a neural network model, and determine its relationship with the word length, word commonness, letter frequency and other attributes to predict the relevant percentage of the future date, and discusses the uncertainty of the model, the accuracy of the classification model and other characteristics of the data set. In terms of accuracy, the random forest algorithm is the best, 72%, and the k-nearest neighbor algorithm is the worst. In terms of F1 score, logistic regression algorithm performs the best conclusion.

Dualistic cascade convolutional neural network dedicated to fully PolSAR image ship detection

Influenced by the imaging mechanism, the occurrence of interference clutter in synthetic aperture radar (SAR) renders the identification of false alarms using detectors challenging. Polarimetric SAR has the potential to improve ship detection performance owing to its distinctive polarization characteristics. The present study proposes a dualistic cascade convolutional neural network (DCCNN) algorithm driven by polarization characteristics for ship detection with fully PolSAR data. First, the new characterizations of fully PolSAR data—Optimized SPAN (OSPAN) and 6-D polarization vector (P6), were mined and defined based on the polarization coherence matrix to introduce more intact information of targets. Then, a backbone feature extraction network with parallel dualistic cascade architecture, basic geometric feature extraction network (BGFENet), and polarization feature enhancement network (PFENet) was specifically constructed, which provided the comprehensive feature representation of targets via feature fusion. Finally, the classification and regression tasks were accomplished in the fully convolutional detection subnetwork relying on extracted multi-scale fusion feature maps, while focusing on target location regression, leading to a decrease in the cost of detection efficiency caused by redundant PFENet. In addition, the corresponding training strategy according to the special architecture of DCCNN was designed to overcome the problem of labeled fully PolSAR data insufficiency and migrate refined geometric knowledge of targets in the single-polarization SAR amplitude image. Experimental results on the established fully polarized SAR dataset show that DCCNN outperforms the competitive CNN-based target detection methods by at least 6.79% in terms of average precision. Moreover, experimental results on the typical large scenes show that DCCNN is at least 0.88% higher than the well-known conventional methods in terms of F1 score.

CAVIDS: Real time intrusion detection system for connected autonomous vehicles using logical analysis of data

In recent times, Connected Autonomous Vehicles (CAVs) have been gaining popularity and have already begun to revolutionize the transport industry, mainly using real-time monitoring, analysis and exchange of data through In-Vehicle Networks (IVN). IVN can also interact with the outside world and roadside infrastructures. With all these advantages, certain weaknesses also creep in as the communication in IVN lacks proper security measures like encryption and authentication. These lack of security measures in IVN makes CAVs vulnerable to cyber-attacks which may have fatal consequences. The de facto standard for communication in IVN is known as Controller Area Network or CAN, and it also lacks said security features to protect CAN from outside threats like cyber attacks, sensor tempering etc. Real-time intrusion detection with minimal computational resources is the need of the hour to prevent disruptive consequences and fatal crashes. In this paper, we propose an intrusion detection method based on Logical Analysis of Data (LAD) which facilitates real-time intrusion detection in CAVs using minimal computational resources. LAD is a two-class classification technique which uses the concept of a partially defined Boolean function (pdBf) to extract different rules for classification from a historical dataset consisting of CAN messages only. Usually, the normal behaviour of any CAV is specified using the extracted rules, and any unusual behaviour that does not conform with the extracted rules is treated as an anomaly. In this paper, we have extracted rules for anomaly and treated all behaviours that do not conform with these extracted rules as normal. The proposed method outperforms many state-of-the-art approaches, which include other rule-based methods, machine learning and deep learning methods both in terms of F1 score and detection time. For CarChallenge 2020 dataset, the proposed method requires lesser than 47.5 μs for detection using a Raspberry Pi device, which is far less than the average time difference between two consecutive CAN messages and thus can be considered as real-time intrusion detection. Real-time intrusion detection is important in preventing any untoward incidents in CAVs.

Machine Learning Classification–Regression Schemes for Desert Locust Presence Prediction in Western Africa

For decades, humans have been confronted with numerous pest species, with the desert locust being one of the most damaging and having the greatest socio-economic impact. Trying to predict the occurrence of such pests is often complicated by the small number of records and observations in databases. This paper proposes a methodology based on a combination of classification and regression techniques to address not only the problem of locust sightings prediction, but also the number of locust individuals that may be expected. For this purpose, we apply different machine learning (ML) and related techniques, such as linear regression, Support Vector Machines, decision trees, random forests and neural networks. The considered ML algorithms are evaluated in three different scenarios in Western Africa, mainly Mauritania, and for the elaboration of the forecasting process, a number of meteorological variables obtained from the ERA5 reanalysis data are used as input variables for the classification–regression machines. The results obtained show good performance in terms of classification (appearance or not of desert locust), and acceptable regression results in terms of predicting the number of locusts, a harder problem due to the small number of samples available. We observed that the RF algorithm exhibited exceptional performance in the classification task (presence/absence) and achieved noteworthy results in regression (number of sightings), being the most effective machine learning algorithm among those used. It achieved classification results, in terms of F-score, around the value of 0.9 for the proposed Scenario 1.

Hybrid Defect Prediction Model Based on Counterfactual Feature Optimization

Software defect prediction is critical to ensuring software quality. Researchers have worked on building various defect prediction models to improve the performance of defect prediction. Existing defect prediction models are mainly divided into two categories: models constructed based on artificial statistical features and models constructed based on semantic features. DP-CNN [Li J, He P, Zhu J, et al. Software defect prediction via convolutional neural network. In: 2017 IEEE international conference on software quality, reliability and security (QRS). IEEE, 2017; 318–328.] is one of the best defect prediction models, because it combines both artificial statistical features and semantic features, so its performance is greatly improved compared to traditional defect prediction models. This paper is based on the DP-CNN model and makes the following two improvements: first, using a new Struc2vec network representation technique to mine existing information between software modules, which specializes in learning node representations from structural identity and can further extract structural features associated with defects. Let the DP-CNN model once again incorporate the newly mined structural features. Then, this paper proposes a feature selection method based on counterfactual explanations, which can determine the importance score of each feature by the feature change rate of counterfactual samples. The origin of these feature importance scores is interpretable. Under the guidance of these interpretable feature importance scores, better feature subsets can be obtained and used to optimize artificial statistical features within the DP-CNN model. Based on the above methods, this paper proposes a new hybrid defect prediction model DPS-CNN-STR. Evaluating our model on six open source projects in terms of F1 score in defect prediction. Experimental results show that DPS-CNN-STR improves the state-of-the-art method by an average of 3.3%.

Expert diagnostic system for detection of hypertension and diabetes mellitus using discrete wavelet decomposition of photoplethysmogram signal and machine learning technique

Hypertension is a leading risk factor for cardiovascular disease (CVD) and the overlap of diabetes mellitus (DM) with hypertension can lead to severe complications. There is a need for early diagnosis and risk stratification to implement an overall risk management strategy. Presently, the conventional method is not suitable for large-scale screening. The primary aim of this study is to develop an automated diagnostic system that uses Photoplethysmogram (PPG) signals for the non-invasive diagnosis of hypertension and DM-II. The proposed model uses a statistical feature extracted by decomposing the PPG signal up to level 11 into a sub-band using Discrete wavelet transform (DWT), and a variety of classifiers are used for the classification of hypertension and detection of DM-II patients. Three feature selection techniques used are Spearman correlation, ReliefF and Minimum Redundancy Maximum Relevance (mRMR) to select 20 top features out of 130 features using correlation with systole blood pressure (SBP), diastole blood pressure (DBP) values and D-II. The highest accuracy attained by the Adaptive neural fuzzy system (ANFIS) for classification categories such as normal (NT) vs prehypertension(PHT), NT vs. hypertension type 1 (HT-I), NT vs hypertension type 2 (HT-II) in terms of F1 score are 92.%, 98.5%, 98.3% (SBP) and 83.1%, 95.6%, 86.8% (DBP),respectively. The accuracy achieved by the adaptive-network-based fuzzy inference system (ANFIS) for the classification of normal (non-diabetic) vs. diabetic patients is 99.1%. The hybrid learning algorithm-based classifier achieved higher accuracy for hypertension risk stratification as compared to the hard computing classifier, which requires parameter tuning and DWT decomposition is robust to the noisy signal, overcoming the limitation of the morphological feature-based model.

EQUI-VOCAL: Synthesizing Queries for Compositional Video Events from Limited User Interactions

We introduce EQUI-VOCAL: a new system that automatically synthesizes queries over videos from limited user interactions. The user only provides a handful of positive and negative examples of what they are looking for. EQUI-VOCAL utilizes these initial examples and additional ones collected through active learning to efficiently synthesize complex user queries. Our approach enables users to find events without database expertise, with limited labeling effort, and without declarative specifications or sketches. Core to EQUI-VOCAL's design is the use of spatio-temporal scene graphs in its data model and query language and a novel query synthesis approach that works on large and noisy video data. Our system outperforms two baseline systems---in terms of F1 score, synthesis time, and robustness to noise---and can flexibly synthesize complex queries that the baselines do not support.