Performance Evaluation Metrics Research Articles

Numerical optimization of drug solubility inside the supercritical carbon dioxide system using different machine learning models

This research comprehensively investigates the solubility characteristics of five distinct drugs including: Nystatin, Niflumic acid, Tolfenamic acid, Glibenclamide, and Rivaroxaban, across a range of pressure (P) and temperature (T) conditions. The solubility is computed in supercritical carbon dioxide as the solvent. It was aimed to build a holistic view of solubility estimation using machine learning technique. To predict drug solubility accurately, three regression models— K-Nearest Neighbors (KNN), Multilayer Perceptron (MLP), and Polynomial Regression (PR)—were employed, with hyperparameter optimization conducted using the Harmony Search (HS) algorithm. Performance evaluation metrics, including R-squared (R2) scores, Root Mean Square Error (RMSE), and Maximum Error, were employed to assess model effectiveness. Notably, HS-PR emerged as the top-performing model, achieving an impressive score of 0.96449 in terms of R2 metric, highlighting its proficiency in modeling drug solubility under varying conditions.

AI and Blockchain-Based Secure Data Dissemination Architecture for IoT-Enabled Critical Infrastructure.

The Internet of Things (IoT) is the most abundant technology in the fields of manufacturing, automation, transportation, robotics, and agriculture, utilizing the IoT's sensors-sensing capability. It plays a vital role in digital transformation and smart revolutions in critical infrastructure environments. However, handling heterogeneous data from different IoT devices is challenging from the perspective of security and privacy issues. The attacker targets the sensor communication between two IoT devices to jeopardize the regular operations of IoT-based critical infrastructure. In this paper, we propose an artificial intelligence (AI) and blockchain-driven secure data dissemination architecture to deal with critical infrastructure security and privacy issues. First, we reduced dimensionality using principal component analysis (PCA) and explainable AI (XAI) approaches. Furthermore, we applied different AI classifiers such as random forest (RF), decision tree (DT), support vector machine (SVM), perceptron, and Gaussian Naive Bayes (GaussianNB) that classify the data, i.e., malicious or non-malicious. Furthermore, we employ an interplanetary file system (IPFS)-driven blockchain network that offers security to the non-malicious data. In addition, to strengthen the security of AI classifiers, we analyze data poisoning attacks on the dataset that manipulate sensitive data and mislead the classifier, resulting in inaccurate results from the classifiers. To overcome this issue, we provide an anomaly detection approach that identifies malicious instances and removes the poisoned data from the dataset. The proposed architecture is evaluated using performance evaluation metrics such as accuracy, precision, recall, F1 score, and receiver operating characteristic curve (ROC curve). The findings show that the RF classifier transcends other AI classifiers in terms of accuracy, i.e., 98.46%.

Addressing agricultural challenges: An identification of best feature selection technique for dragon fruit disease recognition

Dragon fruit is a prominent substance in global agriculture. Despite this, it is gaining popularity and is a viable solution in resource-poor, environmentally degraded areas because of its many health benefits. Nevertheless, many dragon fruit plantations have been impacted by the disease, reducing their yield, and the detection system is still conventional. Farmers’ lack of disease identification and management expertise diminished crop quality and products. As a result, little research was carried out to assist those specific farmers requiring adequate agricultural support. This research has proposed an autonomous agro-based system to recognize dragon diseases using in-depth analysis of feature selection techniques. After the collection of real-time images of the dragon, the images are preprocessed using various image-processing techniques. The two important features are retrieved after segmentation. The analysis of variance (ANOVA) and the least absolute shrinkage and selection operator (LASSO) are used as feature selection techniques to assess the feature rank based on the mutual score. To analyze the effectiveness of the machine learning algorithms that were used, six distinct machine learning classifiers were applied to the top-ranked feature sets, and their performance was measured using seven distinct performance evaluation metrics. AdaBoost and Random Forest classifiers for the LASSO feature ranking approach got the maximum accuracy, which is 96.29%, based on a comparison of classifiers based on the ANOVA and LASSO feature set. Despite this, we have optimized the computational resources of each classifier for the LASSO feature set.

Intelligent Systems in Motion

Simultaneous localization and mapping (SLAM) serves as a cornerstone in autonomous systems and has seen exponential growth in its roles, particularly in facilitating advanced path planning solutions. One emerging avenue of research that is rapidly evolving is the incorporation of multi-sensor fusion techniques to enhance SLAM-based path planning. The paper initiates with a thorough review of various sensor types and their attributes before covering a broad spectrum of both traditional and contemporary algorithms for multi-sensor fusion within SLAM. Performance evaluation metrics pertinent to SLAM and sensor fusion are explored. A special focus is laid on the interconnected roles and applications of multi-sensor fusion in SLAM-based path planning, discussing its significance in navigation scenarios as well as addressing challenges such as computational burden and real-time implementation. This paper sets the stage for future developments in creating more robust, resilient, and efficient SLAM-based path planning systems enabled by multi-sensor fusion.

Ensemble Learning Based Malicious Node Detection in SDN-Based VANETs

Background: The architecture of Software Defined Networking (SDN) integrated with Vehicular Ad-hoc Networks (VANETs) is considered a practical method for handling large-scale, dynamic, heterogeneous vehicular networks, since it offers flexibility, programmability, scalability, and a global understanding. However, the integration with VANETs introduces additional security vulnerabilities due to the deployment of a logically centralized control mechanism. These security attacks are classified as internal and external based on the nature of the attacker. The method adopted in this work facilitated the detection of internal position falsification attacks. Objective: This study aimed to investigate the performance of k-NN, SVM, Naïve Bayes, Logistic Regression, and Random Forest machine learning (ML) algorithms in detecting position falsification attacks using the Vehicular Reference Misbehavior (VeReMi) dataset. It also aimed to conduct a comparative analysis of two ensemble classification models, namely voting and stacking for final decision-making. These ensemble classification methods used the ML algorithms cooperatively to achieve improved classification. Methods: The simulations and evaluations were conducted using the Python programming language. VeReMi dataset was selected since it was an application-specific dataset for VANETs environment. Performance evaluation metrics, such as accuracy, precision, recall, F-measure, and prediction time were also used in the comparative studies. Results: This experimental study showed that Random Forest ML algorithm provided the best performance in detecting attacks among the ML algorithms. Voting and stacking were both used to enhance classification accuracy and reduce time required to identify an attack through predictions generated by k-NN, SVM, Naïve Bayes, Logistic Regression, and Random Forest classifiers. Conclusion: In terms of attack detection accuracy, both methods (voting and stacking) achieved the same level of accuracy as Random Forest. However, the detection of attack using stacking could be achieved in roughly less than half the time required by voting ensemble. Keywords: Machine learning methods, Majority voting ensemble, SDN-based VANETs, Security attacks, Stacking ensemble classifiers, VANETs,

The Evaluation of Logistics Enterprise Performance Index Based on TOPSIS-Grey Relational Analysis

Performance assessment is a pivotal facet within the operational framework of logistics enterprises, functioning as a mechanism to gauge business outcomes and growth potential. For this study, the authors developed a performance evaluation system for logistics enterprises under the paradigm of sustainable development to reveal the fissures and quandaries within the operational milieu by scrutinizing the current state of logistics enterprises. Drawing on pertinent references and empirical inquiries, they employed the entropy weight method to allocate weights to the performance evaluation metrics of logistics enterprises and TOPSIS–grey relational analysis method to comprehensively assess the performance of such enterprises. Empirical findings show that during the period of 2016–2018, a majority of the sampled logistics enterprises demonstrated an ascending trajectory in their comprehensive proximity, and a minority exhibited fluctuating and descending trends. These findings suggest the favorable trajectory of the evolution of logistics enterprises.

Cancer Metastasis Prediction and Genomic Biomarker Identification through Machine Learning and eXplainable Artificial Intelligence in Breast Cancer Research.

Method: This research presents a model combining machine learning (ML) techniques and eXplainable artificial intelligence (XAI) to predict breast cancer (BC) metastasis and reveal important genomic biomarkers in metastasis patients. A total of 98 primary BC samples was analyzed, comprising 34 samples from patients who developed distant metastases within a 5-year follow-up period and 44 samples from patients who remained disease-free for at least 5 years after diagnosis. Genomic data were then subjected to biostatistical analysis, followed by the application of the elastic net feature selection method. This technique identified a restricted number of genomic biomarkers associated with BC metastasis. A light gradient boosting machine (LightGBM), categorical boosting (CatBoost), Extreme Gradient Boosting (XGBoost), Gradient Boosting Trees (GBT), and Ada boosting (AdaBoost) algorithms were utilized for prediction. To assess the models' predictive abilities, the accuracy, F1 score, precision, recall, area under the ROC curve (AUC), and Brier score were calculated as performance evaluation metrics. To promote interpretability and overcome the "black box" problem of ML models, a SHapley Additive exPlanations (SHAP) method was employed. The LightGBM model outperformed other models, yielding remarkable accuracy of 96% and an AUC of 99.3%. In addition to biostatistical evaluation, in XAI-based SHAP results, increased expression levels of TSPYL5, ATP5E, CA9, NUP210, SLC37A1, ARIH1, PSMD7, UBQLN1, PRAME, and UBE2T (p ≤ 0.05) were found to be associated with an increased incidence of BC metastasis. Finally, decreased levels of expression of CACTIN, TGFB3, SCUBE2, ARL4D, OR1F1, ALDH4A1, PHF1, and CROCC (p ≤ 0.05) genes were also determined to increase the risk of metastasis in BC. The findings of this study may prevent disease progression and metastases and potentially improve clinical outcomes by recommending customized treatment approaches for BC patients.

Lightweight and smart data fusion approaches for wearable devices of the Internet of Medical Things

Internet of Medical Things (IoMT) refers to a group of dedicated devices, i.e., wearable sensors attached to the human body for capturing the real-time data, specifically physiological signals, to develop an automated and technology-assisted smart healthcare framework. However, due to the miniaturized nature and their limited capabilities, data captured by these wearable devices need to be consistent, especially in terms of accuracy and precision. Therefore, data must be refined prior to the actual processing, which is a challenging task in a smart healthcare infrastructure. In the literature, various approaches have been reported to address this issue. However, most of these approaches are either domain-specific or extremely difficult to implement, especially in the resource-constrained environments of IoMT. In this paper, we present two-tier lightweight device and server-enabled data fusion approaches for wearable devices and their server modules, respectively. In these approaches, each device is bound to perform an in-node processing, i.e., local data fusion, on the sensed data values prior to initialization of the transmission process. For this purpose, each wearable device is required to use knowledge of the previously transmitted refined data values. Additionally, the proposed approach is pruned against abrupt changing scenarios, e.g. heart attack, where the frequency of sensed data values drastically fluctuates. Moreover, a non-metric-based data fusion, i.e., global data fusion, is presented to enhance the accuracy of sensed data by removing redundant data values, especially those collected by neighboring devices. Simulation results reflect the exceptional performance of these smart data fusion approaches, i.e., both local and global, in the context of numerous performance evaluation metrics, such as accuracy and precision ratio.

Construction of a linguistically interactive thesaurus for English second language acquisition based on an eigenvalue-fitting superiority algorithm

Abstract This paper completes the overall design of a linguistic interactive terminology database based on the characteristics of second language acquisition and terminology and completes the construction of the terminology database by combining a goodness-of-fit detection algorithm based on terminology eigenvalue extraction. The efficiency of terminology information recognition is analyzed and compared with the terminology conversion rate of the eigenvalue goodness-offit algorithm using a neural network learning model of long and short-term memory to optimize the performance of the terminology database. The metric approach's classifier performance evaluation metrics are used to compare the accuracy and recall of the two algorithms accurately. The results show that the accuracy of the fitted superiority classifier with the application of word eigenvalue embedding compared to the LSTM classifier for the classification of electric power terms is improved by about 11% in all categories, and the average accuracy of the classifier exceeds 76.5%.

A novel crude oil futures trading strategy based on volume-price time-frequency decomposition with ensemble deep reinforcement learning

With the deepening financialization of crude oil, an increasing number of investors are becoming involved in trading crude oil futures. However, various factors influence crude oil futures price fluctuations, presenting complex characteristics. For this reason, a trading strategy plays a crucial role in investment. In this paper, we construct an ensemble trading strategy by combining time-frequency feature extraction of crude oil volume-price series with deep reinforcement learning. We decompose the original oil volume-price series by using the optimized variational mode decomposition. Additionally, taking into account the long memory characteristics of the crude oil time series, we employ the Sharpe ratio to filter the appropriate agents corresponding to market volatility. Moreover, the best features of the three agents are effectively combined and used in the next phase of trading, thus robustly adapting to different market volatility situations. Finally, we tested our strategy using Brent crude oil futures. Compared with other benchmark models, the proposed OVMD(V–P)-DRL-Ensemble strategy is suitable for the highly volatile crude oil market and performs well in terms of the corresponding performance evaluation metrics, showing excellent return performance and stability.

A prediction model for neonatal necrotizing enterocolitis in preterm and very low birth weight infants.

Neonatal necrotizing enterocolitis (NEC) is a severe gastrointestinal disease that primarily affects preterm and very low birth weight infants, with high morbidity and mortality. We aim to build a reliable prediction model to predict the risk of NEC in preterm and very low birth weight infants. We conducted a retrospective analysis of medical data from infants (gestational age <32 weeks, birth weight <1,500 g) admitted to Maternal and Child Health Hospital of Guangxi Zhuang Autonomous Region. We collected clinical data, randomly dividing it into an 8:2 ratio for training and testing. Multivariate logistic regression was employed to identify significant predictors for NEC. Principal component analysis was used for dimensionality reduction of numerical variables. The prediction model was constructed through logistic regression, incorporating all relevant variables. Subsequently, we calculated performance evaluation metrics, including Receiver Operating Characteristic (ROC) curves and confusion matrices. Additionally, we conducted model performance comparisons with common machine learning models to establish its superiority. A total of 292 infants were included, with 20% (n = 58) randomly selected for external validation. Multivariate logistic regression revealed the significance of four predictors for NEC in preterm and very low birth weight infants: temperature (P = 0.003), Apgar score at 5 min (P = 0.004), formula feeding (P = 0.007), and gestational diabetes mellitus (GDM, P = 0.033). The model achieved an accuracy of 82.46% in the test set with an F1 score of 0.90, outperforming other machine learning models (support vector machine, random forest). Our logistic regression model effectively predicts NEC risk in preterm and very low birth weight infants, as confirmed by external validation. Key predictors include temperature, Apgar score at 5 min, formula feeding, and GDM. This study provides a vital tool for NEC risk assessment in this population, potentially improving early interventions and child survival. However, clinical validation and further research are necessary for practical application.

BCI-AMSH: A MATLAB based open-source brain computer interface assistive application for mental stress healing

Numerous Brain computer interface (BCI) assistive software applications or toolboxes help to monitor the status of brain through Electroencephalogram (EEG); however, currently there is no application available to alleviate mental stress along with EEG analysis. In this paper, a new MATLAB-based assistive application called BCI-AMSH (Brain-Computer Interface-Assistive Mental Stress Healing) is developed. This application includes EEG signal offline analysis and stress healing techniques, such as guided meditation and singing bowl sound therapy, combined with real-time EEG analysis using the Enobio-8 device. The EEG offline analysis and stress healing modules are designed to be platform-independent.EEG offline analysis module offers signal preprocessing methods, diverse EEG signal analysis techniques, feature extraction methods, such as time and frequency domain analysis, and a range of machine learning and deep learning models for classification, various performance evaluation metrics. Additionally, it generates comprehensive EEG report about the user's mental states such as focus, relax and stress both in real-time and offline analysis. Furthermore, the stress healing module provides a variety of guided breathing practices and singing bowl sound therapy practices to effectively reduce stress levels. This new application is designed to be very simple and user-friendly, making it accessible to users without prior experience. Its potential applications are significant, particularly in the clinical industry for mental health purposes, as well as for novice researchers seeking to explore new possibilities in BCI.

SKEDS — An external knowledge supported logistic regression approach for document-level sentiment classification

Due to the enormous amount of user-generated content being generated on the web, labeling such data is a time-consuming and expensive endeavor. As a result, we have limited annotated data and the vast majority of data are unlabeled. Analysis reveals that extracting (external) knowledge from unlabeled data and integrating it with knowledge extracted from labeled data is a beneficial task for text information processing, in particular text classification. In this paper, we present a hybrid approach for classifying sentiments that employs external knowledge, which is categorized as either general-purpose sentiment knowledge or domain-related knowledge. General-purpose sentiment knowledge is extracted from sentiment lexicons, whereas domain-related knowledge is extracted from unlabeled data from the same or related domains. Similar domains for a given domain are identified based on their similarity score in terms of overlapping features. The proposed approach utilizes both forms of external knowledge and combines them with logistic regression to train an improved classification model. The classification model uses the conventional gradient descent algorithm for optimization, and its convergence analysis indicates that it is convex and converges to the global optimum. The proposed approach is empirically evaluated and compared to three baselines and one state-of-the-art method using standard performance evaluation metrics on a multi-domain sentiment dataset. The experiment results are encouraging, demonstrating that the proposed approach considerably outperforms the baseline approaches and outperforms the state-of-the-art approach by up to 2% in terms of both f-score and accuracy.

Predicting graduation grades using Machine Learning: A case study of Can Tho University students

A number of factors influence a student's attainment of graduation. Besides scholastic performance within the academic curriculum, other variables such as living circumstances, gender, and choice of major significantly contribute to the probability of achieving graduation. The capacity to forecast academic performance at the time of graduation holds profound importance for universities, especially in discerning the influential factors that contribute to a student's successful completion of their educational pursuits. This study employs multiple machine learning algorithms, including K-nearest neighbor, Neural network, Decision tree, Random forest, and Gradient boosting, to prognosticate the graduation outcomes of 7,837 undergraduate students from Can Tho University during the academic year 2022. These selected students were enrolled in 16 colleges and institutes affiliated with Can Tho University. The efficacy of the employed algorithms was assessed through performance evaluation metrics encompassing accuracy, precision, recall, and F-measure. Furthermore, a 15-fold cross-validation technique was employed for validation. The findings revealed that the Random forest model yielded the most reliable predictions. The factors that significantly impact graduation grades comprise GPA, training point, residential address, college, major, and gender. Based on the experimental findings, these factors were ranked to ascertain their effects on student graduation.

The evidence on target-date mutual funds

This paper assimilates the knowledge and evidence on target-date mutual funds (TDFs). It begins with a discussion of the environment that contributes to the tremendous growth of TDFs. Next, a survey of the theory and recommendations on glide paths indicates a trend towards focusing on meeting retirement liabilities, rather than optimizing asset only portfolios. A review of performance evaluation metrics for TDFs shows that none of the available indexes possesses all seven characteristics of an ideal benchmark. Plan sponsors can provide better outcomes by offering multiple risk profile TDFs while researchers can focus on improving glide path and benchmark design.

Deep learning in news recommender systems: A comprehensive survey, challenges and future trends

Nowadays, people prefer to read news articles from online sources worldwide due to their easiness and availability. For the last few years, online searching for required information or content has been replaced by item recommendation, and news recommendation is also not an exception. For news recommendations, News Recommender System (NRS) helps the users to find the appropriate and pertinent content, alleviate the problem of information overload, and propose news that would be of interest to news readers. NRS also assists different users all around the world in this regard by recommending the most recent news articles based on their interests and past preferences. Many techniques such as traditional, Deep Learning (DL), and hybrid have been proposed to solve the NRS challenges and issues. DL techniques are considered one of the best techniques and have been successfully applied in various fields such as Natural Language Processing (NLP) and Computer Vision (CV). This survey article provides a detailed analysis of DL models-based techniques to build NRS. In this regard, firstly, a comprehensive comparison is provided between published survey articles on NRS and this research work. Secondly, it discusses the background of recommendation systems and their techniques. Furthermore, NRS is explored along with its current research challenges. Then background knowledge of DL and its methods have been discussed along with the analysis of year-wise published relevant articles having DL as the applied technique. The survey also presents widely used datasets and performance evaluation metrics used in the relevant literature. Finally, a detailed discussion provides several future directions and open research challenges for the researchers to consider DL applications in NRS.

Dynamically Synthetic Images for Federated Learning of medical images

BackgroundTo develop deep learning models for medical diagnosis, it is important to collect more medical data from several medical institutions. Due to the regulations for privacy concerns, it is infeasible to collect data from various medical institutions to one institution for centralized learning. Federated Learning (FL) provides a feasible approach to jointly train the deep learning model with data stored in various medical institutions instead of collected together. However, the resulting FL models could be biased towards institutions with larger training datasets. MethodologyIn this study, we propose the applicable method of Dynamically Synthetic Images for Federated Learning (DSIFL) that aims to integrate the information of local institutions with heterogeneous types of data. The main technique of DSIFL is to develop a synthetic method that can dynamically adjust the number of synthetic images similar to local data that are misclassified by the current model. The resulting global model can handle the diversity in heterogeneous types of data collected in local medical institutions by including the training of synthetic images similar to misclassified cases in local collections. ResultsIn model performance evaluation metrics, we focus on the accuracy of each client's dataset. Finally, the accuracy of the model of DSIFL in the experiments can achieve the higher accuracy of the FL approach. ConclusionIn this study, we propose the framework of DSIFL that achieves improvements over the conventional FL approach. We conduct empirical studies with two kinds of medical images. We compare the performance by variants of FL vs. DSIFL approaches. The performance by individual training is used as the baseline, whereas the performance by centralized learning is used as the target for the comparison studies. The empirical findings suggest that the DSIFL has improved performance over the FL via the technique of dynamically synthetic images in training.

SQL injection attacks

SQL Injection (SQLi) attacks continue to pose significant threats to modern web applications, compromising data integrity and confidentiality. This research delves into the development and evaluation of methodologies designed to detect and mitigate these malicious attacks. Employing a diverse set of web applications, the study unfolds in a controlled environment, simulating real-world conditions to assess the effectiveness of current defense mechanisms against SQLi. Building upon this baseline, the research introduces a two-pronged defense mechanism: a Static Analysis Tool to pre-emptively identify vulnerabilities in application code and a Runtime Query Sanitizer that employs rule-based patterns and machine learning models to scrutinize and sanitize SQL queries in real-time. Performance evaluation metrics, encompassing detection rate, false positives, response time, and machine learning efficiency, are meticulously documented. Further robustness of these mechanisms is ascertained through real-world simulations involving unsuspecting users and ethical hackers. Initial results indicate promising potential for the introduced methodologies in safeguarding web applications against SQLi attacks. The study's findings serve as a critical step towards fortifying web applications, emphasizing the amalgamation of static analysis and real-time query sanitization as an effective countermeasure against SQLi threats.

Hierarchical Computational Screening of Quantum Metal-Organic Framework Database to Identify Metal-Organic Frameworks for Volatile Organic-Compound Capture from Air.

The design and discovery of novel porous materials that can efficiently capture volatile organic compounds (VOCs) from air are critical to address one of the most important challenges of our world, air pollution. In this work, we studied a recently introduced metal-organic framework (MOF) database, namely, quantum MOF (QMOF) database, to unlock the potential of both experimentally synthesized and hypothetically generated structures for adsorption-based n-butane (C4H10) capture from air. Configurational Bias Monte Carlo (CBMC) simulations were used to study the adsorption of a quaternary gas mixture of N2, O2, Ar, and C4H10 in QMOFs for two different processes, pressure swing adsorption (PSA) and vacuum-swing adsorption (VSA). Several adsorbent performance evaluation metrics, such as C4H10 selectivity, working capacity, the adsorbent performance score, and percent regenerability, were used to identify the best adsorbent candidates, which were then further studied by molecular simulations for C4H10 capture from a more realistic seven-component air mixture consisting of N2, O2, Ar, C4H10, C3H8, C3H6, and C2H6. Results showed that the top five QMOFs have C4H10 selectivities between 6.3 × 103 and 9 × 103 (3.8 × 103 and 5 × 103) at 1 bar (10 bar). Detailed analysis of the structure-performance relations showed that low/mediocre porosity (0.4-0.6) and narrow pore sizes (6-9 Å) of QMOFs lead to high C4H10 selectivities. Radial distribution function analyses of the top materials revealed that C4H10 molecules tend to confine close to the organic parts of MOFs. Our results provided the first information in the literature about the VOC capture potential of a large variety and number of MOFs, which will be useful to direct the experimental efforts to the most promising adsorbent materials for C4H10 capture from air.

Technical performance metrics of a security operations center

This research introduces a novel framework for creating metrics intended for security operations centers (SOCs). The framework is developed using the design science research methodology and has been validated by generating four novel metrics to assess the technical performance of a SOC. Additionally, the study examines the existing landscape of metrics for SOCs and concludes that a majority of the metrics discussed in the literature primarily focus on operational aspects rather than technical performance. The absence of adequate technical performance metrics makes it challenging to accurately evaluate the tangible impact of a SOC on overall cyber defense capabilities. The research also highlights the insufficiency of current methods in constructing metrics and frameworks tailored for measuring SOCs' technical performance. The resulting framework offers SOCs means to create high-quality metrics for performance evaluation. Furthermore, the metrics the framework was validated with offer SOCs an opportunity to enhance their ability to quantify their threat detection capabilities.