Advanced Machine Learning Methodologies Research Articles

Predicting and Monitoring Anxiety and Depression: Advanced Machine Learning Techniques for Mental Health Analysis

Anxiety and depression are considered among the most prevailing mental illnesses; they affect millions in the USA and worldwide. Besides being highly prevalent, these conditions have major implications for individuals and American society as a whole. The prime objective of this research project was to design and evaluate advanced machine learning methodologies for the monitoring and prediction of anxiety and depression. The rise in recent advances in Machine Learning and AI technologies has unleashed tremendous potential in the diagnosis and monitoring of mental health conditions such as anxiety and depression. Predictive models, powered by Machine Learning algorithms, process vast amounts of data and detect patterns that might have evaded human clinicians. This dataset for the current research project was retrieved from the website kaggle.com and shared publicly with anyone by the Harvard Data Verse repository. The dataset contained behavioral, psychophysiological, and demographic data that were collected from 593 participants aged 18-35 years for the prediction of anxiety and depression disorder risk. For this study, three machine learning algorithms were deployed: Logistic Regression, XG-Boost, and Random Forest. To assess and evaluate the performance of the algorithms, two key performance evaluation metrics were utilized MSE & R-squared. By reviewing the performance of the aforementioned three machine learning models, Linear Regression, Random Forest Regressor, and XG-Boost Regressor, using evaluation metrics MSE and R-squared are compared in a tabular form. Retrospectively, all three models performed remarkably well, with very low MSE values and R-squared values close to 1. Linear Regression marginally outperformed the others, but all models were successful in predicting the anxiety or depression indicator accurately. The proposed models are valid and reliable models for predicting mental health, therefore enabling the identification of at-risk individuals well in advance, allowing early intervention to prevent symptom onsets or advancements in their course and thus improve overall outcomes.

Malware authorship attribution: Unmasking the culprits behind malicious software

With the digital age ushering in an unprecedented proliferation of malware, accurately attributing these malicious software variants to their original authors or affiliated groups has emerged as a crucial endeavor in cybersecurity. This study delves into the intricacies of malware authorship attribution by combining traditional analytical techniques with advanced machine learning methodologies. An integrated approach, encompassing static and dynamic analyses, yielded promising results in the challenging realm of malware attribution. Despite the encouraging outcomes, the research highlighted the multifaceted complexities involved, especially considering the sophisticated obfuscation techniques frequently employed by attackers. This paper emphasizes the merits of a holistic attribution model and underscores the importance of continuous innovation in the face of an ever-evolving threat landscape.

Estimation of human age using machine learning on panoramic radiographs for Brazilian patients

This paper addresses a relevant problem in Forensic Sciences by integrating radiological techniques with advanced machine learning methodologies to create a non-invasive, efficient, and less examiner-dependent approach to age estimation. Our study includes a new dataset of 12,827 dental panoramic X-ray images representing the Brazilian population, covering an age range from 2.25 to 96.50 years. To analyze these exams, we employed a model adapted from InceptionV4, enhanced with data augmentation techniques. The proposed approach achieved robust and reliable results, with a Test Mean Absolute Error of 3.1 years and an R-squared value of 95.5%. Professional radiologists have validated that our model focuses on critical features for age assessment used in odontology, such as pulp chamber dimensions and stages of permanent teeth calcification. Importantly, the model also relies on anatomical information from the mandible, maxillary sinus, and vertebrae, which enables it to perform well even in edentulous cases. This study demonstrates the significant potential of machine learning to revolutionize age estimation in Forensic Science, offering a more accurate, efficient, and universally applicable solution.

Chronic Kidney Disease Prediction Using Machine Learning

Abstract: Chronic kidney disease (CKD) is a progressive condition in which the kidneys lose their ability to function effectively over time. Individuals with hypertension, diabetes, or a family history of CKD are at increased risk, emphasizing the importance of early detection for effective intervention and management. Recent research has focused on employing machine learning techniques, including Ant Colony Optimization (ACO) and Support Vector Machine (SVM) classifiers, to predict CKD presence using a minimal set of features. This study aims to optimize predictive accuracy through advanced machine learning methodologies, facilitating timely and targeted healthcare interventions for at-risk individuals. By analyzing relevant clinical data, predictive models developed in this research offer promising avenues for early identification of CKD, enabling proactive disease management strategies. The integration of machine learning techniques in CKD prediction not only enhances predictive accuracy but also contributes to advancing personalized healthcare. Early detection allows for the implementation of preventive measures and personalized treatment plans, ultimately improving patient outcomes and reducing the burden on healthcare systems. The proposed methodology seeks to optimize predictive accuracy, aiding in timely and targeted healthcare interventions.

Machine learning models on a web application to predict short-term postoperative outcomes following anterior cervical discectomy and fusion

BackgroundThe frequency of anterior cervical discectomy and fusion (ACDF) has increased up to 400% since 2011, underscoring the need to preoperatively anticipate adverse postoperative outcomes given the procedure’s expanding use. Our study aims to accomplish two goals: firstly, to develop a suite of explainable machine learning (ML) models capable of predicting adverse postoperative outcomes following ACDF surgery, and secondly, to embed these models in a user-friendly web application, demonstrating their potential utility.MethodsWe utilized data from the National Surgical Quality Improvement Program database to identify patients who underwent ACDF surgery. The outcomes of interest were four short-term postoperative adverse outcomes: prolonged length of stay (LOS), non-home discharges, 30-day readmissions, and major complications. We utilized five ML algorithms - TabPFN, TabNET, XGBoost, LightGBM, and Random Forest - coupled with the Optuna optimization library for hyperparameter tuning. To bolster the interpretability of our models, we employed SHapley Additive exPlanations (SHAP) for evaluating predictor variables’ relative importance and used partial dependence plots to illustrate the impact of individual variables on the predictions generated by our top-performing models. We visualized model performance using receiver operating characteristic (ROC) curves and precision-recall curves (PRC). Quantitative metrics calculated were the area under the ROC curve (AUROC), balanced accuracy, weighted area under the PRC (AUPRC), weighted precision, and weighted recall. Models with the highest AUROC values were selected for inclusion in a web application.ResultsThe analysis included 57,760 patients for prolonged LOS [11.1% with prolonged LOS], 57,780 for non-home discharges [3.3% non-home discharges], 57,790 for 30-day readmissions [2.9% readmitted], and 57,800 for major complications [1.4% with major complications]. The top-performing models, which were the ones built with the Random Forest algorithm, yielded mean AUROCs of 0.776, 0.846, 0.775, and 0.747 for predicting prolonged LOS, non-home discharges, readmissions, and complications, respectively.ConclusionsOur study employs advanced ML methodologies to enhance the prediction of adverse postoperative outcomes following ACDF. We designed an accessible web application to integrate these models into clinical practice. Our findings affirm that ML tools serve as vital supplements in risk stratification, facilitating the prediction of diverse outcomes and enhancing patient counseling for ACDF.

Exploring the influence of crystallization fouling on microscale heat exchangers through machine learning analysis

Crystallization deposition within micro-scale heat exchangers poses significant challenges to their efficiency and functionality, stemming from the accumulation of crystalline residues on heat transfer surfaces. This study employs advanced machine learning methodologies, including GRU, LSTM, RNN, and CNN models, to explore the underlying factors influencing micro heat exchanger performance. Through meticulous analysis of key parameters such as Reynolds number, sedimentation coefficient, flow rate, and channel dimensions, the study aims to delineate the foundational factors shaping heat exchanger performance at the microscopic level. Results reveal the exceptional accuracy of CNN model in forecasting experimental outcomes, surpassing 99% accuracy and demonstrating superior performance compared to traditional numerical methods. Temperature emerges as a pivotal determinant, profoundly influencing crystallization dynamics, with its intricate interplay with solute solubility elucidated through rigorous analysis. Furthermore, comparative assessment of training times highlights the CNN model’s efficiency, attributed to its specialized architecture suited for spatial data processing. This study provides valuable insights into the impact of crystallization deposition on microscale heat exchangers, showcasing the transformative potential of machine learning in optimizing heat exchanger performance and addressing operational challenges in industrial applications.

Performance evaluation of advanced machine learning methodologies in simulating hydrogen chloride (HCl) absorption by deep eutectic solvents

This work aims to construct the physical-based machine learning (ML) approach for estimating the hydrogen chloride (HCl) solubility in eleven different deep eutectic solvents (DESs). All DES-HCl equilibrium records versus temperatures and pressures are gathered from the literature to ensure ML universality. Then, a well-known feature selection technique was applied to identify the physically meaningful descriptors for the hydrogen bond acceptors (HBA) and hydrogen bond donors (HBD). Afterward, temperature, pressure, selected meaningful physical descriptors, and HBA/HBD molar concentrations are utilized to design ML tools to compute HCl solubility in DESs. Diverse MLs, including extreme learning machines, convolutional neural networks, long short-term memory, cascade neural networks, and extreme gradient boosting (XGBoost), are tested, and their accuracy is compared to select the best candidate. The XGBoost model predictions for HCl solubility in DESs are in acceptable agreement with the literature records. This model precisely simulates 417 DES-HCl equilibrium records in the temperature range of 293–348 K and pressures of up to 127 kPa with a correlation coefficient of 0.99621, mean absolute percentage deviation of 3.04, mean absolute error of 0.448, and root mean square error of 0.729. Results of the designed XGBoost model prove that acetamide-ethylene glycol (1:1), which can averagely dissolve up to 32.47 wt% (weight percent) of HCl, is the best DES for the HCl capture and recovery. After approving the XGBoost model reliability with different visual and numerical inspections, the Leverage method also justified its validity domain by more than 97.6%.

Revolutionizing Higher Education Admissions: A Holistic Solution Integrating Machine Learning and Chatbot Technology

In the dynamic landscape of higher education admissions, traditional processes encounter inefficiencies and limitations in providing personalized support to prospective students. This research proposes a comprehensive solution that merges advanced machine learning methodologies with AI-driven chatbot technology to redefine the admission journey. By addressing the challenges faced by both applicants and institutions, this innovative approach aims to streamline the admissions process while nurturing a supportive ecosystem for students. Through predictive modeling and educational chatbots, the integrated system enhances efficiency and provides tailored guidance, facilitating a smoother transition into higher education. This paper explores the potential of this integrated solution, emphasizing its implications for predictive modeling, college prediction, and the overall improvement of the admission experience.

Prediction of pediatric peanut oral food challenge outcomes using machine learning

Clinical testing, including food-specific skin and serum IgE level tests, provides limited accuracy to predict food allergy. Confirmatory oral food challenges (OFCs) are often required, but the associated risks, cost, and logistic difficulties comprise a barrier to proper diagnosis. We sought to utilize advanced machine learning methodologies to integrate clinical variables associated with peanut allergy to create a predictive model for OFCs to improve predictive performance over that of purely statistical methods. Machine learning was applied to the Learning Early about Peanut Allergy (LEAP) study of 463 peanut OFCs and associated clinical variables. Patient-wise cross-validation was used to create ensemble models that were evaluated on holdout test sets. These models were further evaluated by using 2 additional peanut allergy OFC cohorts: the IMPACT study cohort and a local University of Michigan cohort. In the LEAP data set, the ensemble models achieved a maximum mean area under the curve of 0.997, with a sensitivity and specificity of 0.994 and 1.00, respectively. In the combined validation data sets, the top ensemble model achieved a maximum area under the curve of 0.871, with a sensitivity and specificity of 0.763 and 0.980, respectively. Machine learning models for predicting peanut OFC results have the potential to accurately predict OFC outcomes, potentially minimizing the need for OFCs while increasing confidence in food allergy diagnoses.

Safeguarding Cyber Ecosystems: Machine Learning and Data Mining Approaches for Proactive Phishing Detection in Websites

Abstract: The prevalence of cyber threats, notably phishing attacks, presents a significant challenge in the digital realm with the increasing reliance on the internet. Phishing, where victims' credentials are illicitly acquired through deceptive websites resembling legitimate ones, is a particularly concerning form of cybercrime. This paper proposes an innovative system utilizing Machine Learning and Data Mining techniques to detect both established and newly generated phishing URLs without historical behavioral data. The system, embodied in a Chrome browser plugin, operates in real-time to provide immediate protection as users browse web pages. A distinctive feature of this system is its capability to identify phishing websites lacking prior behavioral patterns, ensuring adaptability to evolving cyber threats. Through comprehensive training with an extensive dataset, the model powering the plugin aims for high accuracy in detecting phishing attempts. By deploying advanced Machine Learning methodologies, the system discerns subtle patterns and characteristics intrinsic to phishing URLs, effectively distinguishing them from legitimate websites. The ultimate objective is to enhance cyber security by providing users with a robust and proactive tool against the growing sophistication of phishing attacks, thereby contributing to the ongoing efforts to create a secure online environment amidst the dynamic landscape of cyber threats.

A Comparative Study of Machine Learning Classifiers for Enhancing Knee Osteoarthritis Diagnosis

Knee osteoarthritis (KOA) is a leading cause of disability, particularly affecting older adults due to the deterioration of articular cartilage within the knee joint. This condition is characterized by pain, stiffness, and impaired movement, posing a significant challenge in medical diagnostics and treatment planning, especially due to the current inability for early and accurate detection or monitoring of disease progression. This research introduces a multifaceted approach employing feature extraction and machine learning (ML) to improve the accuracy of diagnosing and classifying KOA stages from radiographic images. Utilizing a dataset of 3154 knee X-ray images, this study implemented feature extraction methods such as Histogram of Oriented Gradients (HOG) with Linear Discriminant Analysis (LDA) and Min–Max scaling to prepare the data for classification. The study evaluates six ML classifiers—K Nearest Neighbors classifier, Support Vector Machine (SVM), Gaussian Naive Bayes, Decision Tree, Random Forest, and XGBoost—optimized via GridSearchCV for hyperparameter tuning within a 10-fold Stratified K-Fold cross-validation framework. An ensemble model has also been made for the already high-accuracy models to explore the possibility of enhancing the accuracy and reducing the risk of overfitting. The XGBoost classifier and the ensemble model emerged as the most efficient for multiclass classification, with an accuracy of 98.90%, distinguishing between healthy and unhealthy knees. These results underscore the potential of integrating advanced ML methodologies for the nuanced and accurate diagnosis and classification of KOA, offering new avenues for clinical application and future research in medical imaging diagnostics.

Guest Editorial: Introduction to the Special Issue on Advanced Machine Learning Methodologies for Underwater Image and Video Processing and Analysis

Guest Editorial: Introduction to the Special Issue on Advanced Machine Learning Methodologies for Underwater Image and Video Processing and Analysis

Investigation of Solar Energy for Urban Areas and Metropolitan Communities Using Machine Learning

This study explores the assessment of solar panels in urban and community settings through the application of machine learning algorithms, specifically Artificial Neural Networks (ANNs) and Support Vector Machine-Long Short-Term Memory (SVM-LSTM). The objective of this evaluation is to determine the extent to which solar technology benefits these communities. Linear regression is a valuable tool for data analysis when there is a clear linear relationship between inputs and outputs. However, when dealing with non-linear datasets, more sophisticated analysis methods are required. Support Vector Machines (SVM), a machine learning technique rooted in statistical learning theory, offer an effective approach for addressing non-linear regression challenges. The study’s central focus lies in leveraging machine learning techniques to evaluate the viability of solar panels in urban and community contexts. This evaluation utilizes two distinct algorithms, ANN and SVM-LSTM, to assess the impact of solar technology on these communities. By employing advanced machine learning methodologies, the study aims to shed light on whether solar energy solutions hold the potential to significantly benefit urban and community environments. The results of this analysis may have far-reaching implications for sustainable energy adoption in urban areas, paving the way for a more environmentally responsible and energy-efficient future

Malware authorship attribution: Unmasking the culprits behind malicious software

With the digital age ushering in an unprecedented proliferation of malware, accurately attributing these malicious software variants to their original authors or affiliated groups has emerged as a crucial endeavor in cybersecurity. This study delves into the intricacies of malware authorship attribution by combining traditional analytical techniques with advanced machine learning methodologies. An integrated approach, encompassing static and dynamic analyses, yielded promising results in the challenging realm of malware attribution. Despite the encouraging outcomes, the research highlighted the multifaceted complexities involved, especially considering the sophisticated obfuscation techniques frequently employed by attackers. This paper emphasizes the merits of a holistic attribution model and underscores the importance of continuous innovation in the face of an ever-evolving threat landscape.

Quantifying Brain and Cognitive Maintenance as Key Indicators for Sustainable Cognitive Aging: Insights from the UK Biobank

Age-related cognitive decline is a global phenomenon that affects individuals worldwide. The course and extent of this decline are influenced by numerous factors, such as genetics, lifestyle, education, and cognitive engagement. The theory of brain and cognitive reserve/maintenance posits that these factors have a significant impact on the degree of cognitive decline and overall brain health. However, the absence of standardized definitions and measurements for these terms creates ambiguity in research. To address this issue, we utilized a robust and systematic experimental paradigm, employing a considerably large subject pool comprising 17,030 participants from the UK Biobank. Utilizing advanced machine learning methodologies, we were able to accurately quantify both brain maintenance (BM) and cognitive maintenance (CM), making use of six distinct MRI modalities and nine distinct cognitive capabilities. Our study successfully identified several significant features that were meaningfully associated with both BM and CM outcomes. The results of our study demonstrate that lifestyle factors play a significant role in influencing both BM and CM through unique and independent mechanisms. Specifically, our study found that health status is a critical determinant of BM, while diabetes was found to be moderately associated with CM. Furthermore, our study revealed a positive correlation between BM/CM and cognitive reserve. By carefully considering the unique and independent mechanisms that govern both BM and CM, as well as their correlation with cognitive reserve, our study has provided valuable insight into the various strategies that may be leveraged to promote sustainable interventions to enhance cognitive and brain health across the lifespan.

Prediction of Heart Disease using Machine Learning Techniques

Abstract: Cardiovascular diseases account for approximately 30% of global deaths, with a significant number resulting from delayed diagnoses. The challenge arises when addressing this issue through data analysis. To address this, the adoption of Machine Learning Techniques has become prevalent, and it was also the chosen approach in our study. Our research paper presents a novel classification model developed after extensive stages of pre-processing. The prediction model incorporates diverse feature combinations and utilizes well-established classification techniques. The selection of these techniques was based on a meticulous comparison of their respective accuracy levels. Our ultimate objective is to achieve the highest level of accuracy when compared to existing models in this domain. By leveraging the power of data analysis and employing advanced machine learning methodologies, we aim to enhance the early detection and prediction of cardiovascular diseases. This can potentially contribute to a significant reduction in mortality rates and pave the way for more effective healthcare interventions in the field of cardiovascular health.

Resume screener system

This research paper introduces a state-of-the-art "Resume Screener System" aimed at revolutionizing and automating the labor-intensive task of resume analysis for recruitment purposes. Developed using Python, the system integrates Artificial Intelligence and Natural Language Processing techniques to streamline the hiring process. Utilizing a dataset sourced from Kaggle, comprising a thousand resumes converted into textual data, the system undergoes comprehensive model training and evaluation. Employing advanced machine learning methodologies such as the Support Vector Classifier (SVC) and Neighbours Classifier, the system rigorously tests and analyzes these models to determine the most effective approach. By evaluating each model's performance against predefined criteria, the system identifies the optimal model for resume screening. The primary objective of this work is to provide recruiters and HR professionals with an innovative tool that efficiently matches job requirements with candidates' skill sets as presented in their resumes. By automating the initial screening phase, the system not only saves time and effort but also ensures a more objective and consistent evaluation of applicants. This research contributes to the advancement of machine learning applications in the field of human resources, illustrating the transformative impact of technology on traditional hiring practices.

Guest Editorial Introduction to the Special Issue on Advanced Machine Learning Methodologies for Large-Scale Video Object Segmentation and Detection

Video object segmentation and detection are two important tasks toward intelligent video content understanding. Due to their wide applications in real-world vision tasks, such as video surveillance and automatic driving, they have recently attracted great attention in the computer vision and multimedia processing communities. Although numerous deep learning-based approaches have been proposed to solve these problems, implementing effective and efficient video object segmentation and detection is still very challenging for now, and the principles of solutions to address the problems are still understudied. On the one hand, the features learned by the current deep models are not strong enough to capture the rich spatial and temporal information from the input videos. On the other hand, the annotation information in the video data (especially for unconstrained online videos) is usually insufficient, unspecific, or even absent, thus challenging the current mainstream learning schemes.

Stopping criteria for ending autonomous, single detector radiological source searches.

While the localization of radiological sources has traditionally been handled with statistical algorithms, such a task can be augmented with advanced machine learning methodologies. The combination of deep and reinforcement learning has provided learning-based navigation to autonomous, single-detector, mobile systems. However, these approaches lacked the capacity to terminate a surveying/search task without outside influence of an operator or perfect knowledge of source location (defeating the purpose of such a system). Two stopping criteria are investigated in this work for a machine learning navigated system: one based upon Bayesian and maximum likelihood estimation (MLE) strategies commonly used in source localization, and a second providing the navigational machine learning network with a “stop search” action. A convolutional neural network was trained via reinforcement learning in a 10 m × 10 m simulated environment to navigate a randomly placed detector-agent to a randomly placed source of varied strength (stopping with perfect knowledge during training). The network agent could move in one of four directions (up, down, left, right) after taking a 1 s count measurement at the current location. During testing, the stopping criteria for this navigational algorithm was based upon a Bayesian likelihood estimation technique of source presence, updating this likelihood after each step, and terminating once the confidence of the source being in a single location exceeded 0.9. A second network was trained and tested with similar architecture as the previous but which contained a fifth action: for self-stopping. The accuracy and speed of localization with set detector and source initializations were compared over 50 trials of MLE-Bayesian approach and 1000 trials of the CNN with self-stopping. The statistical stopping condition yielded a median localization error of ~1.41 m and median localization speed of 12 steps. The machine learning stopping condition yielded a median localization error of 0 m and median localization speed of 17 steps. This work demonstrated two stopping criteria available to a machine learning guided, source localization system.

Enhancing Market Price Decision-Making in Fintech through A Busines¬s Intelligence Technique

The surge of Fintech data and its implications on informed decision-making within the transportation sector have spurred the need for advanced analytical frameworks. This study addresses the challenge of leveraging Fintech data's temporal dynamics to enhance predictive capabilities and decision-making. The methodologies encompass an AutoEncoder (AE) for spatial feature extraction and an Improved Gated Recurrent Unit (IGRU) to capture temporal dependencies. Additionally, the Huber loss function optimizes model parameters, particularly in handling outliers. Integrating these techniques, our study explores Fintech data's spatial and temporal patterns, contributing insights for transportation planners and Fintech industries. Results demonstrate the efficacy of AE in learning spatial features, while IGRU effectively captures temporal dependencies, enabling the prediction of Fintech data with enhanced accuracy. The application of Huber loss ensures robustness by mitigating outlier influence. By the study's end, the model's predictive capabilities foster informed decision-making, offering opportunities to enhance Fintech data quality, reduce congestion, and bolster road safety. Overall, this research underscores the significance of advanced machine learning methodologies in decoding Fintech data's intricacies, laying a foundation for data-driven decision-making in the transportation and Fintech sectors.