A review of approaches to identifying patient phenotype cohorts using electronic health records

As statistical machine learning algorithms and techniques continue to mature, many researchers and developers see statistical machine learning not only as a topic of expert study, but also as a tool for software development. Extensive prior work has studied software development, but little prior work has studied software developers applying statistical machine learning. This paper presents interviews of eleven researchers experienced in applying statistical machine learning algorithms and techniques to human-computer interaction problems, as well as a study of ten participants working during a five-hour study to apply statistical machine learning algorithms and techniques to a realistic problem. We distill three related categories of difficulties that arise in applying statistical machine learning as a tool for software development: (1) difficulty pursuing statistical machine learning as an iterative and exploratory process, (2) difficulty understanding relationships between data and the behavior of statistical machine learning algorithms, and (3) difficulty evaluating the performance of statistical machine learning algorithms and techniques in the context of applications. This paper provides important new insight into these difficulties and the need for development tools that better support the application of statistical machine learning.

BackgroundDengue is a mosquito-borne viral disease that poses a significant public health threat in tropical and subtropical regions worldwide. Accurate forecasting of dengue outbreaks is crucial for effective public health planning and intervention. This study aims to assess the predictive performance and computational efficiency of a number of statistical models and machine learning techniques for dengue forecasting, both with and without the inclusion of climate factors, to inform the design of dengue surveillance systems.MethodsThe dengue forecasting methods comparison in this study considers dengue cases in Rio de Janeiro, Brazil, as well as climate factors known to affect disease transmission. Employing a dynamic window approach, various statistical methods and machine learning techniques were used to generate weekly forecasts at several time horizons. Error measures, uncertainty intervals, and computational efficiency obtained with each method were compared. Statistical models considered were Autoregressive (AR), Moving Average (MA), Autoregressive Integrated Moving Average (ARIMA), and Exponential Smoothing State Space Model (ETS). In addition, models incorporating temperature and humidity as covariates, such as Vector Autoregression (VAR) and Seasonal ARIMAX (SARIMAX), were employed. Machine learning techniques evaluated were Random Forest, XGBoost, Support Vector Machine (SVM), Long–Short-Term Memory (LSTM) networks, and Prophet. Ensemble approaches that integrated the top performing methods were also considered. The evaluated methods also incorporated lagged climatic variables to account for delayed effects.ResultsAmong the statistical models, ARIMA demonstrated the best performance using only historical case data, while SARIMAX significantly improved predictive accuracy by incorporating climate covariates. In general, the LSTM model, particularly when combined with climate covariates, proved to be the most accurate machine learning model, despite being slower to train and predict. For long-term forecasts, Prophet with climate covariates was the most effective. Ensemble models, such as the combination of LSTM and ARIMA, showed substantial improvements over individual models.ConclusionsThis study demonstrates the strengths and limitations of various methods for dengue forecasting across multiple timeframes. It highlights the best-performing statistical and machine learning methods, including their computational efficiency, underscoring the significance of machine learning techniques and the integration of climate covariates to improve forecasts. These findings offer valuable insights for public health officials, facilitating the development of dengue surveillance systems for more accurate forecasting and timely allocation of resources to mitigate dengue outbreaks.

Current and future applications of statistical machine learning algorithms for agricultural machine vision systems

Generative AI (GenAI) and LLMs have started to influence how teachers teach and studentslearn, including the ones in programming languages and tutoring. However, there have been debates onwhether AI could be beneficial to students’ learning or not. One way to see this issue is from the perspectives of thestudents. Therefore, this study explored how students perceive the use of AI in their education. The data wasgathered through interviews with 62 students and other stakeholders, such as instructors and IT specialists. The results showed that the students positively perceived using AI as a tutor. Moreover, this study suggests several things to consider when integrating AI tutors for programming. The findings reveal positive student perceptions regarding AI's potential within the teaching-learning process. Students envision AI tutors offering personalized assistance, adapting to individual learning styles, and providing immediate feedback, potentially augmenting their grasp of programming concepts. We applied Statistical analysis, machine learning, and natural language processing techniques such as PCA, t-SNE, LDA, and sentiment analysis.

Work with Python and powerful open source tools such as Gensim and spaCy to perform modern text analysis, natural language processing, and computational linguistics algorithms.About This BookDiscover the open source Python text analysis ecosystem, using spaCy, Gensim, scikit-learn, and KerasHands-on text analysis with Python, featuring natural language processing and computational linguistics algorithmsLearn deep learning techniques for text analysisWho This Book Is ForThis book is for you if you want to dive in, hands-first, into the interesting world of text analysis and NLP, and you're ready to work with the rich Python ecosystem of tools and datasets waiting for you!What You Will LearnWhy text analysis is important in our modern ageUnderstand NLP terminology and get to know the Python tools and datasetsLearn how to pre-process and clean textual dataConvert textual data into vector space representationsUsing spaCy to process textTrain your own NLP models for computational linguisticsUse statistical learning and Topic Modeling algorithms for text, using Gensim and scikit-learnEmploy deep learning techniques for text analysis using KerasIn DetailModern text analysis is now very accessible using Python and open source tools, so discover how you can now perform modern text analysis in this era of textual data.This book shows you how to use natural language processing, and computational linguistics algorithms, to make inferences and gain insights about data you have. These algorithms are based on statistical machine learning and artificial intelligence techniques. The tools to work with these algorithms are available to you right now - with Python, and tools like Gensim and spaCy.You'll start by learning about data cleaning, and then how to perform computational linguistics from first concepts. You're then ready to explore the more sophisticated areas of statistical NLP and deep learning using Python, with realistic language and text samples. You'll learn to tag, parse, and model text using the best tools. You'll gain hands-on knowledge of the best frameworks to use, and you'll know when to choose a tool like Gensim for topic models, and when to work with Keras for deep learning.This book balances theory and practical hands-on examples, so you can learn about and conduct your own natural language processing projects and computational linguistics. You'll discover the rich ecosystem of Python tools you have available to conduct NLP - and enter the interesting world of modern text analysis.Style and approachThe book teaches NLP from the angle of a practitioner as well as that of a student. This is a tad unusual, but given the enormous speed at which new algorithms and approaches travel from scientific beginnings to industrial implementation, first principles can be clarified with the help of entirely practical examples.

Manganese (Mn) nodules are deep-sea mineral resources that contain critical elements for modern industries, such as nickel (Ni), cobalt (Co), and copper (Cu). As a result, extensive global research has been conducted on the exploration and formation mechanisms of Mn nodules. However, studies investigating the correlation between manganate minerals and critical elements, as well as the deep-sea environment represented by these mineral phases, have been limited. Here, we show the significance of phase transformations in manganate minerals within Mn nodules, in relation to deep-sea formation environments and the geochemical behavior of Ni, Co, and Cu, using basic statistical analysis, machine learning, and hydrothermal experiments. The statistical analysis and machine learning models revealed that the trace amounts of Ni, Co, and Cu exhibit a positive correlation with vernadite, a dominant manganate mineral in Mn nodules. Additionally, hydrothermal experiments demonstrated that under suboxic conditions at temperatures between 120°C and 150°C, Mn nodules undergo mineral phase transformations, forming various minerals, including todorokite, Mn ferrite, fluorapatite, hematite, and goethite. During these transformations, Ni, Co, and Cu were incorporated into the mineral structures of todorokite, fluorapatite, and Mn ferrite, respectively. Therefore, the mineral phases within Mn nodules serve as key indicators not only for understanding their formation environment but also for tracking changes in the geochemical behavior of Ni, Co, and Cu. Furthermore, our methodological approach demonstrates to many researchers how correlations between materials can be validated and utilized using statistical analysis and machine learning techniques.

Predicting rate of penetration (ROP) has always been of fundamental interest to the drilling industry. Early predictions can assist the engineer in changing parameters to reduce non-productive time (NPT) and achieve optimum ROP. This paper illustrates methods to predict the ROP in a computationally efficient manner using only data available at the surface. These methods can then be incorporated into real time drilling operations, first through a passive diagnostic tool, and then an integrated real-time control loop. In this work, statistical learning techniques such as trees, bagged trees, and random forests (RF) are used to predict ROP. Trees provide easy interpretability and hence are favored over other non-linear techniques. However, accuracy is imperative in this procedure. Accuracy can be increased by using bootstrap aggregating (bagging) or Random Forests. These techniques are applied, using the statistical software computing package R and its numerous libraries. Statistical learning techniques have been applied to a data set which had nine predictors. Applying trees to a data set yields great visualization of the data, but the lack of accuracy and can result in substantial overfitting. This shortcoming was rectified using bagging or RF methods to substantially increase accuracy. The results were promising in all cases and acceptable for real time predictions. Scalability is another concern for real time operations. Computational efficiency of the methods were evaluated, and the best method was based on a combination of computational efficiency and accuracy. Potential time savings which would result from applying the model in real-time optimizations and demonstration of the power of machine learning techniques are included in this paper. Future improvements will be incorporated in real-time prediction during drilling. State of the art statistical learning and machine learning techniques are applied to prediction of ROP, whereas previous prediction methods have not been based on real-time drilling data. The result is a computationally efficient model which can determine the right features for prediction at each step, while also incorporating engineering judgement and maintaining integrity of the statistical principles being employed. These methods can easily be extended to other drilling parameters such as MSE or Torque and Drag.

The usage of smart grids is growing steadily around the world. This technology has been proposed as a promising solution to enhance energy efficiency and improve consumption management in buildings. Such benefits are usually associated with the ability of accurately forecasting energy demand. However, the energy consumption series forecasting is a challenge for statistical linear and Machine Learning (ML) techniques due to temporal fluctuations and the presence of linear and non-linear patterns. Traditional statistical techniques are able to model linear patterns, while obtaining poor results in forecasting the non-linear component of the time series. ML techniques are data-driven and can model non-linear patterns, but their feature selection process and parameter specification are a complex task. This paper proposes an Evolutionary Hybrid System (EvoHyS) which combines statistical and ML techniques through error series modeling. EvoHyS is composed of three phases: (i) forecast of the linear and seasonal component of the time series using a Seasonal Autoregressive Integrated Moving Average (SARIMA) model, (ii) forecast of the error series using an ML technique, and (iii) combination of both linear and non-linear forecasts from (i) and (ii) using a a secondary ML model. EvoHyS employs a Genetic Algorithm (GA) for feature selection and hyperparameter optimization in phases (ii) and (iii) aiming to improve its accuracy. An experimental evaluation was conducted using consumption energy data of a smart grid in a one-step-ahead scenario. The proposed hybrid system reaches statistically significant improvements when compared to other statistical, hybrid, and ML approaches from the literature utilizing well known metrics, such as Mean Squared Error (MSE).

The implementation of machine learning models that lie at the heart of traditional financial institutions like banks has primarily remained a "black box". Techniques such as extreme gradient boosting, random forest and support vector machines are challenging to interpret. This restricts the commercial application of these techniques as they lack transparency and often hide potential biases inherited by the machine learning algorithms. Ultimately, this further limits the ability of these techniques to assess an individual’s ability to pay for a loan. The biases lead to credit applications being rejected or individuals being offered "unfair" credit terms based on their credit risk. Recently there have been attempts to predict credit default risk in literature. However, the majority of studies are conducted using data from developed countries, where most individuals are banked and data is complete. Therefore, this study aims to investigate and compare statistical classification techniques and machine learning techniques, with the focus being on interpreting the results and explaining which features contribute to the credit approval processes that may excel in both developed and developing countries. To achieve this goal, we explored three datasets from FinTech organisations, namely Home Credit, Xente, and Super Lender. Two of the datasets are from Uganda and Nigeria (developing countries in Africa) and one from the USA (a developed country in North America). We adopted Shapley Additive exPlanation (SHAP) to interpret the models. Using SHAP, we identified that for all three datasets, the external ratings, total due and loan duration increase the likelihood of an individual defaulting on a loan. Additionally, we also found that the tree-based algorithms, particularly the extreme gradient boosting algorithm outperforms logistic regression, random forest and other statistical learning techniques.

Money laundering has been a global issue for decades, which is one of the major threat for economy and society. Government, regulatory and financial institutions are combating it together in their respective capacity, however still billions of dollars in fines by authorities make the headlines in the news. High-speed internet services have enabled financial institutions to deliver better customer experience through multi-channel engagements, which has led to exponential growth in transactions and new avenues for laundering the money for fraudsters. Literature shows the usage of statistical methods, data mining and Machine Learning (ML) techniques for money laundering detection, but limited research on Deep Learning (DL) techniques, primarily due to lack of model interpretability and explainability of the decisions made. Several studies are conducted on application of ML for Anti-Money Laundering (AML), and Explainable Artificial Intelligence (XAI) techniques in general, but lacks the study on usage of DL techniques together with XAI. This paper aims to review the current state-of-the-art literature on DL together with XAI for identifying suspicious money laundering transactions and identify future research areas. Key findings of the review are, researchers have preferred variants of Convolutional Neural Networks, and AutoEncoder; graph deep learning together with natural language processing is emerging as an important technology for AML; XAI use is not seen in AML domain; 51% ML methods used in AML are non-interpretable, 58% studies used sample of old real data; key challenges for researchers are access to recent real transaction data and scarcity of labelled training data; and data being highly imbalanced. Future research directions are, application of XAI techniques to bring-out explainability, graph deep learning using natural language processing (NLP), unsupervised and reinforcement learning to handle lack of labelled data; and joint research programs between research community and industry to benefit from domain knowledge and controlled access to data.

Aspirin-exacerbated respiratory disease (AERD) is an acquired inflammatory condition characterized by the presence of asthma, chronic rhinosinusitis with nasal polyposis, and respiratory hypersensitivity reactions on ingestion of aspirin or other nonsteroidal anti-inflammatory drugs (NSAIDs). Despite AERD having a classic constellation of symptoms, the diagnosis is often overlooked, with an average of greater than 10 years between the onset of symptoms and diagnosis of AERD. Without a diagnosis, individuals will lack opportunities to receive effective treatments, such as aspirin desensitization or biologic medications. Our aim was to develop a combined algorithm that integrates both natural language processing (NLP) and machine learning (ML) techniques to identify patients with AERD from an electronic health record (EHR). A rule-based decision tree algorithm incorporating NLP-based features was developed using clinical documents from the EHR at Mayo Clinic. From clinical notes, using NLP techniques, 7 features were extracted that included the following: AERD, asthma, NSAID allergy, nasal polyps, chronic sinusitis, elevated urine leukotriene E4 level, and documented no-NSAID allergy. MedTagger was used to extract these 7 features from the unstructured clinical text given a set of keywords and patterns based on the chart review of 2 allergy and immunology experts for AERD. The status of each extracted feature was quantified by assigning the frequency of its occurrence in clinical documents per subject. We optimized the decision tree classifier's hyperparameters cutoff threshold on the training set to determine the representative feature combination to discriminate AERD. We then evaluated the resulting model on the test set. The AERD algorithm, which combines NLP and ML techniques, achieved an area under the receiver operating characteristic curve score, sensitivity, and specificity of 0.86 (95% CI 0.78-0.94), 80.00 (95% CI 70.82-87.33), and 88.00 (95% CI 79.98-93.64) for the test set, respectively. We developed a promising AERD algorithm that needs further refinement to improve AERD diagnosis. Continued development of NLP and ML technologies has the potential to reduce diagnostic delays for AERD and improve the health of our patients.

This paper presents a critical assessment analysis on mental health detection in Online Social Networks (OSNs) based on the data sources, machine learning techniques, and feature extraction method. The appropriateness of the mental health detection was also investigated by identifying its data analysis method, comparison, challenges, and limitations. This study reviewed articles published in major databases between 2007 and 2018 through keyword searches. The articles were screened base on their titles and abstracts before the full texts were reviewed. The articles were coded in accordance with data set (e.g., data sources, keywords, and geographical locations), method of data analysis, machine learning or deep learning technique, classifier performance, and feature extraction method. 22 articles were selected for review from the total of 2770. As OSNs exhibit high potential as a data source in early detection of mental health problems, most researchers used text analysis on a new data set extracted from different OSNs sources. The extracted data were examined using a statistical analysis or machine learning techniques. Several studies also applied multimethod techniques, which included distributing questionnaires while requesting for the respondents' consent to later access and extract information from his/her OSNs account. Big data in OSNs contribute on mental health problem detection. The presented method is an alternative approach to the early detection of mental health problems rather than using traditional strategies, such as collecting data through questionnaires or devices and sensors, which are time-consuming and costly. However, mental health problem detection through OSNs necessitates a comprehensive adoption, innovative algorithms, and computational linguistics to describe its limitations and challenges. Moreover, referrals from mental health specialists as subject matter experts are also required to help obtain accurate and effective information.

Air pollution is a major environmental issue and machine learning techniques play an important role in analyzing and forecasting these data sets. Air quality is an outcome of the complex interaction of several factors involving the chemical reactions, meteorological parameters, and emissions from natural and anthropogenic sources. In this paper, we propose an efficient combined technique that takes the benefits of statistical techniques and machine learning techniques to predict/forecast the Air Quality and Pollution in particular regions. This work also indicates that prediction performance varies over different regions/cities in India. We used time series analysis, regression and Ada-boosting to anticipate PM 2.5 concentration levels in several locations throughout Hyderabad on an annual basis, depending on numerous atmospheric and surface parameters like wind speed, air temperature, pressure, and so on. Dataset for this investigation is taken from Kaggle and experimented with proposed method and comparison results of our experiments are then plotted.

In this paper, we propose a novel methodology for micro-seismic event detection using multiple statistical feature extraction and machine learning techniques. The method is based on amplitude thresholding and is grounded on the idea, that each relevant seismic event crosses a particular threshold value. We have created a unique dataset of movement, hammering and no-movement using off-the-shelve geophones and 16-bit analog to digital converter. The data is sampled at 10000Hz. Each recorded dataset is of duration 2 min. The data recorded is passed through amplitude thresholding method. Multiple statistical features like root mean square, standard deviation, variance, minimum value and maximum value are extracted from the dataset in both time and frequency domain. k-fold cross validation is done to obtain validation accuracy. We have explored various machine learning algorithms and the results are discussed here. Among all classifiers, Random Forest algorithm achieves the overall highest accuracy of 92.76%.

We present a novel approach to homogenize daily GNSS water vapour time series using statistical methods and machine learning techniques. The procedure involves three main steps:Segmentation. The aim is to detect the number and position of change-points in a time series of Integrated Water Vapour (IWV) differences (GNSS minus reference), modelled as a constant (mean) value per segment superposed with a fourth order Fourier series and white noise with a monthly varying variance. The model parameters are estimated by penalized maximum likelihood algorithm, implementing Dynamic Programming search in an iterative scheme [1]. Attribution. The aim is to predict, for each and every change-point, in which of the GNSS (G) or reference (E) series the jump in mean occurred. Information from nearby stations is introduced as additional G' and E' series, which are combined with G and E into six series of differences. A Feasible Generalized Least Squares regression is used to estimate the size of the jumps in the six series and a Random Forest classifier is used to predict which of the four base series caused the jump. The classifier is trained and validated beforehand with a large data set by using a resampling strategy [2]. Correction. The raw G and E series are corrected for the corresponding shifts in mean that were detected and attributed to G and/or E. The paper will present recent improvements of the attribution method, namely: i) the optimization of detection skill scores, both for the training of the classifier and application; ii) the optimization of the sample size for the resampling; iii) a refined nearby-aggregation method based on inverse distance weighting. The method is applied to a new, enhanced, data set based on more than 6000 globally-distributed GNSS stations. The impact of homogenization on IWV trends over the period 1994-2022 is presented.[1] Quarello et al., 2022, https://doi.org/10.3390/rs14143379[2] Nguyen et al., 2024, https://doi.org/10.1002/joc.844