Statistical Learning Procedures Research Articles

Fe recovery from drinking water treatment sludge for reuse in tannery wastewater treatment: Machine learning and statistical modelling

This work experimentally and numerically studied the reusability of Fe recovered from water treatment sludge as a coagulant and compared its performance with a commercial coagulant in tannery wastewater treatment. The performance of the recycled and commercial coagulants in total suspended solid (TSS), chemical oxygen demand (COD), color, chromium (Cr3+) and turbidity removal was compared with considering pH, coagulant type and dose as independent variables. Additionally, the process was modelled by two statistical and machine learning procedures, i.e., response surface methodology (RSM) and artificial neural network (ANN). For RSM model, a 32 run experimental matrix was designed by factorial design, while ANN model was developed by a dataset extracted from the publications inclusive of the 32 experiments. The results showed Fe recovery rate of 181 mg/g dry sludge. The maximum TSS, COD, color, Cr3+ and turbidity removal by recovered and commercial coagulants were (85 %, 40 %, 42 %, 68 %, and 88 %) and (70 %, 35 %, 49 %, 64 % and 88 %) respectively. ANN COD removal model showed a good prediction in all training, validation and testing phases with R 0.891, 0.963 and 0.917 respectively. While the R for TSS, COD, color, Cr3+ and turbidity statistical models were 0.962, 0.866, 0.960, 0.962 and 0.973 respectively.

Scaling by subsampling for big data, with applications to statistical learning

Handling large datasets and calculating complex statistics on huge datasets require important computing resources. Using subsampling methods to calculate statistics of interest on small samples is often used in practice to reduce computational complexity, for instance using the divide and conquer strategy. In this article, we recall some results on subsampling distributions and derive a precise rate of convergence for these quantities and the corresponding quantiles. We also develop some standardisation techniques based on subsampling unstandardised statistics in the framework of large datasets. It is argued that using several subsampling distributions with different subsampling sizes brings a lot of information on the behaviour of statistical learning procedures: subsampling allows to estimate the rate of convergence of different algorithms, to estimate the variability of complex statistics, to estimate confidence intervals for out-of-sample errors and interpolate their values at larger scales. These results are illustrated on simulations, but also on two important datasets, frequently analysed in the statistical learning community, EMNIST (recognition of digits) and VeReMi (analysis of Network Vehicular Reference Misbehavior).

A Random Access Protocol for Crowded Massive MIMO Systems Based on a Bayesian Classifier

The strongest user collision resolution (SUCRe) protocol is an efficient grant-based random access (RA) solution to provide connectivity for large numbers of user equipments (UEs), leveraging massive MIMO propagation features, typically available in fifth-generation and beyond networks. In this letter, we propose to replace the retransmission rule of SUCRe protocol by a Bayesian classifier for identifying the strongest user, aiming to resolve the collisions in a decentralized way, at the UEs’ side. As an offline training stage, we first conduct a statistical learning procedure to obtain the density estimations of the UEs’ decision variables in the SUCRe protocol, both in the cases of the UE being the strongest contender or not. Then, following the maximum a posteriori decision criterion, we determine how the UE can decide if it is the strongest contender (retransmitting the chosen RA pilot) or not (staying idle and trying again later). The numerical results show that our proposed method achieves significant connectivity performance improvements compared with other protocols, without requiring any additional complexity or overhead.

Young minds’ quest for regularity: Evidence from the Turkish causative

AbstractChildren's remarkable ability to generalize beyond the input and the resulting overregularizations/ irregularizations provide a platform for a discussion of whether morphology learning uses analogy-based, rule-based, or statistical learning procedures. The present study, testing 115 children (aged 3 to 10) on an elicited production task, investigated the acquisition of the irregular distribution in the Turkish causative. Results showed that in early acquisition, to pin down the four causative suffixes, children engaged in comparisons between analogous exemplars. Thereafter to tackle the irregularity in two of the suffixes, children entertained competing hypotheses that yielded overregularizations and irregularizations. Overregularizations were instances of abstraction across the input based on type frequency; irregularizations were attempts to default to erroneous micro-generalizations. Negative correlation between errors and verb frequency suggested that recovery from errors was sensitive to token frequency. The overgeneralize-then-recover pattern that emerged in the acquisition of causative supported an integrated account of the roles of analogy, abstraction, and frequency in morphology learning.

Integration of process knowledge and statistical learning for the Dow data challenge problem

In this paper, we propose a statistical learning procedure that integrates process knowledge for the Dow data challenge problem presented in Braun et al. (2020). The task is to build an accurate inferential sensor model to predict the impurity in the product stream with apparent drifts. The proposed method consists of i) process data exploratory analysis, ii) a method for variable selection, iii) a method to deal with non-negative physical property modeling using a softplus function; and iv) a method for online bias updating based on known data. We make use of process operation knowledge in all steps of data analytics, including exploratory analysis and feature selection. We report the detection of equipment-switching operations in the data and interpolations found in the impurity data. Partial least squares (PLS) and least angle regression solution (LARS) are adopted to model the data with strong collinearity. Pros and cons of LARS and PLS are given with practical implications.

Datasailr - An R Package for Row by Row Data Processing, Using DataSailr Script

Data processing and data cleaning are essential steps before applying statistical or machine learning procedures. R provides a flexible way for data processing using vectors. R packages also provide other ways for manipulating data such as using SQL and using chained functions. I present yet another way to process data in a row by row manner using data manipulation oriented script, DataSailr script. This article introduces datasailr package, and shows potential benefits of using domain specific language for data processing.

Cross-level analysis of molecular and neurobehavioral function in a prospective series of patients with germline heterozygous PTEN mutations with and without autism

BackgroundPTEN is a well-established risk gene for autism spectrum disorder (ASD). Yet, little is known about how PTEN mutations and associated molecular processes influence neurobehavioral function in mutation carriers with (PTEN-ASD) and without ASD (PTEN no-ASD). The primary aim of the present study was to examine group differences in peripheral blood-derived PTEN pathway protein levels between PTEN-ASD, PTEN no-ASD, and idiopathic macrocephalic ASD patients (macro-ASD). Secondarily, associations between protein levels and neurobehavioral functions were examined in the full cohort.MethodsPatients were recruited at four tertiary medical centers. Peripheral blood-derived protein levels from canonical PTEN pathways (PI3K/AKT and MAPK/ERK) were analyzed using Western blot analyses blinded to genotype and ASD status. Neurobehavioral measures included standardized assessments of global cognitive ability and multiple neurobehavioral domains. Analysis of variance models examined group differences in demographic, neurobehavioral, and protein measures. Bivariate correlations, structural models, and statistical learning procedures estimated associations between molecular and neurobehavioral variables. To complement patient data, Western blots for downstream proteins were generated to evaluate canonical PTEN pathways in the PTEN-m3m4 mouse model.ResultsParticipants included 61 patients (25 PTEN-ASD, 16 PTEN no-ASD, and 20 macro-ASD). Decreased PTEN and S6 were observed in both PTEN mutation groups. Reductions in MnSOD and increases in P-S6 were observed in ASD groups. Elevated neural P-AKT/AKT and P-S6/S6 from PTEN murine models parallel our patient observations. Patient PTEN and AKT levels were independently associated with global cognitive ability, and p27 expression was associated with frontal sub-cortical functions. As a group, molecular measures added significant predictive value to several neurobehavioral domains over and above PTEN mutation status.LimitationsSample sizes were small, precluding within-group analyses. Protein and neurobehavioral data were limited to a single evaluation. A small number of patients were excluded with invalid protein data, and cognitively impaired patients had missing data on some assessments.ConclusionsSeveral canonical PTEN pathway molecules appear to influence the presence of ASD and modify neurobehavioral function in PTEN mutation patients. Protein assays of the PTEN pathway may be useful for predicting neurobehavioral outcomes in PTEN patients. Future longitudinal analyses are needed to replicate these findings and evaluate within-group relationships between protein and neurobehavioral measures.Trial registrationClinicalTrials.gov Identifier NCT02461446

Possible Atmosphere and Ionospheric Anomalies of the 2019 Pakistan Earthquake Using Statistical and Machine Learning Procedures on MODIS LST, GPS TEC, and GIM TEC

Identifying atmospheric and ionospheric anomalies based on remote sensing satellites has contributed highly to develop the hypothesis of lithosphere-atmosphere-ionosphere coupling over the earthquake (EQ) epicenter during the seismic preparation period. This article has investigated the variations of potential EQ precursor in daytime and nighttime land surface temperature (LST) before and after the 2019 Pakistan EQ from Moderate Resolution Imaging Spectroradiometer (MODIS) satellite. The nighttime LST values of MODIS exhibit temporal anomalies during nighttime period within a time window of five days before and after the main shock day. Furthermore, the LST values predicted by artificial neural network (ANN) validate the significant enhancement in nighttime time series of MODIS. The nighttime LST anomalies obtained from the observation and ANN prediction are more than 20% and 7% of normal distribution beyond the confidence bounds, respectively, within five days after the main shock. Likewise, the ionospheric anomaly from daily total electron content (TEC) values at Sukkur Global Positioning System (GPS) station confirms the EQ associated ionospheric perturbations on the day after the main shock. The Global Ionospheric Maps (GIMs) also show the TEC anomalies during 1000–1400 LT on September 25, 2019.

Statistically learning Archean carbonate diagenesis

Geochemical data are often very noisy due to the large natural heterogeneity of the geological materials as well as oversimplification of rock classification. This has serious repercussions on the precision of our knowledge of the deep past as we often rely solely on geochemical proxies to investigate the geological evolution of Archean and Proterozoic environments. Here statistical learning procedures were applied to achieve unbiased classification of Neoarchean stromatolitic dolostone textures on the basis of the distribution of their trace elements and rare earth elements (REE) investigated through laser ablation induced coupled plasma – mass spectrometry (LA-ICP-MS). Multivariate statistical analyses and supervised statistical learning have revealed that different dolomite fabrics, thought as products of aggrading diagenesis and recrystallization, are in fact chemically indistinguishable. The diagenetic processes that cause the re-crystallization of dolomite and the consequent change of textures, is not affecting the distribution of major and trace elements inherited by the depositional environment or during early stages of diagenesis. At the same time the algorithm has revealed that an optically homogeneous microcrystalline dolomite sample may in fact be geochemically inhomogeneous because of processes of ripening and recrystallization occurred at an early stage of marine diagenesis and that have contributed to element mobilization. Statistical learning has succeeded in recognizing chemofacies which not always overlap with dolomite textures and fabrics highlighting the importance of crystallographic and diagenetic studies before any study of carbonates as geochemical proxies.

Artificial intelligence and avian influenza: Using machine learning to enhance active surveillance for avian influenza viruses

Influenza A viruses are one of the most significant viral groups globally with substantial impacts on human, domestic animal and wildlife health. Wild birds are the natural reservoirs for these viruses, and active surveillance within wild bird populations provides critical information about viral evolution forming the basis of risk assessments and countermeasure development. Unfortunately, active surveillance programs are often resource-intensive, and thus, enhancing programs for increased efficiency is paramount. Machine learning, a branch of artificial intelligence applications, provides statistical learning procedures that can be used to gain novel insights into disease surveillance systems. We use a form of machine learning, gradient boosted trees, to estimate the probability of isolating avian influenza viruses (AIV) from wild bird samples collected during surveillance for AIVs from 2006 to 2011 in the United States. We examined several predictive features including age, sex, bird type, geographic location and matrix gene rRT-PCR results. Our final model had high predictive power and only included geographic location and rRT-PCR results as important predictors. The highest predicted viral isolation probability was for samples collected from the north-central states and the south-eastern region of Alaska. Lower rRT-PCR Ct-values are associated with increased likelihood of AIV isolation, and the model estimated 16% probability of isolating AIV from samples declared negative (i.e., ≥35 Ct-value) using the rRT-PCR screening test and standard protocols. Our model can be used to prioritize previously collected samples for isolation and rapidly evaluate AIV surveillance designs to maximize the probability of viral isolation given limited resources and laboratory capacity.

Filtering for risk assessment of interbank network

Our paper contributes to the recent macroprudential policy addressing the resilience of financial systems in terms of their interconnectedness. We argue that beneath an interbank market, there is a fundamental latent network that affects the liquidity distributions among banks. To investigate the interbank market, we propose a framework that identifies such latent network using a statistical learning procedure. The framework reverse engineers overnight signals observed as banks conduct their reserve management on a daily basis. Our simulation-based results show that possible disruptions in funds supply are highly affected by the interconnectedness of the latent network. Hence, the proposed framework serves as an early warning system for regulators to monitor the overnight market and to detect ex-ante possible disruptions based on the inherent network characteristics.

Modeling Music-Selection Behavior in Everyday Life: A Multilevel Statistical Learning Approach and Mediation Analysis of Experience Sampling Data.

Music listening has become a highly individualized activity with smartphones and music streaming services providing listeners with absolute freedom to listen to any kind of music in any situation. Until now, little has been written about the processes underlying the selection of music in daily life. The present study aimed to disentangle some of the complex processes among the listener, situation, and functions of music listening involved in music selection. Utilizing the experience sampling method, data were collected from 119 participants using a smartphone application. For 10 consecutive days, participants received 14 prompts using stratified-random sampling throughout the day and reported on their music-listening behavior. Statistical learning procedures on multilevel regression models and multilevel structural equation modeling were used to determine the most important predictors and analyze mediation processes between person, situation, functions of listening, and music selection. Results revealed that the features of music selected in daily life were predominantly determined by situational characteristics, whereas consistent individual differences were of minor importance. Functions of music listening were found to act as a mediator between characteristics of the situation and music-selection behavior. We further observed several significant random effects, which indicated that individuals differed in how situational variables affected their music selection behavior. Our findings suggest a need to shift the focus of music-listening research from individual differences to situational influences, including potential person-situation interactions.

Creating glasswing butterfly-inspired durable antifogging superomniphobic supertransmissive, superclear nanostructured glass through Bayesian learning and optimization

Nanostructured superclear, superomniphobic glass fabricated by a statistical machine learning procedure.

Can nonexperts really emulate statistical learning methods? A comment on “The accuracy, fairness, and limits of predicting recidivism”

Recent research has questioned the value of statistical learning methods for producing accurate predictions in the criminal justice context. Using results from respondents on Amazon Mechanical Turk (MTurkers) who were asked to predict recidivism, Dressel and Farid (2018) argue that nonexperts can achieve predictive accuracy and fairness on par with algorithmic approaches that employ statistical learning models. Analyzing the same data from the original study, this comment employs additional techniques and compares the quality of the predicted probabilities output from statistical learning procedures versus the MTurkers’ evaluations. The metrics presented indicate that statistical approaches do, in fact, outperform the nonexperts in important ways. Based on these new analyses, it is difficult to accept the conclusion presented in Dressel and Farid (2018) that their results “cast significant doubt on the entire effort of algorithmic recidivism prediction.”

Machine learning for detection of lymphedema among breast cancer survivors.

In the digital era when mHealth has emerged as an important venue for health care, the application of computer science, such as machine learning, has proven to be a powerful tool for health care in detecting or predicting various medical conditions by providing improved accuracy over conventional statistical or expert-based systems. Symptoms are often indicators for abnormal changes in body functioning due to illness or side effects from medical treatment. Real-time symptom report refers to the report of symptoms that patients are experiencing at the time of reporting. The use of machine learning integrating real-time patient-centered symptom report and real-time clinical analytics to develop real-time precision prediction may improve early detection of lymphedema and long term clinical decision support for breast cancer survivors who face lifelong risk of lymphedema. Lymphedema, which is associated with more than 20 distressing symptoms, is one of the most distressing and dreaded late adverse effects from breast cancer treatment. Currently there is no cure for lymphedema, but early detection can help patients to receive timely intervention to effectively manage lymphedema. Because lymphedema can occur immediately after cancer surgery or as late as 20 years after surgery, real-time detection of lymphedema using machine learning is paramount to achieve timely detection that can reduce the risk of lymphedema progression to chronic or severe stages. This study appraised the accuracy, sensitivity, and specificity to detect lymphedema status using machine learning algorithms based on real-time symptom report. A web-based study was conducted to collect patients' real-time report of symptoms using a mHealth system. Data regarding demographic and clinical information, lymphedema status, and symptom features were collected. A total of 355 patients from 45 states in the US completed the study. Statistical and machine learning procedures were performed for data analysis. The performance of five renowned classification algorithms of machine learning were compared: Decision Tree of C4.5, Decision Tree of C5.0, gradient boosting model (GBM), artificial neural network (ANN), and support vector machine (SVM). Each classification algorithm has certain user-definable hyper parameters. Five-fold cross validation was used to optimize these hyper parameters and to choose the parameters that led to the highest average cross validation accuracy. Using machine leaning procedures comparing different algorithms is feasible. The ANN achieved the best performance for detecting lymphedema with accuracy of 93.75%, sensitivity of 95.65%, and specificity of 91.03%. A well-trained ANN classifier using real-time symptom report can provide highly accurate detection of lymphedema. Such detection accuracy is significantly higher than that achievable by current and often used clinical methods such as bio-impedance analysis. Use of a well-trained classification algorithm to detect lymphedema based on symptom features is a highly promising tool that may improve lymphedema outcomes.

Function Modeling Improves the Efficiency of Spatial Modeling Using Big Data from Remote Sensing

Spatial modeling is an integral component of most geographic information systems (GISs). However, conventional GIS modeling techniques can require substantial processing time and storage space and have limited statistical and machine learning functionality. To address these limitations, many have parallelized spatial models using multiple coding libraries and have applied those models in a multiprocessor environment. Few, however, have recognized the inefficiencies associated with the underlying spatial modeling framework used to implement such analyses. In this paper, we identify a common inefficiency in processing spatial models and demonstrate a novel approach to address it using lazy evaluation techniques. Furthermore, we introduce a new coding library that integrates Accord.NET and ALGLIB numeric libraries and uses lazy evaluation to facilitate a wide range of spatial, statistical, and machine learning procedures within a new GIS modeling framework called function modeling. Results from simulations show a 64.3% reduction in processing time and an 84.4% reduction in storage space attributable to function modeling. In an applied case study, this translated to a reduction in processing time from 2247 h to 488 h and a reduction is storage space from 152 terabytes to 913 gigabytes.

Filtering for Risk Assessment of Interbank Network

We develop a framework for risk assessment in an interbank network, where banks interact with each other via short-term debt contracts. Focusing on the demand side of liquidity and omitting credit rationing, the framework identifies the interbank network structure and its degree of interconnectedness. Identification is facilitated by a statistical learning procedure that reverse engineers signals (transactions) observed in the interbank market and conducts inference about banks’ individual and network-level characteristics. Performing a series of simulation studies after banking network identification, we measure the interconnectedness of the network with respect to its resilience to absorb systemic risk. The findings from the simulations imply that the benefits of integration outweigh the cost of interconnectedness as long as the interconnectedness technology is hedge serving (risk diversifying). If it is not the case, the simulation risk metrics we compute increase exponentially with the level of network interconnectedness. This, as a result, undermines the value of integration, even when the network is identified in its simplest forms.

A connectionist model of the retreat from verb argument structure overgeneralization.

A central question in language acquisition is how children build linguistic representations that allow them to generalize verbs from one construction to another (e.g., The boy gave a present to the girl → The boy gave the girl a present), whilst appropriately constraining those generalizations to avoid non-adultlike errors (e.g., I said no to her → *I said her no). Although a consensus is emerging that learners solve this problem using both statistical and semantics-based learning procedures (e.g., entrenchment, pre-emption, and semantic verb class formation), there currently exist few - if any - proposals for a learning model that combines these mechanisms. The present study used a connectionist model to test an account that argues for competition between constructions based on (a) verb-in construction frequency, (b) relevance of constructions for the speaker's intended message, and (c) fit between the fine-grained semantic properties of individual verbs and individual constructions. The model was able not only (a) to simulate the overall pattern of overgeneralization-then-retreat, but also (b) to use the semantics of novel verbs to predict their argument structure privileges (just as real learners do), and

Testing for Predictability in Financial Returns Using Statistical Learning Procedures

This article examines the ability of recently developed statistical learning procedures, such as random forests or support vector machines, for forecasting the first two moments of stock market daily returns. These tools present the advantage of the flexibility of the considered nonlinear regression functions even in the presence of many potential predictors. We consider two cases: where the agent's information set only includes the past of the return series, and where this set includes past values of relevant economic series, such as interest rates, commodities prices or exchange rates. Even though these procedures seem to be of no much use for predicting returns, it appears that there is real potential for some of these procedures, especially support vector machines, to improve over the standard GARCH(1,1) model the out‐of‐sample forecasting ability for squared returns. The researcher has to be cautious on the number of predictors employed and on the specific implementation of the procedures since using many predictors and the default settings of standard computing packages leads to overfitted models and to larger standard errors.

Privacy Aware Learning

We study statistical risk minimization problems under a privacy model in which the data is kept confidential even from the learner. In this local privacy framework, we establish sharp upper and lower bounds on the convergence rates of statistical estimation procedures. As a consequence, we exhibit a precise tradeoff between the amount of privacy the data preserves and the utility, as measured by convergence rate, of any statistical estimator or learning procedure.