Statistical Learning Applications Research Articles

Concentration bounds for the empirical angular measure with statistical learning applications

Concentration bounds for the empirical angular measure with statistical learning applications

Gaussian Mixture Model-Based Classification of Corrosion Severity in Concrete Structures Using Ultrasonic Imaging

Corrosion in reinforced concrete (RC) structures is associated with a reduction of the rebar diameter, loss of interfacial bond, cracking, and eventual spalling and probable collapse of the structure. The negative effects of corrosion on structural safety, durability, and longevity imposes significant costs on the national economy. Therefore, planned non-destructive testing (NDT) of concrete structures is essential to enhance the safety and economic sustainability of infrastructure. Previous work by the research group has established the capability of the ultrasonic Synthetic Aperture Focusing Technique (SAFT) as a tool for detection of rebar corrosion. This work extends the previous research towards application of statistical learning for ascertaining the corrosion severity through analysis of SAFT images of the rebar. Using features extracted from images, a Gaussian mixture model (GMM) is implemented to classify the severity of corrosion damage to the rebar. The results from the research positively demonstrate the potential of the proposed technique as an enabler for decisions pertaining to maintenance and timely repair of concrete infrastructural assets.

Application of statistical learning and mechanistic modelling towards mapping the substrate electronic space in a Cu-catalyzed Suzuki–Miyaura coupling

DFT and statistical learning allow mapping the electronic substrate effects on a Cu-catalyzed Suzuki–Miyaura reaction between iodobenzenes and aryl boronate esters. The resulting substrate map can be used to predict the outcome of similar reactions.

Statistical Machine Learning for Quantitative Finance

We survey the active interface of statistical learning methods and quantitative finance models. Our focus is on the use of statistical surrogates, also known as functional approximators, for learning input–output relationships relevant for financial tasks. Given the disparate terminology used among statisticians and financial mathematicians, we begin by reviewing the main ingredients of surrogate construction and the motivating financial tasks. We then summarize the major surrogate types, including (deep) neural networks, Gaussian processes, gradient boosting machines, smoothing splines, and Chebyshev polynomials. The second half of the article dives deeper into the major applications of statistical learning in finance, covering ( a) parametric option pricing, ( b) learning the implied/local volatility surface, ( c) learning option sensitivities, ( d) American option pricing, and ( e) model calibration. We also briefly detail statistical learning for stochastic control and reinforcement learning, two areas of research exploding in popularity in quantitative finance.

Application of Statistical Learning to Identify Omicron Mutations in SARS-CoV-2 Viral Genome Sequence Data From Populations in Africa and the United States

With timely collection of SARS-CoV-2 viral genome sequences, it is important to apply efficient data analytics to detect emerging variants at the earliest time. To evaluate the application of a statistical learning strategy (SLS) to improve early detection of novel SARS-CoV-2 variants using viral sequence data from global surveillance. This case series applied an SLS to viral genomic sequence data collected from 63 686 individuals in Africa and 531 827 individuals in the United States with SARS-CoV-2. Data were collected from January 1, 2020, to December 28, 2021. The outcome was an indicator of Omicron variant derived from viral sequences. Centering on a temporally collected outcome, the SLS used the generalized additive model to estimate locally averaged Omicron caseload percentages (OCPs) over time to characterize Omicron expansion and to estimate when OCP exceeded 10%, 25%, 50%, and 75% of the caseload. Additionally, an unsupervised learning technique was applied to visualize Omicron expansions, and temporal and spatial distributions of Omicron cases were investigated. In total, there were 2698 cases of Omicron in Africa and 12 141 in the United States. The SLS found that Omicron was detectable in South Africa as early as December 31, 2020. With 10% OCP as a threshold, it may have been possible to declare Omicron a variant of concern as early as November 4, 2021, in South Africa. In the United States, the application of SLS suggested that the first case was detectable on November 21, 2021. The application of SLS demonstrates how the Omicron variant may have emerged and expanded in Africa and the United States. Earlier detection could help the global effort in disease prevention and control. To optimize early detection, efficient data analytics, such as SLS, could assist in the rapid identification of new variants as soon as they emerge, with or without lineages designated, using viral sequence data from global surveillance.

A First-Order Optimization Algorithm for Statistical Learning with Hierarchical Sparsity Structure

In many statistical learning problems, it is desired that the optimal solution conform to an a priori known sparsity structure represented by a directed acyclic graph. Inducing such structures by means of convex regularizers requires nonsmooth penalty functions that exploit group overlapping. Our study focuses on evaluating the proximal operator of the latent overlapping group lasso developed by Jacob et al. in 2009 . We implemented an alternating direction method of multiplier with a sharing scheme to solve large-scale instances of the underlying optimization problem efficiently. In the absence of strong convexity, global linear convergence of the algorithm is established using the error bound theory. More specifically, the paper contributes to establishing primal and dual error bounds when the nonsmooth component in the objective function does not have a polyhedral epigraph. We also investigate the effect of the graph structure on the speed of convergence of the algorithm. Detailed numerical simulation studies over different graph structures supporting the proposed algorithm and two applications in learning are provided. Summary of Contribution: The paper proposes a computationally efficient optimization algorithm to evaluate the proximal operator of a nonsmooth hierarchical sparsity-inducing regularizer and establishes its convergence properties. The computationally intensive subproblem of the proposed algorithm can be fully parallelized, which allows solving large-scale instances of the underlying problem. Comprehensive numerical simulation studies benchmarking the proposed algorithm against five other methods on the speed of convergence to optimality are provided. Furthermore, performance of the algorithm is demonstrated on two statistical learning applications related to topic modeling and breast cancer classification. The code along with the simulation studies and benchmarks are available on the corresponding author’s GitHub website for evaluation and future use.

Resilient satellite navigation empowers modern science, economy, and society

Established on the exact mathematical principles, understanding of radio-wave propagation, and statistical signal processing and information theory, satellite navigation has become an essential cornerstone of modern civilisation, and an indispensable component of the national infrastructure. The increasing number of both navigation and non-navigation applications of the Global Navigations Satellite System (GNSS) utilise its Positioning, Navigation, and Timing (PNT) service for technology and business development, daily operation of technology and socio-economic systems, and improvement of the quality of life. The inherent shortcomings and limitations of GNSS ask for a transition towards the GNSS resilient to natural and artificial detrimental effects, which degrade the GNSS positioning performance. Here a systematic overview of the causes of the GNSS positioning performance degradation is outlined. Recent developments in mathematics and computer science are discussed as fundamental in the GNSS resilience development. Formulation of the Satellite-Positioning-as-a-Service (SPaaS) is outlined, as the new fundamental paradigm for resilient GNSS. Finally, the effects of the SpaaS on the wide range of GNSS applications in science, economy, and society are discussed. The contributions to SPaaS concept and the related developments through the application of statistical learning, mathematical methods and models development, and applications discussed result from the author’s involvement in numerous international strategic, technology, regulatory, standardisation, business, and academic education development and collaboration activities.

Geometric Morphometric Data Augmentation Using Generative Computational Learning Algorithms

The fossil record is notorious for being incomplete and distorted, frequently conditioning the type of knowledge that can be extracted from it. In many cases, this often leads to issues when performing complex statistical analyses, such as classification tasks, predictive modelling, and variance analyses, such as those used in Geometric Morphometrics. Here different Generative Adversarial Network architectures are experimented with, testing the effects of sample size and domain dimensionality on model performance. For model evaluation, robust statistical methods were used. Each of the algorithms were observed to produce realistic data. Generative Adversarial Networks using different loss functions produced multidimensional synthetic data significantly equivalent to the original training data. Conditional Generative Adversarial Networks were not as successful. The methods proposed are likely to reduce the impact of sample size and bias on a number of statistical learning applications. While Generative Adversarial Networks are not the solution to all sample-size related issues, combined with other pre-processing steps these limitations may be overcome. This presents a valuable means of augmenting geometric morphometric datasets for greater predictive visualization.

Nonconvex robust programming via value-function optimization

Convex programming based robust optimization is an active research topic in the past two decades, partially because of its computational tractability for many classes of optimization problems and uncertainty sets. However, many problems arising from modern operations research and statistical learning applications are nonconvex even in the nominal case, let alone their robust counterpart. In this paper, we introduce a systematic approach for tackling the nonconvexity of the robust optimization problems that is usually coupled with the nonsmoothness of the objective function brought by the worst-case value function. A majorization-minimization algorithm is presented to solve the penalized min-max formulation of the robustified problem that deterministically generates a “better” solution compared with the starting point (that is usually chosen as an unrobustfied optimal solution). A generalized saddle-point theorem regarding the directional stationarity is established and a game-theoretic interpretation of the computed solutions is provided. Numerical experiments show that the computed solutions of the nonconvex robust optimization problems are less sensitive to the data perturbation compared with the unrobustfied ones.

Machine Learning for Occupation Coding—A Comparison Study

Abstract Asking people about their occupation is common practice in surveys and censuses around the world. The answers are typically recorded in textual form and subsequently assigned (coded) to categories, which have been defined in official occupational classifications. While this coding step is often done manually, substituting it with more automated workflows has been a longstanding goal, promising reduced data-processing costs and accelerated publication of key statistics. Although numerous researchers have developed different algorithms for automated occupation coding, the algorithms have rarely been compared with each other or tested on different data sets. We fill this gap by comparing some of the most promising algorithms found in the literature and testing them on five data sets from Germany. The first two algorithms we test exemplify a common practice in which answers are coded automatically according to a predefined list of job titles. Statistical learning algorithms—that is, regularized multinomial regression, tree boosting, or algorithms developed specifically for occupation coding (algorithms three to six)—can improve upon algorithms one and two, but only if a sufficient number of training observations from previous surveys is available. The best results are obtained by merging the list of job titles with coded answers from previous surveys before using this combined training data for statistical learning (algorithm 7). However, the differences between the algorithms are often small compared to the large variation found across different data sets, which we ascribe to systematic differences in the way the data were coded in the first place. Such differences complicate the application of statistical learning, which risks perpetuating questionable coding decisions from the training data to the future.

Smart media and application

Smart media and application

Dataset retrieval system based on automation of data preparation with dataset description model

SummaryData preparation is the most effortful task in the process of statistical learning. Many studies related to data mining are performed without data preparation by assuming that qualified datasets are already prepared. It may hide useful patterns of data, which can result in poor performance and incorrect learning. Automation of data preparation can solve these problems. For automation of data preparation, a few issues should be considered, such as flexible expression of requirements according to the purpose of the learning model, accessibility to data sources, and performance degradation due to automation. In this paper, we propose a dataset description model that can express the requirements for data processing and dataset retrieval system based on automated data preparation. The proposed system makes it possible to provide good quality datasets for statistical learning applications using data preparation methods such as data acquisition, refinement, and organization. In the experiment, we demonstrate that the proposed system doesn't have performance loss as compared to the existing manual systems. Moreover, the quality of the datasets are also improved by using the proposed system.

Restricted strong convexity implies weak submodularity

We connect high-dimensional subset selection and submodular maximization. Our results extend the work of Das and Kempe [In ICML (2011) 1057–1064] from the setting of linear regression to arbitrary objective functions. For greedy feature selection, this connection allows us to obtain strong multiplicative performance bounds on several methods without statistical modeling assumptions. We also derive recovery guarantees of this form under standard assumptions. Our work shows that greedy algorithms perform within a constant factor from the best possible subset-selection solution for a broad class of general objective functions. Our methods allow a direct control over the number of obtained features as opposed to regularization parameters that only implicitly control sparsity. Our proof technique uses the concept of weak submodularity initially defined by Das and Kempe. We draw a connection between convex analysis and submodular set function theory which may be of independent interest for other statistical learning applications that have combinatorial structure.

Measuring the Stability of Results From Supervised Statistical Learning

ABSTRACTStability is a major requirement to draw reliable conclusions when interpreting results from supervised statistical learning. In this article, we present a general framework for assessing and comparing the stability of results, which can be used in real-world statistical learning applications as well as in simulation and benchmark studies. We use the framework to show that stability is a property of both the algorithm and the data-generating process. In particular, we demonstrate that unstable algorithms (such as recursive partitioning) can produce stable results when the functional form of the relationship between the predictors and the response matches the algorithm. Typical uses of the framework in practical data analysis would be to compare the stability of results generated by different candidate algorithms for a dataset at hand or to assess the stability of algorithms in a benchmark study. Code to perform the stability analyses is provided in the form of an R package. Supplementary material for this article is available online.

Fast learning of scale‐free networks based on Cholesky factorization

Recovering network connectivity structure from high-dimensional observations is of increasing importance in statistical learning applications. A prominent approach is to learn a Sparse Gaussian Markov Random Field by optimizing regularized maximum likelihood, where the sparsity is induced by imposing L1 norm on the entries of a precision matrix. In this article, we shed light on an alternative objective, where instead of precision, its Cholesky factor is penalized by the L1 norm. We show that such an objective criterion possesses attractive properties that allowed us to develop a very fast Scale-Free Networks Estimation Through Cholesky factorization (SNETCH) optimization algorithm based on coordinate descent, which is highly parallelizable and can exploit an active set approach. The approach is particularly suited for problems with structures that allow sparse Cholesky factor, an important example being scale-free networks. Evaluation on synthetically generated examples and high-impact applications from a biomedical domain of up to more than 900,000 variables provides evidence that for such tasks the SNETCH algorithm can learn the underlying structure more accurately, and an order of magnitude faster than state-of-the-art approaches based on the L1 penalized precision matrix.

On Reject and Refine Options in Multicategory Classification

ABSTRACTIn many real applications of statistical learning, a decision made from misclassification can be too costly to afford; in this case, a reject option, which defers the decision until further investigation is conducted, is often preferred. In recent years, there has been much development for binary classification with a reject option. Yet, little progress has been made for the multicategory case. In this article, we propose margin-based multicategory classification methods with a reject option. In addition, and more importantly, we introduce a new and unique refine option for the multicategory problem, where the class of an observation is predicted to be from a set of class labels, whose cardinality is not necessarily one. The main advantage of both options lies in their capacity of identifying error-prone observations. Moreover, the refine option can provide more constructive information for classification by effectively ruling out implausible classes. Efficient implementations have been developed for the proposed methods. On the theoretical side, we offer a novel statistical learning theory and show a fast convergence rate of the excess ℓ-risk of our methods with emphasis on diverging dimensionality and number of classes. The results can be further improved under a low noise assumption and be generalized to the excess 0-d-1 risk. Finite-sample upper bounds for the reject and reject/refine rates are also provided. A set of comprehensive simulation and real data studies has shown the usefulness of the new learning tools compared to regular multicategory classifiers. Detailed proofs of theorems and extended numerical results are included in the supplemental materials available online.

PEMBELAJARAN STATISTIKA MELALUI PROJECT BASED LEARNING DENGAN BANTUAN MICROSOFT EXCEL UNTUK MENINGKATKAN HASIL BELAJAR PESERTA DIDIK VIII D SMPN UNGGULAN SINDANG INDRAMAYU

The aim of this research is to improve the learning outcomes of VIIID students of SMP Negeri Unggulan Sindang Indramayu school year 2014/2015 on Statistic material through Project Based Learning Model-Aided Microsoft Excel. The subjects of this research is VIII D students of SMP Negeri Unggulan Sindang Indramayu school year 2014/2015, consist of 30 students, 19 girls and 11 boys. Data collection techniques in this study using observation sheets, tests, observations, questionnaires, interviews and documentation. Statistics attitude towards learning through Project Based Learning Model-Aided Microsoft Excel mostly been good and very good. Can be seen the percentage of successful the first cycle increased from 86.67% to 100% in the second cycle. Percentage mastery learning outcomes in mathematics learning competency skills is increase. The percentage of successful at first cycle increased from 73% to 86.67% in the second cycle. Learning outcomes based on knowledge an increasing number of which reached KKM cycle I to cycle II. KKM percentage reached 76.67% in the first cycle increased to 86.67% in the second cycle. Based on the research that has been conducted, statistics learning using Project Based Learning Model-Aided Microsoft Excel is able to improve the learning outcomes of class VIIID students of SMP Negeri Unggulan Sindang Indramayu school year 2014/2015. The application of statistical learning using Project Based Learning Model-Aided Microsoft Excel as an alternative model of learning that can be applied to topics and other subjects but required good planning and proper time management.

A Dependence Stability Bound based on the VC Dimension for Relational Classification

Relational classification (RC) is concerned with the application of statistical learning to relational data. RC models do not have improved stability to smooth the perturbations generated by variations in the correlation between the relational data. Therefore, few studies have attempted to derive a bound and develop a stability learning framework for RC models. To solve this problem, we derive a learning bound with a new measure dependence stability and a limited Vapnik–Chervonenkis (VC) dimension. Based on the learning bound, we then design a stable learning framework that serves as a guideline for the development of new learning algorithms for a broad class of RC models. Applying a Markov logic network on synthesized and real-world datasets, our experimental results demonstrate that our bound can be tight if the RC model has appropriate dependence stability and limited VC dimension and our learning framework increases the stability of RC models while reducing the deviation between empirical risk and true risk.

Multiple influences on regolith characteristics from continental-scale geophysical and mineralogical remote sensing data using Self-Organizing Maps

We investigate the characteristics of regolith through the application of statistical learning to diverse layers of terrestrial, continental-scale remote sensing data. This combination allows us to explore the multiple influences of bedrock, climate, biota, landscape and time on regolith development and properties: an interdisciplinary geoscience modeling problem. From a wide variety of available data for Australia, we select remotely sensed geophysical, geomorphological and mineralogical inputs with good spatial coverage. We use Self-Organizing Maps (SOM), a topologically constrained unsupervised statistical learning algorithm, to characterize the geophysical and mineralogical signatures of regolith and bedrock. Regolith materials cover more than 80% of the Australian continent, range in age from Precambrian to Quaternary and vary in thickness from less than a meter to more than a kilometer. The diversity of regolith cover type and character across Australia provides an opportunity to demonstrate knowledge discovery from remote sensing data. The outputs of our SOM analysis are combined with ground observations from locations showing naturally occurring anomalous concentrations of nickel, tin and uranium. We identify a minimum number of natural clusters indicating subtle but significant differences in regolith and bedrock mineralization characteristics. Our results show that SOM identifies spatially contiguous regions representing unique regolith and bedrock materials. In the Yilgarn Craton we observe key differences in landscape character, density of the crust, and relative abundance of radioactive elements and alumino-silicate and ferric oxide minerals. These properties discriminate between nickel-prospective residual deeply weathered regolith formed on mafic and/or ultramafic bedrock and uranium-prospective Cainozoic paleochannels containing felsic bedrock source materials. National-scale data are publicly available for many continental regions, as in the Australian example, and our approach has general applicability. We demonstrate that remote sensing data may be used to understand the regolith, revealing the interplay between environmental history and bedrock character at regional scales, and differences between residual and transported regolith, provenance of source materials and their relative ages.

Fast sparse regression and classification

Abstract Many present day applications of statistical learning involve large numbers of predictor variables. Often, that number is much larger than the number of cases or observations available for training the learning algorithm. In such situations, traditional methods fail. Recently, new techniques have been developed, based on regularization, which can often produce accurate models in these settings. This paper describes the basic principles underlying the method of regularization, then focuses on those methods which exploit the sparsity of the predicting model. The potential merits of these methods are then explored by example.