Risk Minimization Method Research Articles

Statistical Learning Theory and Occam’s Razor: The Core Argument

Statistical learning theory is often associated with the principle of Occam’s razor, which recommends a simplicity preference in inductive inference. This paper distills the core argument for simplicity obtainable from statistical learning theory, built on the theory’s central learning guarantee for the method of empirical risk minimization. This core “means-ends” argument is that a simpler hypothesis class or inductive model is better because it has better learning guarantees; however, these guarantees are model-relative and so the theoretical push towards simplicity is checked by our prior knowledge.

Feature selection under budget constraint in medical applications: analysis of penalized empirical risk minimization methods

Feature selection is a crucial step when building supervised predictive models. In many medical applications, features are associated with costs. For example, the diagnostic value extracted by a clinical test is associated with its own cost. Costs can also refer to a non-financial aspects, such as a decision between an invasive exploratory surgery and a simple blood test. Traditional feature selection methods, which ignore costs, aim to choose a subset of features that maximize the accuracy of the corresponding model. However, such a model can be impractical as the total cost of making a prediction may exceed the assumed user-specified budget. In cost-constrained methods, it is necessary to take into account both the relevance of the feature and its cost. We focus on embedded feature selection methods based on a very general penalized empirical risk minimization framework that includes various loss functions. The most natural ℓ0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _0$$\\end{document}-type penalty is computationally intractable and therefore we analyze other penalties such as: the cost-sensitive lasso and adaptive lasso, non-convex penalties and the method based on knockoffs. The experiments performed on real medical datasets, including large database MIMIC, indicate that non-convex penalties give promising results, in particular they allow to achieve high accuracy, especially when the assumed budget is low. Our model achieved AUC 0.88 for the MIMIC-II dataset in which we predict the occurrence of liver diseases based on clinical features with a budget equal to (5%) of the cost of all available features, which is significantly better than the AUC for the traditional method. Moreover, the fraction of feature cost wasted for noisy features in our method is usually lower than for cost-sensitive lasso.

Manifold Regularized Joint Transfer for Open Set Domain Adaptation

Unsupervised Domain Adaptation (UDA) aims to leverage knowledge of a well-labeled source domain to learn an effective classifier for an unlabeled target domain. However, a common scenario in real-world applications is that the target domain contains unknown categories that are not observed in the source domain. This setting is termed as open set domain adaptation (OSDA). Most existing approaches of OSDA can only classify known classes well but fail to recognize unknown samples effectively. In this paper, we propose an effective method, named manifold regularized joint transfer (MRJT), for OSDA. MRJT learns new feature representations by simultaneously reducing distribution discrepancy between domains, increasing compactness of within-class, discriminating different known classes, and distinguishing the unknown from the known. The learned new features are projected onto reproducing kernel Hilbert space. In this space, a weighted structural risk minimization method is integrated with manifold regularization to utilize geometric information sufficiently to learn an effective classifier. Extensive experimental results on four real-world datasets verify the superiority of our method. It can not only classify known samples into the right known classes but also recognize unknown samples effectively.

Distribution matching and structure preservation for domain adaptation

Cross-domain classification refers to completing the corresponding classification task in a target domain which lacks label information, by exploring useful knowledge in a related source domain but with different data distribution. Domain adaptation can deal with such cross-domain classification, by reducing divergence of domains and transferring the relevant knowledge from the source to the target. To mine the discriminant information of the source domain samples and the geometric structure information of domains, and thus improve domain adaptation performance, this paper proposes a novel method involving distribution matching and structure preservation for domain adaptation (DMSP). First, it aligns the subspaces of the source domain and target domain on the Grassmann manifold; and learns the non-distorted embedded feature representations of the two domains. Second, in this embedded feature space, the empirical structure risk minimization method with distribution adaptation regularization and intra-domain graph regularization is used to learn an adaptive classifier, further adapting the source and target domains. Finally, we perform extensive experiments on widely used cross-domain classification datasets to validate the superiority of DMSP. The average classification accuracy of DMSP on these datasets is the highest compared with several state-of-the-art domain adaptation methods.

Adversarially robust subspace learning in the spiked covariance model

AbstractWe study the problem of robust subspace learning when there is an adversary who can attack the data to increase the projection error. By deriving the adversarial projection risk when data follows the multivariate Gaussian distribution with the spiked covariance, or so‐called the Spiked Covariance model, we propose to use the empirical risk minimization method to obtain the optimal robust subspace. We then find a non‐asymptotic upper bound of the adversarial excess risk, which implies the empirical risk minimization estimator is close to the optimal robust adversarial subspace. The optimization problem can be solved easily by the projected gradient descent algorithm for the rank‐one spiked covariance model. However, in general, it is computationally intractable to solve the empirical risk minimization problem. Thus, we propose to minimize an upper bound of the empirical risk to find the robust subspace for the general spiked covariance model. Finally, we conduct numerical experiments to show the robustness of our proposed algorithms.

The Role of Risk Forecast and Risk Tolerance in Portfolio Management: A Case Study of the Chinese Financial Sector

Portfolio decisions are affected by the volatility of financial markets and investors’ risk tolerance levels. To better allocate portfolios; we introduce risk tolerance into the portfolio management problem by considering the risk contribution of portfolio components. In this paper, portfolio weights are allocated to two stages. In the first stage, the portfolio risks and the risk contribution of each share are forecasted. In the second stage, we put forward three weighting techniques—“aggressive”, “moderate” and “conservative”, according to three standard levels of risk tolerance. In addition, a new risk measure called “joint extreme risk probability” (JERP), with risk tolerance taken into account, is proposed. A case study of the Chinese financial industry is conducted to verify the performance of our methods. The empirical results demonstrate that weighting techniques constrained by risk tolerance lead to higher gains in a normal market and less loss when a market is risky. Compared with risk-tolerance-adjusted strategies, the relationship between the performance of the traditional conditional value at risk (CVaR) minimization method and the market risk level is less obviously demonstrated. Viewed from the results, JERP functions as an effective signal that helps investors to deal with potential market risks.

Feature Selection Under Budget Constraint in Medical Applications: Analysis of Penalized Empirical Risk Minimization Methods

Feature Selection Under Budget Constraint in Medical Applications: Analysis of Penalized Empirical Risk Minimization Methods

Anti-Corruption Tools of Financial Risk Management in Public Administration

Every country is concerned with the problem of corruption. Selfish misuse of public office undermines the people's confidence in government and institutions, makes public policies less effective and fair, and misuses taxpayer funds that could be used to build schools, roads and hospitals. Financial risk management in public administration is based on risk assessments and finding tools to help influence them. To develop practical tools for financial risk management, the authors studied the economic practises that determine the features of financial risk assessment. They identified the elements that can motivate financial risks, including three stages of identification and analysis of risks, analyzed the main stages and indicators of assessing an organization's financial condition based on the balance sheet, produced a risk assessment algorithm and risk minimization methods in managing financial activities, and conducted a comparative analysis of financial risk assessment methods. Based on the Professional Integrity Framework proposed by the Organization for Economic Cooperation and Development, the authors developed basic measures to mitigate the potential negative consequences and proposed seven financial risk management tools for public administration. The proposed anti-corruption tools will help significantly reduce financial risks in public administration and corruption in general.

Statistical generalization performance guarantee for meta-learning with data dependent prior

Meta-learning aims to leverage experience from previous tasks to achieve an effective and fast adaptation ability when encountering new tasks. However, it is unclear how the generalization property applies to new tasks. Probably approximately correct (PAC) Bayes bound theory provides a theoretical framework to analyze the generalization performance for meta-learning with an explicit numerical generalization error upper bound. A tighter upper bound may achieve better generalization performance. However, for the PAC-Bayes meta-learning bound, the prior distribution is selected randomly which results in poor generalization performance.In this paper, we derive three novel generalization error upper bounds for meta-learning based on the PAC-Bayes relative entropy bound. Furthermore, in order to avoid randomly prior distribution, based on the empirical risk minimization (ERM) method, a data-dependent prior for the PAC-Bayes meta-learning bound algorithm is developed and the sample complexity and computational complexity are analyzed. The experiments illustrate that the proposed three PAC-Bayes bounds for meta-learning achieve a competitive generalization guarantee, and the extended PAC-Bayes bound with a data-dependent prior can achieve rapid convergence ability.

Weighted distributed differential privacy ERM: Convex and non-convex

Distributed machine learning allows different parties to learn a single model over all data sets without disclosing their own data. In this paper, we propose a weighted distributed differentially private (WD-DP) empirical risk minimization (ERM) method to train a model in distributed setting, considering different weights of different clients. For the first time, we theoretically analyze the benefits brought by weighted paradigm in distributed differentially private machine learning. Our method advances the state-of-the-art differentially private ERM methods in distributed setting. By detailed theoretical analysis, we show that in distributed setting, the noise bound and the excess empirical risk bound can be improved by considering different weights held by multiple parties. Additionally, in some situations, the constraint: strongly convexity of the loss function in ERM is not easy to achieve, so we generalize our method to the condition that the loss function is not restricted to be strongly convex but satisfies the Polyak-Łojasiewicz condition. Experiments on real data sets show that our method is more reliable and we improve the performance of distributed differentially private ERM, especially in the case that data scales on different clients are uneven. Moreover, it is an attractive result that our distributed method achieves almost the same theoretical and experimental results as previous centralized methods.

Classification From Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization.

Pairwise similarities and dissimilarities between data points are often obtained more easily than full labels of data in real-world classification problems. To make use of such pairwise information, an empirical risk minimization approach has been proposed, where an unbiased estimator of the classification risk is computed from only pairwise similarities and unlabeled data. However, this approach has not yet been able to handle pairwise dissimilarities. Semisupervised clustering methods can incorporate both similarities and dissimilarities into their framework; however, they typically require strong geometrical assumptions on the data distribution such as the manifold assumption, which may cause severe performance deterioration. In this letter, we derive an unbiased estimator of the classification risk based on all of similarities and dissimilarities and unlabeled data. We theoretically establish an estimation error bound and experimentally demonstrate the practical usefulness of our empirical risk minimization method.

Hedging strategy for unit-linked life insurance contracts with self-exciting jump clustering

<p style='text-indent:20px;'>This paper studies the hedging problem of unit-linked life insurance contracts in an incomplete market presence of self-exciting (clustering) effect, which is described by a Hawkes process. Applying the local risk-minimization method, we manage to obtain closed-form expressions of the locally risk-minimizing hedging strategies for both pure endowment and term insurance contracts. Besides, we demonstrate the existence of the minimal martingale measure and perform numerical analyses. Our numerical results indicate that jump clustering has a significant impact on the optimal hedging strategies.</p>

Methods of risk minimization in investment and construction projects

The paper puts forward a hypothesis about the possible integration of risk assessment methods in order to minimize them in investment and construction projects, for that the problems of risk minimization in the management and implementation of construction projects are considered. A critical review of existing methods of risk analysis is carried out and a method of risk minimization is proposed based on the creation of a “Construction risk Informatoria” and methods for optimizing the reallocation of resources according to the minimum time criterion, which determines the risk of lost profits. The example of the use of risk minimization methods in the optimization procedure for network models of low-rise building construction is given, and the composition of the application of mathematical methods and models that contribute to increasing the economic effect in this area is described. As a result, effective recommendations were developed in the form of tools for monitoring the dynamics of construction work to prevent risk situations.

Beyond risk parity – A machine learning-based hierarchical risk parity approach on cryptocurrencies

It has long been known that estimating large empirical covariance matrices can lead to very unstable solutions, with estimation errors more than offsetting the benefits of diversification. In this study, we employ the Hierarchical Risk Parity approach, which applies state-of-the-art mathematics including graph theory and unsupervised machine learning to a large portfolio of cryptocurrencies. An out-of-sample comparison with traditional risk-minimization methods reveals that Hierarchical Risk Parity outperforms in terms of tail risk-adjusted return, thereby working as a potential risk management tool that can help cryptocurrency investors to better manage portfolio risk. The results are robust to different rebalancing intervals, covariance estimation windows and methodologies.

Risk minimization methods for loading dangerous goods into tanker trains

Risk minimization methods for loading dangerous goods into tanker trains

Random distributional response model based on spline method

The aim of this paper is to develop a random distribution on scalar regression model framework to account for the entire subject-specific distribution of the outcome, and to relate these distributions to a set of covariates. We develop a basis representation by approximating quantile functions of distributions measured at the grid points in [0,1], based on the non-decreasing basis function. Then, we estimate the basis coefficients of covariates by using an empirical risk minimization method, with the restriction that all subject-specific quantile functions have their monotone increasing property in the original data space. Finally, we conduct a test procedure to identify the association between the distribution and a set of covariates. We systematically investigate the rate of convergence of our estimator and its asymptotic distribution under the nullity. We also show our estimation and test procedures through simulations and an analysis of Canadian weather data obtained from a historical database.

Fair dynamic valuation of insurance liabilities: Merging actuarial judgement with market- and time-consistency

In this paper, we investigate the fair valuation of insurance liabilities in a dynamic multi-period setting. We define a fair dynamic valuation as a valuation which is actuarial (mark-to-model for claims independent of financial market evolutions), market-consistent (mark-to-market for any hedgeable part of a claim) and time-consistent, extending the work of Dhaene et al. (2017) and Barigou and Dhaene (2019). We provide a complete hedging characterization for fair dynamic valuations. Moreover, we show how to implement fair dynamic valuations through a backward iterations scheme combining risk minimization methods from mathematical finance with standard actuarial techniques based on risk measures.

Fair Dynamic Valuation of Insurance Liabilities: Merging Actuarial Judgement With Market- and Time-Consistency

In this paper, we investigate the fair valuation of insurance liabilities in a dynamic multi-period setting. We define a fair dynamic valuation as a valuation which is actuarial (mark-to-model for claims independent of financial market evolutions), market-consistent (mark-to-market for any hedgeable part of a claim) and time-consistent, extending the work of Dhaene et al. (2017) and Barigou and Dhaene (2018). We provide a complete hedging characterization for fair dynamic valuations. Moreover, we show how to implement fair dynamic valuations through a backward iterations scheme combining risk minimization methods from mathematical finance with standard actuarial techniques based on risk measures.

Knowledge acquisition and decision making based on Bayes risk minimization method

There are two central parts in multiple attribute decision making (MADM), which are weight assignment and attribute selection. However, attribute selection is usually ignored in the existing researches, which will result in the difficulty of knowledge acquisition and the error of decision making. In addition, with respect to the data set with labels, the existing methods of weight assignment usually neglect or do not take full advantage of the supervisory function of labels, which may also lead to some decision making mistakes. To make up for these deficiencies, this paper proposes a method for knowledge acquisition and decision making based on Bayes risk minimization. In this method, a novel Bayes risk model based on neighborhood and Gaussian kernel is raised, and a heuristic forward greedy algorithm is designed for attribute selection. Finally, a number of experiments, including the comparison experiments on University of California Irvine (UCI) data and the effectiveness evaluation of fighter, are carried out to illustrate the superiority and applicability of the proposed method.

Statistical learning with group invariance: problem, method and consistency

Statistical learning theory (SLT) provides the theoretical basis for many machine learning algorithms (e.g. SVMs and kernel methods). Invariance, as one type of popular prior knowledge in pattern analysis, has been widely incorporated into various statistical learning algorithms to improve learning performance. Though successful in some applications, existing invariance learning algorithms are task-specific, and lack a solid theoretical basis including consistency. In this paper, we first propose the problem of statistical learning with group invariance (or group invariance learning in short) to provide a unifying framework for existing invariance learning algorithms in pattern analysis by exploiting group invariance. We then introduce the group invariance empirical risk minimization (GIERM) method to solve the group invariance learning problem, which incorporates the group action on the original data into empirical risk minimization (ERM). Finally, we investigate the consistency of the GIERM method in detail. Our theoretical results include three theorems, covering the necessary and sufficient conditions of consistency, uniform two-sided convergence and uniform one-sided convergence for the group invariance learning process based on the GIERM method.