Effect Of Outliers Research Articles

A systematic review and meta-analysis of the prevalence of childhood undernutrition in North Africa.

Undernutrition (stunting, wasting and underweight) among children remains a public health concern in North Africa, especially following recent conflicts in the region. Therefore, this paper systematically reviews and meta-analyses the prevalence of undernutrition among children under five in North Africa to determine whether efforts to reduce undernutrition are on track to achieving the Sustainable Development Goals (SDGs) by 2030. Eligible studies published between 1st January 2006 and 10th April 2022 were searched for, using five electronic bibliographic databases (Ovid MEDLINE, Web of Science, Embase (Ovid), ProQuest and CINAHL). The JBI critical appraisal tool was used, and a meta-analysis was conducted using the 'metaprop' command in STATA, to estimate the prevalence of each undernutrition indicator in the seven North African countries (Egypt, Sudan, Libya, Algeria, Tunisia, Morocco, and Western Sahara). Due to the significant heterogeneity among studies (I2 >50%), a random effect model and sensitivity analysis were conducted to examine the effect of outliers. Out of 1592 initially identified, 27 met the selection criteria. The prevalence of stunting, wasting and being underweight were 23.5%, 7.9% and 12.9%, respectively. Significant variations between the countries with the highest rates of stunting and wasting were reported in Sudan (36%, 14.1%), Egypt (23.7%, 7.5%), Libya (23.1%, 5.9%), and Morocco (19.9%, 5.1%). Sudan also had the highest prevalence of underweight (24.6%), followed by Egypt (7%), Morocco (6.1%), and Libya (4.3%) with more than one in ten children in Algeria and Tunisia having stunted growth. In conclusion, undernutrition is widespread in the North African region, particularly in Sudan, Egypt, Libya, and Morocco, making it challenging to meet the SDGs by 2030. Nutrition monitoring and evaluation in these countries is highly recommended.

From share of choice to buyers' welfare maximization: Bridging the gap through distributionally robust optimization

We study the product design problem where a decision maker selects the features of a product from a set of feasible options. We focus on two widely studied objectives in this field, that is, the share of choice (SOC) and buyers' welfare (BW). The two objectives are vulnerable to different types of customer preference misspecification, that is, deviation from the nominal utility distribution and effects of outliers, respectively. We formulate a distributionally robust optimization (DRO) SOC maximization model and a winsorized BW maximization model to obtain robust solutions to the two problems. Interestingly, we show that the two robust models are equivalent in a certain sense—for appropriate choices of robustness parameters, both models return the same solution. This observation has important ramifications. For instance, it indicates that a product designed to yield higher BW is more robust to the deviation from the nominal utility distribution for the SOC problem, while that with higher SOC is less sensitive to the effect of outliers for the BW problem. Last but not least, we use the equivalence to develop a new approach to solve the DRO model for the SOC problem, using a version of the winsorized BW model. Extensive numerical experiments demonstrate the superior performance of the proposed approach.

The effects of outdated data and outliers on Kenya’s 2019 Global Food Security Index score and rank

The effects of outdated data and outliers on Kenya’s 2019 Global Food Security Index score and rank

A INFLUÊNCIA DOS INVESTIMENTOS PÚBLICOS ESTADUAIS EM CIÊNCIA, TECNOLOGIA E INOVAÇÃO NO DESENVOLVIMENTO REGIONAL

Investments in Science, Technology and Innovation (ST&I) are important for increasing economic competitiveness and for choosing the development process. This article addresses the influence of state public investments in ST&I on regional development. For this purpose, an econometric model of panel analysis was used with data from 2000 to 2012, with 13 annual observations for the variables of each UF. A proxy for economic growth was used as a dependent variable as explanatory variables were the value of public expenditure used. in Research and Development (R&D) for each state (DISP) and the proxy for innovation, represented by a state science and technology indicator, in addition to the human capital control (KH) variables, the population of each of the Brazilian states (POP) and a Dummy for mitigate the effects of outliers. By estimating the panel data model, it was possible to prove the hypothesis that state public investment in R&D is a necessary condition to generate further innovation in the private sector and promote economic growth, even though this effect is asymmetric and concentrated.

An Optimization-Based Robust Dynamic State Estimation for Power Systems with Synchronized Phasor Measurement Units, Involving Disturbance Rejection

State estimation provides the best estimate of the system states using a set of measurements that is divided into two main classes; static and dynamic types. The state estimator should provide an acceptable response in the presence of system changes and/or disturbances. To achieve this aim, considering the high rate of data changes and the high volume of exchanging information, the phasor measurement units could play an outstanding role. Furthermore, due to the inevitable errors of the measurements, and to provide an authentic estimate of the system states using the measured data, a state estimator should be able to eliminate or decrease the undesirable effect of bad data or outliers on the final output of the process. This type of state estimator is called a robust type. In this paper, a dynamic state estimator for a system with the synchronized phasor measurement units is utilized, which is capable of decreasing the effect of disturbances. Simultaneously, the robustness of the state estimation process in the presence of outliers is considered, which enables the state estimation process to reduce the negative effect of outliers on its final output. Therefore, the implemented process is called a robust dynamic state estimation method. By defining different scenarios and utilizing numerical indices, simulation results are thoroughly analyzed based on an explanatory three-bus network and the IEEE standard 14-bus and 57-bus networks using the MATPOWER tool.

A Unified Multi-View Clustering Method Based on Non-Negative Matrix Factorization for Cancer Subtyping

Non-negative matrix factorization (NMF) has gained sustaining attention due to its compact leaning ability. Cancer subtyping is important for cancer prognosis analysis and clinical precision treatment. Integrating multi-omics data for cancer subtyping is beneficial to uncover the characteristics of cancer at the system-level. A unified multi-view clustering method was developed via adaptive graph and sparsity regularized non-negative matrix factorization (multi-GSNMF) for cancer subtyping. The local geometrical structures of each omics data were incorporated into the procedures of common consensus matrix learning, and the sparsity constraints were used to reduce the effect of noise and outliers in bioinformatics datasets. The performances of multi-GSNMF were evaluated on ten cancer datasets. Compared with 10 state-of-the-art multi-view clustering algorithms, multi-GSNMF performed better by providing significantly different survival in 7 out of 10 cancer datasets, the highest among all the compared methods.

Outlier detection based on the functional coefficient of variation

The coefficient of the variation function is a useful descriptive statistic, especially when comparing the variability of more than two curve groups, even when they have significantly different mean curves. Since the coefficient of variation function is the ratio of the mean and standard deviation functions, its particular property is that it shows the acceleration more explicitly than the standard deviation function. The aim of the study is twofold: to show that the functional coefficient of variation is more sensitive to abrupt changes than the functional standard deviation and to propose the utilisation of the functional coefficient of variation as an outlier detection tool. Several simulation trials have shown that the coefficient of the variation function allows the effects of outliers to be seen explicitly.

Incomplete multi-view clustering network via nonlinear manifold embedding and probability-induced loss

Incomplete multi-view clustering, which included missing data in different views, is more challenging than multi-view clustering. For the purpose of eliminating the negative influence of incomplete data, researchers have proposed a series of solutions. However, the present incomplete multi-view clustering methods still confront three major issues: (1) The interference of redundant features hinders these methods to learn the most discriminative features. (2) The importance role of local structure is not considered during clustering. (3) These methods fail to utilize data distribution information to guide models update to decrease the effects of outliers and noise. To address above issues, a novel deep clustering network which exerted on incomplete multi-view data was proposed in this paper. We combine multi-view autoencoders with nonlinear manifold embedding method UMAP to extract latent consistent features of incomplete multi-view data. In the clustering method, we introduce Gaussian Mixture Model (GMM) to fit the complex distribution of data and deal with the interference of outliers. In addition, we reasonably utilize the probability distribution information generated by GMM, using probability-induced loss function to integrate feature learning and clustering as a joint framework. In experiments conducted on multiple benchmark datasets, our method captures incomplete multi-view data features effectively and perform excellent.

Effects of outliers on remote sensing‐assisted forest biomass estimation: A case study from the United States national forest inventory

Abstract Large‐scale ecological sampling networks, such as national forest inventories (NFIs), collect in situ data to support biodiversity monitoring, forest management and planning, and greenhouse gas reporting. Data harmonization aims to link auxiliary remotely sensed data to field‐collected data to expand beyond field sampling plots, but outliers that arise in data harmonization—questionable observations because their values differ substantially from the rest—are rarely addressed. In this paper, we review the sources of commonly occurring outliers, including random chance (statistical outliers), definitions and protocols set by sampling networks, and temporal and spatial mismatch between field‐collected and remotely sensed data. We illustrate different types of outliers and the effects they have on estimates of above‐ground biomass population parameters using a case study of 292 NFI plots paired with airborne laser scanning (ALS) and Sentinel‐2 data from Sawyer County, Wisconsin, United States. Depending on the criteria used to identify outliers (sampling year, plot location error, nonresponse, presence of zeros and model residuals), as many as 53 of the 292 Forest Inventory and Analysis plot observations (18%) were identified as potential outliers using a single criterion and 111 plot observations (38%) if all criteria were used. Inclusion or removal of potential outliers led to substantial differences in estimates of mean and standard error of the estimate of biomass per unit area. The simple expansion estimator, which does not rely on ALS or other auxiliary data, was more sensitive to outliers than model‐assisted approaches that incorporated ALS and Sentinel‐2 data. Including Sentinel‐2 predictors showed minimal increases to the precision of our estimates relative to models with ALS predictors alone. Outliers arise from many causes and can be pervasive in data harmonization workflows. Our review and case study serve as a note of caution to researchers and practitioners that the inclusion or removal of potential outliers can have unintended consequences on population parameter estimates. When used to inform large‐scale biomass mapping, carbon markets, greenhouse gas reporting and environmental policy, it is necessary to ensure the proper use of NFI and remotely sensed data in geospatial data harmonization.

Localization Verifiability Under Outlier Measurements With Linear Programming

The problem of localizing a set of nodes from relative pairwise measurements appears in different fields such as computer vision, sensor networks, and robotics. In practice, the measurements might be contaminated by noise and outliers that lead to erroneous localization. Previous work has empirically shown that robust algorithms can, in some situations, almost completely cancel the effect of outliers. However, there is a theoretical gap in answering the following question: under what conditions on the number, magnitude, and arrangement of the outlier measurements can we guarantee that a robust algorithm will recover the ground truth locations from the relative measurements alone? We denote this concept as <i>verifiability</i>, and answer the question for the case of an <inline-formula><tex-math notation="LaTeX">$\ell _{1}$</tex-math></inline-formula>-norm robust optimization formulation, with translation measurements that are affected only by large-magnitude outliers and no small-magnitude noise. We prove that verifiability depends only on the topology of the graph, the location of the edges affected by the outliers, and the sign of the outliers, while it is independent of the (a priori unknown) true location of the nodes, and the magnitude of the outliers. We present an algorithm based on the dual simplex algorithm that checks the verifiability of a problem, and, if not verifiable, completely characterizes the space of equivalent solutions that are consistent with the given pairwise measurements. Our theory and algorithms can be used to compute the <i>a priori</i> probability of recovering a solution congruent or equivalent to the ground truth, without having access to the true locations.

A Measurement of Social Cohesion in Poland’s NUTS2 Regions in the Period 2010–2019 by Applying Dynamic Relative Taxonomy to Interval-Valued Data

Composite indicators play an important role in the analysis of socio-economic phenomena. A number of different approaches to constructing composite indicators have been proposed in the literature. Depending on the degree of compensation, they can be divided into compensatory, partially compensatory, and non-compensatory. The following article focuses on the method of relative taxonomy and its dynamic modification. While this method is typically applied to metric data, the authors propose using the dynamic approach for interval-valued data, which describes objects of interest more precisely. Metric data are of an atomic nature; i.e., an observation of each variable is expressed as one real number. In contrast, each observation of an interval-valued variable is expressed as an interval. By making use of interval-valued data, it is possible to assess objects not only at the regional level but also at a lower level of territorial aggregation, taking into account spatial variation across districts that make up each region. The study described in the article was conducted by applying relative taxonomy in its dynamic approach to interval-valued data in order to measure the level of social cohesion in Poland’s NUTS2 regions during the period 2010–2019. The target dataset was obtained by aggregating numeric data about social cohesion in districts (LAU1) at the level of regions. The lower and upper limit of the interval for each region was based on district-level data and corresponded to the 2nd and 8th decile, respectively (60% of observations), which helped to mitigate the effect of outliers. By applying dynamic relative taxonomy to interval-valued data, it was possible to graphically represent changes in the level of social cohesion that took place across 17 Poland’s NUTS2 regions between 2010 and 2019. It was found that during the reference period, the level of social cohesion in the regions systematically improved. Despite the observed variation, the distance between the regions consistently decreased over time. The level of social cohesion was found to be higher in regions that had received more EU funding to support regional development.

An Online Adaptive Policy Iteration-Based Reinforcement Learning for a Class of a Nonlinear 3D Overhead Crane

We consider an online policy iteration-based reinforcement learning for a class of a nonlinear three-dimensional overhead crane with bounded uncertainties. Under the assumption that the rope length is fixed with small swing angles, a linearized model is derived. The system has four states; two actuated states: position x and y, and two un-actuated states, which are the rope angles θx and θy. The adaptive reinforcement learning controller is designed to handle the effects of measurement noises and outliers. We propose a model-free; hence it does not require precise knowledge of the system dynamics. When the state information is not available, a Kalman filter estimator is equipped with a dynamical saturation function to attenuate the effects of measurement noises and to remove outliers. A simulation study is established to illustrate the influence and robustness of the developed controller, and it can enhance the tracking trajectory under different scenarios to test the scheme.

Robust M-estimators and Machine Learning Algorithms for Improving the Predictive Accuracy of Seaweed Contaminated Big Data

A common problem in regression analysis using ordinary least squares (OLS) is the effect of outliers or contaminated data on the estimates of the parameters. A robust method that is not sensitive to outliers and can handle contaminated data is needed. In this study, the objective is to determine the significant parameters that determine the moisture content of the seaweed after drying and develop a hybrid model to reduce the outliers. The data were collected with sensors from the v-Groove Hybrid Solar Drier (v-GHSD) at Semporna, South-Eastern Coast of Sabah, Malaysia. After the second order interaction, we have 435 drying parameters, each parameter has 1914 observations. First, we used four machine learning algorithms, such as random forest, support vector machine, bagging and boosting to determine the significant parameters by selecting 15, 25, 35 and 45 parameters. Second, we developed the hybrid model using robust methods such as M. Bi-Square, M. Hampel and M. Huber. The results show that there is a significant improvement in the reduction of the number of outliers and better prediction using hybrid model for the contaminated seaweed big data. For the highest variable importance of 45 significant drying parameters of seaweed, the hybrid model bagging M Bi-square performs better because it has the lowest percentage of outliers of 4.08 %.

Efficient iteratively reweighted algorithms for robust hyperbolic localization

Hyperbolic localization utilizes the time-difference-of-arrival (TDOA) measurements obtained from multiple coordinated sensors to locate an emitting source. A challenge in such an area, is that adverse factors like the non-line-of-sight propagation in typical environments may lead to outlying sensor observations and therefore deteriorated positioning performance. To mitigate the negative effects of outliers in the raw TDOA-based range-difference (RD) data on localization accuracy, the conventional ℓ2-norm based RD linear least squares (RD-LLS) location estimator is robustified here using the Geman–McClure (GM) and correntropy-induced loss functions, respectively. Two soft-redescending M-estimators are consequently developed, whereafter two iterative algorithms are presented to handle the corresponding optimization formulations. The first method applies the iteratively reweighted least squares to implement GM loss minimization, whereas the second employs the technique of half-quadratic optimization, through which the correntropy loss is maximized in an alternating manner. Either way the problem comes down to updating a series of weight-like variables and tackling a simplifed formulation that is closely related to RD-LLS at each iteration, thereby leading to fairly computationally efficient iterative solutions. Numerical results confirm the capabilities of the proposed schemes to outperform several existing hyperbolic positioning approaches.

Multisource localization based on angle distribution of time–frequency points using an FOA microphone

AbstractMultisource localization occupies an important position in the field of acoustic signal processing and is widely applied in scenarios, such as human‐machine interaction and spatial acoustic parameter acquisition. The direction‐of‐arrival (DOA) of a sound source is convenient to render spatial sound in the audio metaverse. A multisource localization method in a reverberation environment is proposed based on the angle distribution of time–frequency (TF) points using a first‐order ambisonics (FOA) microphone. The method is implemented in three steps. 1) By exploring the angle distribution of TF points, a single‐source zone (SSZ) detection method is proposed by using a standard deviation‐based measure, which reveals the degree of convergence of TF point angles in a zone. 2) To reduce the effect of outliers on localization, an outlier removal method is designed to remove the TF points whose angles are far from the real DOAs, where the median angle of each detected zone is adopted to construct the outlier set. 3) DOA estimates of multiple sources are obtained by postprocessing of the angle histogram. Experimental results in both the simulated and real scenarios verify the effectiveness of the proposed method in a reverberation environment, which also show that the proposed method outperforms reference methods.

Capped L21-norm-based common spatial patterns for EEG signals classification applicable to BCI systems.

The common spatial patterns (CSP) technique is an effective strategy for the classification of multichannel electroencephalogram (EEG) signals. However, the objective function expression of the conventional CSP algorithm is based on the L2-norm, which makes the performance of the method easily affected by outliers and noise. In this paper, we consider a new extension to CSP, which is termed capped L21-norm-based common spatial patterns (CCSP-L21), by using the capped L21-norm rather than the L2-norm for robust modeling. L21-norm considers the L1-norm sum which largely alleviates the influence of outliers and noise for the sake of robustness. The capped norm is further used to mitigate the effects of extreme outliers whose signal amplitude is much higher than that of the normal signal. Moreover, a non-greedy iterative procedure is derived to solve the proposed objective function. The experimental results show that the proposed method achieves the highest average recognition rates on the three real data sets of BCI competitions, which are 91.67%, 85.07%, and 82.04%, respectively. Capped L21-norm-based common spatial patterns-a robust model for EEG signals classification.

Data-driven two-stage distributionally robust optimization for refinery planning under uncertainty

This work investigates the refinery planning problem under uncertainty in product prices. A novel data-driven Wasserstein distributionally robust optimization framework is proposed for handling uncertainties in the refinery-wide planning operations. A data-driven ambiguity set is constructed based on the Wasserstein metric to model the distributional uncertainty. The robust kernel density estimation technique is adopted to establish the support set to reduce the effect of the potential outliers. Based on the derived support set and ambiguity set, a data-driven two-stage distributionally robust optimization model for refinery planning is developed. Then, the robust counterpart of the proposed model is formulated to make the problem computationally tractable. Finally, a real-world case study on a petroleum refinery is presented to illustrate the effectiveness and applicability of the proposed framework.

Unpacking Outlier with Weight Least Square (Implemented on Pepper Plantations Data)

Outliers in regression analysis can cause large residuals, the diversity of the data becomes greater, causing the data to be heterogenous. If an outlier is caused by an error in recording observations or an error in preparing equipment, the outlier can be ignored or discarded before data analysis is carried out. However, if outliers exist not because of the researcher's error, but are indeed information that cannot be provided by other data, then the outlier data cannot be ignored and must be included in data analysis. There are several methods to deal with outliers. The Weight Least Square method produces good results and is quite resistive to outliers. The WLS method is used to overcome the regression model with non-constant error variance, because WLS has the ability to neutralize the consequences of violating the normality assumption caused by the presence of outliers and can eliminate the nature of unusualness and consistency of the OLS estimate. To compare the level of estimator accuracy between regression models, the mean absolute percentage error (MAPE) is used. Based on the results of this study, it was concluded that the WLS method produced a smaller Mean Absolute Percentage Error value so that the use of this method was more appropriate because it was not susceptible to the effect of outliers.

Auto-outlier Fusion Technique for Chest X-ray Classification with Multi-head Attention Mechanism

A chest X-ray is one of the most widely available radiological examinations for diagnosing and detecting various lung illnesses. The National Institutes of Health (NIH) provides an extensive database, ChestX-ray8 and ChestXray14, to help establish a deep learning community for analysing and predicting lung diseases. ChestX-ray14 consists of 112,120 frontal-view X-ray images of 30,805 distinct patients with text-mined fourteen disease image labels, where each image has multiple labels and has been utilised in numerous research in the past. To our current knowledge, no previous study has investigated outliers and multi-label impact for a single X-ray image during the preprocessing stage. The effect of outliers is mitigated in this paper by our proposed auto-outlier fusion technique. The image label is regenerated by concentrating on a particular factor in one image. The final cleaned dataset will be used to compare the mechanisms of multi-head self-attention and multi-head attention with generalised max-pooling.

Auto-outlier Fusion Technique for Chest X-ray Classification with Multi-head Attention Mechanism

A chest X-ray is one of the most widely available radiological examinations for diagnosing and detecting various lung illnesses. The National Institutes of Health (NIH) provides an extensive database, ChestX-ray8 and ChestXray14, to help establish a deep learning community for analysing and predicting lung diseases. ChestX-ray14 consists of 112,120 frontal-view X-ray images of 30,805 distinct patients with text-mined fourteen disease image labels, where each image has multiple labels and has been utilised in numerous research in the past. To our current knowledge, no previous study has investigated outliers and multi-label impact for a single X-ray image during the preprocessing stage. The effect of outliers is mitigated in this paper by our proposed auto-outlier fusion technique. The image label is regenerated by concentrating on a particular factor in one image. The final cleaned dataset will be used to compare the mechanisms of multi-head self-attention and multi-head attention with generalised max-pooling.