Outlier Deletion Research Articles

Financial inclusion through Kiosk-based banking services: a study with reference to business correspondent models in the state of Kerala

PurposeThe paper aims to study the outreach and performance of business correspondent (BC) models, which are implemented as a subsidiary agent of banks to accelerate the financial inclusion (FI) mission in India. In this regard, the study illustrates BC's products and services rendered to customers, forms of delivery channels and BC's view on banking services and Kiosk-based BC programs.Design/methodology/approachThe current paper is an empirical study based on surveying 200 Kiosk-based BCs working in the state of Kerala. After the preliminary screening analysis of the data with outlier deletion, removal of missing values and normality test, both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were executed followed by reliability test, convergent and discriminant validity tests. Covariance-based structural equation modeling (CBSEM) was performed for CFA and inferential tests were carried out by using statistical package for the social sciences (SPSS) and analysis of a moment structures (AMOS) and Eviews.FindingsChiefly, eight operational forms of BCs were found from the field survey. Hypothetical tests show the significant impact of the serviceability of banks on BC's profitability. Validity tests such as average variance extracted (AVE), composite reliability (CR), maximum shared variance (MSV) and average shared variance (ASV) were established after the removal of the cross-loaded items of the questionnaire from the rotated component matrix. BCs perform main banking services especially bank account opening facility and Akshaya E-Centers are widely used for this model as Kiosk banking in the surveyed state.Originality/valueSo far, no study has encompassed empirical research on performance analysis and outreach of the BC model in the state of Kerala where this BC model well functions. Since the study is a novel form of banking channelization for FI, the study can contribute to understanding the further feasibility and future dimension of the model based on experimental views of BCs.

Semi-supervised support vector regression based on data similarity and its application to rock-mechanics parameters estimation

Rock-mechanics parameters such as Young’s modulus and Poisson’s ratio are critical to geomechanical analysis and resource exploration. Because these parameters come from laboratory measurement, they present some characteristics such as insufficient samples and contamination of outliers. In this paper, a novel semi-supervised support vector machine soft sensor is devised considering the characteristics of the parameters. First, it takes into account data similarity and selects labeled data set that are most similar to the continuous unlabeled data set at each iteration to improve estimation performance. Meanwhile, an outlier deletion algorithm is developed for a better similarity comparison. After that, a semi-supervised approach is presented for the estimation of rock-mechanics parameters, it can leverage continuous unlabeled data to train the model dynamically. Finally, the verification of our method is carried out on data set from UCI (University of California, Irvine) and several drilling sites. The results demonstrate that our method outperforms eight well-known methods in estimation accuracy.

Two-phase deep learning model for short-term wind direction forecasting

Accurate and reliable wind direction prediction is important for improving wind power conversion efficiency and operation safety. In this paper, a two-phase deep learning model is proposed and constructed for high-performance short-term wind direction forecasting. In the first phase, a hybrid data processing strategy, including data reconstruction, outlier deletion, dimension reduction, and sequence decomposition, is proposed to extract the most meaningful information from practical data. Then, in the second phase, a robust echo state network is developed for wind direction forecasting. In addition, its hyper-parameters are optimized using an improved flower pollination algorithm (IFPA) to achieve high efficiency. Experiments conducted on data from real wind farms validate the proposed hybrid data processing method. Finally, comparisons with benchmark prediction models show that the proposed network achieves superior performance.

Detecting Lake Level Change From 1992 to 2019 of Zhari Namco in Tibet Using Altimetry Data of TOPEX/Poseidon and Jason-1/2/3 Missions

Zhari Namco, a large lake in the Tibetan Plateau (TP), is sensitive to climate and environmental change. However, it is difficult to retrieve accurate and continuous lake levels for Zhari Namco. A robust strategy, including atmospheric delay correction, waveform retracking, outlier deletion, and inter-satellite adjustment, is proposed to generate a long-term series of lake levels for Zhari Namco through multi-altimeter data. Apparent biases are found in troposphere delay correction from different altimeter products and adjusted using an identical model. The threshold (20%) algorithm is employed for waveform retracking. The two-step method combining a sliding median filter and 2σ criterion is used to eliminate outliers. Tandem mission data of altimeters are used to estimate inter-satellite bias. Finally, a 27-year-long lake level time series of Zhari Namco are constructed using the TOPEX/Poseidon-Jason1/2/3 (T/P-Jason1/2/3) altimeter data from 1992 to 2019, resulting in an accuracy of 10.1 cm for T/P-Jason1/2/3. Temperature, precipitation, lake area, equivalent water height, and in situ gauge data are used for validation. The correlation coefficient more than 0.90 can be observed between this result and in situ gauge data. Compared with previous studies and existing database products, our method yields sequences with the best observational quality and the longest continuous monitoring in Zhari Namco. The time series indicates that the lake level in Zhari Namco has increased by ∼ 5.7 m, with an overall trend of 0.14 ± 0.01 m/yr, showing a fluctuating rate (1992–1999: −0.25 ± 0.05 m/yr, 2000–2008: 0.26 ± 0.04 m/yr, 2009–2016: −0.05 ± 0.03 m/yr, 2017–2019: 1.34 ± 0.34 m/yr). These findings will enhance the understanding of water budget and the effect of climate change in the TP.

Outlier detection in test samples and supervised training set selection

‎Outlier detection is a technique for recognizing samples out of the main population within a data set‎. ‎Outliers have negative impacts on classification‎. ‎The recognized outliers are deleted to improve the classification power generally‎. ‎This paper proposes a method for outlier detection in test samples besides a supervised training set selection‎. ‎Training set selection is done based on the intersection of three well known similarity measures namely‎, ‎jacquard‎, ‎cosine‎, ‎and dice‎. ‎Each test sample is evaluated against the selected training set for possible outlier detection‎. ‎The selected training set is used for a two-stage classification‎. ‎The accuracy of classifiers are increased after outlier deletion‎. ‎The majority voting function is used for further improvement of classifiers‎.

Outlier Data Treatment Methods Toward Smart Grid Applications

In a smart grid environment, advanced metering infrastructure (AMI) and intelligent sensors have been deployed extensively. As a result, large-scale and fine-grained smart grid data are more convenient to be collected, in which outliers exist pervasively, caused by system failures, environmental effects, and human interventions. Outlier deletion is always implemented in data preprocessing for improving data quality. However, due to the fact that real records that reflect rare and unusual patterns are also recognized as outliers, outlier mining is necessary to be carried out with the aim of discovering knowledge on abnormal patterns in power generation, transmission, distribution, transformation, and consumption. To the best of our knowledge, a comprehensive and systematic review of outlier data treatment methods is still lacked in the smart grid environment. We, in this paper, aim at presenting the review of outlier data treatment methods toward smart grid applications and categorize them into outlier rejection and outlier mining groups. Since we do this survey from the perspective of data-driven analytics and data mining methods, information security technologies are barely discussed in this paper. Based on a general overview of outlier data treatment methods, we make the contribution of providing the application scenarios of outlier rejection and outlier mining in the smart grid environment. With the construction of smart grid throughout the world, dealing with outlier data has become more crucial for the security and reliability of power system operation. Therefore, we also discuss some future challenges of outlier data treatment toward smart energy management.

EasyPCC: Benchmark Datasets and Tools for High-Throughput Measurement of the Plant Canopy Coverage Ratio under Field Conditions.

Understanding interactions of genotype, environment, and management under field conditions is vital for selecting new cultivars and farming systems. Image analysis is considered a robust technique in high-throughput phenotyping with non-destructive sampling. However, analysis of digital field-derived images remains challenging because of the variety of light intensities, growth environments, and developmental stages. The plant canopy coverage (PCC) ratio is an important index of crop growth and development. Here, we present a tool, EasyPCC, for effective and accurate evaluation of the ground coverage ratio from a large number of images under variable field conditions. The core algorithm of EasyPCC is based on a pixel-based segmentation method using a decision-tree-based segmentation model (DTSM). EasyPCC was developed under the MATLAB® and R languages; thus, it could be implemented in high-performance computing to handle large numbers of images following just a single model training process. This study used an experimental set of images from a paddy field to demonstrate EasyPCC, and to show the accuracy improvement possible by adjusting key points (e.g., outlier deletion and model retraining). The accuracy (R2 = 0.99) of the calculated coverage ratio was validated against a corresponding benchmark dataset. The EasyPCC source code is released under GPL license with benchmark datasets of several different crop types for algorithm development and for evaluating ground coverage ratios.

Genetic Algorithm for Initial Orbit Determination with Too Short Arc (Continued)

When the genetic algorithm is used to solve the problem of too short-arc (TSA) orbit determination, due to the difference of computing process between the genetic algorithm and the classical method, the original method for outlier deletion is no longer applicable. In the genetic algorithm, the robust estimation is realized by introducing different loss functions for the fitness function, then the outlier problem of the TSA orbit determination is solved. Compared with the classical method, the genetic algorithm is greatly simplified by introducing in different loss functions. Through the comparison on the calculations of multiple loss functions, it is found that the least median square (LMS) estimation and least trimmed square (LTS) estimation can greatly improve the robustness of the TSA orbit determination, and have a high breakdown point.

Outlier deletion based improvement on the StOMP algorithm for sparse solution of large-scale underdetermined problems

StOMP algorithm is well suited to large-scale underdetermined applications in sparse vector estimations. It can reduce computation complexity and has some attractive asymptotical statistical properties. However, the estimation speed is at the cost of accuracy violation. This paper suggests an improvement on the StOMP algorithm that is more efficient in finding a sparse solution to the large-scale underdetermined problems. Also, compared with StOMP, this modified algorithm can not only more accurately estimate parameters for the distribution of matched filter coefficients, but also improve estimation accuracy for the sparse vector itself. Theoretical success boundary is provided based on a large-system limit for approximate recovery of sparse vector by modified algorithm, which validates that the modified algorithm is more efficient than StOMP. Actual computations with simulated data show that without significant increment in computation time, the proposed algorithm can greatly improve the estimation accuracy.

Memetic Pareto differential evolutionary neural network used to solve an unbalanced liver transplantation problem

Donor---recipient matching constitutes a complex scenario difficult to model. The risk of subjectivity and the likelihood of falling into error must not be underestimated. Computational tools for the decision-making process in liver transplantation can be useful, despite the inherent complexity involved. Therefore, a multi-objective evolutionary algorithm and various techniques to select individuals from the Pareto front are used in this paper to obtain artificial neural network models to aid decision making. Moreover, a combination of two pre-processing methods has been applied to the dataset to offset the existing imbalance. One of them is a resampling method and the other is a outlier deletion method. The best model obtained with these procedures (with AUC = 0.66) give medical experts a probability of graft survival at 3 months after the operation. This probability can help medical experts to achieve the best possible decision without forgetting the principles of fairness, efficiency and equity.

A 274-lake calibration data-set and inference model for chironomid-based summer air temperature reconstruction in Europe

A 274-lake calibration data-set for chironomid-based temperature reconstruction is presented which is based on the taxonomic amalgamation of a 117-lake data-set from Switzerland and a 157-lake data-set from Norway and Svalbard. Taxonomic consistency of the two data-sets was ensured by joint microscope sessions by the two involved analysts, re-identifying chironomid assemblages in the Swiss data-set to reach an identical taxonomic resolution as in the Norwegian data, and by double-checking selected samples of the Norwegian calibration data-set. The combined Swiss-Norwegian calibration data-set contains information on the distribution of 154 chironomid taxa over a July air temperature range of 3.5–18.4 °C, a pH range of 4.7–8.8, and an altitudinal range of 5–2815 m asl from lakes in temperate, subarctic, arctic, and alpine environments. Inference models developed based on this data-set using weighted averaging-partial least squares (WA-PLS) regression outperformed inference models based on maximum likelihood regression. After outlier deletion WA-PLS regression predicted July air temperature with a bootstrapped (cross-validated) root mean squared error of prediction (RMSEP) of 1.40 °C. Inference models developed from the separate regional data-sets have an RMSEP of 1.16 and 1.43 °C for the Norwegian and the Swiss calibration data-set, respectively. The WA-PLS inference model based on the Norwegian data-set adequately predicted July air temperature based on chironomid assemblages in the Swiss data-set (RMSEP 2.05 °C; r 2 0.74). In contrast, the WA-PLS model based on the Swiss chironomid assemblages performed poorly in inferring July air temperatures based on the Norwegian chironomid assemblages (RMSEP 2.70 °C; r 2 0.39). We attribute this discrepancy to the large proportion of chironomid taxa in Norwegian samples not represented in the Swiss data-set, the larger range of both pH and lake types included in the Norwegian calibration data-set, and the lack of some chironomid taxa with an arctic and boreal distribution in the Swiss data-set. The WA-PLS inference model based on the combined Swiss-Norwegian calibration data-set is the largest taxonomically consistent chironomid-based inference model available to date and outperforms other existing models if the r 2 and RMSEP relative to the overall temperature gradient are examined. We demonstrate that the combined model can reconstruct a larger range of temperatures when applied to fossil assemblages than models based on the Norwegian or Swiss calibration data-sets only. The combined data-set includes the majority of chironomid taxa expected in late Pleistocene and Holocene sediments of lakes from Northern, Eastern, and Central Europe as well as from southern European mountain lakes. The newly developed WA-PLS inference model is therefore well suited for developing continental-scale reconstructions of late Quaternary temperature change based on chironomid records from different parts of Europe.

Comportamento proativo nas organizações: o efeito dos valores pessoais

O comportamento proativo é definido como um conjunto de comportamentos extrapapel em que o trabalhador busca espontaneamente mudanças no seu ambiente de trabalho, soluciona e antecipa-se aos problemas, visando a metas de longo prazo que beneficiam a organização. Este estudo teve por objetivo investigar a relação entre os valores pessoais e o comportamento proativo nas organizações. Foram utilizados como instrumentos de medida o Personal Values Questionnaire e a Escala de Comportamento Proativo nas Organizações, ambos já validados para o Brasil. Após a eliminação dos casos extremos, a amostra ficou constituída por 325 funcionários de diferentes organizações. A análise de regressão linear revelou que os valores predizem significativamente os comportamentos proativos, apontando uma relação positiva do tipo motivacional estimulação (B= 0,205, p<0,01) e universalismo/benevolência (B=0,302, p<0,01) com proatividade, e negativa com o tipo motivacional tradição (B= -0,189, p<0,01), de acordo com o previsto pelo referencial teórico. As implicações para os estudos na área são discutidas.

Association Between Staff Factors and Levels of Conflict and Containment on Acute Psychiatric Wards in England

Association Between Staff Factors and Levels of Conflict and Containment on Acute Psychiatric Wards in England

Association Between Staff Factors and Levels of Conflict and Containment on Acute Psychiatric Wards in England

Conflict (for example, aggression, substance use, and absconding) threatens the safety of patients and staff, and containment use (for example, coerced medication, special observation, and manual restraint) arouses strong feelings and is controversial. Previous work suggests that three staff factors have an impact on rates of these events: the positive appreciation of patients by staff, the staff's management of their own emotional reactions to patient behavior, and the provision of an effective structure. The aim of this study was to test this theory. A multivariate cross-sectional design was utilized. Data were collected for a six-month period on 136 acute psychiatric wards in 26 National Health Service Trusts in England. Multiple regression was conducted to determine the factors most strongly associated with total conflict and containment rates. Provision of an effective structure of rules and routines for patients was the staff feature most strongly and consistently associated with lower conflict and containment rates, with other elements of the theory less well supported by the data. Other patient and ward factors were also significantly associated with differences in conflict and containment rates between wards, in particular the proportion of patients formally detained under mental health legislation, the quality of the physical environment, and staff and patient race-ethnicity. Conflict and containment are reliable and meaningful concepts. Staff factors are relevant in the determination of conflict and containment rates on wards, but the most important of these is the provision of an effective, well-organized structure of rules and daily routines. There are a number of ways in which conflict on wards might be reduced, including a greater emphasis on the production of effective structure and order on the ward and work in the community to reduce the numbers of compulsory admissions.

DGPS 방식에 의한 위성의 정밀궤도 결정을 위한 GPS 원시 자료 전처리

위성에 탑재된 GPS 수신기의 관측 자료를 활용하여 저궤도위성의 정밀궤도 결정을 위해서 반드시 필요한 관측 자료 전처리에 관련된 연구를 수행하였다. 전처리 과정에서는 반송파 위상 자료와 코드 자료에 있는 사이클 슬립, 시계 오차, 불량 관측값, 이온층 지연 효과 등을 제거하거나 보정하여 일정한 간격으로 재정렬된 이중차분 자료를 생성한다. DGPS 방식을 이용하여 저궤도 위성의 정밀궤도결정을 수행하면 그 정밀도가 수 미터에서 수 센티미터 수준에 달하기 때문에 전처리 과정에서도 그 정밀도에 영향을 미치지 않을 정도로 관측 자료의 편집이 정밀하게 수행되어야 한다. 그러나 GPS수신기가 자료를 수집하는 시간간격에 따라 관측 자료를 분해할 수 있는 한계가 달라지기 때문에 자료의 수신시간간격은 전처리의 성능과 직결된다. 또한 수신기의 성능과 수신기를 탑재한 위성의 고도에 따라서 자료의 질이 달라지기도 하므로, 이 논문에서는 DGPS 방식에 의한 위성의 정밀궤도결정을 수행하기 위한 전처리 과정에서 수신시간간격, 수신기의 성능과 위성의 고도에 따른 전처리의 성능을 분석하여 시간간격과 수신기의 종류에 따라 사용할 수 있는 전처리 방법을 제안하였다. This article investigates the problem of data preprocessing for the precision orbit determination (POD) of low earth orbit satellite using GPS .aw data. Several data preprocessing algorithms have been developed to edit the GPS data automatically such that outlier deletion, cycle slip identification and correction, and time tag error correction. The GPS data are precisely edited for the accuracy of POD. Some methods of data preprocessing are restricted to the rate of the collections of the pseudorange and carrier phase measurements. This study considers the preprocessing efficiency varied with the rate, the quality of receiver and the altitude of the satellite's orbit. We also propose the proper methods in accordance with the rate for single frequency and dual frequency receivers.

Regression residuals and test statistics: Assessing naive outlier deletion

AbstractThe authors derive the joint distributions of a studentized deleted residual and various regression quantities, calculated with all the data or with one case deleted. They show that the correlation between the studentized deleted residual and the deleted test statistic has an interesting interpretation in terms of well‐known regression quantities. These results allow them to examine the effect of applying some naive outlier deletion methods before making inferences about a regression parameter.

An Automatic Editing Algorithm for GPS data

An algorithm has been developed to edit automatically Global Positioning System data such that outlier deletion, cycle slip identification and correction are independent of clock instability, selective availability, receiver–satellite kinematics, and tropospheric conditions. This algorithm, called TurboEdit, operates on undifferenced, dual frequency carrier phase data, and requires (1) the use of P code pseudorange data and (2) a smoothly varying ionospheric electron content. The latter requirement can be relaxed if the analysis software incorporates ambiguity resolution techniques to estimate unresolved cycle slip parameters. TurboEdit was tested on the large data set from the CASA Uno experiment, which contained over 2500 cycle slips. Analyst intervention was required on 1% of the station–satellite passes, almost all of these problems being due to difficulties in extrapolating variations in the ionospheric delay. The algorithm is presently being adapted for real time data editing in the Rogue receiver for continuous monitoring applications.

Errata: Is It Outlier Deletion or Is It Sample Truncation? Notes on Science and Sexuality

Errata: Is It Outlier Deletion or Is It Sample Truncation? Notes on Science and Sexuality

Is It Outlier Deletion or Is It Sample Truncation? Notes on Science and Sexuality

Is It Outlier Deletion or Is It Sample Truncation? Notes on Science and Sexuality