Kind Of Outliers Research Articles

Deep learning for ultrasound surface echo detection

Ultrasound array imaging techniques are often applied using a coupling medium to transmit waves from the transducer to the scanned component. In this type of tests, the echoes generated by the wave reflection at the component surface contain information that can be used to estimate the surface shape and its position relative to the probe, which are needed to compute the imaging focal laws. The most interesting feature to extract from surfaces echoes is the Time of Flight (TOF) for each reception channel. TOF measurement in acoustic signals is a problem that frequently arises not only in Non Destructive Testing (NDT), but also in medical imaging, geophysics and other fields. The surface echo in a signal is usually identified as the first high amplitude peak, and is traditionally detected with standard methods as threshold crossing, peak search or matched filters. All these methods are easily affected by Signal to Noise Ratio, which might be high in some channels, for example if the ray arrives at the corresponding element at a low sensibility angle. Moreover, spurious echoes from bubbles, the emission signal tail or echoes from other parts of the component can be incorrectly labelled as surface echoes. This kind of outliers are difficult to filter by conventional means and, thus, there is a need for more robust surface estimation methods. In this work, we study the application of a deep 3D Convolutional Neural Network (CNN) to detect surface echoes in Full Matrix Capture (FMC) ultrasound data. The CNN is trained using signals acquired with a matrix array and a reference component, using a robotic arm set-up to measure the probe position and orientation relative to the component. Thus, with knowledge of the set-up geometry, a model is used to compute the theoretical surface echo TOFs, which are used to label the data for training. The model was tested with a validation set, presenting good agreement with the ground truth. The results were compared with the threshold crossing method observing a significant reduction of outliers in the TOF estimation.

M. Hubert, P. Rousseeuw and P. Segaert: Multivariate functional outlier detection

Firstly we congratulate the authors on a wonderful paper full of new nice ideas, which can be considered as a major breakthrough in the functional outlier detection using visual procedures. These ideas include a taxonomy of outliers, the definition of bag distance and the centrality-stability plots. Indeed, each of the last two ideas is fundamental in the development of each of the two corresponding procedures to detect multivariate functional outliers introduced in the paper. The first procedure consists in constructing a heat map using the functional bag distance based on the Tukey depth while the second in a scatter-plot based on the skew-adjusted projection depth, SPD, named the centrality-stability plot, CSP. Moreover, both procedures complement themselves because the heat maps are good in detecting all kind of outliers excepting the shape outliers, but those are clearly identified with the CSP’s. Our discussion focus, firstly, on shedding light on the behaviour of the proposed procedures when applied to multivariate functional data whose dimension is entitled to be extremely high. Secondly, a simplification of the CSP is proposed. Furthermore, we encourage the authors to comment on the advantages/disadvantages of applying, what they call, the MFSPD versus, what they call, the 1/(1+ FAO), as the difference between them just lies in the reverse order of the integral and inverse functionals.

Robust Transition Estimation for Community Evolution and Evolutionary Outlier Detection

PDF HTML阅读 XML下载 导出引用 引用提醒 社群演化的稳健迁移估计及演化离群点检测 DOI: 10.3724/SP.J.1001.2013.04477 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: 国家自然科学基金(61375069,61105069);中国博士后科学基金(2011M500846);江苏省科技支撑计划(BE2012181);江苏省教育厅自然科学基金(11KJB520001);江苏高校优势学科建设工程资助项目;计算机软件新技术国家重点实验室自主课题(ZZKT2013B11) Robust Transition Estimation for Community Evolution and Evolutionary Outlier Detection Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:时序数据集中的社群演化模式是网络行为动力学研究与应用的重要领域.基于社群演化的离群点检测不仅能够发现新颖的异常行为模式,同时也有利于更准确地理解社群的演化趋势.运用成员关于社群隶属关系的变化,提出了社群演化迁移矩阵的概念,研究并揭示了迁移矩阵的若干性质及其与社群结构演化之间的关系.在采用稳健回归M-估计方法进一步优化迁移矩阵降低异常点干扰的同时,对社群演化离群点加以刻画和定义.鉴于复杂网络包含大量随机游走的边缘个体,所定义的离群点综合考虑其在社群中角色的变化和相对于社群总体迁移模式的差异.基于上述思想提出的演化离群点检测算法能够适应各类社群演化趋势,更有效地聚焦和发现大规模社会网络中重要成员的异常演化行为.实验结果表明,所提出的方法能够从大规模社会网络演化序列中发现重要的离群演化模式,并在现实中找到合理的解释. Abstract:Community evolutionary pattern analysis in temporal datasets is a key issue in social network dynamics research and applications. Identifying outlying objects against main community evolution trends is not only meaningful by itself for the purpose of finding novel evolution behaviors, but also helpful for better understanding the mainstream of community evolution. Upon giving the belonging matrix of community members, this study defines a type of transition matrix to characterize the pattern of the evolutionary dynamic between two consecutive belonging snapshots. A set of properties about the transition matrix is discussed, which reveals its close relation to the gradual community structural change in quantity. The transition matrix is further optimized using M-estimator Robust Regression methods by minimizing the disturbance incurred by the outliers, and the abnormality of the outlier objects can then be computed at the same time. Considering that large proportion of trivial but nomadic objects may exist in large datasets like those of complex social networks, focus is placed only on the community evolutionary outliers that show remarkable difference from the main bodies of their communities and sharp change of their membership role within the communities. A definition on such type of local and global outliers is given, and an algorithm on detection such kind of outliers is proposed in this paper. Experimental results on both synthetic and real datasets show that the proposed approach is highly effective in discovering interesting evolutionary community outliers. 参考文献 相似文献 引证文献

Locally centred Mahalanobis distance: A new distance measure with salient features towards outlier detection

Outlier detection is a prerequisite to identify the presence of aberrant samples in a given set of data. The identification of such diverse data samples is significant particularly for multivariate data analysis where increasing data dimensionality can easily hinder the data exploration and such outliers often go undetected. This paper is aimed to introduce a novel Mahalanobis distance measure (namely, a pseudo-distance) termed as locally centred Mahalanobis distance, derived by centering the covariance matrix at each data sample rather than at the data centroid as in the classical covariance matrix. Two parameters, called as Remoteness and Isolation degree, were derived from the resulting pairwise distance matrix and their salient features facilitated a better identification of atypical samples isolated from the rest of the data, thus reflecting their potential application towards outlier detection. The Isolation degree demonstrated to be able to detect a new kind of outliers, that is, isolated samples within the data domain, thus resulting in a useful diagnostic tool to evaluate the reliability of predictions obtained by local models (e.g. k-NN models).To better understand the role of Remoteness and Isolation degree in identification of such aberrant data samples, some simulated and published data sets from literature were considered as case studies and the results were compared with those obtained by using Euclidean distance and classical Mahalanobis distance.

Data Dimension Reduction and Visualization with Application to Multi-Dimensional Gearbox Diagnostics Data: Comparison of Several Methods

In recent years we face the growing interest in building automated diagnosis systems detecting ‘normal’ or ‘abnormal’ functioning of a system. But little is known about the distribution and shape of the data describing 'normal' functioning. The shape of the data is of paramount importance in determining the mathematical model of the data serving for the diagnosis. We got real industrial data by gathering vibration signals of a gearbox working in a mine excavator operating in time-varying conditions. The main considered problems are: what is the shape of the recorded 15-dimensional data and what kind of outliers may be found there? We have used for this purpose pseudo grand tour, PCA and simple auto-associative neural network. The methods used proved to be very effective in answering our questions.

Outliers in some Faces and non-Faces data

We look for outliers in graphical data containing n = 6977 faces or non-faces images from Seung's collection. Our concern is: what kind of outliers may be found in such graphical data. To obtain the global geometrical characteristics, the Pseudo Grand Tour and Kohonens's self-organising maps are applied. We define as outliers those images which reproduce themselves badly from K principal components, with K denoting intrinsic dimension. The concept of mild and severe outliers, and own and alien principal components is also introduced.

Reliabilities of χ2- and F Tests in Gauss-Markov Model

Power and level breakdown points are used in robust statistics to determine the global reliability of a test. A method for calculating the finite-sample versions of these for tests such as χ2 or F is given here when observations have the Gauss-Markov model. Outliers are compared with the generating model using a coordinate transformation and simulated samples. Outliers are considered in two groups, namely, “doubtful” and “clear.” The reliability of a χ2- or F test changes with the magnitude, number, and kind of outliers (i.e., random outliers or influential outliers), and also with the level of significance, α. The χ2- and F tests are generally insensitive to both doubtful and clear outliers. The χ2 test can reliably reject the null hypothesis at α = 0.05, when the observations contain multiple clear random outliers, only if the magnitude of one of them is at least greater than 5σ. Also, the F test can reliably reject the null hypothesis at α = 0.05, when the observations contain multiple clear random outliers, only if the magnitude of one of them is at least greater than 6σ. The greater the magnitude and number of clear outliers that are contained in the observations, the more successful is the χ2- or F test in rejecting the null hypothesis. The reliabilities of the χ2- and F tests decrease as the number of unknowns increases. For this case, the power and level finite-sample breakdown points of the χ2- and F tests are estimated here to be approximately 1/2.