Automatic Threshold Selection Research Articles

Automated surface texture analysis via Discrete Cosine Transform and Discrete Wavelet Transform

Surface roughness and texture are critical to the functional performance of engineering components. The ability to analyze roughness and texture effectively and efficiently is much needed to ensure surface quality in many surface generation processes, such as machining, surface mechanical treatment, etc. Discrete Wavelet Transform (DWT) and Discrete Cosine Transform (DCT) are two commonly used signal decomposition tools for surface roughness and texture analysis. Both methods require selecting a threshold to decompose a given surface into its three main components: form, waviness, and roughness. However, although DWT and DCT are part of the ISO surface finish standards, there exists no systematic guidance on how to compute these thresholds, and they are often manually selected on case by case basis. This makes utilizing these methods for studying surfaces dependent on the user's judgment and limits their automation potential. Therefore, we present two automatic threshold selection algorithms based on information theory and signal energy. We use machine learning to validate the success of our algorithms both using simulated surfaces as well as digital microscopy images of machined surfaces. Specifically, we generate feature vectors for each surface area or profile and apply supervised classification. Comparing our results with the heuristic threshold selection approach shows good agreement with mean accuracies as high as 95%. We also compare our results with Gaussian filtering (GF), and show that while GF results for areas can yield slightly higher accuracies, our results outperform GF for surface profiles. We further show that our automatic threshold selection has significant advantages in terms of computational time as evidenced by decreasing the number of mode computations by an order of magnitude compared to the heuristic thresholding for DCT.

Agglomerative Clustering with Threshold Optimization via Extreme Value Theory

Clustering is a critical part of many tasks and, in most applications, the number of clusters in the data are unknown and must be estimated. This paper presents an Extreme Value Theory-based approach to threshold selection for clustering, proving that the “correct” linkage distances must follow a Weibull distribution for smooth feature spaces. Deep networks and their associated deep features have transformed many aspects of learning, and this paper shows they are consistent with our extreme-linkage theory and provide Unreasonable Clusterability. We show how our novel threshold selection can be applied to both classic agglomerative clustering and the more recent FINCH (First Integer Neighbor Clustering Hierarchy) algorithm. Our evaluation utilizes over a dozen different large-scale vision datasets/subsets, including multiple face-clustering datasets and ImageNet for both in-domain and, more importantly, out-of-domain object clustering. Across multiple deep features clustering tasks with very different characteristics, our novel automated threshold selection performs well, often outperforming state-of-the-art clustering techniques even when they select parameters on the test set.

Best-Arm Identification Using Extreme Value Theory Estimates of the CVaR

We consider a risk-aware multi-armed bandit framework with the goal of avoiding catastrophic risk. Such a framework has multiple applications in financial risk management. We introduce a new conditional value-at-risk (CVaR) estimation procedure combining extreme value theory with automated threshold selection by ordered goodness-of-fit tests, and we apply this procedure to a pure exploration best-arm identification problem under a fixed budget. We empirically compare our results with the commonly used sample average estimator of the CVaR, and we show a significant performance improvement when the underlying arm distributions are heavy-tailed.

Urban Impervious Surface Automatic Threshold Detection Model Derived From Multitemporal Landsat Images

Accurate mapping urban impervious surfaces (UIS) from satellite images is crucial for understanding urban land cover change directions and addressing UIS change-induced environmental issues. The UIS index is an effective way to reduce the computational cost of processing remote sensing imageries although continuous UIS detection from multitemporal Landsat images, repeated fine-tuning, and reselection for a proper threshold should be considered. In this article, we proposed a novel UIS index and an improved particle swarm optimization (PSO)-based UIS automatic threshold selection model (UISAT) to overcome those shortages when detecting UIS derived from multitemporal Landsat images. The proposed UIS index transformed different temporal Landsat images into index images with distinguishable pixel values for each gray pixel. In this way, the pixel gray values representing UIS were enhanced, and non-UIS was weakened with lower gray scales in the transformed index images. Next, an automatic threshold selection strategy was developed to enhance the applicability of the PSO model in terms of threshold fine-tuning and reselection. Then, we performed comprehensive assessments for the UISAT model, including in comparison to an improved convolutional deep belief network for relative accuracy verification, random points matching with Google Earth images, and comparing to other typical indices. The results demonstrated that the UISAT model provided the best performance in UIS detection from multitemporal Landsat images. The producer’s accuracies (PAs) of UISAT results in Beijing, Hefei, and Guangzhou, China, were 91.73%, 93.87%, and 92.84%, while the user’s accuracies (UA) were at 91.51%, 93.35%, and 92.24%. The proposed model outperformed the existing commonly used UIS indices [e.g., the Biophysical Composition Index (BCI), the Combinational Build-up Index (CBI), the Genetic Algorithm-based Urban Cluster Automatic Threshold (GA-UCAT), and the Gaussian-based Automatic Threshold Modified Normalized Difference Impervious Surface Index (G-MNDISI)] by an average increase of 6.51% in overall accuracy.

Image Representation Method Based on Complex Wavelet Transform and Phase Congruency, with Automatic Threshold Selection

Image representation is an active area of research with increasing applications in military and defense. Image representation aims at representing an image with lesser number of coefficients than the actual image, without affecting the image quality. It is the first step in image compression. Once the image is represented by using some set of coefficients, it is further encoded using various compression algorithms. This paper proposes an adaptive method for image representation, which uses Complex Wavelet transform and the concept of phase congruency, where the number of coefficients used for image representation depends on the information content in the input image. The efficiency of the proposed method has been assessed by comparing the number of coefficients used to represent the image using the proposed method with that used when Complex Wavelet transform is used for image representation. The resultant image quality is determined by computing the PSNR values and Normalized Cross Correlation. Experiments carried out show highly promising results, in terms of the reduction in the number of coefficients used for image representation and the quality of the resultant image.

Comparison of annual maximum and peaks-over-threshold methods with automated threshold selection in flood frequency analysis: a case study for Australia

Flood frequency analysis (FFA) involves fitting of a probability distribution to observed flood data. Two main models, annual maximum (AM) and peaks over threshold (POT), are generally adopted in FFA. The POT model is underemployed due to its complexity and uncertainty associated with threshold selection and meeting independence criteria in selecting POT data series. This study evaluates the POT and AM models using data from 188 gauged stations in southeast Australia. The POT model adopted in this study applies different average numbers of events per year fitted with generalized Pareto (GP) distribution with an automated threshold detection method. For the AM model, the GP distribution is also adopted, and flood quantiles estimated by the two models are compared. It has been found that there are notable differences in design flood estimates between the AM and POT models. The study uses catchment characteristics data to understand the differences in quantile estimates between the AM and POT models. The percentage differences between the AM and POT models can be explained by mean annual rainfall (MAR) and mean annual evapotranspiration (MAE) (ARIs of 1.01–5 years), MAR (10-year ARI), stream density (SDEN) (for 20- and 50-year ARIs) and SDEN, main stream slope (S1085) and MAE (for 100-year ARI). Stations showing smaller % differences have relatively higher MAR and smaller MAE (indicating wetter condition); conversely, stations showing higher % differences have smaller MAR and higher MAE (drier condition).

Uncertainty in denoising of MRSI using low‐rank methods

PurposeLow-rank denoising of MRSI data results in an apparent increase in spectral SNR. However, it is not clear if this translates to a lower uncertainty in metabolite concentrations after spectroscopic fitting. Estimation of the true uncertainty after denoising is desirable for downstream analysis in spectroscopy. In this work, the uncertainty reduction from low-rank denoising methods based on spatiotemporal separability and linear predictability in MRSI are assessed. A new method for estimating metabolite concentration uncertainty after denoising is proposed. Automatic rank threshold selection methods are also assessed in simulated low SNR regimes.MethodsAssessment of denoising methods is conducted using Monte Carlo simulation of proton MRSI data and by reproducibility of repeated in vivo acquisitions in 5 subjects.ResultsIn simulated and in vivo data, spatiotemporal based denoising is shown to reduce the concentration uncertainty, but linear prediction denoising increases uncertainty. Uncertainty estimates provided by fitting algorithms after denoising consistently underestimate actual metabolite uncertainty. However, the proposed uncertainty estimation, based on an analytical expression for entry-wise variance after denoising, is more accurate. It is also shown automated rank threshold selection using Marchenko-Pastur distribution can bias the data in low SNR conditions. An alternative soft-thresholding function is proposed.ConclusionLow-rank denoising methods based on spatiotemporal separability do reduce uncertainty in MRS(I) data. However, thorough assessment is needed as assessment by SNR measured from residual baseline noise is insufficient given the presence of non-uniform variance. It is also important to select the right rank thresholding method in low SNR cases.

Automatic threshold selection algorithm to distinguish a tissue chromophore from the background in photoacoustic imaging.

The adaptive matched filter (AMF) is a method widely used in spectral unmixing to classify different tissue chromophores in photoacoustic images. However, a threshold needs to be applied to the AMF detection image to distinguish the desired tissue chromophores from the background. In this study, we propose an automatic threshold selection (ATS) algorithm capable of differentiating a target from the background, based on the features of the AMF detection image. The mean difference between the estimated thickness, using the ATS algorithm, and the known values was 0.17 SD (0.24) mm for the phantom inclusions and -0.05 SD (0.21) mm for the tissue samples of malignant melanoma. The evaluation shows that the thickness and the width of the phantom inclusions and the tumors can be estimated using AMF in an automatic way after applying the ATS algorithm.

SVM Kernels comparison for brain tumor diagnosis using MRI

Magnetic Resonance Image (MRI) brain images have an essential role in medical analysis and cancer identification .In this paper multi kernel SVM algorithm is used for MRI brain tumor detection. The proposed work is involving the following stages: image acquisition, image preprocessing, feature extraction and tumor classification. An automatic threshold selection region based segmentation method called Otsu is used for thresholding during preprocessing stage. SVM classification algorithm with four different kernels are used to determine the normal and abnormal images. SVM with quadratic kernel results in best classification accuracy of 86.5%.

An automatic three-way clustering method based on sample similarity

The three-way clustering is an extension of traditional clustering by adding the concept of fringe region, which can effectively solve the problem of inaccurate decision-making caused by inaccurate information or insufficient data in traditional two-way clustering methods. The existing three-way clustering works often select the appropriate number of clusters and the thresholds for three-way partition according to subjective tuning. However, the method of fixing the number of clusters and the thresholds of the partition cannot automatically select the optimal number of clusters and partition thresholds for different data sets with different sizes and densities. To address the above problem, this paper proposed an improved three-way clustering method. First, we define the roughness degree by introducing the sample similarity to measure the uncertainty of the fringe region. Moreover, based on the roughness degree, we define a novel partitioning validity index to measure the clustering partitions and propose an automatic threshold selection method. Second, based on the concept of sample similarity, we introduce the intra-class similarity and the inter-class similarity to describe the quantitative change of the relationship between the sample and the clusters, and define a novel clustering validity index to measure the clustering performance under different numbers of clusters through the integration of the above two kinds of similarities. Furthermore, we propose an automatic cluster number selection method. Finally, we give an automatic three-way clustering approach by combining the proposed threshold selection method and the cluster number selection method. The comparison experiments demonstrate the effectiveness of our proposal.

Cost-Sensitive Ensemble Feature Ranking and Automatic Threshold Selection for Chronic Kidney Disease Diagnosis

Automated medical diagnosis is one of the important machine learning applications in the domain of healthcare. In this regard, most of the approaches primarily focus on optimizing the accuracy of classification models. In this research, we argue that, unlike general-purpose classification problems, medical applications, such as chronic kidney disease (CKD) diagnosis, require special treatment. In the case of CKD, apart from model performance, other factors such as the cost of data acquisition may also be taken into account to enhance the applicability of the automated diagnosis system. In this research, we proposed two techniques for cost-sensitive feature ranking. An ensemble of decision tree models is employed in both the techniques for computing the worth of a feature in the CKD dataset. An automatic threshold selection heuristic is also introduced which is based on the intersection of features’ worth and their accumulated cost. A set of experiments are conducted to evaluate the efficacy of the proposed techniques on both tree-based and non tree-based classification models. The proposed approaches were also evaluated against several comparative techniques. Furthermore, it is demonstrated that the proposed techniques select around 1/4th of the original CKD features while reducing the cost by a factor of 7.42 of the original feature set. Based on the extensive experimentation, it is concluded that the proposed techniques employing feature-cost interaction heuristic tend to select feature subsets that are both useful and cost-effective.

A Comparative Study of Water Indices and Image Classification Algorithms for Mapping Inland Surface Water Bodies Using Landsat Imagery

A comparative study of water indices and image classification algorithms for mapping inland water bodies using Landsat imagery was carried out through obtaining 24 high-resolution (≤5 m) and cloud-free images archived in Google Earth with the same (or ±1 day) acquisition dates as the Landsat-8 OLI images over 24 selected lakes across the globe, and developing a method to generate the alternate ground truth data from the Google Earth images for properly evaluating the Landsat image classification results. In addition to the commonly used green band-based water indices, Landsat-8 OLI’s ultra-blue, blue, and red band-based water indices were also tested in this research. Two unsupervised (the zero-water index threshold H0 method and Otsu’s automatic threshold selection method) and one supervised (the k-nearest neighbor (KNN) method) image classification algorithms were employed for conducting the image classification. Through comparing a total of 2880 Landsat image classification results with the alternate ground truth data, this study showed that (1) it is not necessary to use some supervised image classification methods for extracting water bodies from Landsat imagery given the high computational cost associated with the supervised image classification algorithms; (2) the unsupervised classification algorithms such as the H0 and Otsu methods could achieve comparable accuracy as the KNN method, although the H0 method produced more large error outliers than the Otsu method, thus the Otsu method is better than the H0 method; and (3) the ultra-blue band-based AWEInsuB is the best water index for the H0 method, and the ultra-blue band-based MNDWI2uB is the best water index for both the Otsu and KNN methods.

L-moments for automatic threshold selection in extreme value analysis

In extreme value analysis, sensitivity of inference to the definition of extreme event is a paramount issue. Under the peaks-over-threshold approach, this translates directly into the need of fitting a Generalized Pareto distribution to observations above a suitable level that balances bias versus variance of estimates. Selection methodologies established in the literature face recurrent challenges such as an inherent subjectivity or high computational intensity. We suggest a truly automated method for threshold detection, aiming at time efficiency and elimination of subjective judgment. Based on the well-established theory of L-moments, this versatile data-driven technique can handle batch processing of large collections of extremes data, while also presenting good performance on small samples. The technique’s performance is evaluated in a large simulation study and illustrated with significant wave height data sets from the literature. We find that it compares favorably to other state-of-the-art methods regarding the choice of threshold, associated parameter estimation and the ultimate goal of computationally efficient return level estimation.

The risk function of the goodness-of-fit tests for tail models

This paper contributes to answering a question that is of crucial importance in risk management and extreme value theory: How to select the threshold above which one assumes that the tail of a distribution follows a generalized Pareto distribution. This question has gained increasing attention, particularly in finance institutions, as the recent regulative norms require the assessment of risk at high quantiles. Recent methods answer this question by multiple uses of the standard goodness-of-fit tests. These tests are based on a particular choice of symmetric weighting of the mean square error between the empirical and the fitted tail distributions. Assuming an asymmetric weighting, which rates high quantiles more than small ones, we propose new goodness-of-fit tests and automated threshold selection procedures. We consider a parameterized family of asymmetric weight functions and calculate the corresponding mean square error as a loss function. We then explicitly determine the risk function as the finite sample expected value of the loss function. Finally, the risk function can be used to discuss the question of which symmetric or asymmetric weight function and, thus, which goodness-of-fit test should be used in a new method for determining the threshold value.

Parameter estimation and threshold selection uncertainty in extreme wind speed distribution – A frequentist approach with generalized Pareto distribution using automatic threshold selection

Parameter estimation and threshold selection uncertainty in extreme wind speed distribution – A frequentist approach with generalized Pareto distribution using automatic threshold selection

Medical image segmentation based on fuzzy 2-partition Kapur entropy using fast recursive algorithm

Segmentation is a foremost part of computer-aided diagnosis (CAD) and medical image analysis. This paper provides an automatic threshold selection approach for the segmentation of medical images based on fuzzy 2-partition using Kapur entropy. The fuzzy 2-partition Kapur entropy approach converts the image into fuzzy 2-partition by applying two parameterised fuzzy membership functions. The optimal threshold is attained by searching for an ideal combination of the parameter for the fuzzy membership functions so that the fuzzy 2-partition Kapur entropy is maximised. The complexity of searching the optimal combination of parameters is reduced by integrating the concept of fast recursive algorithm and Kapur entropy. The proposed approach performance is measured using several medical images and it is found that results are considerable encouraging. The proposed approach could be put up as a component of a CAD system for early detection of diseases like cancer.

Future floods using hydroclimatic simulations and peaks over threshold: An alternative to nonstationary analysis inferred from trend tests

Nonstationary flood frequency analysis (NFFA) has increasingly been applied to predict future floods under climate change. The inference of nonstationarity from trend tests (INTT) on historical floods is a widespread practice used to justify the application of NFFA; however, its reliability has seldom been investigated. This research examines future floods from interpretations of INTT compared to those obtained from cause-and-effect processes by a hydrological model (INCE). The study uses INCE to quantify the changes in the regime and magnitude of floods due to potential climate change in the mid-21st century using multi-model ensemble simulations under two greenhouse gas emissions scenarios (i.e., RCP 2.6 and RCP 8.5). Independent peaks over threshold from simulated streamflow were used to estimate the changes in future (2040–2064) flood regimes and magnitudes compared to their historical (1983–2007) counterparts for 29 unregulated catchments across Alberta, Canada. Separation of extreme events and their fittings to generalized Pareto distributions (GPD) were based on a hybrid approach that combined two developed automated threshold selection methods and four estimators for the parameters of the GPD. Based on comparing the results of INTT and INCE, we show that future changes in floods contradict. We also show that the future frequency curves shifted differently at different return periods compared to historical curves, while in some instances, future climate tended to decrease small floods and increase larger floods or vice versa. Finally, flood magnitudes in 2/3 of the studied catchments in Alberta are predicted to intensify, accompanied by increases in the rate of occurrence and earlier shifts in the timing of floods for both climate scenarios, whereas no considerable change in the duration was recognized.

Automatic defect identification technology of digital image of pipeline weld

Digital image of pipeline weld is an important basis for the reliability management of pipeline welds. However, the error rate of artificial discrimination is high. In order to increase the defect identification accuracy of digital image of pipeline weld, we adopted several methods (e.g. multiple edge detection, detection channel and threshold segmentation) to carry out image processing on the image defects of pipeline welds. Then, a defect characteristic database on the digital images of pipeline welds was constructed, including grayscale difference, equivalent area (S/C), circularity, entropy, correlation and other parameters. Furthermore, a multi-classifier construction (SVM) model was established. Thus, the classification and evaluation on the defects in the digital images of pipeline welds were realized. Finally, an automatic defect identification software for digital image of pipeline weld was developed and verified on site. And the following research results were obtained. First, after image processing, the edge detection results obtained by Canny and other algorithms are satisfactory when there is no noise. In the case of noise, however, pseudo-edge emerges in the detection results. In this case, the automatic threshold selection method shall be adopted to detect the image edge to obtain the rational threshold. Second, there are 14 parameters in the defect characteristic database, including shape characteristic, lamination characteristic and image length pixel. Third, by virtue of the SVM classification model, the shape characteristics of each type of defect can be clarified, and the defect characteristics can be identified, such as crack, slag inclusion, air hole, incomplete penetration, non-fusion and strip. Based on field application, the following results were obtained. First, this automatic defect identification technology is applicable to quality identification and evaluation of various defects in pipeline welds. Second, its identification accuracy is higher than 90%. Third, by virtue of this technology, automatic defect identification and evaluation of digital image of pipeline weld is realized. In conclusion, these research results help to ensure the safe operation of pipelines.

Simulation of multi-platform LiDAR for assessing total leaf area in tree crowns

LiDAR (Light Detection and Ranging) technology has been increasingly implemented to assess the biophysical attributes of forest canopies. However, LiDAR-based estimation of tree biophysical attributes remains difficult mainly due to the occlusion of vegetative elements in multi-layered tree crowns. In this study, we developed a new algorithm along with a multiple-scan methodology to analyse the impact of occlusion on LiDAR-based estimates of tree leaf area. We reconstructed five virtual tree models using a computer graphic-based approach based on in situ measurements from multiple tree crowns, for which the position, size, orientation and area of all leaves were measured. Multi-platform LiDAR simulations were performed on these 3D tree models through a point-line intersection algorithm. An approach based on the Delaunay triangulation algorithm with automatic adaptive threshold selection was proposed to construct the scanned leaf surface from the simulated discrete LiDAR point clouds. In addition, the leaf area covered by laser beams in each layer was assessed in combination with the ratio and number of the scanned points. Quantitative comparisons of LiDAR scanning for the occlusion effects among various scanning approaches, including fixed-position scanning, multiple terrestrial LiDAR scanning and airborne-terrestrial LiDAR cross-scanning, were assessed on different target trees. The results showed that one simulated terrestrial LiDAR scan alongside the model tree captured only 25–38% of the leaf area of the tree crown. When scanned data were acquired from three simulated terrestrial LiDAR scans around one tree, the accuracy of the leaf area recovery rate reached 60–73% depending on the leaf area index, tree crown volume and leaf area density. When a supplementary airborne LiDAR scanning was included, occlusion was reduced and the leaf area recovery rate increased to 72–90%. Our study provides an approach for the measurement of total leaf area in tree crowns from simulated multi-platform LiDAR data and enables a quantitative assessment of occlusion metrics for various tree crown attributes under different scanning strategies.

Automatic localization of the rotor-stator rubbing fault based on acoustic emission method and higher-order statistics

Localization to the rotor-stator rubbing fault of rotating machinery such as aero-engines has been attracting plenty of attentions in fault diagnosis field. Present study attempts to effectively and accurately localize such rubbing sources based on modal AE method automatically. In order to explore the propagation characteristic of rubbing signals, theoretical analysis and finite element simulation on a stator case are conducted. And the dependence of accuracy of time difference of arrival (TDOA) method on the propagation characteristic is investigated. Based on the analysis result, a higher-order statistics (HOS) algorithm is introduced into acoustic emission (AE) to automatically identify the arrival time of rubbing signals and localize the rubbing source via automatic threshold selection. To verify the effectiveness of the proposed AE method to recognize the rubbing fault, a series of rubbing tests are carried out. It is demonstrated by the experimental results that the accuracy of the proposed method is improved compared with conventional TDOA methods since more than half of the sources localized at the rubbing region can be identified in almost all cases and the identification accuracy is up to 80 % at threshold of 85 % of amplitude.