Erroneous Features Research Articles

ChatDiff: A ChatGPT-based diffusion model for long-tailed classification

Long-tailed data distributions have been a major challenge for the practical application of deep learning. Information augmentation intends to expand the long-tailed data into uniform distribution, which provides a feasible way to mitigate the data starvation of underrepresented classes. However, most existing augmentation methods face two significant challenges: (1) limited diversity in generated samples, and (2) the adverse effect of generated negative samples on downstream classification performance. In this paper, we propose a novel information augmentation method, named ChatDiff, to provide diverse positive samples for underrepresented classes, and eliminate generated negative samples. Specifically, we start with a prompt template to extract textual prior knowledge from the ChatGPT-3.5 model, enhancing the feature space for underrepresented classes. Then using this prior knowledge, a conditional diffusion model generates semantic-rich image samples for tail classes. Moreover, the proposed ChatDiff leverages a CLIP-based discriminator to screen and remove generated negative samples. This process avoids neural network learning the invalid or erroneous features, and further, improves long-tailed classification performance. Comprehensive experiments conducted on long-tailed benchmarks such as CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist 2018, validate the effectiveness of our ChatDiff method.

An improved KAZE algorithm for SAR image matching

ABSTRACT The KAZE algorithm has been popularly used to match the remote sensing images. However, it has limited performance when its standard form is directly applied to match the synthetic aperture radar (SAR) images containing significant multiplicative speckle noise. In this paper, an effective SAR image matching algorithm is proposed, which is a combination of KAZE, phase congruency, and speckle noise removing anisotropic diffusion. In this algorithm, the ratio-based Phase Congruency (PC) information is used to eliminate the erroneous features, which improves the repeatability rate of the KAZE features. In addition, the scale layers of the input SAR images are constructed by speckle noise removing anisotropic diffusion method which significantly reduces the effect of noise. The proposed method can improve the repeatability of the extracted KAZE features. Moreover, it can give better recall and precision values than the state-of-the-art methods. It gives the root mean square error (RMSE) values in the range of 0.881 to 1.104 pixels in SAR image matching. Experiments on multi-temporal SAR images demonstrate the applicability of the proposed method.

Reconstructing Static Memories from the Brain with EEG Feature Extraction and Generative Adversarial Networks

Forensic tools and facial recognition techniques have implemented state memory reconstruction in the court of law. Recent research has incorporated machine learning into this field in order to fulfill the rising demand for this service. However, the suggested solutions require expensive cutting-edge equipment such as fMRI and CT scanners. The goal of the present study is to recreate images using user electroencephalography (EEG) input patterns. We combined a discriminator and generator network in order to decrease blur, face distortion, and erroneous features in the ImageNet dataset. An EEG feature matrix was produced using a convolutional neural network (CNN) encoder from spectrogram inputs, which are visual representations of the spectrum of electrical frequencies observed at each electrode. The encoder produced an 85% accuracy when linking spectrograms to the labels of the relevant images. The feature matrix was passed into the GAN during training. The viability of image reconstruction and coherent image representation from the brain were both proved feasible to a dependable degree due to our GAN output images’ resemblance to the original dataset. The present work could find applications in future studies as a non-invasive, more affordable option to recreate memories from brain signals.

New patch-based strategy for COVID-19 automatic identification using chest x-ray images.

The development of a robust model for automatic identification of COVID-19 based on chest x-rays has been a widely addressed topic over the last couple of years; however, the scarcity of good quality images sets, and their limited size, have proven to be an important obstacle to obtain reliable models. In fact, models proposed so far have suffered from over-fitting erroneous features instead of learning lung features, a phenomenon known as shortcut learning. In this research, a new image classification methodology is proposed that attempts to mitigate this problem. To this end, annotation by expert radiologists of a set of images was performed. The lung region was then segmented and a new classification strategy based on a patch partitioning that improves the resolution of the convolution neural network is proposed. In addition, a set of native images, used as an external evaluation set, is released. The best results were obtained for the 6-patch splitting variant with 0.887 accuracy, 0.85 recall and 0.848 F1score on the external validation set. The results show that the proposed new strategy maintains similar values between internal and external validation, which gives our model generalization power, making it available for use in hospital settings. The online version contains supplementary material available at 10.1007/s12553-022-00704-4.

Deep learning models for COVID-19 chest x-ray classification: Preventing shortcut learning using feature disentanglement.

In response to the COVID-19 global pandemic, recent research has proposed creating deep learning based models that use chest radiographs (CXRs) in a variety of clinical tasks to help manage the crisis. However, the size of existing datasets of CXRs from COVID-19+ patients are relatively small, and researchers often pool CXR data from multiple sources, for example, using different x-ray machines in various patient populations under different clinical scenarios. Deep learning models trained on such datasets have been shown to overfit to erroneous features instead of learning pulmonary characteristics in a phenomenon known as shortcut learning. We propose adding feature disentanglement to the training process. This technique forces the models to identify pulmonary features from the images and penalizes them for learning features that can discriminate between the original datasets that the images come from. We find that models trained in this way indeed have better generalization performance on unseen data; in the best case we found that it improved AUC by 0.13 on held out data. We further find that this outperforms masking out non-lung parts of the CXRs and performing histogram equalization, both of which are recently proposed methods for removing biases in CXR datasets.

Using the atmospheric CO2 growth rate to constrain the CO2 flux from land use and land cover change since 1900.

We explore the ability of the atmospheric CO2 record since 1900 to constrain the source of CO2 from land use and land cover change (hereafter "land use"), taking account of uncertainties in other terms in the global carbon budget. We find that the atmospheric constraint favors land use CO2 flux estimates with lower decadal variability and can identify potentially erroneous features, such as emission peaks around 1960 and after 2000, in some published estimates. Furthermore, we resolve an offset in the global carbon budget that is most plausibly attributed to the land use flux. This correction shifts the mean land use flux since 1900 across 20 published estimates down by 0.35 PgC year-1 to 1.04 ± 0.57 PgC year-1 , which is within the range but at the low end of these estimates. We show that the atmospheric CO2 record can provide insights into the time history of the land use flux that may reduce uncertainty in this term and improve current understanding and projections of the global carbon cycle.

To Fix Those Plots, Use Limits!

Summary Professional mathematical software may produce plots of elementary functions that are in error by orders of magnitude or show erroneous features. The same software may give a correct limit near an endpoint, but such a limit yields at most a single point on the graph. In contrast, proofs or derivations of the same limit provide information to guide the software toward a sketch that shows correct trends and stays within correct bounds.

Learning non-stationary Langevin dynamics from stochastic observations of latent trajectories

Many complex systems operating far from the equilibrium exhibit stochastic dynamics that can be described by a Langevin equation. Inferring Langevin equations from data can reveal how transient dynamics of such systems give rise to their function. However, dynamics are often inaccessible directly and can be only gleaned through a stochastic observation process, which makes the inference challenging. Here we present a non-parametric framework for inferring the Langevin equation, which explicitly models the stochastic observation process and non-stationary latent dynamics. The framework accounts for the non-equilibrium initial and final states of the observed system and for the possibility that the system’s dynamics define the duration of observations. Omitting any of these non-stationary components results in incorrect inference, in which erroneous features arise in the dynamics due to non-stationary data distribution. We illustrate the framework using models of neural dynamics underlying decision making in the brain.

An Integrated Model for Transformer Fault Diagnosis to Improve Sample Classification near Decision Boundary of Support Vector Machine

Support vector machine (SVM), which serves as one kind of artificial intelligence technique, has been widely employed in transformer fault diagnosis when involving dissolved gas analysis (DGA). However, when using SVM, it is easy to misclassify samples which are located near the decision boundary, resulting in a decrease in the accuracy of fault diagnosis. Given this issue, this paper proposed a genetic algorithm (GA) optimized probabilistic SVM (GAPSVM) integrated with the fuzzy three-ratio (FTR) method, in which the GAPSVM can judge whether a sample is near the decision boundary according to its output probabilities and diagnose the samples which are not near the decision boundary. Then, FTR is used to diagnose the samples which are near the decision boundary. Combining GAPSVM and FTR, the integrated model can accurately diagnose samples near the decision boundary of SVM. In addition, to avoid redundant and erroneous features, this paper also used GA to select the optimal DGA features. The diagnostic accuracy of the proposed GAPSVM integrated with the FTR fault diagnosis method reached 86.80% after 10 repeated calculations using 118 groups of IEC technical committee (TC) 10 samples. Moreover, the robustness is also proven through 30 groups of DGA samples from the State Grid Co. of China and 15 practical cases with missing values.

A semi-supervised learning algorithm via adaptive Laplacian graph

Many semi-supervised learning methods have been developed in recent years, especially graph-based approaches, which have achieved satisfactory performance in the practical applications. There are two points that need to be noticed. Firstly, the quality of the graph directly affects the final classification accuracy. However, graph-based algorithms mostly use k-Nearest Neighbor to construct the graph. And the directly constructed graph is inaccurate due to outliers and erroneous features in the data. Secondly, the amount of labeled data is a small part of all data. It cannot be guaranteed that all categories of data are included in the labeled data and the labels of data are not totally correct in practice. To address the aforementioned problems, we propose a new graph-based semi-supervised method named ALGSSL via adaptive Laplacian graph. In the algorithm, we adaptively update the graph to reduce the sensitiveness of the construction of initial graph. Meanwhile, we use the regularization parameters to set confidence on existing labels, which can reduce the impact of the error labels on the result and discover the new category. Experiments on three toy datasets and nine benchmark datasets demonstrate the proposed method can achieve good performance.

Automatic grading software for 2D CAD files

AbstractUndergraduate computer‐aided design (CAD) courses can have large enrollments, making it difficult for instructors to thoroughly assess each student's assignments and provide individualized feedback. While in some subject areas textbook companies are developing online platforms that adapt problem sets to meet each individual student's needs, there has been apparently little attention to the area of individualized CAD instruction. This paper presents a program that compares students’ two‐dimensional CAD drawings to a solution, computes the number of missing or erroneous features in each student's drawing, and generates a visual comparison of each student's drawing with the solution. The system is implemented both as an offline version for batch processing and an online version for immediate student feedback on assignments. While the system lacks the “smartness” of a human grader, the offline version substantially reduces time spent grading exams and improves grading accuracy and consistency, and the online version encourages students to focus on important details of drawing and provides immediate online feedback for students as they complete assignments, as well as providing after‐the‐fact feedback on exams.

Differential evolution for feature selection: a fuzzy wrapper–filter approach

The selection of an optimal feature subset from all available features in the data is a vital task of data pre-processing used for several purposes such as the dimensionality reduction, the computational complexity reduction required for data processing (e.g., clustering, classification and regression) and the performance enhancement of a data processing technique. To serve such purposes, feature selection approaches which are fundamentally categorized into filters and wrappers try to eliminate irrelevant, redundant and erroneous features in the data. Each category comes with its own advantages and disadvantages. While wrappers can generally provide higher classification performance than filters, filters are computationally more efficient than wrappers. In order to bring the advantages of wrappers and filters together, i.e., to get higher classification performance with smaller feature subset size in a shorter time, this paper proposes a differential evolution approach combining filter and wrapper approaches through an improved information theoretic local search mechanism which is based on the concepts of fuzziness to cope with both continuous and discrete datasets. To show the superiority of the proposed approach, it is examined and compared with traditional and recent evolutionary feature selection approaches on several benchmarks from different well-known data repositories.

Artefact detection in global digital elevation models (DEMs): The Maximum Slope Approach and its application for complete screening of the SRTM v4.1 and MERIT DEMs

Despite post-processing efforts by space agencies and research institutions, contemporary global digital elevation models (DEMs) may contain artefacts, i.e., erroneous features that do not exist in the actual terrain, such as spikes, holes and line errors. The goal of the present paper is to illuminate the artefact issue of current global DEM data sets that might be an obstacle for any geoscience study using terrain information. We introduce the Maximum Slope Approach (MSA) as a technique that uses terrain slopes as indicator to detect and localize spurious artefacts. The MSA relies on the strong sensitivity of terrain slopes for sudden steps in the DEM that is a direct feature of larger artefacts. In a numerical case study, the MSA is applied for globally complete screening of two SRTM-based 3 arc-second DEMs, the SRTM v4.1 and the MERIT-DEM. Based on 0.1° × 0.1° sub-divisions and a 5 m/m slope threshold, 1341 artefacts were detected in SRTM v4.1 vs. 108 in MERIT. Most artefacts spatially correlate with SRTM voids (and thus with the void-filling) and not with the SRTM-measured elevations. The strong contrast in artefact frequency (factor ~12) is attributed to the SRTM v4.1 hole filling. Our study shows that over parts of the Himalaya Mountains the SRTM v4.1 data set is contaminated by step artefacts where the use of this DEM cannot be recommended. Some caution should be exercised, e.g., over parts of the Andes and Rocky Mountains. The same holds true for derived global products that depend on SRTM v4.1, such as gravity maps. Primarily over the major mountain ranges, the MERIT model contains artefacts, too, but in smaller numbers. As a conclusion, globally complete artefact screening is recommended prior to the public release of any DEM data set. However, such a quality check should also be considered by users before using DEM data. MSA-based artefact screening is not only limited to DEMs, but can be applied as quality assurance measure to other gridded data sets such as digital bathymetric models or gridded physical quantities such as gravity or magnetics.

Signal processing methods in fault detection in manufacturing systems

The paper gives a short introduction to the problem of fault detection in manufacturing systems using digital signal processing methods. Usually, in manufacturing systems faults can occur in electrical drives, transmission lines, power management systems and can be detected through sensor data acquisition. Important task of the diagnosis is to differentiate normal operating condition from faulty condition. Detection of occurred faults in manufacturing systems depends on how efficiently erroneous features are extracted from acquired signals. This work focuses on signal processing based methods using Discrete Wavelet and Wavelet Packet Transforms for detection and classification the occurred faults. The faults are simulated using test signals with different time and frequency properties and the results obtained from different approaches are evaluated and compared. The simulation results prove that the proposed techniques handle the problem of fault detection and may even predict abnormalities exploring long term tendencies of the detected signals.

Crosstalk Effect in SNPP VIIRS

An investigation has been carried out to examine the crosstalk contamination in the Visible Infrared Imaging Radiometer Suite (VIIRS) onboard the Suomi National Polar-orbiting Partnership (SNPP) spacecraft. Prior to this study, the cause of the pronounced striping in Earth View (EV) images and obvious discontinuity in the EV brightness temperature (BT) of the thermal emissive bands (TEB) during black body (BB) warm-up cool-down (WUCD) calibration observed since launch has not been identified. Meanwhile, it has been recently demonstrated in the MODerate-resolution Imaging Spectroradiometer (MODIS) long-wave infrared (LWIR) photovoltaic (PV) bands that the crosstalk effect induces the same erroneous features. In this investigation, it is shown that the established lunar imagery analysis indeed verifies the existence of crosstalk contamination in SNPP VIIRS TEB. The crosstalk effect is quantitatively characterized by deriving the crosstalk coefficients from the scheduled lunar observations. The magnitude of the effect is comparatively smaller than that in MODIS LWIR PV bands, but is of a large enough magnitude to induce the aforementioned artificial features. Among all SNPP VIIRS TEB, Band M14 has the largest crosstalk contamination from Band M15, while Bands M13, M15, M16, and I5 have pronounced crosstalk effects as well. One new detail of the crosstalk effect specific to SNPP VIIRS, differing from the MODIS result, is the distinctive two-group pattern of odd and even detectors for each affected band due to the arrangement of the detector on the focal plane assembly (FPA). This is fully consistent with the earlier finding that this odd-even detector arrangement contributes to striping in the sea surface temperature (SST) products. Our analyses additionally suggest an explanation of the large temperature anomalies appearing during the WUCD time periods. The parallel effort examining the potential crosstalk contamination in SNPP VIIRS reflective solar bands, however, reveals no observable effect.

Genix: a new online automated pipeline for bacterial genome annotation.

Next-generation sequencing has significantly reduced the cost of genome-sequencing projects, resulting in an expressive increase in the availability of genomic data in public databases. The cheaper and easier is to sequence new genomes, the more accurate the annotation steps have to be to avoid both the loss of information and the accumulation of erroneous features that may affect the accuracy of further analysis. In the case of bacteria genomes, a range of web annotation software has been developed; however, many applications have yet to incorporate the steps required to improve their result, including the removal of false-positive/spurious and a more complete identification of non-coding features. We present Genix, a new web-based bacterial genome annotation pipeline. A comparison of the results generated by Genix for four reference genomes against those generated by other annotation tools indicated that our pipeline is able to provide results that are closer to the reference genome annotation, with a smaller amount of false-positive proteins and missing functional annotated proteins. Additionally, the metrics obtained by Genix were slightly better than those obtained by Prokka, a state-of-art standalone annotation system. Our results indicate that Genix is a useful tool that is able to provide a more refined result, and may be a user-friendly way to obtain high-quality results.

A DEM of the 2010 surface topography of Storglaciären, Sweden

ABSTRACTDuring the summer of 2010 the surface elevation of Storglaciären in northern Sweden was measured using high-precision GNSS and reflectorless Total Station surveys. The DEM created from these data contain less noise than those created from orthophotographic methods over snow covered glaciers and is therefore smoother, with fewer erroneous features in the data. The principal, though not sole, intended use for the DEM is in the calculation of surface mass balance, which has influenced decisions on what constitutes a functional part of a glacier, leading to the exclusion of features such as snow aprons and perennial ice above the bergschrund. Other peripheral features have changed since the previous, aerial survey from 1999 leading to a reduction in size of approximately 0.17 km2.

Artifacts in Stage IV NWS Real-Time Multisensor Precipitation Estimates and Impacts on Identification of Maximum Series

AbstractIn recent years, there has been a growing interest by different user communities to use radar-based rainfall products for identifying and quantifying heavy precipitation and other related information. A potentially viable product for such analyses is the National Weather Service (NWS) Stage IV Quantitative Precipitation Estimate (QPE), which covers the entire conterminous United States and is available at 1, 6, and 24 h temporal resolutions. The current article focuses on data artifacts in the Stage IV product over its full record period (2002–2013) and assesses their impacts on the analysis and derivation of heavy rainfall statistics. Examining annual maximum series extracted from the Stage IV 1-h accumulation product revealed the presence of numerous artifacts that were mostly linked to erroneous rain gauge reports and which can be mistaken as rainfall maxima. Artifacts were found over many parts of the United States, including the west, south, southeast, and northeast. Other erroneous features ...

Vermiculate artefacts in image analysis of granular materials

Some reported analyses of images of deforming granular materials have generated surprising vermiculate strain features which are difficult to reconcile with the mechanics of deformation of granular matter. Detailed investigation using synthetic images and improved processing of images of laboratory experiments indicates that such features can emerge as a consequence of the image acquisition (sensor, contrast, resolution), the subsequent image correlation implementation, and the user’s choice of processing parameters. The two principal factors are: (i) the texture and resolution of the images and (ii) the algorithm used to achieve sub-pixel displacement resolution. Analysis of the images using a sub-pixel interpolation algorithm that is more robust than that used originally eliminates the vermiculate features for images with moderate resolution and texture. However, erroneous features persist in images with low resolution and poor texture. Guidance is provided on ways in which such artefacts can be avoided through improved experimental and image analysis techniques.

Simultaneous feature selection and weighting – An evolutionary multi-objective optimization approach

Selection of feature subset is a preprocessing step in computational learning, and it serves several purposes like reducing the dimensionality of a dataset, decreasing the computational time required for classification and enhancing the classification accuracy of a classifier by removing redundant and misleading or erroneous features. This paper presents a new feature selection and weighting method aided with the decomposition based evolutionary multi-objective algorithm called MOEA/D. The feature vectors are selected and weighted or scaled simultaneously to project the data points to such a hyper space, where the distance between data points of non-identical classes is increased, thus, making them easier to classify. The inter-class and intra-class distances are simultaneously optimized by using MOEA/D to obtain the optimal features and the scaling factor associated with them. Finally, k-NN (k-Nearest Neighbor) is used to classify the data points having the reduced and weighted feature set. The proposed algorithm is tested with several practical datasets from the well-known data repositories like UCI and LIBSVM. The results are compared with those obtained with the state-of-the-art algorithms to demonstrate the superiority of the proposed algorithm.