Large Training Data Research Articles

Infrared spectroscopy for ploidy estimation: An example in two species of Veronica using fresh and herbarium specimens.

Polyploidy has become a central factor in plant evolutionary biological research in recent decades. Methods such as flow cytometry have revealed the widespread occurrence of polyploidy; however, its inference relies on expensive lab equipment and is largely restricted to fresh or recently dried material. Here, we assess the applicability of infrared spectroscopy to infer ploidy in two related species of Veronica (Plantaginaceae). Infrared spectroscopy relies on differences in the absorbance of tissues, which could be affected by primary and secondary metabolites related to polyploidy. We sampled 33 living plants from the greenhouse and 74 herbarium specimens with ploidy known through flow cytometrical measurements and analyzed the resulting spectra using discriminant analysis of principal components (DAPC) and neural network (NNET) classifiers. Living material of both species combined was classified with 70% (DAPC) to 75% (NNET) accuracy, whereas herbarium material was classified with 84% (DAPC) to 85% (NNET) accuracy. Analyzing both species separately resulted in less clear results. Infrared spectroscopy is quite reliable but is not a certain method for assessing intraspecific ploidy level differences in two species of Veronica. More accurate inferences rely on large training data sets and herbarium material. This study demonstrates an important way to expand the field of polyploid research to herbaria.

Tropical Cyclone Wind Speed Estimation: A Large Scale Training Data Set and Community Benchmarking

AbstractTropical cyclones (TCs) cause significant disruptions to infrastructure and livelihood. The scale of loss due to TCs may be mitigated by prompt and accurate advisories about TC wind speed. Current advisories are consensus based and have a time delay of about 6 hr between each new update. As part of efforts to increase the frequency of wind speed estimations without compromising on the accuracy, we curate a high‐quality hurricane image and wind speed data set and develop a machine learning (ML) model for frequent wind speed estimation. The image and wind speed pairings are consolidated from different sources. The wind speeds are interpolated hourly to curate the TC wind speed estimation data set. The data set is supplemented with metadata that eases adoption of the data set by the ML community. We have also designed a competition to garner interest and encourage community involvement to build state‐of‐the‐art ML models to estimate wind speeds from satellite images of TCs. The competition winners were able to improve the wind speed estimation by almost 50% over the benchmark model.

Few-shot learning using explainable Siamese twin network for the automated classification of blood cells.

Automated classification of blood cells from microscopic images is an interesting research area owing to advancements of efficient neural network models. The existing deep learning methods rely on large data for network training and generating such large data could be time-consuming. Further, explainability is required via class activation mapping for better understanding of the model predictions. Therefore, we developed a Siamese twin network (STN) model based on contrastive learning that trains on relatively few images for the classification of healthy peripheral blood cells using EfficientNet-B3 as the base model. Hence, in this study, a total of 17,092 publicly accessible cell histology images were analyzed from which 6% were used for STN training, 6% for few-shot validation, and the rest 88% for few-shot testing. The proposed architecture demonstrates percent accuracies of 97.00, 98.78, 94.59, 95.70, 98.86, 97.09, 99.71, and 96.30 during 8-way 5-shot testing for the classification of basophils, eosinophils, immature granulocytes, erythroblasts, lymphocytes, monocytes, platelets, and neutrophils, respectively. Further, we propose a novel class activation mapping scheme that highlights the important regions in the test image for the STN model interpretability. Overall, the proposed framework could be used for a fully automated self-exploratory classification of healthy peripheral blood cells. The whole proposed framework demonstrates the Siamese twin network training and 8-way k-shot testing. The values indicate the amount of dissimilarity.

Machine learning-based inversion for acoustic impedance with large synthetic training data: Workflow and data characterization

Where wells are sparse or training data are difficult to label with high-quality wireline-derived impedance logs, machine learning (ML)-based inversion of acoustic impedance typically depends on small training data sets, leading to biased prediction. We have advanced a novel workflow that applies large synthetic seismic training data to reduce facies-related bias. Using a geologically realistic model as the truth model, we randomly select sparse seed wells to perform sequential Gaussian simulation (SGS) for impedance models of the same geometry and simulate facies variability. We implement random forest regression on 30 features extracted from the synthetic volume. We observe that more seed wells tend to reduce facies-induced bias by sampling more types of facies, resulting in a better prediction. We then focus on the responses of SGS models to facies changes, the number of seed wells necessary for a useful synthetic model, and how much a synthetic model can help ML-based inversion. We observe that the SGS synthetic training model outperforms well-direct training in general. For modeled clastic shore-zone systems in Miocene Gulf of Mexico, two or more seed wells are necessary for a significant reduction of root-mean-square error and outliners, and improvement of facies imaging. In a field-data test, we apply a similar workflow to quantitatively predict acoustic impedance, which is then converted to a sand-volume map at a high-frequency sequence (10–100 m), revealing detailed facies and sandstone patterns. Such results are valuable in many geologic and engineering applications, such as hydrocarbon and CO2 reservoir prospecting, reserve estimation, simulation, etc.

NuKit: A deep learning platform for fast nucleus segmentation of histopathological images.

Nucleus segmentation represents the initial step for histopathological image analysis pipelines, and it remains a challenge in many quantitative analysis methods in terms of accuracy and speed. Recently, deep learning nucleus segmentation methods have demonstrated to outperform previous intensity- or pattern-based methods. However, the heavy computation of deep learning provides impression of lagging response in real time and hampered the adoptability of these models in routine research. We developed and implemented NuKit a deep learning platform, which accelerates nucleus segmentation and provides prompt results to the users. NuKit platform consists of two deep learning models coupled with an interactive graphical user interface (GUI) to provide fast and automatic nucleus segmentation "on the fly". Both deep learning models provide complementary tasks in nucleus segmentation. The whole image segmentation model performs whole image nucleus whereas the click segmentation model supplements the nucleus segmentation with user-driven input to edits the segmented nuclei. We trained the NuKit whole image segmentation model on a large public training data set and tested its performance in seven independent public image data sets. The whole image segmentation model achieves average [Formula: see text] and [Formula: see text]. The outputs could be exported into different file formats, as well as provides seamless integration with other image analysis tools such as QuPath. NuKit can be executed on Windows, Mac, and Linux using personal computers.

Physics-driven self-supervised learning system for seismic velocity inversion

Seismic velocity inversion plays a vital role in various applied seismology processes. A series of deep learning methods have been developed that rely purely on manually provided labels for supervision; however, their performances depend heavily on using large training data sets with corresponding velocity models. Because no physical laws are used in the training phase, it is usually challenging to generalize trained neural networks to a new data domain. To mitigate these issues, we have embedded a seismic forward modeling step at the end of a network to remap the inversion result back to seismic data and thus train the neural network through self-supervised loss, i.e., the misfit between the network input and output. As a result, we eliminate the need for many labeled velocity models, and physical laws are introduced when back-propagating gradients through the seismic forward modeling step. We verify the effectiveness of our approach through comprehensive experiments on synthetic data sets, where self-supervised learning outperforms the fully supervised approach, which accesses much more labeled data. The superior performance is even more significant when compared with a new data domain that has velocity models with faults and more geologic layers. Finally, in case of unknown and more complex data types, we develop a network-constrained full-waveform inversion (FWI) method. This method refines the initial prediction of the network by iteratively optimizing network parameters other than the velocity model, as found with the conventional FWI method, and demonstrates clear advantages in terms of interface and velocity accuracy. With these measures (self-supervised learning and network-constrained FWI), our physics-driven self-supervised learning system successfully mitigates issues such as the dependence on large labeled data sets, the absence of physical laws, and the difficulty in adapting to new data domains.

Rethinking Collaborative Metric Learning: Toward an Efficient Alternative Without Negative Sampling.

The recently proposed Collaborative Metric Learning (CML) paradigm has aroused wide interest in the area of recommendation systems (RS) owing to its simplicity and effectiveness. Typically, the existing literature of CML depends largely on the negative sampling strategy to alleviate the time-consuming burden of pairwise computation. However, in this work, by taking a theoretical analysis, we find that negative sampling would lead to a biased estimation of the generalization error. Specifically, we show that the sampling-based CML would introduce a bias term in the generalization bound, which is quantified by the per-user Total Variance (TV) between the distribution induced by negative sampling and the ground truth distribution. This suggests that optimizing the sampling-based CML loss function does not ensure a small generalization error even with sufficiently large training data. Moreover, we show that the bias term will vanish without the negative sampling strategy. Motivated by this, we propose an efficient alternative without negative sampling for CML named Sampling-Free Collaborative Metric Learning (SFCML), to get rid of the sampling bias in a practical sense. Finally, comprehensive experiments over seven benchmark datasets speak to the supriority of the proposed algorithm.

Web crippling design of cold-formed stainless steel channels under interior-two-flange loading condition using deep belief network

This research presents a deep-learning framework, namely a deep belief network (DBN), for analyzing the interior-two-flange web crippling performance of cold-formed stainless steel channels with centered and offset web holes. An elastoplastic finite element (FE) model, validated using 101 experimental results which were previously reported in the literature, generates a total of 43,200 data points for training the DBN. When compared to a total of 54 experimental results published in the literature, the DBN predictions were shown to be approximately 10% more conservative. Using the same large training data, the developed DBN model outperformed the Backpropagation Neural Network (a typical shallow artificial neural network) and the PaddlePaddle-based linear regression model. A parametric analysis was then performed using the DBN predictions to explore the effect of section size, web holes and bearing length. Design equations for (reduced) web crippling strength are proposed for the cold-formed stainless steel perforated channels, and the feasibility of the proposed equations was assessed by the conducted reliability analysis.

Automatic diagnosis of cardiovascular diseases using wavelet feature extraction and convolutional capsule network

Cardiovascular diseases (CVDs) are the primary cause of high mortality levels worldwide with coronary artery disease (CAD) being the largest contributor. Delay in diagnosis of CAD may progress and lead to myocardial infarction and congestive heart failure. Thus, timely diagnosis is a crucial key to prevent the development of serious complications and allow an optimum medication. This paper intends to design a diagnosis system to classify-eight different diseases using a capsule network with wavelet decomposed images of short ECG segments. It is noteworthy that in contrast to the previous studies that require large training data to efficiently predict unseen data, our designed models achieve high performance with less training data and training time. In the proposed system, firstlyECG signals are preprocessed and segmented into beats segments. Then, the continuous wavelet transform is used to transform the segments into2D scalograms. These latter are decomposed using discrete wavelet transform to obtain wavelet coefficient imageswhich are fed to the reformed capsule networks. The focal loss was applied to mitigate the class-imbalance issue. The proposed model was trained and tested using 5-fold cross-validation and train-test split techniques. The best performance parameters were achieved in the train-test split approach with an accuracy of 98.6%, sensitivity of 99.6%, specificity of 98.7%, precision of 99.1%, F1 score of 99.4% and area under the roc value of 0.99.Furthermore, experimental results demonstrated the superiority of our approach over state-of-the-art techniques using the same dataset.

Accuracy and data efficiency in deep learning models of protein expression

Synthetic biology often involves engineering microbial strains to express high-value proteins. Thanks to progress in rapid DNA synthesis and sequencing, deep learning has emerged as a promising approach to build sequence-to-expression models for strain optimization. But such models need large and costly training data that create steep entry barriers for many laboratories. Here we study the relation between accuracy and data efficiency in an atlas of machine learning models trained on datasets of varied size and sequence diversity. We show that deep learning can achieve good prediction accuracy with much smaller datasets than previously thought. We demonstrate that controlled sequence diversity leads to substantial gains in data efficiency and employed Explainable AI to show that convolutional neural networks can finely discriminate between input DNA sequences. Our results provide guidelines for designing genotype-phenotype screens that balance cost and quality of training data, thus helping promote the wider adoption of deep learning in the biotechnology sector.

Direct estimation of porosity from seismic data using rock- and wave-physics-informed neural networks

Petrophysical inversion is an important aspect of reservoir modeling. However, due to the lack of a unique and straightforward relationship between seismic traces and rock properties, predicting petrophysical properties directly from seismic data is a complex task. Many studies have attempted to identify the direct end-to-end link using supervised machine learning techniques, but they face challenges such as lack of a large petrophysical training data set or estimates that may not conform with physics or depositional history of the rocks. We present a rock- and wave-physics-informed neural network (RW-PINN) model that can estimate porosity directly from seismic image traces with no wells or with a limited number of wells and with predictions that are consistent with rock physics and geologic knowledge of deposition. The RW-PINN takes advantage of auto-differentiation to compute the gradients across the rock- and wave-physics models. As an example, we use the uncemented-sand rock-physics model and normal-incidence wave physics to guide the learning of the RW-PINN to eventually get good estimates of porosities from normal-incidence seismic traces and limited well data. Training the RW-PINN with few wells (weakly supervised scenario) helps in tackling the problem of nonuniqueness as different porosity logs can give similar seismic traces. We use a weighted normalized root mean square error loss function to train the weakly supervised network and demonstrate the impact of different weights on porosity predictions. The RW-PINN's estimated porosities and seismic traces are compared to predictions from a completely supervised model, which gives slightly better porosity estimates but matches the seismic traces poorly and requires a large amount of labeled training data. We demonstrate the complete workflow for executing petrophysical inversion of seismic data using self-supervised or weakly supervised RW-PINNs.

A New Cycle-consistent Adversarial Networks With Attention Mechanism for Surface Defect Classification With Small Samples

Surface defect detection is the essential process to ensure the quality of products. Surface defect classification (SDC) based on deep learning (DL) has shown its great potential. However, the well-trained SDC model usually requires large training data, and the small intraclass differences between the defect and normal samples also degrades the performance of SDC model. To overcome these drawbacks, this article proposed a new cycle-consistent adversarial networks with attention mechanism (AttenCGAN). First, AttenCGAN is used for synthesizing defect samples to enlarge the samples volume. Second, the attention mechanism is adopted for the feature enhancement by finding the discriminative parts of the samples and enlarging the differences among the samples. AttenCGAN is tested on KolektorSDD and DAGM2007 datasets, and its accuracies are 98.53% and 99.57% with only a few samples. The experiment results show that AttenCGAN outperforms other published SDC methods based on DL and machine learning, which validates its potential.

Neural machine translation for Sanskrit to Malayalam using morphology and evolutionary word sense disambiguation

Neural machine translation (NMT) is a fast-evolving MT paradigm and showed good results, particularly in large training data circumstances, for several language pairs. In this paper, we have utilized Sanskrit to Malayalam language pair neural machines translation. The attention-based mechanism for the development of the machine translation system was particularly exploited. Word sense disambiguation (WSD) is a phenomenon for disambiguating the text to let the machine infer the proper definition of the particular word. Sequential deep learning approaches such as a recurrent neural network (RNN), a gated recurrent unit (GRU), a long short term memory (LSTM), and a bi-directional LSTM (BLSTM) were used to analyze the tagged data. By adding morphological elements and evolutionary word sense disambiguation, the suggested common character-word embedding-based NMT model gives a BLEU score of 38.58 which was higher than the others.

Micro-network Based Convolutional Neural Network with Integration of Multilayer Feature Fusion Strategy for Human Activity Recognition

Convolutional neural networks (CNN) have shown remarkable performance in enormous computer vision applications over the years, and many works have been done for human activity recognition (HAR) using CNN. However, most of deep learning-based architectures require large training data and plenty of computational resources. Therefore, we proposed a simple and efficient deep learning model for human activity recognition, which works on complex visual data and require less computational resources. In the proposed work, we designed a novel CNN architecture by stacking the repeated components called micro-networks to incorporate multiscale processing in the network layers. We have used a feature fusion strategy to pass the previous layer’s abstract complementary information to the next adjacent layer, believing that each layer encapsulates specific feature maps. Therefore, the hidden complementary information can potentially enhance the feature discrimination capacity of the network and help in learning the network. The proposed architecture is fully trained from scratch with stochastic gradient descent (SGD) optimizer at 0.05 initial learning rate and a softmax classifier is used for activity recognition. The merit of the proposed method over standard deep learning models is its computational efficiency in terms of learnable parameters and computational resources. The proposed model gives good performance on small as well as large size datasets. For authentication of the proposed method several extensive experiments are conducted on publically available datasets, namely UCF-101, HMDB-51, YouTube, and IXMAS datasets. The results have shown the outperformance of the proposed method over the existing state-of-the-art methods.

Classification of animal sounds in a hyperdiverse rainforest using convolutional neural networks with data augmentation

To protect tropical forest biodiversity, we need to be able to detect it reliably, cheaply, and at scale. Automated detection of sound producing animals from passively recorded soundscapes via machine-learning approaches is a promising technique towards this goal, but it is constrained by the necessity of large training data sets. Using soundscapes from a tropical forest in Borneo and a Convolutional Neural Network model (CNN), we investigate i) the minimum viable training data set size for accurate prediction of call types (‘sonotypes’), and ii) the extent to which data augmentation and transfer learning can overcome the issue of small and imbalanced training data sets. We found that even relatively high sample sizes (>80 per sonotype) lead to mediocre accuracy, which however improved significantly with data augmentation and transfer learning, including at extremely small sample sizes (3 per sonotype), regardless of taxonomic group or call characteristics. Neither transfer learning nor data augmentation alone achieved high accuracy. Our results suggest that transfer learning and data augmentation could make the use of CNNs to classify species’ vocalizations feasible even for small soundscape-based projects with many rare species. Retraining our open-source model requires only basic programming skills which makes it possible for individual conservation initiatives to match their local context, in order to enable more evidence-informed management of biodiversity.

Ultrasound Shear Wave Elasticity Imaging With Spatio-Temporal Deep Learning.

Ultrasound shear wave elasticity imaging is a valuable tool for quantifying the elastic properties of tissue. Typically, the shear wave velocity is derived and mapped to an elasticity value, which neglects information such as the shape of the propagating shear wave or push sequence characteristics. We present 3D spatio-temporal CNNs for fast local elasticity estimation from ultrasound data. This approach is based on retrieving elastic properties from shear wave propagation within small local regions. A large training data set is acquired with a robot from homogeneous gelatin phantoms ranging from 17.42 kPa to 126.05 kPa with various push locations. The results show that our approach can estimate elastic properties on a pixelwise basis with a mean absolute error of 5.01(437) kPa. Furthermore, we estimate local elasticity independent of the push location and can even perform accurate estimates inside the push region. For phantoms with embedded inclusions, we report a 53.93% lower MAE (7.50 kPa) and on the background of 85.24% (1.64 kPa) compared to a conventional shear wave method. Overall, our method offers fast local estimations of elastic properties with small spatio-temporal window sizes.

Chemical transformer compression for accelerating both training and inference of molecular modeling

Transformer models have been developed in molecular science with excellent performance in applications including quantitative structure-activity relationship (QSAR) and virtual screening (VS). Compared with other types of models, however, they are large and need voluminous data for training, which results in a high hardware requirement to abridge time for both training and inference processes. In this work, cross-layer parameter sharing (CLPS), and knowledge distillation (KD) are used to reduce the sizes of transformers in molecular science. Both methods not only have competitive QSAR predictive performance as compared to the original BERT model, but also are more parameter efficient. Furthermore, by integrating CLPS and KD into a two-state chemical network, we introduce a new deep lite chemical transformer model, DeLiCaTe. DeLiCaTe accomplishes 4× faster rate for training and inference, due to a 10- and 3-times reduction of the number of parameters and layers, respectively. Meanwhile, the integrated model achieves comparable performance in QSAR and VS, because of capturing general-domain (basic structure) and task-specific knowledge (specific property prediction). Moreover, we anticipate that the model compression strategy provides a pathway to the creation of effective generative transformer models for organic drugs and material design.

A Hybrid Method of Enhancing Accuracy of Facial Recognition System Using Gabor Filter and Stacked Sparse Autoencoders Deep Neural Network

Face recognition has grown in popularity due to the ease with which most recognition systems can find and recognize human faces in images and videos. However, the accuracy of the face recognition system is critical in ascertaining the success of a person’s identification. A lack of sufficiently large training datasets is one of the significant challenges that limit the accuracy of face recognition systems. Meanwhile, machine learning (ML) algorithms, particularly those used for image-based face recognition, require large training data samples to achieve a high degree of face recognition accuracy. Based on the above challenge, this research proposes a method for improving face recognition precision and accuracy by employing a hybrid approach of the Gabor filter and a stacked sparse autoencoders (SSAE) deep neural network. The face image datasets from Olivetti Research Laboratory (OLR) and the Extended Yale-B databases were used to evaluate the proposed hybrid model’s performance. All face image datasets used in our experiments are grayscale image type with a resolution of 92 × 112 for the OLR database and a resolution 192 × 168 for the Extended Yale-B database. Our experimental results showed that the proposed method improved face recognition accuracy by approximately 100% for the two databases used at a significantly reduced feature extraction time compared to the current state-of-art face recognition methods for all test cases. The SSAE approach can explore large and complex datasets with minimal computation time. In addition, the algorithm minimizes the false acceptance rate and improves recognition accuracy. This implies that the proposed method is promising and has the potential to enhance the performance of face recognition systems.

A rough set approach for classification of fabric defects

PurposeIn recent years, rough set theory has evolved as one of the most promising classification techniques. One of the cardinal uses of rough set theory is its application for rule generation. The purpose of this paper is to propose a real-time fabric inspection technique. This work deals with the multi-class classification of fabric defects using rough set theory.Design/methodology/approachThis technique focuses on the classification of fabric defects using the effective decision rules envisaged by rough set theory. In the proposed work, the six features of 50 images have been used for multiclass classification of fabric defects.FindingsIn this work, 40 images were used for generation of decision rules and 10 unseen images were used for validation out of which nine images are accurately predicted by the proposed technique.Originality/valueThe proposed method accurately identified 9 out of 10 testing defects. The obtained decision rules provide an insight about the classification method which ensures that the prediction accuracy can be improved further by framing more robust decision rules with the help of a large training data set. Thus, with the support of modern computational systems this method is potent in getting recognition from the textile industry as a real-time classification technique.

Efficient data acquisition and training of collisional-radiative model artificial neural network surrogates through adaptive parameter space sampling

Effective plasma transport modeling of magnetically confined fusion devices relies on having an accurate understanding of the ion composition and radiative power losses of the plasma. Generally, these quantities can be obtained from solutions of a collisional-radiative (CR) model at each time step within a plasma transport simulation. However, even compact, approximate CR models can be computationally onerous to evaluate, and in-situ evaluation of these models within a larger plasma transport code can lead to a rigid bottleneck. As a way to bypass this bottleneck, we propose deploying artificial neural network (ANN) surrogates to allow rapid evaluation of the necessary plasma quantities. However, one issue with training an accurate ANN surrogate is the reliance on a sufficiently large and representative training and validation data set, which can be time-consuming to generate. In this work we explore a data-driven active learning and training routine to allow autonomous adaptive sampling of the problem parameter space to ensure a sufficiently large and meaningful set of training data is assembled for the network training. As a result, we can demonstrate approximately order-of-magnitude savings in required training data samples to produce an accurate surrogate.