Scale Datasets Research Articles

A transfer learning method to assimilate numerical data with experimental data for effusion cooling

Effusion cooling was one of the most important external cooling technologies for airfoils in gas turbines and aeroengines. Due to the complicated flow field of effusion cooling, simulations were in lack of fidelity, while experimental measurements were expensive and slow. Therefore, the industry has been continuously seeking for mathematical tools that could integrate simulation data with experimental data for effusion cooling, to achieve high fidelity and fast prediction. Deep learning techniques, maturing in recent years, are potential tools to fulfill such demands. However, it was also known that generalization accuracy of machine learning models was still insufficient for small scale datasets. The present study proposed a transfer learning method based on an iterative neural operator to assimilate numerical data with experimental data for effusion cooling. Reynolds Averaged Navier Stokes simulations were conducted to collect source data. A pre-trained machine learning model was built up on the source dataset and transferred to three separately sets of target data. The target datasets included an experimental dataset obtained in this study and two experimental datasets in the literature, each with less than 20 data, different data size and different variables. Ten non-transferred models and three transferred models were evaluated for the accuracy, training speed and data dependence. Results showed that the iterative neural operator model could precisely capture the nonlinear characteristic of effusion, and predict the local effusion cooling effectiveness of random hole configurations with a high quality. Compared with a direct machine learning approach, transfer learning significantly reduced the requirements for the number of training samples (reduced by 2–3 times) and training epochs (reduced by 5–6 times) to reach the same accuracy. Meanwhile, when the data cost and training cost were identical, transfer learning reduced the errors by 63.7% for experimental data, 41.8% for literature dataset 1, and 46.2% for literature dataset 2 respectively. The efforts were expected to provide a robust solution to modelling experimental data of effusion cooling using limited sample number and limited time with the aid of relatively bigger numerical datasets.

Learning rate selection in stochastic gradient methods based on line search strategies

Finite-sum problems appear as the sample average approximation of a stochastic optimization problem and often arise in machine learning applications with large scale data sets. A very popular approach to face finite-sum problems is the stochastic gradient method. It is well known that a proper strategy to select the hyperparameters of this method (i.e. the set of a-priori selected parameters) and, in particular, the learning rate, is needed to guarantee convergence properties and good practical performance. In this paper, we analyse standard and line search based updating rules to fix the learning rate sequence, also in relation to the size of the mini batch chosen to compute the current stochastic gradient. An extensive numerical experimentation is carried out in order to evaluate the effectiveness of the discussed strategies for convex and non-convex finite-sum test problems, highlighting that the line search based methods avoid expensive initial setting of the hyperparameters. The line search based approaches have also been applied to train a Convolutional Neural Network, providing very promising results.

Neural Machine Translation Models with Attention-Based Dropout Layer

In bilingual translation, attention-based Neural Machine Translation (NMT) models are used to achieve synchrony between input and output sequences and the notion of alignment. NMT model has obtained state-of-the-art performance for several language pairs. However, there has been little work exploring useful architectures for Urdu-to-English machine translation. We conducted extensive Urdu-to-English translation experiments using Long short-term memory (LSTM)/Bidirectional recurrent neural networks (Bi-RNN)/Statistical recurrent unit (SRU)/Gated recurrent unit (GRU)/Convolutional neural network (CNN) and Transformer. Experimental results show that Bi-RNN and LSTM with attention mechanism trained iteratively, with a scalable data set, make precise predictions on unseen data. The trained models yielded competitive results by achieving 62.6% and 61% accuracy and 49.67 and 47.14 BLEU scores, respectively. From a qualitative perspective, the translation of the test sets was examined manually, and it was observed that trained models tend to produce repetitive output more frequently. The attention score produced by Bi-RNN and LSTM produced clear alignment, while GRU showed incorrect translation for words, poor alignment and lack of a clear structure. Therefore, we considered refining the attention-based models by defining an additional attention-based dropout layer. Attention dropout fixes alignment errors and minimizes translation errors at the word level. After empirical demonstration and comparison with their counterparts, we found improvement in the quality of the resulting translation system and a decrease in the perplexity and over-translation score. The ability of the proposed model was evaluated using Arabic-English and Persian-English datasets as well. We empirically concluded that adding an attention-based dropout layer helps improve GRU, SRU, and Transformer translation and is considerably more efficient in translation quality and speed.

Graph-of-Code: Semantic Clone Detection Using Graph Fingerprints

The code clone detection issue has been researched using a number of explicit factors based on the tokens and contents and found effective results. However, exposing code contents may be an impractical option because of privacy and security factors. Moreover, the lack of scalability of past methods is an important challenge. The code flow states can be inferred by code structure and implicitly represented using empirical graphs. The assumption is that modelling of the code clone detection problem can be achieved without the content of the codes being revealed. Here, a Graph-of-Code concept for the code clone detection problem is introduced, which represents codes into graphs. While Graph-of-Code provides structural properties and quantification of its characteristics, it can exclude code contents or tokens to identify the clone type. The aim is to evaluate the impact of graph-of-code structural properties on the performance of code clone detection. This work employs a feature extraction-based approach for unlabelled graphs. The approach generates a ‘Graph Fingerprint’ which represents different topological feature levels. The results of code clone detection indicate that code structure has a significant role in detecting clone types. We found different GoC-models outperform others. The models achieve between 96% to 99% in detecting code clones based on recall, precision, and F1-Score. The GoC approach is capable in detecting code clones with scalable dataset and with preserving codes privacy.

A Balanced Triplet Loss for Person Re-Identification

Compared to cross-entropy in deep learning, triplet loss is less affected by the biased label information and widely used in fine-grained visual tasks. Especially in person re-identification (re-ID), triplet loss is improved with batch-hard sampling which only selects the hardest samples during the training process to reduce invalid triplets involved in the loss computation. The hardest samples’ loss computation can provide a more intense gradient descent than raw samples. However, the batch-hard triplet loss discards multiple samples with important information, which can negatively impact feature learning. Besides, the hardest samples cause loss stuck problems frequently in training. In this work, we propose a balanced triplet loss for comprehensive feature learning and stable model convergence. The balanced triplet loss only mines the hardest negative samples of each category within a mini-batch. Compared with batch-hard triplet loss, it preserves the features of all the negative categories rather than one negative category with the hardest negative sample. It achieves a balance between triplet selection and information loss. The experiments show that our method can produce competitive results in re-ID tasks. In addition, we analyze the correlation between the intensity of data mining and the granularity of feature learning and further adapt the balanced triplet loss to general fine-grained image classification. The experiments prove the adapted balanced triplet loss also outperforms cross-entropy in multiple datasets of different scales.

POSTPARTUM DEPRESSION PREDICTION USING MACHINE LEARNING

After giving birth, postpartum depression is a serious mental health problem that affects moms. Symptoms may include trouble bonding with the child, difficulty sleeping, loss of appetite, and extreme irritability. Among 50 % of the mothers experience major changes in their mental health and approximately 1 in 10 women will seek help.The typical duration of postpartum depression is 3 to 6 months, however this might vary depending on a number of factors. The Edinburgh Postnatal Depression Scale (EPDS) data set, which is gathered after a week from the mothers who gave birth to their children, is used in this study to train the model using Random forest algorithm.This model has two phases, where the first phase predicts the status of the mother in any one of the four ranges (Depressed, Most Likely Depressed, Likely Depressed, No Depression) based on the trained data. Where the second phase predicts the level of depression(0-4 none, 5-9 mild, 10-14 Moderate, 15-19 moderately severe, 20- 27 severe.) based on the PHQ-9 Questionnaire, and it is suggested to take post-6 weeks of delivery. The results of phase one and two will be sent to the user via mail when they take the assessments in week one and week 6 respectively. The key benefit of this model is that there is no need for a third party because user and model evaluations take place directly between them. Since postpartum depression is not properly understood, many mothers do not receive treatment for their sickness, which can result in terrible circumstances and can be avoided by following this approach.

Large Scale Fish Images Classification and Localization using Transfer Learning and Localization Aware CNN Architecture

Building an automatic fish recognition and detection system for large-scale fish classes is helpful for marine researchers and marine scientists because there are large numbers of fish species. However, it is quite difficult to build such systems owing to the lack of data imbalance problems and large number of classes. To solve these issues, we propose a transfer learning-based technique in which we use Efficient-Net, which is pre-trained on ImageNet dataset and fine-tuned on QuT Fish Database, which is a large scale dataset. Furthermore, prior to the activation layer, we use Global Average Pooling (GAP) instead of dense layer with the aim of averaging the results of predictions along with having more information compared to the dense layer. To check the validity of our model, we validate our model on the validation set which achieves satisfactory results. Also, for the localization task, we propose an architecture that consists of localization aware block, which captures localization information for better prediction and residual connections to handle the over-fitting problem. Actually, the residual connections help the layer to combine missing information with the relevant one. In addition, we use class weights and Focal Loss (FL) to handle class imbalance problems along with reducing false predictions. Actually, class weights assign less weights to classes having fewer instances and large weights to classes having more number of instances. During the localization, the qualitative assessment shows that we achieve 57% Mean Intersection Over Union (IoU) on testing data, and the classification results show 75% precision, 70% recall, 78% accuracy and 74% F1-Score for 468 fish species.

Sarcasm detection using news headlines dataset

Sarcasm has been an elusive concept for humans. Due to interesting linguistic properties, sarcasm detection has gained traction of the Natural Language Processing (NLP) research community in the past few years. However, the task of predicting sarcasm in a text remains a difficult one for machines as well, and there are limited insights into what makes a sentence sarcastic. Past studies in sarcasm detection either use large scale datasets collected using tag-based supervision or small scale manually annotated datasets. The former category of datasets are noisy in terms of labels and language, whereas the latter category of datasets do not have enough instances to train deep learning models reliably despite having high-quality labels. To overcome these shortcomings, we introduce a high-quality and relatively larger-scale dataset which is a collection of news headlines from a sarcastic news website and a real news website. We describe the unique aspects of our dataset and compare its various characteristics with other benchmark datasets in sarcasm detection domain. Furthermore, we produce insights into what constitute as sarcasm in a text using a Hybrid Neural Network architecture. First released in 2019, we dedicate a section on how the NLP research community has extensively relied upon our contributions to push the state of the art further in the sarcasm detection domain. Lastly, we make the dataset as well as framework implementation publicly available to facilitate continued research in this domain.

Advanced Meta-Heuristic Algorithm Based on Particle Swarm and Al-Biruni Earth Radius Optimization Methods for Oral Cancer Detection

Oral cancer is a deadly form of cancerous tumor that is widely spread in low and middle-income countries. An early and affordable oral cancer diagnosis might be achieved by automating the detection of precancerous and malignant lesions in the mouth. There are many research attempts to develop a robust machine-learning model that can detect oral cancer from images. However, these are still lacking high precision in oral cancer detection. Therefore, this work aims to propose a new approach capable of detecting oral cancer in medical images with higher accuracy. In this work, a novel and robust oral cancer detection based on a convolutional neural network (CNN) and optimized deep belief network (DBN). The design parameters of CNN and DBN are optimized using a new optimization algorithm, which is developed as a hybrid of Particle Swarm Optimization (PSO) and Al-Biruni Earth Radius (BER) Optimization algorithms and is denoted by (PSOBER). Using a standard biomedical images dataset available on the Kaggle repository, the proposed approach shows promising results outperforming various competing approaches with an accuracy of 97.35%. In addition, a set of statistical tests, such as One-way analysis-of-variance (ANOVA) and Wilcoxon signed-rank tests, are conducted to prove the significance and stability of the proposed approach. The proposed methodology is solid and efficient, and specialists can adopt it. However, additional research on a larger scale dataset is required to confirm the findings and highlight other oral features that can be utilized for cancer detection.

A Pilot Study Leveraging Large Scale Datasets from Internet-Connected Transcranial Magnetic Stimulators: Circadian Modulation of Cortical Excitability

A Pilot Study Leveraging Large Scale Datasets from Internet-Connected Transcranial Magnetic Stimulators: Circadian Modulation of Cortical Excitability

FS3change: A Scalable Method for Change Pattern Mining

Mining change patterns can give unique understanding on the evolution of dynamically changing systems like social relation graphs, weblinks, hardware descriptions and models. A more recent focus is source code change pattern mining that may qualitatively justify expected or uncover unexpected patterns. These patterns then offer a basis, e.g., for program language evolution or auto-completion support. We present a change pattern mining method that greatly expands the limits of input data and pattern complexity, over existing methods. We propose scalability solutions on conceptual and algorithmic level, thereby evolving the state-of-the-art sampling-based frequent subgraph mining method FS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> , resulting in 75% reduction in memory consumption and a speedup of 6500 for a large scale dataset. Patterns can have 100,000s of occurrences for which manual review is impossible and may lead to misinterpretation. We propose the novel content track approach for interactively exploring pattern contents in context, based on marginal distributions. We evaluate our approach by mining 1,000 open source projects contributing a total of 558 million changes and 2 billion contextual connections among them, thereby, demonstrating its scalability. A manual interpretation of 19 patterns shows sensible mined patterns allowing to deduct implications for language design and demonstrating the soundness of the approach.

2EPV‐ApproCom: Enhanced Effective, Private, and Verifiable Approximate Similarity Computation with Server Aided over Scalable Datasets for IoT

In big data analytics, Jaccard similarity is a widely used block for scalable similarity computation. It is broadly applied in the Internet of Things (IoT) applications, such as credit system, social networking, epidemic tracking, and so on. However, with the increasing privacy concerns of user’s sensitive data for IoT, it is intensively desirable and necessary to investigate privacy‐preserving Jaccard similarity computing over two users’ datasets. To boost the efficiency and enhance the security, we propose two methods to measure Jaccard similarity over private sets of two users under the assistance of an untrusted cloud server in this paper. Concretely, by leveraging an effective Min‐Hash algorithm on encrypted datasets, our protocols output an approximate similarity, which is very close to the exact value, without leaking any additional privacy to the cloud. Our first solution is under a semihonest cloud server, and our enhanced solution introduced the consistency-check mechanism to achieve verifiability in malicious model. For efficiency, the first solution only need about 6 minutes for billion‐element sets. Furthermore, as far as we know, the consistency-check mechanism is proposed for the first time to achieve an effective verifiable approximate similarity computation.

Price Forecasting on a Large Scale Data Set using Time Series and Neural Network Models

Price Forecasting on a Large Scale Data Set using Time Series and Neural Network Models

The Mesozoic West and Central Africa Rift System (WCARS) and the older Kandi Shear Zone (KSZ): Rifting and tectonics of North Africa and South America and fragmentation of Gondwana based on geophysical investigations

Intra- and trans-continental tectonic lineaments and tectonic structures cutting Africa and Gondwana have long been identified by many authors. These structures were initially identified from topography, coast lines and geological data but more recently from remote sensing and geophysical data. Here gravity and aeromagnetic data for Africa and South America have been used to identify, track and model these structures at depth within the crust/upper mantle. The two structures investigated are the West and Central African Rift System (WCARS) and the Kandi Shear Zone (KSZ). WCARSthe nature and spatial extent of the WCARS have been identified using gravity data. In its simplest form, the WCARS consists of a set of strike-slip shear zones linked to extensional rift basins that can be traced as continuous structures from the Atlantic coast of Nigeria to southern Algeria and from the coast of Cameroon to Kenya. The rift system developed as a set of discrete tectonic pulses throughout the Mesozoic at the same time as the Central, South and Equatorial Atlantic Oceans developed into a single unified spreading centre. Each tectonic pulse had a different basin response due to the changing orientation of the plate tectonic controlled displacement field. The African rift basins formed by crustal extension involving brittle failure and ductile flow of the upper and lower crust respectively, resulting in the isostatic subsidence of the surface and rapid infilling of the basins with mainly continental derived sediments. Gravity and seismological studies support this crustal basin model originally proposed by McKenzie. This contribution shows that the evolution of the WCARS is clearly linked to the oceanic plate tectonic processes. New analysis of the fracture zone geometry in the Central Atlantic shows that these rift pulses involving crustal extension, strike-slip faulting, stratigraphic unconformities and short episodes of crustal contraction e.g. Santonian and Maastrichtian events, which can be identified and correlated with changes in the azimuth of the fracture zone geometry. KSZThe second, shorter part of the paper uses predominantly aeromagnetic data to track the KSZ from the Atlantic coast of Benin north into central Algeria before splaying out beneath the Saharan Platform. To the south the KSZ is shown, prior to the formation of the Equatorial Atlantic, to link with the Trans-Brazilian Lineament but its continuation south of Brazil still remain highly speculative.Finally, this review raises new insights and questions that are ideal for future research topics, both in developing the analytic methodologies and their application to other continental scale datasets.

Research on Aircraft image recognition based on transfer learning and improved YOLOv5 model

Effective differentiation of aircraft types using images is important for providing military combat information as well as civilian aircraft operations. Aircraft image recognition has many difficulties such as large variations of target scale, complex backgrounds, and difficult data set acquisition, which lead to the low recognition accuracy of existing models. To address the problem of low recognition accuracy caused by the above difficulties. This paper proposes the improved YOLOv5 model for the recognition of aircraft images. First, this paper designs the CSPResNet50dCA network as the backbone of the YOLOv5 model to enhance the feature extraction capability for small target aircraft in images. By introducing the coordinate attention mechanism and the CSP structure, the feature-focusing capability and the computing speed of the model are enhanced. Afterward, we use data enhancement to expand the data set and transfer learning to improve the generalization ability and convergence speed of the model, so as to improve its robustness. The experimental results show that the improved YOLOv5 model has significantly improved the recognition accuracy of aircraft targets， and significantly enhanced feature extraction ability for small target aircraft with good generalization ability.

Adaptive Graph Auto-Encoder for General Data Clustering.

Graph-based clustering plays an important role in the clustering area. Recent studies about graph neural networks (GNN) have achieved impressive success on graph-type data. However, in general clustering tasks, the graph structure of data does not exist such that GNN can not be applied to clustering directly and the strategy to construct a graph is crucial for performance. Therefore, how to extend GNN into general clustering tasks is an attractive problem. In this paper, we propose a graph auto-encoder for general data clustering, AdaGAE, which constructs the graph adaptively according to the generative perspective of graphs. The adaptive process is designed to induce the model to exploit the high-level information behind data and utilize the non-euclidean structure sufficiently. Importantly, we find that the simple update of the graph will result in severe degeneration, which can be concluded as better reconstruction means worse update. We provide rigorous analysis theoretically and empirically. Then we further design a novel mechanism to avoid the collapse. Via extending the generative graph models to general type data, a graph auto-encoder with a novel decoder is devised and the weighted graphs can be also applied to GNN. AdaGAE performs well and stably in different scale and type datasets. Besides, it is insensitive to the initialization of parameters and requires no pretraining.

Comparative analysis of deep convolution neural network models on small scale datasets

Deep Learning is a representational model that proves significant impact in computer vision applications by automating the feature extraction process with the advancement and availability of high computing power. Conventional machine learning algorithm works efficiently on large scale data and trained for specific applications that cannot be used for other task. Deep neural networks require large amounts of data, resources, time and high computing devices. Complex deep learning models for a new application domain could be developed using transfer learning with less resources and few labeled training data. An important challenge is negative transfer, which means the initial and target problems are similar. Also, there are no research findings on choosing the right pre-trained model which are closely related to the target dataset and there is no proper measure to analyze the performance. Convolutional Neural Networks models impart significant advancement to the ImageNet visual recognition challenge in the last decade. This work focuses on the use of pre-trained models for small datasets. It analyzes the initialization of parameters, tuning of hyperparameters, parameter optimization, regularization methods, and computation complexity of pre-trained convolutional neural network models. Experiments are performed on three different data sets COREL1k, COIL20 and Garbage images. It shows that the time to re-train the existing pre-trained models was greatly reduced by transfer learning. Transferring weights, instances and layers from models trained in ImageNet dataset could produce better performance in similar small scale datasets, but the performance of dataset which differs in ImageNet dataset was not satisfactory.

WORD: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from CT image.

Whole abdominal organ segmentation is important in diagnosing abdomen lesions, radiotherapy, and follow-up. However, oncologists' delineating all abdominal organs from 3D volumes is time-consuming and very expensive. Deep learning-based medical image segmentation has shown the potential to reduce manual delineation efforts, but it still requires a large-scale fine annotated dataset for training, and there is a lack of large-scale datasets covering the whole abdomen region with accurate and detailed annotations for the whole abdominal organ segmentation. In this work, we establish a new large-scale Whole abdominal ORgan Dataset (WORD) for algorithm research and clinical application development. This dataset contains 150 abdominal CT volumes (30495 slices). Each volume has 16 organs with fine pixel-level annotations and scribble-based sparse annotations, which may be the largest dataset with whole abdominal organ annotation. Several state-of-the-art segmentation methods are evaluated on this dataset. And we also invited three experienced oncologists to revise the model predictions to measure the gap between the deep learning method and oncologists. Afterwards, we investigate the inference-efficient learning on the WORD, as the high-resolution image requires large GPU memory and a long inference time in the test stage. We further evaluate the scribble-based annotation-efficient learning on this dataset, as the pixel-wise manual annotation is time-consuming and expensive. The work provided a new benchmark for the abdominal multi-organ segmentation task, and these experiments can serve as the baseline for future research and clinical application development.

Dense Relational Image Captioning via Multi-Task Triple-Stream Networks.

We introduce dense relational captioning, a novel image captioning task which aims to generate multiple captions with respect to relational information between objects in a visual scene. Relational captioning provides explicit descriptions for each relationship between object combinations. This framework is advantageous in both diversity and amount of information, leading to a comprehensive image understanding based on relationships, e.g., relational proposal generation. For relational understanding between objects, the part-of-speech (POS; i.e., subject-object-predicate categories) can be a valuable prior information to guide the causal sequence of words in a caption. We enforce our framework to learn not only to generate captions but also to understand the POS of each word. To this end, we propose the multi-task triple-stream network (MTTSNet) which consists of three recurrent units responsible for each POS which is trained by jointly predicting the correct captions and POS for each word. In addition, we found that the performance of MTTSNet can be improved by modulating the object embeddings with an explicit relational module. We demonstrate that our proposed model can generate more diverse and richer captions, via extensive experimental analysis on large scale datasets and several metrics. Then, we present applications of our framework to holistic image captioning, scene graph generation, and retrieval tasks.

TM3Loc: Tightly-Coupled Monocular Map Matching for High Precision Vehicle Localization

Vision-based map-matching with HD map for high precision vehicle localization has gained great attention for its low-cost and ease of deployment. However, its localization performance is still unsatisfactory in accuracy and robustness in numerous real applications due to the sparsity and noise of the perceived HD map landmarks. This article proposes the tightly-coupled monocular map-matching localization algorithm (TM3Loc) for monocular-based vehicle localization. TM3Loc introduces semantic chamfer matching (SCM) to model monocular map-matching problem and combines visual features with SCM in a tightly-coupled manner. By applying the sliding window-based optimization technique, the historical visual features and HD map constraints are also introduced, such that the vehicle poses are estimated with an abundance of visual features and multi-frame HD map landmark features, rather than with single-frame HD map observations in previous works. Experiments are conducted on large scale dataset of 15 km long in total. The results show that TM3Loc is able to achieve high precision localization performance using a low-cost monocular camera, largely exceeding the performance of the previous state-of-the-art methods, thereby promoting the development of autonomous driving.