Scale Datasets Research Articles

A novel neural grey system model with Bayesian regularization and its applications

Most machine learning models are essentially “Black-box” models, of which the performance heavily relies on large scale data sets. In this work the idea of “Grey-box” modelling is adopted in order to take most advantage of known information represented by deterministic structure, and then the neural grey system model is developed. Levenberg-Marquardt algorithm is used to train the proposed model, and the Bayesian regularization is used to tune the regularized parameter automatically. Six real world case studies are presented to show the performance of the proposed model, comparing to 6 existing machine learning models and 17 grey system models. The results show that the proposed model can significantly overperform the other models and has very good generality, illustrating its high potential in a wide variety of real world applications and the efficiency of the proposed modelling method.

Merging Statistical Feature via Adaptive Gate for Improved Text Classification

Currently, text classification studies mainly focus on training classifiers by using textual input only, or enhancing semantic features by introducing external knowledge (e.g., hand-craft lexicons and domain knowledge). In contrast, some intrinsic statistical features of the corpus, like word frequency and distribution over labels, are not well exploited. Compared with external knowledge, the statistical features are deterministic and naturally compatible with corresponding tasks. In this paper, we propose an Adaptive Gate Network (AGN) to consolidate semantic representation with statistical features selectively. In particular, AGN encodes statistical features through a variational component and merges information via a well-designed valve mechanism. The valve adapts the information flow into the classifier according to the confidence of semantic features in decision making, which can facilitate training a robust classifier and can address the overfitting caused by using statistical features. Extensive experiments on datasets of various scales show that, by incorporating statistical information, AGN can improve the classification performance of CNN, RNN, Transformer, and Bert based models effectively. The experiments also indicate the robustness of AGN against adversarial attacks of manipulating statistical information.

Hierarchical Information Passing Based Noise-Tolerant Hybrid Learning for Semi-Supervised Human Parsing

Deep learning based human parsing methods usually require a large amount of training data to reach high performance. However, it is costly and time-consuming to obtain manually annotated high quality labels for a large scale dataset. To alleviate annotation efforts, we propose a new semi-supervised human parsing method for which we only need a small number of labels for training. First, we generate high quality pseudo labels on unlabeled images using a hierarchical information passing network (HIPN), which reasons human part segmentation in a coarse to fine manner. Furthermore, we develop a noise-tolerant hybrid learning method, which takes advantage of positive and negative learning to better handle noisy pseudo labels. When evaluated on standard human parsing benchmarks, our HIPN achieves a new state-of-the-art performance. Moreover, our noise-tolerant hybrid learning method further improves the performance and outperforms the state-of-the-art semi-supervised method (i.e. GRN) by 4.47 points w.r.t mIoU on the LIP dataset.

Improving Adversarial Robustness via Probabilistically Compact Loss with Logit Constraints

Convolutional neural networks (CNNs) have achieved state-of-the-art performance on various tasks in computer vision. However, recent studies demonstrate that these models are vulnerable to carefully crafted adversarial samples and suffer from a significant performance drop when predicting them. Many methods have been proposed to improve adversarial robustness (e.g., adversarial training and new loss functions to learn adversarially robust feature representations). Here we offer a unique insight into the predictive behavior of CNNs that they tend to misclassify adversarial samples into the most probable false classes. This inspires us to propose a new Probabilistically Compact (PC) loss with logit constraints which can be used as a drop-in replacement for cross-entropy (CE) loss to improve CNN's adversarial robustness. Specifically, PC loss enlarges the probability gaps between true class and false classes meanwhile the logit constraints prevent the gaps from being melted by a small perturbation. We extensively compare our method with the state-of-the-art using large scale datasets under both white-box and black-box attacks to demonstrate its effectiveness. The source codes are available at https://github.com/xinli0928/PC-LC.

Author Homepage Discovery in CiteSeerX

Scholarly digital libraries provide access to scientific publications and comprise useful resources for researchers. CiteSeerX is one such digital library search engine that provides access to more than 10 million academic documents. We propose a novel search-driven approach to build and maintain a large collection of homepages that can be used as seed URLs in any digital library including CiteSeerX to crawl scientific documents. Precisely, we integrate Web search and classification in a unified approach to discover new homepages: first, we use publicly-available author names and research paper titles as queries to a Web search engine to find relevant content, and then we identify the correct homepages from the search results using a powerful deep learning classifier based on Convolutional Neural Networks. Moreover, we use Self-Training in order to reduce the labeling effort and to utilize the unlabeled data to train the efficient researcher homepage classifier. Our experiments on a large scale dataset highlight the effectiveness of our approach, and position Web search as an effective method for acquiring authors' homepages. We show the development and deployment of the proposed approach in CiteSeerX and the maintenance requirements.

NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Search

Neural Architecture Search (NAS) is an open and challenging problem in machine learning. While NAS offers great promise, the prohibitive computational demand of most of the existing NAS methods makes it difficult to directly search the architectures on large-scale tasks. The typical way of conducting large scale NAS is to search for an architectural building block on a small dataset (either using a proxy set from the large dataset or a completely different small scale dataset) and then transfer the block to a larger dataset. Despite a number of recent results that show the promise of transfer from proxy datasets, a comprehensive evaluation of different NAS methods studying the impact of different source datasets has not yet been addressed. In this work, we propose to analyze the architecture transferability of different NAS methods by performing a series of experiments on large scale benchmarks such as ImageNet1K and ImageNet22K. We find that: (i) The size and domain of the proxy set does not seem to influence architecture performance on the target dataset. On average, transfer performance of architectures searched using completely different small datasets (e.g., CIFAR10) perform similarly to the architectures searched directly on proxy target datasets. However, design of proxy sets has considerable impact on rankings of different NAS methods. (ii) While different NAS methods show similar performance on a source dataset (e.g., CIFAR10), they significantly differ on the transfer performance to a large dataset (e.g., ImageNet1K). (iii) Even on large datasets, random sampling baseline is very competitive, but the choice of the appropriate combination of proxy set and search strategy can provide significant improvement over it. We believe that our extensive empirical analysis will prove useful for future design of NAS algorithms.

Unsupervised Learning of Discourse Structures using a Tree Autoencoder

Discourse information, as postulated by popular discourse theories, such as RST and PDTB, has been shown to improve an increasing number of downstream NLP tasks, showing positive effects and synergies of discourse with important real-world applications. While methods for incorporating discourse become more and more sophisticated, the growing need for robust and general discourse structures has not been sufficiently met by current discourse parsers, usually trained on small scale datasets in a strictly limited number of domains. This makes the prediction for arbitrary tasks noisy and unreliable. The overall resulting lack of high-quality, high-quantity discourse trees poses a severe limitation to further progress. In order the alleviate this shortcoming, we propose a new strategy to generate tree structures in a task-agnostic, unsupervised fashion by extending a latent tree induction framework with an auto-encoding objective. The proposed approach can be applied to any tree-structured objective, such as syntactic parsing, discourse parsing and others. However, due to the especially difficult annotation process to generate discourse trees, we initially develop a method to generate larger and more diverse discourse treebanks. In this paper we are inferring general tree structures of natural text in multiple domains, showing promising results on a diverse set of tasks.

NPU RGBD Dataset and a Feature-Enhanced LSTM-DGCN Method for Action Recognition of Basketball Players+

Computer vision-based action recognition of basketball players in basketball training and competition has gradually become a research hotspot. However, owing to the complex technical action, diverse background, and limb occlusion, it remains a challenging task without effective solutions or public dataset benchmarks. In this study, we defined 32 kinds of atomic actions covering most of the complex actions for basketball players and built the dataset NPU RGB+D (a large scale dataset of basketball action recognition with RGB image data and Depth data captured in Northwestern Polytechnical University) for 12 kinds of actions of 10 professional basketball players with 2169 RGB+D videos and 75 thousand frames, including RGB frame sequences, depth maps, and skeleton coordinates. Through extracting the spatial features of the distances and angles between the joint points of basketball players, we created a new feature-enhanced skeleton-based method called LSTM-DGCN for basketball player action recognition based on the deep graph convolutional network (DGCN) and long short-term memory (LSTM) methods. Many advanced action recognition methods were evaluated on our dataset and compared with our proposed method. The experimental results show that the NPU RGB+D dataset is very competitive with the current action recognition algorithms and that our LSTM-DGCN outperforms the state-of-the-art action recognition methods in various evaluation criteria on our dataset. Our action classifications and this NPU RGB+D dataset are valuable for basketball player action recognition techniques. The feature-enhanced LSTM-DGCN has a more accurate action recognition effect, which improves the motion expression ability of the skeleton data.

Threats of illegal, unregulated, and unreported fishing to biodiversity and food security in the Republic of the Congo.

Illegal, unregulated, and unreported (IUU) fishing poses a major threat to effective management of marine resources, affecting biodiversity and communities dependent on these coastal resources. Spatiotemporal patterns of industrial fisheries in developing countries are often poorly understood, and global efforts to describe spatial patterns of fishing vessel activity are currently based on automatic identification system (AIS) data. However, AIS is often not a legal requirement on fishing vessels, likely resulting in underestimates of the scale and distribution of legal and illegal fishing activity, which could have significant ramifications for targeted enforcement efforts and the management of fisheries resources. To help address this knowledge gap, we analyzed 3 years of vessel monitoring system (VMS) data in partnership with the national fisheries department in the Republic of the Congo to describe the behavior of national and distant-water industrial fleets operating in these waters. We found that the spatial footprint of the industrial fisheries fleet encompassed over one-quarter of the Exclusive Economic Zone. On average, 73% of fishing activity took place on the continental shelf (waters shallower than 200 m). Our findings highlight that VMS is not acting as a deterrent or being effectively used as a proactive management tool. As much as 33% (13% on average) of fishing effort occurred in prohibited areas set aside to protect biodiversity, including artisanal fisheries resources, and the distant-water fleet responsible for as much as 84% of this illegal activity. Given the growth in industrial and distant-water fleets across the region, as well as low levels of management and enforcement, these findings highlight that there is an urgent need for the global community to help strengthen regional and national capacity to analyze national scale data sets if efforts to combat IUU fishing are to be effective.

Collecting a Large Scale Dataset for Classifying Fake News Tweets Using Weak Supervision

The problem of automatic detection of fake news in social media, e.g., on Twitter, has recently drawn some attention. Although, from a technical perspective, it can be regarded as a straight-forward, binary classification problem, the major challenge is the collection of large enough training corpora, since manual annotation of tweets as fake or non-fake news is an expensive and tedious endeavor, and recent approaches utilizing distributional semantics require large training corpora. In this paper, we introduce an alternative approach for creating a large-scale dataset for tweet classification with minimal user intervention. The approach relies on weak supervision and automatically collects a large-scale, but very noisy, training dataset comprising hundreds of thousands of tweets. As a weak supervision signal, we label tweets by their source, i.e., trustworthy or untrustworthy source, and train a classifier on this dataset. We then use that classifier for a different classification target, i.e., the classification of fake and non-fake tweets. Although the labels are not accurate according to the new classification target (not all tweets by an untrustworthy source need to be fake news, and vice versa), we show that despite this unclean, inaccurate dataset, the results are comparable to those achieved using a manually labeled set of tweets. Moreover, we show that the combination of the large-scale noisy dataset with a human labeled one yields more advantageous results than either of the two alone.

Genetic and Transcriptomic Biomarkers in Neurodegenerative Diseases: Current Situation and the Road Ahead.

Neurodegenerative diseases are etiologically and clinically heterogeneous conditions, often reflecting a spectrum of disease rather than well-defined disorders. The underlying molecular complexity of these diseases has made the discovery and validation of useful biomarkers challenging. The search of characteristic genetic and transcriptomic indicators for preclinical disease diagnosis, prognosis, or subtyping is an area of ongoing effort and interest. The next generation of biomarker studies holds promise by implementing meaningful longitudinal and multi-modal approaches in large scale biobank and healthcare system scale datasets. This work will only be possible in an open science framework. This review summarizes the current state of genetic and transcriptomic biomarkers in Parkinson’s disease, Alzheimer’s disease, and amyotrophic lateral sclerosis, providing a comprehensive landscape of recent literature and future directions.

Deep Learning-Based Building Footprint Extraction With Missing Annotations

Most state-of-the-art deep learning-based methods for extraction of building footprints are aimed at designing proper convolutional neural network (CNN) architectures or loss functions able to effectively predict building masks from remote sensing (RS) images. To properly train such CNN models, large-scale and pixel-level building annotations are required. One common approach to obtain scalable benchmark data sets for the segmentation of buildings is to register RS images with auxiliary geospatial information data, such as those available from OpenStreetMaps (OSM). However, due to land-cover changes, urban construction, and delayed geospatial information updating, some building annotations may be missing in the corresponding ground-truth building mask layers. This will likely introduce confusion in the training of CNN models for discriminating between background and building pixels. To solve this important issue, we first formulate the problem as a long-tailed classification one. Then, we introduce a new joint loss function based on three terms: 1) logit adjusted cross entropy (LACE) loss, aimed at discriminating between building and background pixels from a long-tailed label distribution; 2) weighted dice loss, aimed at increasing the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F_{1}$ </tex-math></inline-formula> scores of the predicted building masks; and 3) boundary (BD) alignment loss, which is optimized for preserving the fine-grained structure of building boundaries. Our experiments, conducted on two benchmark building segmentation data sets, validate the effectiveness of our newly proposed loss with respect to other state-of-the-art losses commonly used for extracting building footprints. The codes of this letter will be publicly available from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/jiankang1991/GRSL_BFE_MA</uri> .

Abstract PO-071: Characterizing transcriptomic indicators of radiosensitivity in cancer and identifying sensitizing therapeutic agents

Abstract While radiation therapy is an integral method for cancer treatment, clinical choices are not currently informed by the genetic and molecular profile of a patient’s tumor. Though it has been shown that genetic features implicate variability, the exact relationship between these features and radiosensitivity is poorly understood. Our work focuses on predicting radiosensitivity of squamous cell carcinoma from molecular features. First, we employ RadioGx, a computational platform for integrative analysis of radiogenomic datasets, to identify a gene expression signature predictive of radiosensitivity. To mitigate the high-dimensionality of gene expression data, we employ pathway-level transcriptomic modeling methods to identify cellular programs associated with radiation sensitivity. Our pancancer radiosensitivity model achieves a Pearson’s correlation of 0.47 with measured radiosensitivity from a radiogenomic dataset using only high level transcriptomic features. Second, we use that gene expression signature to query the Connectivity Map via the PharmacoGx resource and identify candidate radiosensitizers - compounds that may induce the radiosensitivity signature. Further validation is necessary to evaluate the efficacy of these compounds at improving radiosensitivity in both model systems and patient contexts. These results implicate cellular processes in radiosensitivity and advance an approach for identifying radiosensitizing agents through integration of large scale datasets. Citation Format: Ian Smith, Rachel Bell, Meghan Lambie, Benjamin Haibe-Kains, Scott Bratman. Characterizing transcriptomic indicators of radiosensitivity in cancer and identifying sensitizing therapeutic agents [abstract]. In: Proceedings of the AACR Virtual Special Conference on Radiation Science and Medicine; 2021 Mar 2-3. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(8_Suppl):Abstract nr PO-071.

Earthquake magnitude and location estimation from real time seismic waveforms with a transformer network

SUMMARY Precise real time estimates of earthquake magnitude and location are essential for early warning and rapid response. While recently multiple deep learning approaches for fast assessment of earthquakes have been proposed, they usually rely on either seismic records from a single station or from a fixed set of seismic stations. Here we introduce a new model for real-time magnitude and location estimation using the attention based transformer networks. Our approach incorporates waveforms from a dynamically varying set of stations and outperforms deep learning baselines in both magnitude and location estimation performance. Furthermore, it outperforms a classical magnitude estimation algorithm considerably and shows promising performance in comparison to a classical localization algorithm. Our model is applicable to real-time prediction and provides realistic uncertainty estimates based on probabilistic inference. In this work, we furthermore conduct a comprehensive study of the requirements on training data, the training procedures and the typical failure modes. Using three diverse and large scale data sets, we conduct targeted experiments and a qualitative error analysis. Our analysis gives several key insights. First, we can precisely pinpoint the effect of large training data; for example, a four times larger training set reduces average errors for both magnitude and location prediction by more than half, and reduces the required time for real time assessment by a factor of four. Secondly, the basic model systematically underestimates large magnitude events. This issue can be mitigated, and in some cases completely resolved, by incorporating events from other regions into the training through transfer learning. Thirdly, location estimation is highly precise in areas with sufficient training data, but is strongly degraded for events outside the training distribution, sometimes producing massive outliers. Our analysis suggests that these characteristics are not only present for our model, but for most deep learning models for fast assessment published so far. They result from the black box modeling and their mitigation will likely require imposing physics derived constraints on the neural network. These characteristics need to be taken into consideration for practical applications.

Assessing digital elevation model resolution for soil organic carbon prediction

Baseline estimations of soil organic carbon (SOC) have been made globally using SOC models, earth system models, and digital soil mapping techniques. Digital elevation models (DEMs) underpin these analyses to incorporate the effect of topography on SOC. Here, we test the effect of DEM resolution on the relationships between topography and SOC across catchment to hillslope scales. Samples were collected using a nested field sampling approach in a grazed catchment on Eastern Australia. Topographic attributes were derived from 5 m, 25 m, 30 m, and 90 m resolution DEMs and then used to predict SOC using a Random Forest model. This was trained on a catchment-wide dataset and tested using the repeat samples of this dataset and finer scale field data. SOC was able to be predicted using topography in the catchment-wide datasets (model R2 = 0.37–0.51), however not in the finer-scale data (model R2 = 0–0.21). Variable importance was calculated from the modelling process, with elevation (as a surrogate for climate) being the main driver at the catchment-scale. In the finer scale datasets, topographic variables linked to soil redistribution were more important. As a result, SOC estimation methods using coarse resolution DEM data and large-scale sampling may be limited in capturing the effect of topography, having implications for SOC management and modelling.

WITHDRAWN: Integration patterns of MongoDB GridFS for advanced data science and big data processing

The volume of unstructured data is increasing rapidly with velocity and variety generating enormous large scale datasets which are very difficult in terms of analysis and further extraction of knowledge. Such issues related to the storage, processing and knowledge discovery from huge amount of unstructured data is treated under big data analytics. There are various applications and cases where the volume, velocity and variety of data are increasing very frequently and a significant research work is going on to cope up with the aspects of Big Data processing. Big data is still data, but is of immense scale. Big Data is a concept used to describe an incredibly large volume set of data that is exponentially increasing over time. In short, these files are so massive and complex that no standard data processing systems can store or handle them effectively. In current scenarios, there is need to handle big data with advanced data science approaches and algorithms. In this manuscript, the integration patterns and implementation scenarios of MongoDB GridFS is presented for the advanced applications of data science with big data processing.

Vehicle operating state anomaly detection and results virtual reality interpretation

Nowadays, with the increasing complexity of vehicle operating environment, vehicle operating reliability becomes growing essential and has attracted significant attention in recent decades. The majority of vehicle operating activities are organized in the format of time series data. Many conventional approaches are proposed to deal with time-series data. Autoregressive Integrated Moving Average (ARIMA) approach is one of the most widely used methods to analyze time-series data. In this paper, an ARIMA-based anomaly detection framework is developed to identify abnormal states of the vehicles based on the multiple-channel operating time series data. The state anomaly is captured by the deviation of real-time values at different channels from the predictions. After abnormal operating states are identified, a novel immersive Virtual Reality (VR) tool is applied to visualize the difference between normal and abnormal status. The combination of ARIMA anomaly detection and VR visualization for vehicle multiple-channel operating time-series data would provide a robust way to present the results of vehicle operating status. A large scale data set of 14 days operational performance channels of a specific vehicle is utilized to validate the performance of the proposed approach.

The Implications of Interrelated Assumptions on Estimates of Divergence Times and Rates of Diversification.

Phylogenies are increasingly being used as a basis to provide insight into macroevolutionary history. Here, we use simulation experiments and empirical analyses to evaluate methods that use phylogenies as a basis to make estimates of divergence times and rates of diversification. This is the first study to present a comprehensive assessment of the key variables that underpin analyses in this field-including substitution rates, speciation rates, and extinction, plus character sampling and taxon sampling. We show that in unrealistically simplistic cases (where substitution rates and speciation rates are constant, and where there is no extinction), increased character and taxon sampling lead to more accurate and precise parameter estimates. By contrast, in more complex but realistic cases (where substitution rates, speciation rates, and extinction rates vary), gains in accuracy and precision from increased character and taxon sampling are far more limited. The lack of accuracy and precision even occurs when using methods that are designed to account for more complex cases, such as relaxed clocks, fossil calibrations, and models that allow speciation rates and extinction rates to vary. The problem also persists when analyzing genomic scale data sets. These results suggest two interrelated problems that occur when the processes that generated the data are more complex. First, methodological assumptions are more likely to be violated. Second, limitations in the information content of the data become more important.[Divergence time estimation; diversification rates; macroevolution; phylogeny.].

A fuzzy approach for assessment of smart socio-cultural attributes of a historic urban landscape: Case study of Alwar walled city in India

Smartness is a vague concept with different meanings for different people. It is imperative to harness the latent potential of existing settlements for an inclusive smart urban development. The study aims to assess the socio-cultural attributes of an Indian historic urban landscape with a thriving residential culture. Since data collection of neighborhood-level urban communities is not feasible in India, a structured questionnaire was used to conduct household surveys. Such real-world phenomena have inherent imprecision and ambiguity associated with human judgments. Therefore, the survey items are assumed as fuzzy linguistic variables, and the raw dataset is transformed into triangular fuzzy numbers. Fuzzy arithmetic and weighted averaging operators are applied for the hierarchical evaluation of indicators and variables. A robust algorithm is used for the dimension reduction of a fuzzy coded dataset while dealing with subjective responses. The aggregated fuzzy scores show an overall better performance of traditional communities with respect to their socio-cultural attributes, such as a sense of safety and collective efficacy. The application of fuzzy logic in urban planning and allied behavioral studies can effectively and pragmatically deal with the inherent uncertainties in a humanistic system. Future researchers may explore fuzzy multi-criteria evaluation approaches for ordinal scale datasets.

Video-based Person Re-identification Based on Distributed Cloud Computing

Person re-identification(Re-ID) has been a hot research issues in the field of computer vision. In recent years, with the maturity of the theory, a large number of excellent methods have been proposed. However,Large scale data sets and huge networks make training a time-consuming process. At the same time, the parameters and their values generated during the training process also take up a lot of computer resources.Therefore, we apply distributed cloud computing method to perform person re-identification task. Using distributed data storage method, pedestrian data sets and parameters are stored in cloud nodes.In order to speed up operational efficiency and increase fault tolerance, we add data redundancy mechanism to copy and store data blocks to different nodes,and proposed a hash loop optimization algorithm to optimize the data distribution process. Moreover,we assign different layers of the Re-ID network to different nodes to complete the training in the way of model parallelism. By comparing and analyzing the accuracy and operation speed of the distributed model on the video-based data set MARS, the results show that our distributed model has a faster training speed.