Data Size Requirements Research Articles

Survey on Systematic Analysis of Deep Learning Models Compare to Machine Learning

This survey provides a comprehensive analysis of the systematic differences and advancements between deep learning (DL) and traditional machine learning (ML) models. By examining a wide array of research papers, the study highlights the unique strengths and applications of both methodologies. Deep learning, with its multi-layered neural networks, excels in handling large, unstructured datasets, making significant strides in image and speech recognition, natural language processing, and complex pattern recognition tasks. Conversely, traditional machine learning models, which rely on feature extraction and simpler algorithms, remain highly effective in structured data scenarios such as classification, regression, and clustering problems. The survey elucidates the criteria for choosing between DL and ML, focusing on factors like data size, computational resources, and specific application requirements. Furthermore, it discusses the evolving landscape of hybrid models that integrate DL and ML techniques to leverage the strengths of both approaches. This analysis provides valuable insights for researchers and practitioners aiming to deploy the most suitable AI models for their specific needs, emphasizing the importance of contextual understanding in the rapidly advancing field of artificial intelligence.

Deep learning‐based grain‐size decomposition model: A feasible solution for dealing with methodological uncertainty

ABSTRACTTerrigenous clastic sediments cover a large area of the Earth's surface and provide valuable insights into the Earth's evolution and environmental change. Sediment grain‐size decomposition has been widely used as an effective approach to inferring changes in sediment sources, transport processes and depositional environments. Several algorithms, such as single sample unmixing, end‐member modelling analysis and the universal decomposition model, have been developed for grain‐size decomposition. The performance of these algorithms is highly dependent on parameter selections, introducing subjective uncertainty. This uncertainty could undermine the reliability of decomposition results, limit the application of grain‐size decomposition techniques and reduce comparability across different studies. To mitigate the methodological uncertainty, a novel deep learning‐based framework for grain‐size decomposition of terrigenous clastic sediments is proposed. First, an improved universal decomposition model is used to analyse the collected grain‐size data, in order to provide training sets for the end‐to‐end decomposers. To meet the data size requirements of supervised learning, generative adversarial networks are also trained for data augmentation. The performance of the new framework is then evaluated using a small‐scale dataset (73 393 samples from 18 sites) of three sedimentary types (loess, fluvial and lake delta deposits). The decomposed grain‐size results demonstrate high feasibility and great potential of the framework in constructing a robust grain‐size decomposition model. Finally, it is proposed that future grain‐size research should aim to establish guidelines for grain‐size data sharing and produce a big grain‐size database for deep learning.

Communication and computational resource optimization for Industry 5.0 smart devices empowered by MEC

Smart devices in Industry 5.0, such as sensors and robots, are often limited by low battery life and finite computational resources, hindering their ability to perform complex tasks. By offloading computation-intensive tasks to Mobile Edge Cloud Computing (MEC) servers at the network’s edge, businesses can achieve real-time data processing and analysis, reducing communication latency, quicker response times, and improved system reliability. This work presents an integrated framework for MEC and Industry 5.0, aimed at enhancing the performance, efficiency, and flexibility of industrial processes. In particular, we propose a joint optimization problem that maximizes computational energy efficiency by optimally allocating resources, such as processing power and computational resources, as well as device association, in the most efficient manner possible. The problem is formulated as nonconvex/nonlinear, which is intractable and poses high complexity. To solve this challenging problem, we first transform and decouple the original optimization problem into a series of subproblems using the block coordinate descent method. Then, we iteratively obtain an efficient solution using convex optimization methods. In addition, our work sheds light on the fundamental trade-off between local computation and partial offloading schemes. The results show that for small data size requirements, the performance is comparable among different schemes. However, as data size increases, our proposed hybrid scheme, which includes a partial offloading scheme, outperforms others, highlighting the effectiveness of the proposed joint optimization scheme.

Energy Efficient MAC for M2M Communication Using Dynamic TDMA in IoT Network

In the Internet of Things (IoT) context, the relevance of M2M communication increased, creating the need for practical solutions. MTCDs, or Machine Type Communication Devices, frequently encounter issues such as collisions and delays while transmitting data, ensuring network scalability and maintaining the quality of service (QoS) for all devices participating in the communication. To address these challenges, this paper presents the protocol for switching MAC that optimizes energy usage with Dynamic Time Division Multiple Access (EMAC-DTDMA). TheEMAC-DTDMA protocol involves grouping, dynamic MAC switching, and timeslot allocation. The Harris Hawks Optimization (HHO) algorithm is applied to choose the cluster head of the machine type communication devices (MTCH) while the CSMA/CA and CSMA/CARP protocols with different backoff times are used to minimize collisions based on device density, backlog, and active nodes. The timeslots are allocated based on data size and QoS requirements using Dynamic TDMA. The Markov chain model is employed to overcome synchronization issues with traditional TDMA. The EMAC-DTDMA's performance is evaluated through simulation using a network simulator tool, considering access delay, energy usage, collision probability, and throughput.

Delay-Aware Edge-Terminal Collaboration in Green Internet of Vehicles: A Multiagent Soft Actor-Critic Approach

The high performance requirement of task processing and low energy consumption requirement of network operation result in a contradiction for the Internet of Vehicles (IoV). This paper addresses the contradiction and proposes an efficient edge-terminal collaboration scheme to sustain the diverse task requirements for a green IoV network. Specifically, two types of tasks are first modeled in terms of data size and delay requirements and then an energy minimization problem was formulated under the diverse task requirements. Thereafter, a multi-agent reinforcement learning framework is constructed, based on which the time-varying environment of the IoV can be modeled as a partially observable Markov decision process. Furthermore, a multi-agent soft actor-critic (MASAC) scheme is proposed to train the edge-terminal collaboration policy in terms of task offloading, spectrum sharing as well as vehicle power control. Finally, the effectiveness of the proposed scheme is validated by comparing with other baseline schemes through extensive simulation experiments.

Fair Computation Efficiency for OFDMA-Based Multiaccess Edge Computing Systems

This letter investigates the joint optimization of task offloading and resource allocation for OFDMA-based multi-access edge computing systems. This joint optimization aims at maximizing the minimum computation efficiency, which is a ratio between the task data size to energy consumption, of all users to ensure fairness via adjusting transmit power, computation capacity and sub-channel allocations under the concrete delay, data size and energy consumption requirements. The formulated minimum computation efficiency maximization problem is a non-convex mixed-integer problem. To solve this problem, an iterative algorithm is proposed by alternatively solving the continuous sub-problem and the discrete sub-problem. For the continuous sub-problem about joint transmit power and computation capacity, a low-complexity bisection method is proposed to obtain the optimal solution by solving a series of feasibility problems. Then, the dual method is adopted to solve the discrete sub-problem about sub-channel association. Moreover, we acquire the closed-form optimal solution at each iteration. Simulation results verify that the proposed algorithm is superior over baseline schemes in terms of minimum computation efficiency.

MSMC-TTS: Multi-Stage Multi-Codebook VQ-VAE Based Neural TTS

This paper aims to improve neural TTS with vector-quantized, compact speech representations. We propose a Vector-Quantized Variational AutoEncoder (VQ-VAE) based feature analyzer to encode acoustic features into sequences with different time resolutions, and quantize them with multiple VQ codebooks to form the Multi-Stage Multi-Codebook Representation (MSMCR). The TTS system, MSMC-TTS, is proposed to predict better speech via this representation. In prediction, the multi-stage predictor is trained to map the input text sequence to MSMCRs in stages, by minimizing Euclidean distance and “triplet loss”. In synthesis, the neural vocoder converts ground-truth or predicted MSMCRs into speech waveforms. The proposed system is trained with single-speaker TTS datasets and tested in various scenarios for comprehensive evaluation. In TTS evaluation, MSMC-TTS obtains MOS of 4.34 and 4.10 on English and Chinese datasets, which significantly outperforms VITS with scores of 3.78 and 3.90. Meanwhile, compared with Mel-Spectrograms, the domain discrepancy between prediction and ground truth is lower in MSMCRs with the higher Domain-classification Error Rate (DER). Furthermore, this system shows lower modeling complexity and data size requirements, preserving excellent performance even with fewer model parameters or training data. The noticeable improvement in analysis-synthesis and TTS from multiple codebooks and stages also validate them as vital components in seeking a more profitable speech representation and building high-performance neural TTS.

Rotating machinery faults detection method based on deep echo state network

This paper aims to develop an accurate and efficient end-to-end fault detection model trained by small-scale data for the rotating machinery. The echo state network (ESN) is promising thanks to the training process by linear regression, but it struggles in mining spatial information. Thus, a deep ESN based on fixed convolution kernels (FCK-DESN) is proposed. The Prewitt, the Sobel, and the Gaussian lowpass filters are designed as the fixed convolution kernels for spatial feature extraction without training. The one hidden layer autoencoder is built to compress the dimensionality and improve the applicability. Based on the pre-process modules, the ESN could realize pattern recognition under complex conditions. The fault detection approach is then constructed based on the time–frequency information provided by the smoothed pseudo-Wigner–Ville distribution. Case studies of a rotor-bearing system and a diesel engine show that the proposed FCK-DESN approach has better recognition rates than popular deep learning methods with high efficiency and lower data size requirements, which has more practical significance.

UAV-assisted data offloading for smart container in offshore maritime communications

Smart containers have been extensively applied in the maritime industry by embracing the Internet of Things to realize container status monitoring and data offloading without human intervention. However, the offloading rate and delay in the offshore region are limited by the coverage of the onshore base station (BS). In this paper, we investigate the unmanned aerial vehicle (UAV)-assisted data offloading for smart containers in offshore maritime communications where the UAV is as a relay node between smart containers and onshore BS. We first consider the mobility of container vessel in the offshore region and establish a UAV-assisted data offloading model. Based on this model, a data offloading algorithm is proposed to reduce the average offloading delay under data-size requirements and available energy constraints of smart containers. Specifically, the convex-concave procedure is used to update time-slot assignment, offloading approach selection, and power allocation in an iterative manner. Simulation results show that the proposed algorithm can efficiently reduce average offloading delay and increase offloading success ratio. Moreover, it is shown that the UAV relay cannot always bring the performance gain on offloading delay especially in the close-to-shore area, which could give an insight on the deployment of UAV relay in offshore communications.

Estimation of macro- and micro-genetic environmental sensitivity in unbalanced datasets

Genotype-by-environment interaction is caused by variation in genetic environmental sensitivity (GES), which can be subdivided into macro- and micro-GES. Macro-GES is genetic sensitivity to macro-environments (definable environments often shared by groups of animals), while micro-GES is genetic sensitivity to micro-environments (individual environments). A combined reaction norm and double hierarchical generalised linear model (RN-DHGLM) allows for simultaneous estimation of base genetic, macro- and micro-GES effects. The accuracy of variance components estimated using a RN-DHGLM has been explicitly studied for balanced data and recommendation of a data size with a minimum of 100 sires with at least 100 offspring each have been made. In the current study, the data size (numbers of sires and progeny) and structure requirements of the RN-DHGLM were investigated for two types of unbalanced datasets. Both datasets had a variable number of offspring per sire, but one dataset also had a variable number of offspring within macro-environments. The accuracy and bias of the estimated macro- and micro-GES effects and the estimated breeding values (EBVs) obtained using the RN-DHGLM depended on the data size. Reasonably accurate and unbiased estimates were obtained with data containing 500 sires with 20 offspring or 100 sires with 50 offspring, regardless of the data structure. Variable progeny group sizes, alone or in combination with an unequal number of offspring within macro-environments, had little impact on the dispersion of the EBVs or the bias and accuracy of variance component estimation, but resulted in lower accuracies of the EBVs. Compared to genetic correlations of zero, a genetic correlation of 0.5 between base genetic, macro- and micro-GES components resulted in a slight decrease in the percentage of replicates that converged out of 100 replicates, but had no effect on the dispersion and accuracy of variance component estimation or the dispersion of the EBVs. The results show that it is possible to apply the RN-DHGLM to unbalanced datasets to obtain estimates of variance due to macro- and micro-GES. Furthermore, the levels of accuracy and bias of variance estimates when analysing macro- and micro-GES simultaneously are determined by average family size, with limited impact from variability in family size and/or cohort size. This creates opportunities for the use of field data from populations with unbalanced data structures when estimating macro- and micro-GES.

Deep learning\u2013assisted prostate cancer detection on bi-parametric MRI: minimum training data size requirements and effect of prior knowledge

ObjectivesTo assess Prostate Imaging Reporting and Data System (PI-RADS)–trained deep learning (DL) algorithm performance and to investigate the effect of data size and prior knowledge on the detection of clinically significant prostate cancer (csPCa) in biopsy-naïve men with a suspicion of PCa.MethodsMulti-institution data included 2734 consecutive biopsy-naïve men with elevated PSA levels (≥ 3 ng/mL) that underwent multi-parametric MRI (mpMRI). mpMRI exams were prospectively reported using PI-RADS v2 by expert radiologists. A DL framework was designed and trained on center 1 data (n = 1952) to predict PI-RADS ≥ 4 (n = 1092) lesions from bi-parametric MRI (bpMRI). Experiments included varying the number of cases and the use of automatic zonal segmentation as a DL prior. Independent center 2 cases (n = 296) that included pathology outcome (systematic and MRI targeted biopsy) were used to compute performance for radiologists and DL. The performance of detecting PI-RADS 4–5 and Gleason > 6 lesions was assessed on 782 unseen cases (486 center 1, 296 center 2) using free-response ROC (FROC) and ROC analysis.ResultsThe DL sensitivity for detecting PI-RADS ≥ 4 lesions was 87% (193/223, 95% CI: 82–91) at an average of 1 false positive (FP) per patient, and an AUC of 0.88 (95% CI: 0.84–0.91). The DL sensitivity for the detection of Gleason > 6 lesions was 85% (79/93, 95% CI: 77–83) @ 1 FP compared to 91% (85/93, 95% CI: 84–96) @ 0.3 FP for a consensus panel of expert radiologists. Data size and prior zonal knowledge significantly affected performance (4%, p<0.05).ConclusionPI-RADS-trained DL can accurately detect and localize Gleason > 6 lesions. DL could reach expert performance using substantially more than 2000 training cases, and DL zonal segmentation.Key Points• AI for prostate MRI analysis depends strongly on data size and prior zonal knowledge.• AI needs substantially more than 2000 training cases to achieve expert performance.

Parametric Scenario Optimization under Limited Data

We consider optimization problems with uncertain constraints that need to be satisfied probabilistically. When data are available, a common method to obtain feasible solutions for such problems is to impose sampled constraints following the so-called scenario optimization approach. However, when the data size is small, the sampled constraints may not statistically support a feasibility guarantee on the obtained solution. This article studies how to leverage parametric information and the power of Monte Carlo simulation to obtain feasible solutions for small-data situations. Our approach makes use of a distributionally robust optimization (DRO) formulation that translates the data size requirement into a Monte Carlo sample size requirement drawn from what we call a generating distribution. We show that, while the optimal choice of this generating distribution is the one eliciting the data or the baseline distribution in a nonparametric divergence-based DRO, it is not necessarily so in the parametric case. Correspondingly, we develop procedures to obtain generating distributions that improve upon these basic choices. We support our findings with several numerical examples.

Integrated Modeling of Transfer Learning and Intelligent Heuristic Optimization for Steam Cracking Process.

The construction and expansion of steam cracking plants and feedstock diversification have resulted in a significant demand for the numerical simulation and optimization of models to achieve molecular refining and intelligent manufacturing. However, the existing models cannot be widely applied in industrial practice because of the high computational expense, time-consumption, and data size requirements. In this paper, a high-performance optimization process, which integrates transfer learning and a heuristic algorithm, is proposed for the optimization of furnaces for various feedstocks. An effective transfer learning structure, based on motif feature of the reaction network, is designed and subsequent product distribution prediction program is compiled. Then a hybrid genetic algorithm and particle swarm optimization method is applied for the coil outlet temperature (COT) curve optimization using the derived prediction model, and the results are obtained for different pricing policies of products. The results are determined based on the weight coefficients of prices for different products, and could be further explained by the yield distribution pattern and reaction mechanism.

Guidance on field survey programme design for basement flooding assessment

ABSTRACT Flood-related water damages have become the largest home insurance claims in North America in recent years. In response, considerable effort is being focused on options to reduce flood damages, including reducing sanitary sewer backups contributing to basement flooding. The expenditure to carry out field surveys to establish the effectiveness of alternatives, such as the impact of reducing rainfall-derived inflows (RDI), is substantial. Considering this, a guidance methodology on minimum data size requirements is developed. To accomplish this, a statistical and mathematical methodology is developed as a guide to design field programmes, enabling an assessment of the amount of data collection required. The results show that collecting data for approximately 39 storm events yields an acceptable level of accuracy for estimating RDI, providing valuable guidance for field collection programmes.

Introduction to machine and deep learning for medical physicists.

Recent years have witnessed tremendous growth in the application of machine learning (ML) and deep learning (DL) techniques in medical physics. Embracing the current big data era, medical physicists equipped with these state-of-the-art tools should be able to solve pressing problems in modern radiation oncology. Here, a review of the basic aspects involved in ML/DL model building, including data processing, model training, and validation for medical physics applications is presented and discussed. Machine learning can be categorized based on the underlying task into supervised learning, unsupervised learning, or reinforcement learning; each of these categories has its own input/output dataset characteristics and aims to solve different classes of problems in medical physics ranging from automation of processes to predictive analytics. It is recognized that data size requirements may vary depending on the specific medical physics application and the nature of the algorithms applied. Data processing, which is a crucial step for model stability and precision, should be performed before training the model. Deep learning as a subset of ML is able to learn multilevel representations from raw input data, eliminating the necessity for hand crafted features in classical ML. It can be thought of as an extension of the classical linear models but with multilayer (deep) structures and nonlinear activation functions. The logic of going "deeper" is related to learning complex data structures and its realization has been aided by recent advancements in parallel computing architectures and the development of more robust optimization methods for efficient training of these algorithms. Model validation is an essential part of ML/DL model building. Without it, the model being developed cannot be easily trusted to generalize to unseen data. Whenever applying ML/DL, one should keep in mind, according to Amara's law, that humans may tend to overestimate the ability of a technology in the short term and underestimate its capability in the long term. To establish ML/DL role into standard clinical workflow, models considering balance between accuracy and interpretability should be developed. Machine learning/DL algorithms have potential in numerous radiation oncology applications, including automatizing mundane procedures, improving efficiency and safety of auto-contouring, treatment planning, quality assurance, motion management, and outcome predictions. Medical physicists have been at the frontiers of technology translation into medicine and they ought to be prepared to embrace the inevitable role of ML/DL in the practice of radiation oncology and lead its clinical implementation.

Data Size Requirement for Forecasting Daily Crude Oil Price with Neural Networks

When the literature regarding applications of neural networks is investigated, it appears that a substantial issue is what size the training data should be when modelling a time series through neural networks. The aim of this paper is to determine the size of training data to be used to construct a forecasting model via a multiple-breakpoint test and compare its performance with two general methods, namely, using all available data and using just two years of data. Furthermore, the importance of the selection of the final neural network model is investigated in detail. The results obtained from daily crude oil prices indicate that the data from the last structural change lead to simpler architectures of neural networks and have an advantage in reaching more accurate forecasts in terms of MAE value. In addition, the statistical tests show that there is a statistically significant interaction between data size and stopping rule.

Deep Learning for Plant Stress Phenotyping: Trends and Future Perspectives.

Deep learning (DL), a subset of machine learning approaches, has emerged as a versatile tool to assimilate large amounts of heterogeneous data and provide reliable predictions of complex and uncertain phenomena. These tools are increasingly being used by the plant science community to make sense of the large datasets now regularly collected via high-throughput phenotyping and genotyping. We review recent work where DL principles have been utilized for digital image-based plant stress phenotyping. We provide a comparative assessment of DL tools against other existing techniques, with respect to decision accuracy, data size requirement, and applicability in various scenarios. Finally, we outline several avenues of research leveraging current and future DL tools in plant science.

Operation Status Monitoring and Analysis Based on the Architecture of Stream Computing and Memory Computing

The user's real-time electricity consumption data has become an important data source to measure the operation of power grid enterprises. Based on the user's real-time electricity consumption data, combined with discrete Fourier transform, we can monitor the abnormal behavior of the user. Based on the constructed user behavior characteristics, the K-Means clustering algorithm is used to classify the users' behavior categories. Considering the data size and real-time requirements of practical application, stream computing and memory computing technology together constitute the system framework, which is used in real-time monitoring and analysis system. Finally, through the analysis of the results of electricity behavior, the architecture of stream computing and memory computing is compared with the traditional data analysis platform.

Consistent estimation of complete neuronal connectivity in large neuronal populations using sparse "shotgun" neuronal activity sampling.

We investigate the properties of recently proposed "shotgun" sampling approach for the common inputs problem in the functional estimation of neuronal connectivity. We study the asymptotic correctness, the speed of convergence, and the data size requirements of such an approach. We show that the shotgun approach can be expected to allow the inference of complete connectivity matrix in large neuronal populations under some rather general conditions. However, we find that the posterior error of the shotgun connectivity estimator grows quickly with the size of unobserved neuronal populations, the square of average connectivity strength, and the square of observation sparseness. This implies that the shotgun connectivity estimation will require significantly larger amounts of neuronal activity data whenever the number of neurons in observed neuronal populations remains small. We present a numerical approach for solving the shotgun estimation problem in general settings and use it to demonstrate the shotgun connectivity inference in the examples of simulated synfire and weakly coupled cortical neuronal networks.

Research on Multipath Data Transmission Protocol Based on Wireless Sensor Networks

The issue of data transmission in wireless sensor networks was studied. The video data collection was chosen as the object of the study, and the quality of service (QoS) problem of video transmission was analyzed. Video applications were generally characterized by large data size and many QoS requirements, which made the transmission of video always a hard problem. In this dissertation, a cross-layer and multi-path video transmission strategy (MVTS) was presented. MVTS introduced the idea of differentiated service and multipath routing, and operated in both application layer and network layer. In application layer, different types of video frames were distinguished and marked with different tags. Then in network layer these frames were identified by the tags and MVTS forwarded them in different paths. In this way, the key frames could be sent to the sink node reliably. Simulation tests were carried out, and simulation results showed that MVTS extended the network lifetime greatly and provided better assurance for quality of video applications.