Structure Of Input Data Research Articles

A review of three dimensional reconstruction techniques

Three dimensional (3D) modeling is an important stereoscopic representation of an object for multiple viewpoints aggregation and geometrical information. A general 3D modeling pipeline consists of data acquisition, 3D reconstruction and surface reconstruction. The core computational process in 3D modeling is always associated with the 3D reconstruction, which can be categorized into three types, i.e. statistical models, discriminative learning models and generative learning models. Statistical models derive handcrafted feature descriptor from mathematical theory to extract the spatial and geometric features of 3D data. A corresponding matching is performed between multiple viewpoints 3D data to search for the maximum region likelihood across datasets and compute the best match of affine transformation. Discriminative learning models learn spatial coherent of 3D data through data-driven training that leads to the computation of affine transformation with data inferencing. Generative models, on the other hand, have the unsupervised capability of ingesting raw 3D data directly to learn latent representation of input 3D data and later generate ambient output sample from the latent representation. In this paper, a detailed comparison on the three types of 3D reconstruction techniques are reviewed in term of input data structure, correspondence accuracy, precision and recall using four benchmark datasets, i.e. ModelNet10/40, ICL-NUIM, and Semantic3D. The advantages and disadvantages of 3D reconstruction techniques are highlighted for implementation guideline and future improvement.

Profile and population dynamics of Aceh cattle in livestock breeding and forage centre, Indrapuri

This study aims to determine the profile and population dynamics of Aceh cattle in Livestock Breeding and Forage Center (BPTU-HPT), Indrapuri. On the Aceh cattle profile, qualitative data consisting of the colour and form of the horns and quantitative data consisting of the weight and body size were used. The data for population dynamics was the livestock recording data, which includes population structure of Aceh cattle, death, births, expenditure and animal input data for five years (2015 to 2019). Population dynamics have been predicted by time series analysis. The results showed Aceh cattle in BPTU-HPT Indrapuri have a predominantly brick red colour, with a lower body weight and size compared to other local Indonesian cattle. Furthermore, it was projected that the population dynamics of Aceh cattle will increase by an average of 65 heads or 6.02 per cent from 2020 to 2024. In conclusion, Aceh cattle have different profile from other local Indonesian cattle, with a smaller phenotype in particular, but a strong resistance to bad environments. Furthermore, based on the estimates of population dynamics analysis, it has the potential to continue evolving as breeding stock source in BPTU-HPT, Indrapuri.

RF Canceller Tuning Acceleration Using Neural Network Machine Learning for In-Band Full-Duplex Systems

The fifth-generation wireless system framework provides the option to evaluate the performance of in-band full-duplex (IBFD) operation through flexible duplexing. The resulting self-interference, however, must be mitigated within a fraction of a symbol duration for successful communication. This paper introduces the use of neural network machine learning to accelerate the tuning of multi-tap adaptive RF cancellers. Additionally, the optimal network configurations, input data structures and training dataset densities that optimize the performance of this technique are presented. The tuning results of a prototype system using a two-tap canceller were measured over 20 and 100 MHz bandwidths centered at 2.5 GHz, and demonstrated averages of 40 dB cancellation and 6 tuning iterations. These results are compared to a survey of previously-reported adaptive cancellers, and illustrate that this novel application of machine learning to RF canceller tuning provides the fastest convergence speed to date, which can enable IBFD operation in dynamic interference environments.

How to design co-programs

Abstract The observation that program structure follows data structure is a key lesson in introductory programming: good hints for possible program designs can be found by considering the structure of the data concerned. In particular, this lesson is a core message of the influential textbook “How to Design Programs” by Felleisen, Findler, Flatt, and Krishnamurthi. However, that book discusses using only the structure of input data for guiding program design, typically leading towards structurally recursive programs. We argue that novice programmers should also be taught to consider the structure of output data, leading them also towards structurally corecursive programs.

Robust Local Preserving and Global Aligning Network for Adversarial Domain Adaptation

Unsupervised domain adaptation (UDA) requires source domain samples with clean ground truth labels during training. Accurately labeling a large number of source domain samples is time-consuming and laborious. An alternative is to utilize samples with noisy labels for training. However, training with noisy labels can greatly reduce the performance of UDA. In this paper, we address the problem that learning UDA models only with access to noisy labels and propose a novel method called robust local preserving and global aligning network (RLPGA). RLPGA improves the robustness of the label noise from two aspects. One is learning a classifier by a robust informative-theoretic-based loss function. The other is constructing two adjacency weight matrices and two negative weight matrices by the proposed local preserving module to preserve the local topology structures of input data. We conduct theoretical analysis on the robustness of the proposed RLPGA and prove that the robust informative-theoretic-based loss and the local preserving module are beneficial to reduce the empirical risk of the target domain. A series of empirical studies show the effectiveness of our proposed RLPGA.

A TOPSIS-Assisted Feature Selection Scheme and SOM-Based Anomaly Detection for Milling Tools Under Different Operating Conditions

Anomaly detection modeled as a one-class classification is an essential task for tool condition monitoring (TCM) when only the normal data are available. To confront with the real-world settings, it is crucial to take the different operating conditions, e.g., rotation speed, into account when approaching TCM solutions. This work mainly addresses issues related to multi-operating-condition TCM models, namely the varying discriminability of sensory features with different operating conditions; the overlap between normal and anomalous data; and the complex structure of input data. A feature selection scheme is proposed in which the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is presented as a tool to aid the multi-objective selection of sensory features. In addition, four anomaly detection approaches based on Self-Organizing Map (SOM) are studied. To examine the stability of the four approaches, they are applied on different single-operating-condition models. Further, to examine their robustness when dealing with complex data structures, they are applied on multi-operating-condition models. The experimental results using the NASA Milling Data Set showed that all the studied anomaly detection approaches achieved a higher assessment accuracy with our feature selection scheme as compared to the Principal Component Analysis (PCA), Laplacian Score (LS), and extended LS in which we added a final step to the original LS method in order to eliminate redundant features.

Short-Term Travel Speed Prediction for Urban Expressways: Hybrid Convolutional Neural Network Models

Deep learning models for short-term travel speed prediction on urban expressways, such as the convolutional neural network (CNN), still present several limitations in multiscale spatiotemporal feature extraction. Hence, in this paper, three hybrid CNN models are proposed to improve the basic CNN model with regard to three target aspects for short-term (i.e., 5 min) travel speed prediction on urban expressways. More specifically, long short-term memory (LSTM), AutoEncoder (AE), and Inception module are incorporated into the basic CNN model to capture multiscale spatiotemporal features of travel speed data effectively and improve the accuracy and robustness of the basic CNN model. Based on loop detector data collected on the Yan’an expressway in Shanghai, the proposed hybrid CNN models are trained and tuned. To validate the improvements on the target aspects, a comprehensive comparison is conducted using a classical statistical model (i.e., autoregressive integrated moving average), a typical shallow neural network model (i.e., artificial neural network), and two basic deep learning models (i.e., recurrent neural network and CNN). Results show that the prediction accuracies of all the proposed hybrid CNN models exceed 96% and the mean absolute errors are less than 2.5 km/h, which are superior to other models. In terms of target improving aspects, two new metrics were introduced, and the proposed models, especially the AE–CNN model, showed better robustness under various input data structures and traffic states. The LSTM–CNN model outperformed the other models in learning time-series features, and the Inception–CNN model is superior in reproducing the dynamics of traffic congestion patterns on urban expressways.

Thermodynamic cost of edge detection in artificial neural network (ANN)-based processors

ABSTRACT Architecture-based heat dissipation analyses allow us to reveal fundamental sources of inefficiency in a given processor and thereby provide us with road-maps to design less dissipative computing schemes independent of technology-base used to implement them. In this work, we study architectural-level contributions to energy dissipation in an Artificial Neural Network (ANN)-based processor that is trained to perform edge-detection task. We compare the training and information processing cost of ANN to that of conventional architectures and algorithms using 64-pixel binary image. Our results reveal the inherent efficiency advantages of an ANN network trained for specific tasks over general-purpose processors based on von Neumann architecture. We also compare the proposed performance improvements to that of Cellular Array Processors (CAPs) and illustrate the reduction in dissipation for special purpose processors. Lastly, we calculate the change in dissipation as a result of input data structure and show the effect of randomness on energetic cost of information processing. The results we obtained provide a basis for comparison for task-based fundamental energy efficiency analyses for a range of processors and therefore contribute to the study of architecture-level descriptions of processors and thermodynamic cost calculations based on physics of computation.

Use of color parameters in the grouping of soil samples produces more accurate predictions of soil texture and soil organic carbon

Prediction of soil properties such as texture and soil organic carbon (SOC) content by reflectance spectroscopy (RS) is influenced by the heterogeneity of soil samples used to calibrate multivariate models. These soil properties are directly related to color, which, in turn, can be estimated by color parameters derived from the visible (Vis) spectrum at no additional cost. At present, only a few publications have addressed the effect that input data structure and soil heterogeneity have on model performance. Therefore, the objectives of this study were to use Vis-based-color parameters combined with multivariate statistical techniques to group soil samples and comparing the results of different SOC, clay, sand and silt prediction models. Soil sampling was conducted over an area of approximately 500 ha in the region of São Joaquim National Park, Santa Catarina State, Brazil, where a total of 260 soil samples were collected. Soil reflectance data were obtained by Vis-NIR-SWIR spectroscopy in the laboratory, through a spectroradiometer that covers the 350–2500 nm. Soil organic carbon content was determined by dry combustion in an elemental analyzer. Sand, silt and clay fractions were determined using the pipette method. Twenty-two components of color parameters were derived from the Vis spectrum with the use of colorimetry models. For the definition of the most appropriate number of soil sample clusters, two multivariate statistical analyzes: principal component analysis (PCA) and cluster analysis of the samples were applied to the color parameter values. Partial least squares regression (PLSR) and support vector machines (SVM) multivariate models were calibrated for each cluster and also for the models without stratification using 260 soil samples and 95 selected samples (MWS-260 and MWS-95). Overall, the PLSR model performed better than the SVM model, as confirmed by the statistical difference between RMSE results. Multivariate statistical analyzes applied to color parameters were able to group soil samples with similar characteristics, reducing data amplitude and improving the accuracy of soil property predictions. These analyzes demonstrated that the use of Vis-based-color parameters to group soil samples can be a quick and inexpensive way to increase the potential of spectroscopy to accurately predict soil physical and chemical properties.

Adaptive Consistency Propagation Method for Graph Clustering

Graph clustering plays an important role in data mining. Based on an input data graph, data points are partitioned into clusters. However, most existing methods keep the data graph fixed during the clustering procedure, so they are limited to exploit the implied data manifold and highly dependent on the initial graph construction. Inspired by the recent development on manifold learning, this paper proposes an Adaptive Consistency Propagation (ACP) method for graph clustering. In order to utilize the features captured from different perspectives, we further put forward the Multi-view version of the ACP model (MACP). The main contributions are threefold: (1) the manifold structure of input data is sufficiently exploited by propagating the topological connectivities between data points from near to far; (2) the optimal graph for clustering is learned by taking graph learning as a part of the optimization procedure; and (3) the negotiation among the heterogeneous features is captured by the multi-view clustering model. Extensive experiments on real-world datasets validate the effectiveness of the proposed methods on both single- and multi-view clustering, and show their superior performance over the state-of-the-arts.

Prediction of soil texture classes through different wavelength regions of reflectance spectroscopy at various soil depths

The demand for quality and low-cost soil information is growing due to the demands of land use planning and precision agriculture. Soil texture is one of the key soil properties, as it determines other vital soil characteristics such as soil structure, water and thermal regime, diversity of living organisms, plant growth, as well as the soil quality in general. It is usually not constant over an area, varying in space and with soil depth. Routine soil texture analysis is, however, time consuming and expensive. Because of this, the success of proximal soil sensing techniques in estimate soil properties using the VIS-NIR-SWIR and MIR regions is increasing. Advantages of soil spectroscopy include time efficiency, economic convenience, non-destructive application and freeing of chemical agents involved. Therefore, the objectives of this study were: (a) to explore the potential of clay, sand and silt prediction using reflectance spectroscopy; (b) assess the performance of predictive models in different spectral regions, i.e. VIS-NIR-SWIR and MIR; (c) assess the effect of different soil depths on predictive models; and finally (d) explain the differences in prediction accuracy in the means of the input data structure. Soil samples were collected at three depths (0–20, 20–40 and 40–60 cm) at 70 sampling sites over a study area located in the State of Rio Grande do Sul (Brazil). The content of soil texture was determined by Pipette method, and soil spectra were obtained with FieldSpec Pro (VIS-NIR-SWIR) and by Alpha Sample Compartment RT (MIR). Cubist regression algorithm was applied to train predictive models in three separate modeling modes differing in spectral region: (i) VIS-NIR-SWIR, (ii) MIR and (iii) VIS-NIR-SWIR plus MIR. The results showed that the combination of all three soil depths led to a more accurate prediction of soil texture compared to subdivided soil depths. This was explained by variability of the data, which was larger for the total dataset than for the depth-specific data. Consequently, we suggested that no precise comparison between different studies can be made without a proper description of the input data. For all-depths models, the MIR calibration obtained the best accuracy, which was explained due to more information comprised in the MIR region against the VIS-NIR-SWIR. The bands that were more important in predicting soil texture in MIR are related to mineralogy, specifically to kaolinite. This study demonstrated that the MIR spectroscopy technique is capable to complement the standard soil particle size analysis, specially where a large number of soil samples need to be treated in a short period of time.

Multimodal Deep Learning for Heterogeneous GNSS-R Data Fusion and Ocean Wind Speed Retrieval

The comprehensiveness of the raw input data and the effectiveness of feature engineering are two key factors affecting the performance of machine learning. To improve the data comprehensiveness for Global Navigation Satellite System Reflectometry (GNSS-R) ocean wind speed retrieval, this article introduces a new input data structure, which is composed of Delay–Doppler maps (DDM) and all satellite receiver status (SRS) parameters. Then, to overcome the difficulty of handcrafted feature engineering and effectively fusion the information of DDM and SRS, we presented a heterogeneous multimodal deep learning (HMDL) method to retrieve the wind speed according to the heterogeneity of the input data. The proposed model is verified by the performance evaluation of realistic data sets obtained from TDS-1. The new input data structure improves the prediction accuracy at 13.5% to 30.7% on mean absolute error (MAE) at 10.6% to 29.5% on the root mean square error (RMSE). The HMDL improves the prediction accuracy at 7.7% on MAE and 7.1% on RMSE. The whole proposed solution improves the prediction accuracy at 36.3% on MAE and 36.8% on RMSE, comparing with the traditional neural network-based solution. The results clearly show that both the introduction of the new input data structure and HMDL effectively improve the accuracy and robustness of GNSS-R wind speed retrieval.

Examining Naturogenic Processes and Anthropogenic Influences on Tree Growth and Development via Stem Analysis: Data Processing and Computational Analytics

The objective of this study was to develop a stem analysis data processing and computational algorithm and associated software suite that was (1) applicable to temperate and boreal forest tree species, (2) mathematically consistent with excurrent tree stem geometric and allometric principles, (3) compatible with data structures obtained using proprietary and non-proprietary imaging systems, and (4) executable on Windows®-based operating systems. Computationally, the suite denoted SAP (Stem Analysis Program), deployed sectional-specific formulae that were in accord with the following geometric assumptions: (1) stump section was treated as a solid of revolution resembling a cylinder; (2) sections between the stump and the tip were treated as a solid of revolution resembling a frustum of a cone for sections with continuous annual increments, otherwise treated as a cone; and (3) tip section was treated as a solid of revolution resembling a cone. The algorithm also corrected for the slant-based sectional length measurements using Pythagorean Theorem and eliminated the requirement to predict age-specific apex height development through the use of a linear interpolation procedure. Based on input data structures consisting of annual ring-width xylem sequences measured from cross-sectional disk samples acquired at multiple positions along the tree’s main stem, the suite produces a broad array of output, inclusive of radial and longitudinal ring-width sequences, apical growth increments, annual and cumulative sectional and cumulative volume production patterns, and historically reconstructed stem taper profiles. In total, the SAP creates six output data files for each tree analyzed: (1) input data reference summary (e.g., geometric mean ring-widths and resultant radii for each cross-section); (2) radial growth patterns for the cross-section sampled at breast-height (e.g., absolute and relative diameter and basal area growth estimates); (3) sectional (vertical) profiles of volume growth patterns (e.g., absolute and relative growth estimates within each section (bolt)); (4) cumulative volume growth patterns for the entire tree; (5) historical taper profile estimates (e.g., heights and diameters by year); and (6) texturally-labeled compendium of all output files generated. Additionally, real-time graphical output was produced for the purposes of data assessment and verification during the radial sequence data acquisition stage (e.g., graphical presentation of annual ring-width sequences by radii and disk, for use in validating input data structures and increment measurements derived from the imaging system), and interpreting growth and development patterns (e.g., vertical growth layer and specific volume increment profiles by age or year). The utility of the SAP suite was exemplified by processing WindendroTM-based annual ring-width xylem sequences obtained from cross-sectional disks extracted from a jack pine (Pinus banksiana Lamb.) tree via percent-height destructive stem analysis, and subsequently elucidating growth and developmental patterns within the context of silviculture treatment effects (thinning). The SAP suite provides the conceptual and logistical foundation for the continued deployment of the stem analysis approach in a wide range of investigations, including those examining the effect of naturogenic processes and anthropogenic influences on tree growth and development.

Track based alignment procedure for the CBM silicon tracking detector

The CBM (Compressed Baryonic Matter) experiment at FAIR is being designed for the study of the QCD phase diagram in the region of the high baryon chemical potential at relatively moderate temperatures. The Silicon Tracking System (STS) is the central detector for momentum reconstruction of the produced charged particles in the CBM experiment. It consists of 8 layers of altogether ∼ 900 double sided silicon micro strip sensors. Limited mechanical precision(> 100μm) during the mounting, temperature differences during the experiment, influence of the applied magnetic field result in misalignment to the detector component positions. Therefore, the intrinsic spatial resolution(∼ 20μm) of the detector components has to be recovered by a track based alignment method. In this conference paper, the current status of implementation of the alignment algorithm is presented. For this work, the GBL (General broken line) track refit model is employed to create the necessary input data structure to provide the input to the standalone PEDE part of the χ2 minimization based MILLEPEDE II alignment algorithm.

Deep Convolutional Generative Adversarial Network (dcGAN) Models for Screening and Design of Small Molecules Targeting Cannabinoid Receptors.

A deep convolutional generative adversarial network (dcGAN) model was developed in this study to screen and design target-specific novel compounds for cannabinoid receptors. In the adversarial process of training, two models, the discriminator D and the generator G, are iteratively trained. D is trained to discover the hidden patterns among the input data to have the accurate discrimination of the authentic compounds and the "fake" compounds generated by G; G is trained to generate "fake" compounds to fool the well-trained D by optimizing the weights for matrix multiplication of data sampling. In order to determine the appropriate architecture and the input data structure for the involved convolutional neural networks (CNNs), the combinations of various network architectures and molecular fingerprints were explored. Well-developed CNN models including LeNet-5, AlexNet, ZFNet, and VGGNet were investigated. Four types of fingerprints, including MACCS, ECFP6, AtomPair, and AtomPair Count, were calculated to describe the small molecules with diverse structural characteristics. The limitation of generating fingerprints as output remains that the concrete molecular structures cannot be converted directly, while the generative models with convolutional networks provide promising opportunities to the screening of molecules and rational modifications afterward. This study demonstrated how computer-aided drug discovery could benefit from the recent advances in deep learning.

PanoView: An iterative clustering method for single-cell RNA sequencing data.

Single-cell RNA-sequencing (scRNA-seq) provides new opportunities to gain a mechanistic understanding of many biological processes. Current approaches for single cell clustering are often sensitive to the input parameters and have difficulty dealing with cell types with different densities. Here, we present Panoramic View (PanoView), an iterative method integrated with a novel density-based clustering, Ordering Local Maximum by Convex hull (OLMC), that uses a heuristic approach to estimate the required parameters based on the input data structures. In each iteration, PanoView will identify the most confident cell clusters and repeat the clustering with the remaining cells in a new PCA space. Without adjusting any parameter in PanoView, we demonstrated that PanoView was able to detect major and rare cell types simultaneously and outperformed other existing methods in both simulated datasets and published single-cell RNA-sequencing datasets. Finally, we conducted scRNA-Seq analysis of embryonic mouse hypothalamus, and PanoView was able to reveal known cell types and several rare cell subpopulations.

Intelligent pulse analysis of high-speed electrical discharge machining using different RNNs

High-speed electrical discharge machining (EDM) is a nontraditional machining method using high electric energy to efficiently remove materials. In this paper, a novel pulse classification method was proposed based on the recurrent neural network (RNN) for high-speed EDM pulse analysis. This study is the first time that an RNN has been applied in high-speed EDM pulse analysis. Different from traditional EDM, discharge pulses of high-speed EDM were classified into five types during the machining process: open, spark, arc, partially short and short. Models based on three different RNNs including the traditional RNN, LSTM (long short-term memory) and IndRNN (independently recurrent neural network) with different activation functions were built to analyze the discharge pulses in the research. A new input data structure based on the minimum signal change period was proposed in the classification method to simplify the model structure and improve accuracy at the same time. Without setting thresholds, the highest classification accuracy of the proposed model is up to 97.85%, which can simultaneously classify discharge pulses based on 10,000 orders of magnitude including various current values. The proposed method was effectively adapted to the complicated machining conditions and the compound power source of the high-speed EDM. The optimal model was used to analyze the distribution of discharge pulses during the machining process under different currents, fluxes and feeding speeds. The proportion of the discharge pulses was clearly predicted. Through analyzing the discharge pulses of long machining time, the regulation of discharge under different machining parameters was revealed more reliably, providing valuable information for the improvement of high-speed EDM servo systems.

A PCA-Based Framework for Determining Remotely Sensed Geological Surface Orientations and Their Statistical Quality.

The orientations of planar rock layers are fundamental to our understanding of structural geology and stratigraphy. Remote sensing platforms including satellites, unmanned aerial vehicles, and Light Detection and Ranging scanners are increasingly used to build three‐dimensional models of structural features on Earth and other planets. Remotely gathered orientation measurements are straightforward to calculate but subject to uncertainty inherited from input data, differences in viewing geometry, and the plane‐fitting process, complicating geological interpretation. Here, we improve upon the present state of the art by developing a generalized means for computing and reporting errors in strike‐dip measurements from remotely sensed data. We outline a general framework for representing the error space of uncertain orientations in Cartesian and spherical coordinates and develop a principal component analysis (PCA) regression method, which captures statistical errors independent of viewing geometry and input data structure. We also introduce graphical techniques to visualize the uniqueness and quality of orientation measurements and a process to increase statistical power by jointly fitting bedding planes under the assumption of parallel stratigraphy. These new techniques are validated by comparison of field‐gathered orientation measurements with those derived from minimally processed satellite imagery of the San Rafael Swell, Utah, and unmanned aerial vehicle imagery from the Naukluft Mountains, Namibia. We provide software packages supporting planar fitting and visualization of error distributions. This method increases the precision and comparability of structural measurements gathered using a new generation of remote sensing techniques.

RESEARCH OF METHODS OF DEVELOPMENT OF SOFTWARE COMPUTER ENGINEERING BASED ON TYPICAL SOFTWARE ELEMENTS

The paper deals with the problems of increasing the effectiveness of the development of IS, and, in particular, the issues of reducing the development time of the software package of IS. The analysis of technology development software in the life cycle of IS. A structural programming approach suggested decomposing programs in a step-by-step manner. The development of program structures is carried out using the construction of input and output data structures, identification of processing links between these data, formation of a program structure based on data structures and detected matches. It is possible to overcome the complexity factor if we deviate from a straightforward approach to solving the problem posed, consisting in sequential and linear extension of the source code of the program operator-by-operator, resulting in one long and amorphous program. Here, the modularity principle is effective: the initial problem is divided into relatively independent parts; they are implemented by separate software modules, which are then linked into a single unit at the layout stage. The features of the technology of automated program synthesis, namely, the technology of assembling programs from typical program elements, are highlighted. The basic concepts of the above technology have been identified, a study has been conducted to design programs from blocks and a problem area has been identified. The initial data for the formulation and solution of problems for the synthesis of a system of program modules are the set of information arrays of the system, for which there are defined: input, output and intermediate data; many alternative data processing procedures; sequence of procedures in the processing; ways of sharing with external memory. The approach to the selection of typical program elements that meet certain criteria is considered. On the basis of the considered approach, and also taking into account its shortcomings, an improved method was proposed for classifying typical program elements and a method for designing software based on them, taking into account minimizing the time and cost of the project.

SPMgr: Dynamic workflow manager for sampling and filtering data streams over Apache Storm

In this article, we address dynamic workflow management for sampling and filtering data streams in Apache Storm. As many sensors generate data streams continuously, we often use sampling to choose some representative data or filtering to remove unnecessary data. Apache Storm is a real-time distributed processing platform suitable for handling large data streams. Storm, however, must stop the entire work when it changes the input data structure or processing algorithm as it needs to modify, redistribute, and restart the programs. In addition, for effective data processing, we often use Storm with Kafka and databases, but it is difficult to use these platforms in an integrated manner. In this article, we derive the problems when applying sampling and filtering algorithms to Storm and propose a dynamic workflow management model that solves these problems. First, we present the concept of a plan consisting of input, processing, and output modules of a data stream. Second, we propose Storm Plan Manager, which can operate Storm, Kafka, and database as a single integrated system. Storm Plan Manager is an integrated workflow manager that dynamically controls sampling and filtering of data streams through plans. Third, as a key feature, Storm Plan Manager provides a Web client interface to visually create, execute, and monitor plans. In this article, we show the usefulness of the proposed Storm Plan Manager by presenting its design, implementation, and experimental results in order.