Individual Data Streams Research Articles

Double-coupling learning for multi-task data stream classification

Data stream classification methods exploiting cohesion in a single data stream have demonstrated promising performance. However, scenarios involving multiple data streams are indeed common in practice. They involve several correlated data streams and can be viewed as multi-task data streams. Instead of considering them separately as individual data streams, it is beneficial to leverage the correlations among the multi-task data streams in data stream modeling. In this regard, a novel classification method called double-coupling support vector machines (DC-SVM) is proposed to classify multiple data streams simultaneously, where both the external correlations between multiple data streams and the internal relationships within each individual data stream are considered. Experimental results on synthetic and real-world multi-task data streams show that the proposed method outperforms traditional data stream classification methods.

Unified IRS-Aided MIMO Transceiver Designs via Majorization Theory

In this paper, we develop a unified framework for IRS-aided transceiver designs under general power constraints in multiple-input multiple-output (MIMO) systems which implement interference (pre-)subtraction via Tomlinson-Harashima precoding (THP) or Decision Feedback Equalization (DFE) technologies. Armed with majorization theory, two fundamental classes of performance criteria, namely $K$ -increasing Schur-concave and Schur-convex functions of the logarithm of Mean Square Error (MSE) of the data stream, are investigated in depth. Firstly, we propose a simplified counterpart of the optimal transceiver design under general power constraints, with equivalence guaranteed by Pareto optimization theory and Lagrange duality. Moreover, the optimal semi-closed form solution to this simplified transceiver design can be attained using the modified subgradient method. Next, we prove that for any Schur-concave objective, the optimal nonlinear THP (DFE) design is in essence the linear precoding (equalization). For any Schur-convex objective, the optimal transceiver design results in individual data streams with equal MSEs, and thereby reduces to the Gaussian mutual information maximization based design. Based on the above conclusions, we further propose an efficient alternating optimization algorithm to decouple the optimization of the transmit precoder and the IRS reflection coefficients, where the classical successive convex approximation (SCA) technique is applied to fight against non-convex subproblems. From the low computational complexity perspective, a two-stage scheme is also developed inspired by the capability of the IRS in constructing favorable wireless links. Finally, numerical results show the global optimality of the modified subgradient method and the excellent performance of the proposed alternating optimization algorithm and two-stage scheme.

Performance Analysis of Diversity Selection Combining on Radio Frequency for Image Transmission

This paper describes a simulation of the image transmission system with diversity selection combining (DSC) method on radio frequency. DSC is used to obtain multiple data streams corresponding to the transmitted image at the receiver in order to combat errors during image transmission on wireless channels. These individual image data streams are combined to form a composite image with higher perceptual quality. When DSC is used, it is expected that it can affect the number of errors, so that the signal transmitted to the wireless channels becomes more robust against interference caused by transmission channels. By using DSC technique, it can increase the quality of image transmission at the receiver. The result of the experiment which was carried out to test bit error rate (BER) and peak signal to noise ratio (PSNR) on Lena image with rate 0.8 bpp by using DSC on radio frequency in Rayleigh fading channel showed that there was a decrease in BER at the average of 0.0152 or 3.39%, while for PSNR there was an increase at the average of 1.6737 dB or 6.73%when comparing to the system without DSC.

Development of a Machine Learning Model Using Multiple, Heterogeneous Data Sources to Estimate Weekly US Suicide Fatalities

Suicide is a leading cause of death in the US. However, official national statistics on suicide rates are delayed by 1 to 2 years, hampering evidence-based public health planning and decision-making. To estimate weekly suicide fatalities in the US in near real time. This cross-sectional national study used a machine learning pipeline to combine signals from several streams of real-time information to estimate weekly suicide fatalities in the US in near real time. This 2-phase approach first fits optimal machine learning models to each individual data stream and subsequently combines predictions made from each data stream via an artificial neural network. National-level US administrative data on suicide deaths, health services, and economic, meteorological, and online data were variously obtained from 2014 to 2017. Data were analyzed from January 1, 2014, to December 31, 2017. Longitudinal data on suicide-related exposures were obtained from multiple, heterogeneous streams: emergency department visits for suicide ideation and attempts collected via the National Syndromic Surveillance Program (2015-2017); calls to the National Suicide Prevention Lifeline (2014-2017); calls to US poison control centers for intentional self-harm (2014-2017); consumer price index and seasonality-adjusted unemployment rate, hourly earnings, home price index, and 3-month and 10-year yield curves from the Federal Reserve Economic Data (2014-2017); weekly daylight hours (2014-2017); Google and YouTube search trends related to suicide (2014-2017); and public posts on suicide on Reddit (2 314 533 posts), Twitter (9 327 472 tweets; 2015-2017), and Tumblr (1 670 378 posts; 2014-2017). Weekly estimates of suicide fatalities in the US were obtained through a machine learning pipeline that integrated the above data sources. Estimates were compared statistically with actual fatalities recorded by the National Vital Statistics System. Combining information from multiple data streams, the machine learning method yielded estimates of weekly suicide deaths with high correlation to actual counts and trends (Pearson correlation, 0.811; P < .001), while estimating annual suicide rates with low error (0.55%). The proposed ensemble machine learning framework reduces the error for annual suicide rate estimation to less than one-tenth of that of current forecasting approaches that use only historical information on suicide deaths. These findings establish a novel approach for tracking suicide fatalities in near real time and provide the potential for an effective public health response such as supporting budgetary decisions or deploying interventions.

Performance evaluation of image transmission using diversity selection combining technique

This paper discussed a simulation of SPIHT image compression transmission on wireless channels by using RS code and diversity selection combining technique in wavelet domain to get better reconstruction image by minimizing errors that caused by fading and noise. RS Code and Diversity Selection Combining Method were used to combat errors during image transmission on wireless channels. When the bit stream data arrived at the receiver, the coding was used to recover the errors in the bitstream. Diversity was used to obtain multiple data streams corresponding to the transmitted image at the receiver. These individual image data streams were combined to form a composite image with the better quality. The results show both methods demonstrated quality improvement of the received image.

Evaluation of simulation models to mimic the distortions introduced into squiggles by nanopore sequencers and segmentation algorithms.

Nucleotides ratcheted through the biomolecular pores of nanopore sequencers generate raw picoamperage currents, which are segmented into step-current level signals representing the nucleotide sequence. These ‘squiggles’ are a noisy, distorted representation of the underlying true stepped current levels due to experimental and algorithmic factors. We were interested in developing a simulation model to support a white-box approach to identify common distortions, rather than relying on commonly used black box neural network techniques for basecalling nanopore signals. Dynamic time warped-space averaging (DTWA) techniques can generate a consensus from multiple noisy signals without introducing key feature distortions that occur with standard averaging. As a preprocessing tool, DTWA could provide cleaner and more accurate current signals for direct RNA or DNA analysis tools. However, DTWA approaches need modification to take advantage of the a-priori knowledge regarding a common, underlying gold-standard RNA / DNA sequence. Using experimental data, we derive a simulation model to provide known squiggle distortion signals to assist in validating the performance of analysis tools such as DTWA. Simulation models were evaluated by comparing mocked and experimental squiggle characteristics from one Enolase mRNA squiggle group produced by an Oxford MinION nanopore sequencer, and cross-validated using other Enolase, Sequin R1_71_1 and Sequin R2_55_3 mRNA studies. New techniques identified high inserted but low deleted base rates, generating consistent x1.7 squiggle event to base called ratios. Similar probability density and cumulative distribution functions, PDF and CDF, were found across all studies. Experimental PDFs were not the normal distributions expected if squiggle distortion arose from segmentation algorithm artefacts, or through individual nucleotides randomly interacting with individual nanopores. Matching experimental and mocked CDFs required the assumption that there are unique features associated with individual raw-current data streams. Z-normalized signal-to-noise ratios suggest intrinsic sensor limitations being responsible for half the gold standard and noisy squiggle DTW differences.

Opportunistic occupancy-count estimation using sensor fusion: A case study

Estimation of occupancy counts in commercial and institutional buildings enables enhanced energy-use management and workspace allocation. This paper presents the analysis of cost-effective, opportunistic data streams from an academic office building to develop occupancy-count estimations for HVAC control purposes. Implicit occupancy sensing via sensor fusion is conducted using available data from Wi-Fi access points, CO2 sensors, PIR motion detectors, and plug and light electricity load meters, with over 200 h of concurrent ground truth occupancy counts. Multiple linear regression and artificial neural network model formalisms are employed to blend these individual data streams in an exhaustive number of combinations. The findings suggest that multiple linear regression models are the superior model formalism when model transferability between floors is of high value in the case study building. Wi-Fi enabled device counts are shown to have high utility for occupancy-count estimations with a mean R2 of 80.1–83.0% compared to ground truth counts during occupied hours. Aggregated electrical load data are shown to be of higher utility than separately submetered plug and lighting load data.

BER‐optimal selection of peak frequency deviation for RDS2

RDS2 is a newly proposed enhancement of the radio data system (RDS) for frequency modulation (FM) broadcasting, aiming at increasing the data rate. This Letter investigates the BER of the data streams of RDS2. Specifically, the authors propose a selection criterion for the peak frequency deviations of the individual data streams, based on a minimax-BER approach. They also theoretically verify that symbol shifting reduces the peak deviation of the RDS2 data signal by an amount of 25%. Simulation results confirm their theoretical findings.

Программный комплекс для выявления недекларированных возможностей в условиях отсутствия исходного кода.

One of the main problem of a binary code security analysis is a revealing of malicious behavior in an untrusted program. This task is hard to automate, and it requires a participation of a cybersecurity expert. Existing solutions are aimed on the analyst manual work; automation they provide does not demonstrate a system approach. In case where needed analysis tools are absent, the analyst loses the proper support and he is forced to develop tools on one's own. This greatly slows down him from obtaining the practical results. The paper presents a software complex to solve a revealing of malicious behavior problem as a whole: from creating a controlled execution environment to man guided preparing a high-level description of an analyzed algorithm. A QEMU Developer Toolkit (QDT) is introduced, offering support for the domain specific development life cycle. QDT is especially suited for QEMU virtual machine development, including specialized testing and debugging technologies and tools. A high-level hierarchical flowchart-based representation of a program algorithm is presented, as well as an algorithm for its construction. The proposed representation is based on a hypergraph and it allows both automatic and manual data flow analysis at various detail levels. The developed representation is suitable for automatic analysis algorithms implementation. An approach to improve the quality of the resulting representation of the algorithm is proposed. The approach combines individual data streams into the one that links separate logical modules of the algorithm. A test set based on real programs and model examples has been developed to evaluate the result of constructing the proposed high-level algorithm representation.

АНАЛІЗ МОДЕЛЕЙ ТА ПРОГНОЗУВАННЯ РИЗИКІВ ФУНКЦІОНУВАННЯ СИСТЕМИ УПРАВЛІННЯ SDN АРХІТЕКТУРИ

The article deals with the analysis of models and algorithms of the system control SDN of architecture for providing of more quality work of network. Great diversity of most hardware implementation of the data devices SDN architecture leads to the fact that different switches may not support some features or support them with limited performance. In the process network, this can significantly affect the throughput of an individual data flow or whole network domains. Routing flow transfer is carried out by criterion of quality of service and the criterion of balanced loads of network resources the network. Improving the quality of service is subject to the classification of the flow that greatly restricts the ability of flow control. Most models of such networks do not take into account the characteristics of multi-service data flow, which leads to deterioration in the quality of service and increase the likelihood of blocking the channels of transmission network. The inability to implement differentiated management for individual data streams for individual clients and take them into account quality requirements, leads to low efficiency of channel routing, sub-optimal distribution of network load, the deterioration of the quality of service high-priority data streams. Tools control over the process of transmission of individual data flow do not exist, whereby the control system has the ability to determine service degradation for these data streams, and therefore will not be able to guarantee the necessary level of control. New models and algorithms of functioning of a software – managed network will allow for routing data streams of network, differentiating them according to the sensitivity to mixing of packet and connection drops. At created networks try to dissociate C-plane from the plane of transmission and put considerable efforts to overcoming of the nascent problems related to complications of migration from traditional networks to architecture of SDN. The aim of this article is realization of analysis of models and algorithms of functioning of control system of architecture of SDN for more reliable work of network. It gives possibility to the users to get services with necessary quality, by authenticity and will allow overcoming the nascent problems related to complications of migration from traditional networks to architecture of SDN.

Long-Term Synchronization of Hybrid Sensors Networks

This article presents synchronization of a hybrid distributed sensor network with wired and wireless sensors. Authors present an implementation of a sleep monitoring system as a hybrid sensor network that combines wireless inertial sensors controlled by a custom smartphone application as an extension of the polysomnographic (PSG) monitor to improve user's comfort. The authors developed an original method of synchronization of wireless sensor data with the PSG records using an auxiliary audio synchronization signal generated by the smartphone. The timestamps on the smartphone are synchronized with the timestamps from inertial sensors, and time of generated synchronization pulses recorded by the PSG. The individual data streams were synchronized using the Dynamic Time Warping (DTW) mechanism. Authors present the system organization and the results of analysis of the whole night monitoring, including the analysis of channel reliability and clock drift. Clock drift has been reduced from 10-30 seconds to 5.1± 3 milliseconds which is with an improved accuracy as compare to existing methods.

Telemetry Fault-Detection Algorithms: Applications for Spacecraft Monitoring and Space Environment Sensing

Algorithms have been developed that identify unusual behavior in satellite health telemetry. Telemetry from solid-state power amplifiers and amplifier thermistors from 32 geostationary Earth orbit communications satellites from 1991 to 2015 are examined. Transient event detection and change-point event detection techniques that use a sliding window-based median are used, statistically evaluating the telemetry stream compared to the local norm. This approach allows application of the algorithms to any spacecraft platform because there is no reliance in the algorithms on satellite- or component-specific parameters, and it does not require a priori knowledge about the data distribution. Individual telemetry data streams are analyzed with the event detection algorithms, resulting in a compiled list of unusual events for each satellite. This approach identifies up to six events of up to six events that affect 51 of 53 telemetry streams at once, indicative of a spacecraft system-level event. In two satellites, the same top event date (4 December 2008) occurs over more than 10 years of telemetry from both satellites. Of the five spacecraft with known maneuvers, the algorithms identify the maneuvers in all cases. Event dates are compared to known operational activities, space weather events, and available anomaly lists to assess the use of event detection algorithms for spacecraft monitoring and sensing of the space environment.

Polarization of Narrowband VLF Transmitter Signals as an Ionospheric Diagnostic

AbstractVery low frequency (VLF, 3–30 kHz) transmitter remote sensing has long been used as a simple yet useful diagnostic for the D region ionosphere (60–90 km). All it requires is a VLF radio receiver that records the amplitude and/or phase of a beacon signal as a function of time. During both ambient and disturbed conditions, the received signal can be compared to predictions from a theoretical model to infer ionospheric waveguide properties like electron density. Amplitude and phase have in most cases been analyzed each as individual data streams, often only the amplitude is used. Scattered field formulation combines amplitude and phase effectively, but does not address how to combine two magnetic field components. We present polarization ellipse analysis of VLF transmitter signals using two horizontal components of the magnetic field. The shape of the polarization ellipse is unchanged as the source phase varies, which circumvents a significant problem where VLF transmitters have an unknown source phase. A synchronized two‐channel MSK demodulation algorithm is introduced to mitigate 90° ambiguity in the phase difference between the horizontal magnetic field components. Additionally, the synchronized demodulation improves phase measurements during low‐SNR conditions. Using the polarization ellipse formulation, we take a new look at diurnal VLF transmitter variations, ambient conditions, and ionospheric disturbances from solar flares, lightning‐ionospheric heating, and lightning‐induced electron precipitation, and find differing signatures in the polarization ellipse.

M48. Digital Phenotyping in Schizophrenia Using Smartphones

Background: The purpose of our study is to determine if self-reported symptom surveys and passive data (GPS, accelerometer, voice samples, call logs, text logs, and phone use data) collected from patients with schizophrenia in real time on their personal smartphone may be useful in predicting reoccurrence of psychotic symptoms. Methods: Research subjects are adults (age 18–45, both sexes) who have been diagnosed with schizophrenia or schizoaffective disorder and are currently in treatment. Inclusion criteria include owning a smartphone capable of running the study application. Subjects use the smartphone app for a 3-month period. During this period the app will always be constantly collecting data, up to 1 million data points per day per subject as outlined in the background section. In addition, subjects will have psychiatric batteries including the PANSS, CGI, BPRS, and GAF completed as part of monthly check-up meetings with study staff. Outcome measures include (1) Adherence to and Acceptability of the Smartphone Application. (2) Comparison of Smartphone Application Data to Clinician Collected Metrics 3) Sensitivity of the Smartphone Application in Predicting Symptom Change Through Individual Data Streams (Surveys, GPS, Accelerometer, Voice, Call Logs, Text Logs, Screen Data) and in combination. Results: To date 14 of 20 subjects have been recruited into the study. Three have completed the study, 2 have dropped out, and 9 are currently active. The subjects who dropped out of the study did so related to factors other than smartphone and app use. No subjects have complained about the app use or its tracking. We plan to recruit 6 more subjects in the next month. From data pooled from all subjects to date, the total duration of minutes not using the phone shows a negative correlation with the warning signs scale (P = .0001) as well as the length of outgoing phone calls (P = .0024). Taking medications is correlated with the number of outgoing text messages sent (P = .0004). Conclusion: Preliminary sensor data indicates that there are small but statistically significant correlations with psychiatric symptoms for patients with schizophrenia. Results to date also suggest that tracking patients with schizophrenia via their personal smartphone is acceptable in a research context and there is no evidence of harm to date. Using patients’ own phones to assess symptoms offers a potentially cost effective and scalable means to gather new streams of objective data regards patient’s state and real time functioning.

Large-Scale Multi-Stream Quickest Change Detection via Shrinkage Post-Change Estimation

The quickest change detection problem is considered in the context of monitoring large-scale independent normal distributed data streams with possible changes in some of the means. It is assumed that for each individual local data stream, either there are no local changes, or there is a "big" local change that is larger than a pre-specified lower bound. Two different kinds of scenarios are studied: one is the sparse post-change case when the unknown number of affected data streams is much smaller than the total number of data streams, and the other is when all local data streams are affected simultaneously although not necessarily identically. We propose a systematic approach to develop efficient global monitoring schemes for quickest change detection by combining hard thresholding with linear shrinkage estimators to estimating all post-change parameters simultaneously. Our theoretical analysis demonstrates that the shrinkage estimation can balance the tradeoff between the first-order and second-order terms of the asymptotic expression on the detection delays, and our numerical simulation studies illustrate the usefulness of shrinkage estimation and the challenge of Monte Carlo simulation of the average run length to false alarm in the context of online monitoring large-scale data streams.

Sensing Abnormal Resource Flow Using Adaptive Limit Process Charts in a Complex Supply Network*

ABSTRACTSupply networks are becoming increasingly complex with multiple overlapping relationships between firms that may span across industries. Consequently, inventory management is becoming more difficult as managers have to cope with variability in the supply flows that originate from different parts of the network. Managers that quickly sense abnormal flows may intervene and adapt their inventory policies in response to system changes. In this article, we present a framework for sensing abnormal flows originating within the upstream supply network of a focal organization. Our framework combines time series modeling with process charts to identify abnormal flow patterns in the incoming supply streams. It is a flexible framework that uses off‐the‐shelf technology to provide managers with a process that can be employed for monitoring multiple individual or aggregated data streams originating within any complex system such as complex adaptive supply networks. We illustrate our framework on four years of longitudinal supply data from the second largest food bank in the United States. We identify multiple instances of abnormal supply flows and validate our results through rigorous inventory analysis as well as field‐based expert interviews. We discuss the implications of our findings for inventory management in complex supply networks, both from academic and practitioner points of view.

By-Passing Infected Areas in Wireless Sensor Networks using BPR

Abnormalities in sensed data streams indicate the spread of malicious attacks, hardware failure and software corruption among the different nodes in a wireless sensor network. These factors of node infection can affect generated and incoming data streams resulting in high chances of inaccurate data, misleading packet translation, wrong decision making and severe communication disruption. This problem is detrimental to real-time applications having stringent quality-of-service (QoS) requirements. The sensed data from other uninfected regions might also get stuck in an infected region should no prior alternative arrangements are made. Although several existing methods (BOUNDHOLE and GAR) can be used to mitigate these issues, their performance is bounded by some limitations, mainly the high risk of falling into routing loops and involvement in unnecessary transmissions. This paper provides a solution to by-pass the infected nodes dynamically using a twin rolling balls technique and also divert the packets that are trapped inside the identified area. The identification of infected nodes is done by adapting a Fuzzy data clustering approach which classifies the nodes based on the fraction of anomalous data that is detected in individual data streams. This information is then used in the proposed by-passed routing (BPR) which rotates two balls in two directions simultaneously: clockwise and counter-clockwise. The first node that hits any ball in any direction and is uninfected, is selected as the next hop. We are also concerned with the incoming packets or the packets-on-the-fly that may be affected when this problem occurs. Besides solving both of the problems in the existing methods, the proposed BPR technique has greatly improved the studied QoS parameters as shown by almost 40 percent increase in the overall performance.

Nephele streaming: stream processing under QoS constraints at scale

The ability to process large numbers of continuous data streams in a near-real-time fashion has become a crucial prerequisite for many scientific and industrial use cases in recent years. While the individual data streams are usually trivial to process, their aggregated data volumes easily exceed the scalability of traditional stream processing systems. At the same time, massively-parallel data processing systems like MapReduce or Dryad currently enjoy a tremendous popularity for data-intensive applications and have proven to scale to large numbers of nodes. Many of these systems also provide streaming capabilities. However, unlike traditional stream processors, these systems have disregarded QoS requirements of prospective stream processing applications so far. In this paper we address this gap. First, we analyze common design principles of today's parallel data processing frameworks and identify those principles that provide degrees of freedom in trading off the QoS goals latency and throughput. Second, we propose a highly distributed scheme which allows these frameworks to detect violations of user-defined QoS constraints and optimize the job execution without manual interaction. As a proof of concept, we implemented our approach for our massively-parallel data processing framework Nephele and evaluated its effectiveness through a comparison with Hadoop Online. For an example streaming application from the multimedia domain running on a cluster of 200 nodes, our approach improves the processing latency by a factor of at least 13 while preserving high data throughput when needed.

Energy and Spectral Efficiency of Very Large Multiuser MIMO Systems

A multiplicity of autonomous terminals simultaneously transmits data streams to a compact array of antennas. The array uses imperfect channel-state information derived from transmitted pilots to extract the individual data streams. The power radiated by the terminals can be made inversely proportional to the square-root of the number of base station antennas with no reduction in performance. In contrast if perfect channel-state information were available the power could be made inversely proportional to the number of antennas. Lower capacity bounds for maximum-ratio combining (MRC), zero-forcing (ZF) and minimum mean-square error (MMSE) detection are derived. An MRC receiver normally performs worse than ZF and MMSE. However as power levels are reduced, the cross-talk introduced by the inferior maximum-ratio receiver eventually falls below the noise level and this simple receiver becomes a viable option. The tradeoff between the energy efficiency (as measured in bits/J) and spectral efficiency (as measured in bits/channel use/terminal) is quantified for a channel model that includes small-scale fading but not large-scale fading. It is shown that the use of moderately large antenna arrays can improve the spectral and energy efficiency with orders of magnitude compared to a single-antenna system.

Efficient scalable schemes for monitoring a large number of data streams

The sequential changepoint detection problem is studied in the context of global online monitoring of a large number of independent data streams. We are interested in detecting an occurring event as soon as possible, but we do not know when the event will occur, nor do we know which subset of data streams will be affected by the event. A family of scalable schemes is proposed based on the sum of the local cumulative sum, cusum, statistics from each individual data stream, and is shown to asymptotically minimize the detection delays for each and every possible combination of affected data streams, subject to the global false alarm constraint. The usefulness and limitations of our asymptotic optimality results are illustrated by numerical simulations and heuristic arguments. The Appendices contain a probabilistic result on the first epoch to simultaneous record values for multiple independent random walks.