Stream Merging Research Articles

Competitive on-line stream merging algorithms for media-on-demand

We consider the problem of minimizing the bandwidth needed by media-on-demand servers that use stream merging. We consider the on-line case where client requests are not known ahead of time. To facilitate stream merging, clients have the ability to receive data from two streams simultaneously and can buffer up to half of a full stream. We present a new family of on-line stream merging algorithms called dynamic tree algorithms. The bandwidth requirements of the best of these, the dynamic Fibonacci tree algorithms, are within a factor of the minimum between log φ ( n)+ O(1) and log φ (1/(2 D))+ O(1) from the off-line optimal, where n is the number of requests, D is the guaranteed maximum start-up delay measured as a fraction of the time for a full stream, and φ=(1+ 5 )/2 . The new on-line algorithms use a dynamic Fibonacci tree to control how new arrivals should merge with existing streams. Empirical studies show that the dynamic Fibonacci tree algorithms perform much better than indicated by the analysis.

A new patching channel schedule scheme for video multicast

A new patching schedule scheme for video multicast is proposed The response to requests is modeled as a renewal process, and we prove that there exists an optimal value for the renewal period; in addition, a searching approach is presented to find this optimal value. The successive requests in a period are divided into a set of groups, based on which the patching streams are further patched in each group. Simulation results show that the proposed patching scheme outperforms the conventional patching technique by a significant margin in reducing the server network-I/O bandwidth. It even performs better than the dynamic skyscraper algorithm over a wide range of client request rates. Furthermore, it is reasonably competitive with hierarchical multicast stream merging (HMSM) at low to modest client request rates. Most importantly, the implementation complexity of our algorithm is much lower than the skyscraper and HMSM.

Plane mixing layers from parallel and non-parallel merging of two streams

Two types of mixing layers produced from two streams merging at 0° and 18° have been investigated. Each type of mixing layer was produced with velocity ratios 0.7, 0.8 and 0.9 and measurements were taken at six streamwise locations. The boundary layers were untripped and initially turbulent in all cases. Both types of mixing layers were found to attain a self-similar state for velocity ratios 0.7 and 0.8 but failed for 0.9 within the measurement domain. It appears that the mixing layer flow becomes self-similar earlier when merging at 18° than at 0°. With increasing velocity ratio, the development distance was increased and the splitter wake played a dominant role in the development of the mixing layers. The mixing layers from non-parallel merging streams (18°) were found to have higher growth in the near-field than those from parallel merging streams (0°). Both types of mixing layers were found to decrease in growth with increasing velocity ratio, though they spread more at the high-speed side.

On-line stream merging in a general setting

This paper is concerned with on-line scheduling algorithms for merging streams in a video-on-demand system so as to minimize the server bandwidth. We present the first algorithm that has a constant competitive factor (precisely, 5). Our algorithm, unlike previous ones, is not limited to the scenario where clients are equipped with large buffer and client receiving bandwidth. It remains 5-competitive in all settings of buffer size and receiving bandwidth. Technically speaking, our algorithm is based on a novel observation that the behavior of any schedule can be modeled by a rectilinear (binary) tree on a grid. This observation eases the analysis of our algorithm as well as the optimal algorithm.

Scalability of multicast delivery for non-sequential streaming access

To serve asynchronous requests using multicast, two categories of techniques---stream merging and periodic broadcasting---have been proposed. For sequential streaming access, where requests are uninterrupted from the beginning to the end of an object, these techniques are highly scalable: the required server bandwidth for stream merging grows logarithmically as request arrival rate, and the required server bandwidth for periodic broadcasting varies logarithmically as the inverse of start-up delay. A sequential access model, however, is inappropriate to model partial requests and client interactivity observed in various streaming access workloads. This paper analytically and experimentally studies the scalability of multicast delivery under a non-sequential access model where requests start at random points in the object. We show that the required server bandwidth for any protocol providing immediate service grows at least as the square root of request arrival rate, and the required server bandwidth for any protocol providing delayed service grows linearly with the inverse of start-up delay. We also investigate the impact of limited client receiving bandwidth on scalability. We optimize practical protocols which provide immediate service to non-sequential requests. The protocols utilize limited client receiving bandwidth, and they are near-optimal in that the required server bandwidth is very close to its lower bound.

The Dyadic Stream Merging Algorithm

We study the stream merging problem for media-on-demand servers. Clients requesting media from the server arrive by a Poisson process, and delivery to the clients starts immediately. Clients are prepared to receive up to two streams at any time, one or both being fed into a buffer cache. We present an on-line algorithm, the dyadic stream merging algorithm, whose recursive structure allows us to derive a tight asymptotic bound on stream merging performance. In particular, let λ be the Poisson request arrival rate, and let L be the fixed media length. Then the long-time ratio of the expected total stream length under the dyadic algorithm to that under an algorithm with no merging is asymptotically equal to 3log(λL)2λL. Furthermore, we establish the near-optimality of the dyadic algorithm by comparisons with experimental results obtained for an optimal algorithm constructed as a dynamic program. The dyadic algorithm and the best on-line algorithm of those recently proposed differ by less than a percent in their comparison with an off-line optimal algorithm. Finally, the worst-case performance of our algorithm is shown to be no worse than that of earlier algorithms. Thus, the dyadic algorithm appears to be the first near optimal algorithm that admits a rigorous average-case analysis.

Blocking of Dynamic Multicast Connections

Multicast connections have a bandwidth saving nature. This means that a multicast connection --- in taking the form of a tree with streams merging at the nodes --- requires much less capacity from the network links than a bunch of separate point-to-point connections providing the same connectivity. In this paper, we consider dynamic multicast connections that can be used to model, for example, TV or radio delivery on a telecommunications network, such as an ATM network with virtual circuits. We show how to calculate the blocking probabilities of requests to join such a tree. First, we consider the blocking probabilities occurring in a single link. The resulting model is able to capture heterogeneous capacity requirements for different multicast channels. Then we extend the results to a whole network using the reduced load approximation. The accuracy of the approximation method is studied by simulations.

Minimizing bandwidth requirements for on-demand data delivery

Two recent techniques for multicast or broadcast delivery of streaming media can provide immediate service to each client request, yet achieve considerable client stream sharing which leads to significant server and network bandwidth savings. The paper considers: 1) how well these recently proposed techniques perform relative to each other and 2) whether there are new practical delivery techniques that can achieve better bandwidth savings than the previous techniques over a wide range of client request rates. The principal results are as follows: First, the recent partitioned dynamic skyscraper technique is adapted to provide immediate service to each client request more simply and directly than the original dynamic skyscraper method. Second, at moderate to high client request rates, the dynamic skyscraper method has required server bandwidth that is significantly lower than the recent optimized stream tapping/patching/controlled multicast technique. Third, the minimum required server bandwidth for any delivery technique that provides immediate real-time delivery to clients increases logarithmically (with constant factor equal to one) as a function of the client request arrival rate. Furthermore, it is (theoretically) possible to achieve very close to the minimum required server bandwidth if client receive bandwidth is equal to two times the data streaming rate and client storage capacity is sufficient for buffering data from shared streams. Finally, we propose a new practical delivery technique, called hierarchical multicast stream merging (HMSM), which has a required server bandwidth that is lower than the partitioned dynamic skyscraper and is reasonably close to the minimum achievable required server bandwidth over a wide range of client request rates.

Random Queues in Signalized Road Networks

Average delay is an important criterion of evaluating traffic signals. Traffic signals, stream merges, and stream splits influence vehicular delay in signalized networks through influencing random fluctuation of traffic. This paper proposes a method of estimating the random component of queues and delays at signalized intersections inside a network of urban streets. Equations describing the effect of signals and traffic merging and splitting are derived, as well as a comprehensive equation combining the considered effects. Fixed speeds and capacities are assumed. The derived comprehensive equation is used to discuss certain cases of signalized streets. The equation is evaluated using the Monte Carlo technique. The proposed method can be incorporated into existing procedures for evaluating and optimizing traffic signals.

Competitive Parallel Disk Prefetching and Buffer Management

We provide a competitive analysis framework for online prefetching and buffer management algorithms in parallel I/O systems, using a read-once model of block references. This has widespread applicability to key I/O-bound applications such as external merging and concurrent playback of multiple video streams. Two realistic lookahead models, global lookahead and local lookahead, are defined. Algorithms NOM and GREED, based on these two forms of lookahead are analyzed for shared buffer and distributed buffer configurations, both of which occur frequently in existing systems. An important aspect of our work is that we show how to implement both of the models of lookahead in practice using the simple techniques of forecasting and flushing.Given a D-disk parallel I/O system and a globally shared I/O buffer that can hold up to M disk blocks, we derive a lower bound of Ω(D) on the competitive ratio of any deterministic online prefetching algorithm with O(M) lookahead. NOM is shown to match the lower bound using global M-block lookahead. In contrast, using only local lookahead results in an Ω(D) competitive ratio. When the buffer is distributed into D portions of M/D blocks each, the algorithm GREED based on local lookahead is shown to be optimal, and NOM is within a constant factor of optimal. Thus we provide a theoretical basis for the intuition that global lookahead is more valuable for prefetching in the case of a shared buffer configuration, whereas it is enough to provide local lookahead in the case of a distributed configuration. Finally, we analyze the performance of these algorithms for reference strings generated by a uniformly-random stochastic process and we show that they achieve the minimal expected number of I/Os. These results also give bounds on the worst-case expected performance of algorithms which employ randomization in the data layout.

Correlations between dispersion and structure in porous media probed by nuclear magnetic resonance

Magnetic Resonance Imaging (MRI) and Pulsed Gradient Stimulated Echo (PGSTE) Nuclear Magnetic Resonance (NMR) experiments have been used to probe structure and dispersion in a model porous system formed from a packed bed of spherical particles. The structure of the pore space is characterized from the MRI images by determining radial distribution functions for the pore space. The PGSTE experiments yield an average displacement propagator from which the variance of the average molecular displacement and dispersion coefficient are determined at a range of times over which the flow develops. Experiments have been performed over a wide range of Péclet number and flow velocity. The behavior of the dispersion is shown to be closely related to the structure of the pore space. Dispersion transverse to the direction of superficial flow is dominated by Taylor dispersion. In the axial direction, non-Fickian effects become important, even for flow within a single pore. The axial dispersion coefficient approaches an asymptotic limit characterized by a scale length that corresponds to the pore-to-packing correlation length in the direction of flow, suggesting that the non-Fickian dispersion reaches a limiting behavior after just one mixing between streams merging from two interconnected pores.

CHRONOMERGE: An Application for the Merging and Display of Multiple Time-Stamped Data Streams

CHRONOMERGE is a database application that facilitates merging and display of multiple time-stamped data streams. Each stream is a table containing time-stamped values of one or more parameters (such as a panel of laboratory tests) for multiple patients, and is typically created by querying a clinical data respiratory. The data within a single stream therefore represent a pool of multiple time series. The merge operation is complex because of the numerous options to be considered, such as the granularity of the time interval for merge, and the choice of statistical aggregates. CHRONOMERGE combines multiple streams into a single stream based on patient and time, or time alone (if aggregates are to be computed across patients). It allows specification of various options through a graphical user interface and generates appropriate SQL code (or invokes procedural routines) to perform the merge. The resultant stream, or subsets of it, can then be displayed graphically. CHRONOMERGE is intended to facilitate the analysis of time-stamped data that have been extracted from repositories when standard tools (such as the time-series modules of statistics packages) are inadequate.

Concavity of queueing systems with NBU service times

This paper establishes structural properties for the throughput of a large class of queueing networks with i.i.d. new-better-than-used service times. The main result obtained in this paper is applied to a wide range of networks, including tandems, cycles and fork-join networks with general blocking and starvation (as well as certain networks with splitting and merging of traffic streams), to deduce the concavity of their throughput as a function of system parameters, such as buffer and initial job configurations, and blocking and starvation parameters. These results have important implications for the optimal design and control of such queueing networks by providing exact solutions, reducing the search space over which optimization need be performed, or establishing the convergence of optimization algorithms. In order to obtain results for such disparate networks in a unified manner, we introduce the framework of constrained discrete event systems (CDES), which enables us to characterize any permutable and non-interruptive queueing network through its constraint set. The main result of this paper establishes comparison properties of the event occurrence processes of CDES as a function of the constraint sets, which are then translated into the above-mentioned concavity of the throughput as a function of system parameters in the context of queueing networks.

Concavity of queueing systems with NBU service times

This paper establishes structural properties for the throughput of a large class of queueing networks with i.i.d. new-better-than-used service times. The main result obtained in this paper is applied to a wide range of networks, including tandems, cycles and fork-join networks with general blocking and starvation (as well as certain networks with splitting and merging of traffic streams), to deduce the concavity of their throughput as a function of system parameters, such as buffer and initial job configurations, and blocking and starvation parameters. These results have important implications for the optimal design and control of such queueing networks by providing exact solutions, reducing the search space over which optimization need be performed, or establishing the convergence of optimization algorithms. In order to obtain results for such disparate networks in a unified manner, we introduce the framework of constrained discrete event systems (CDES), which enables us to characterize any permutable and non-interruptive queueing network through its constraint set. The main result of this paper establishes comparison properties of the event occurrence processes of CDES as a function of the constraint sets, which are then translated into the above-mentioned concavity of the throughput as a function of system parameters in the context of queueing networks.

Merging video streams in a multimedia storage server: complexity and heuristics

Due to recent advances in network, storage and data compression technologies, video-on-demand (VOD) service has become economically feasible. It is a challenging task to design a video storage server that can efficiently service a large number of concurrent requests on demand. One approach to accomplishing this task is to reduce the I/O demand to the VOD server through data- and resource-sharing techniques. One form of data sharing is the stream-merging approach proposed in [5]. In this paper, we formalize a static version of the stream-merging problem, derive an upper bound on the I/O demand of static stream merging, and propose efficient heuristic algorithms for both static and dynamic versions of the stream-merging problem.

A three‐dimensional analysis of coextrusion

AbstractThe coextrusion of two polymers in a square channel is presented in this paper. Fluids are fed into the channel through different passages with a plane separating them partway through the channel. Downstream of the separation plane, the streams merge forming an interface between them, which is normally distorted before reaching the exit of the channel. These results show that the degree of distortion depends upon factors in addition to the ratio of the viscosity of the two fluids. The ratio of the flow rates in addition to the shape and location of the separation plane are also important.

Determination of highly concentrated Na, K, Mg and Ca in dialysis solution with flow injection on-line dilution and flame atomic absorption spectrometry

An on-line flow injection dilution method for flame atomic absorption spectrometry was developed. This method is based on the cascade method of dilution. The method combines the technology of merging streams with on-line sample volume reduction using stream splitting to obtain precise dilution factors: a series of split and confluence points are traversed by the injected sample zone. At each split point, the sample volume is reduced and at each confluence point dilution occurs to an extent governed by the relative flow rates in the two merging streams. Using various configurations of the manifold and by changing the flow rate of the carrier stream, dilution factors ranging from 10 to 10 000 are obtained. Normally, to achieve high dilution factors the carrier flow rate must be reduced. However, the dilution performance depends critically on the carrier flow rate; when the latter is low, the throughput is correspondingly reduced as well. In this work, it is shown that a reasonably high dilution factor is achievable when using relatively high carrier flow rates. Higher carrier flow rate provides precise dilution factors with the use of peristaltic pumps. To combine precise performance with high sample throughput, the dilution range, using this method, is recommended at 10-3000. To demonstrate the performance of this manifold, high concentrations of Na, K, Mg and Ca were measured in a dialysis solution. Using a 1000-fold dilution, the sample throughput for a fully automated analysis was 70 measurements per hour, with a precision of approximately 2% (RSD n = 11). The automated results were checked with a conventional manual dilution method and the results between the two methods were comparable and with no significant difference found at the 95% level ( t-test) for analyte concentration range from 64 000 to 450 mg l −1.

Prediction of propagation and non-propagation of short fatigue cracks at notches under mean stresses: Akiniwa, Y., Tanaka, K. and Taniguchi, N.JSME Int. J. July 1990 I 33, (3), 288–296

Fatigue even increases the complexity of the pedestrian dynamics which is regarded as a kind of nonlinear system, and might have a significant negative impact on the crowd evacuation. However, it has never been investigated completely and properly. Thus, the fine discrete floor field cellular automata model is modified by integrating the fatigue function to explore the influence of fatigue on the crowd ascending evacuation. The simulation fits well with the empirical data and the observations quantitatively and qualitatively, indicating the model captures the main features of evacuation considering fatigue impact. As a prediction, without merging streams, compared with the case of walking in constant speed, when fatigue is considered, it takes 71.4% longer for all persons to enter the stairs and 87.2% longer to evacuate. With merging streams, fatigue has little impact on the inflow, while it makes the total evacuation time 84.2% longer.

Extending on-line dilution steady-state concentration range by modification of the merging stream and tandem injection continuous-flow methods

The range of steady-state concentrations, which had been achieved previously by on-line dilution constant-flow methods including flow injection, has been extended by the introduction of new flow configurations. The extent of dilution achieved depends on their combined arrangement. The replacement of constant-flow channels by either completely or partially variable flow channels has extended the dilution range. Moreover, a flexible flow approach for on-line dilution is also achieved. With a totally variable speed flow, an on-line dilution system can be operated at a constant over-all flow-rate over all of the achievable dilutions. Hence the precision of the steady-state concentration section is improved, as the choice of a single efficient coil or coil assembly becomes possible. The most effective means of extending the dilution range is stream flow splitting of the diluted sample produced by the merging stream of tandem injection approach followed by remerging the split stream with the diluent. This combination can in practice provide any desired dilution irrespective of whether constant or partially variable speed flow is adopted. On-line dilution of more than 300 000 is demonstrated. Hydrodynamic injection was as effective in performing tandem injection on-line dilution as conventional tandem injection and yielded identical precision of the steady-state concentration. Each of these dilution techniques can be treated as a module with specific dilution features which can be combined with each other to provide specific steady-state concentration on-line dilution.

On-line dilution, steady-state concentrations by tandem injection and merging stream. Application to inductively coupled plasma atomic emission spectrometry sequential multi-element soil analysis

Recently developed flow injection on-line dilution methods, i.e., the merging stream and tandem injection, which yield steady-state concentrations, have been applied to the analysis of reference soil sample digests by inductively coupled plasma atomic emission spectrometry. Direct or on-line dilution determinations were applied, with dilutions ranging from 1.8 to 15 for both dilution methods. Higher dilutions are not feasible, especially when trace elements are involved. A variable speed peristaltic pump was used to limit the total flow-rate by the merging-stream dilution method to 2 ml min–1. This permitted the design of a flow module with one set of coils and a flow configuration suitable for all dilutions. Varying the dilution required only the variation of the peristaltic pump flow tube diameters. Selecting the required dilution for the tandem injection method was carried out with no manual operations and involved only pre-programming of the injection time and the injection cycle period. The results obtained exhibit comparable accuracy and precision for major and minor elements to those obtained by direct determination. By varying the dilution for the determination of trace elements, screening the optimum dilution(s) suitable for the determination of each trace element is possible.