Real-time Deadlines Research Articles

A Foundation for Real-time Applications onFunction-as-a-Service

Serverless (or Function-as-a-Service) compute model enables new applications with dynamic scaling. However, all current Serverless systems are best-effort, and as we prove this means they cannot guarantee hard real-time deadlines, rendering them unsuitable for such real-time applications. We analyze a proposed extension of the Serverless model that adds a guaranteed invocation rate to the serverless model called Real-time Serverless. This approach aims to meet real-time deadlines with dynamically allocated function invocations. We first prove that the Serverless model does not support real-time guarantees. Next, we analyze Real-time Serverless, showing it can guarantee application real-time deadlines for rate-monotonic real-time workloads. Further, we derive bounds on the required invocation rate to meet any set of workload runtimes and periods. Subsequently, we explore an application technique, pre-invocation, and show that it can reduce the required guaranteed invocation rate. We derive bounds for the feasible rate guarantee reduction, and corresponding overhead in wasted compute resources. Finally, we apply the theoretical results to improve the experience quality of a distributed virtual reality/ augmented reality application as well as simplify the application design and resource management.

BDPC: Controlling application delay in 6TiSCH networks for the industrial Internet of Things

One of the essential requirements of wireless industrial Internet of Things (IoT) systems is to have an extremely high packet delivery rate, generally over 99.9% and comply with realtime deadline constraints. In industrial IoT networks, packets arriving after the deadline become part of packet loss and lose meaning when they arrive late. However, currently available industrial IoT proposals aim to minimize End-to-End delay without taking into account simultaneous realtime and reliability constraints. In this paper, we propose a new mechanism, called BDPC (Bounded Delay Packet Control) to tackle this challenge. BDPC combines the knowledge of a node’s traffic delay to the destination (root) with the time budget of a data packet traversing the industrial IoT network, to allocate network resources to comply the system maximum delay requirements using an adaptive and distributed algorithm. Unlike the general aim to minimze end-to-end delay, we propose that data packets must arrive before the deadline, but not faster. Our results show, for example, that by using BDPC, the number of packets arriving before the deadline can be improved more than 2.6 times compared to the case when using the default Minimal Scheduling Function from the standard. As a further advantage, BDPC involves minor modifications to the 6TiSCH protocol stack, which makes it compatible with current implementations.

Optimize Image Processing Algorithm on ARM Cortex-A72 and A53

This work presents a technique to optimize processing image algorithms. The increasing demand for video applications like context-aware computing on mobile embedded systems requires the use of computationally intensive image processing algorithms. The system engineer has the mandate to optimize them so as to meet real-time deadlines. A methodology to take advantage of the asymmetric dual-core processor, which includes an ARM supported by shared memory, is presented with implementation details. The target platform chosen is the NanoPi M4V2. It has a dual-core and quad-core architecture with an ARM Cortex-A72 and Cortex-A53. The basic image correlation algorithm is chosen for benchmarking as it finds widespread application for various template-matching tasks such as face recognition. The basic algorithm prototypes conform to OpenCV, a popular computer vision library. OpenCV algorithms can be easily ported to the ARM core, which runs a popular operating system such as Linux. The algorithms are tested on a variety of images, and performance results are presented, measuring the speedup obtained due to dual-core and quad-core implementation. A major advantage of this approach is that it allows the ARM processor to perform important real-time tasks.

Botnets and how to automatic detect them: exploring new ways of dealing with botnet classification

Threats such as Botnets have become very popular in the current usage of the Internet, such as attacks like distributed denial of services (DoS) which can cause a significant impact on the use of technology. One way to mitigate such issues can be a focus on using intelligent models that can attempt to identify the existence of Botnets in the network traffic early. Thus, this work aims to evaluate the current state of the art on threats related to Botnets and how intelligent technology has been used in real-world restrictions such as real-time deadlines and increased network traffic. From our findings, we have indications that Botnet detection in real-time still is a more significant challenge because the computation power has not grown at the same rate that Internet traffic. This has pointed out other restrictions that must be considered, like privacy legislation and employing cryptography methods for all communications. In this context, we discuss the following steps to deal with the identified issues.

Rational Allocation of Guaranteed Time Slots to Support Real-Time Traffic in Wireless Body Area Networks

Background: The main requirement of Wireless Body Area Network (WBAN) is on-time delivery of vital signs that are sensed through the delay-sensitive biological sensors that are implanted in the body of the patient being monitored by the central gateway. The Medium Access Control (MAC) protocol standard IEEE 802.15.4 supports real-time data delivery through its unique feature called Guaranteed Time Slot (GTS) under its beacon-enabled mode. This protocol is considered suitable for the WBAN Scenario. However, as per the standard, IEEE 802.15.4 uses a simple and straightforward First Come First Served (FCFS) mechanism to distribute GTS slots among the contender nodes. This kind of blindfolded allocation of GTS slots results in poor utilization of bandwidth and also prevents the delay-sensitive sensor nodes from effectively utilizing the contention-free slots. Objective: The main objective of this work is to provide a solution for the unfair allocation of GTS slots in the beacon-enabled mode of the IEEE 802.15.4 standard in WBAN. Method: We propose a Rational Allocation of Guaranteed Time Slot (RAGTS) protocol that distributes the available GTS slots based on the delay-sensitivity of the contender nodes. Results: A series of simulation experiments has been performed to assess the performance of our proposed RAGTS protocol. The simulations capture the dynamic nature of the real-time deadlines associated with sensor traffic. Through simulations, we show that our proposed RAGTS protocol appears to be more stable in terms of various performance metrics than that of the FCFS nature of the GTS allocation technique. Conclusion: In this article, we introduced the RAGTS scheme that enhances the real-time traffic feature of the beacon-enabled mode of IEEE 802.15.4 MAC.

Anytime parallel tempering

Developing efficient MCMC algorithms is indispensable in Bayesian inference. In parallel tempering, multiple interacting MCMC chains run to more efficiently explore the state space and improve performance. The multiple chains advance independently through local moves, and the performance enhancement steps are exchange moves, where the chains pause to exchange their current sample amongst each other. To accelerate the independent local moves, they may be performed simultaneously on multiple processors. Another problem is then encountered: depending on the MCMC implementation and inference problem, local moves can take a varying and random amount of time to complete. There may also be infrastructure-induced variations, such as competing jobs on the same processors, which arises in cloud computing. Before exchanges can occur, all chains must complete the local moves they are engaged in to avoid introducing a potentially substantial bias (Proposition 1). To solve this issue of randomly varying local move completion times in multi-processor parallel tempering, we adopt the Anytime Monte Carlo framework of (Murray, L. M., Singh, S., Jacob, P. E., and Lee, A.: Anytime Monte Carlo. arXiv preprintarXiv:1612.03319, (2016): we impose real-time deadlines on the parallel local moves and perform exchanges at these deadlines without any processor idling. We show our methodology for exchanges at real-time deadlines does not introduce a bias and leads to significant performance enhancements over the naïve approach of idling until every processor’s local moves complete. The methodology is then applied in an ABC setting, where an Anytime ABC parallel tempering algorithm is derived for the difficult task of estimating the parameters of a Lotka–Volterra predator-prey model, and similar efficiency enhancements are observed.

Cloud-backed mobile cognition

Low-power embedded technology offers a roadmap for enabling deep learning (DL) applications in mobile scenarios, like future autonomous vehicles. However, the lack of breakthrough power efficiency improvements can jeopardize the realization of truly “cognitive” mobile systems that meet real-time deadlines. This work focuses on the new generation cloud-backed mobile cognition system architecture where vehicles execute DL applications with dynamic assistance from the cloud. We unveil opportunities for power-efficient inferencing at the edge through a technique that balances inference execution across the cloud and the vehicle. This level of adaptation results in significant power efficiency improvements compared to all or nothing solutions, where inferences execute either completely on the vehicle or completely in the cloud. In addition, the cloud can have an active role in helping the vehicle to improve its DL capabilities by communicating relevant model updates, with up to 63% bandwidth savings and negligible accuracy degradation when the proposed relevance-driven federated learning technique is used. Finally, the cloud-backed mobile cognition concept is extended to the case of “flying clouds” where vehicles connect to flying drones that provide services while in flight. Although their capabilities are not on par with the stationary cloud, the flying cloud reduces services’ latency significantly and enables critical functionalities.

Real-time Optimisation for Industrial Internet of Things (IIoT): Overview, Challenges and Opportunities

Industrial Internet-of-Things (IIoT) with massive data transfers and huge numbers of connected devices, incombination with the high demand for greater quality-of-services, signal processing is no longer producing small data sets but rather, very large ones (measured in gigabytes or terabytes), or even higher. This has posed critical challenges in the context of optimisation. Communication scenarios such as online applications come with the need for real-time optimisation. In such scenarios, often under a dynamic environment, a strict real-time deadline is the most important requirement to be met. To this end, embedded convex optimisation, which can be redesigned and updated within a fast time-scale given sufficient computing power, is a candidate to deal with the challenges in real-time optimisation applications. Real-time optimisation is now becoming a reality in signal processing and wireless networks of IIoT. Research into new technologies to meet future demands is receiving urgent attention on a global scale, especially when 5G networks are expected to be in place in 2020. This work addresses the fundamentals, technologies and practically relevant questions related to the many challenges arising from real-time optimisation communications for industrial IoT.

SPECTRUM

Wireless communication standards such as Long-term Evolution (LTE) are rapidly changing to support the high data-rate of wireless devices. The physical layer baseband processing has strict real-time deadlines, especially in the next-generation applications enabled by the 5G standard. Existing basestation transceivers utilize customized DSP cores or fixed-function hardware accelerators for physical layer baseband processing. However, these approaches incur significant non-recurring engineering costs and are inflexible to newer standards or updates. Software-programmable processors offer more adaptability. However, it is challenging to sustain guaranteed worst-case latency and throughput at reasonably low-power on shared-memory many-core architectures featuring inherently unpredictable design choices, such as caches and Network-on-chip (NoC). We propose SPECTRUM , a predictable, software-defined many-core architecture that exploits the massive parallelism of the LTE/5G baseband processing workload. The focus is on designing scalable lightweight hardware that can be programmed and defined by sophisticated software mechanisms. SPECTRUM employs hundreds of lightweight in-order cores augmented with custom instructions that provide predictable timing, a purely software-scheduled NoC that orchestrates the communication to avoid any contention, and per-core software-controlled scratchpad memory with deterministic access latency. Compared to many-core architecture like Skylake-SP (average power 215 W) that drops 14% packets at high-traffic load, 256-core SPECTRUM by definition has zero packet drop rate at significantly lower average power of 24 W. SPECTRUM consumes 2.11× lower power than C66x DSP cores+accelerator platform in baseband processing. We also enable SPECTRUM to handle dynamic workloads with multiple service categories present in 5G mobile network (Enhanced Mobile Broadband (eMBB), Ultra-reliable and Low-latency Communications (URLLC), and Massive Machine Type Communications (mMTC)), using a run-time scheduling and mapping algorithm. Experimental evaluations show that our algorithm performs task/NoC mapping at run-time on fewer cores compared to the static mapping (that reserves cores exclusively for each service category) while still meeting the differentiated latency and reliability requirements.

Exploring compression and parallelization techniques for distribution of deep neural networks over Edge–Fog continuum – a review

PurposeThe trend of “Deep Learning for Internet of Things (IoT)” has gained fresh momentum with enormous upcoming applications employing these models as their processing engine and Cloud as their resource giant. But this picture leads to underutilization of ever-increasing device pool of IoT that has already passed 15 billion mark in 2015. Thus, it is high time to explore a different approach to tackle this issue, keeping in view the characteristics and needs of the two fields. Processing at the Edge can boost applications with real-time deadlines while complementing security.Design/methodology/approachThis review paper contributes towards three cardinal directions of research in the field of DL for IoT. The first section covers the categories of IoT devices and how Fog can aid in overcoming the underutilization of millions of devices, forming the realm of the things for IoT. The second direction handles the issue of immense computational requirements of DL models by uncovering specific compression techniques. An appropriate combination of these techniques, including regularization, quantization, and pruning, can aid in building an effective compression pipeline for establishing DL models for IoT use-cases. The third direction incorporates both these views and introduces a novel approach of parallelization for setting up a distributed systems view of DL for IoT.FindingsDL models are growing deeper with every passing year. Well-coordinated distributed execution of such models using Fog displays a promising future for the IoT application realm. It is realized that a vertically partitioned compressed deep model can handle the trade-off between size, accuracy, communication overhead, bandwidth utilization, and latency but at the expense of an additionally considerable memory footprint. To reduce the memory budget, we propose to exploit Hashed Nets as potentially favorable candidates for distributed frameworks. However, the critical point between accuracy and size for such models needs further investigation.Originality/valueTo the best of our knowledge, no study has explored the inherent parallelism in deep neural network architectures for their efficient distribution over the Edge-Fog continuum. Besides covering techniques and frameworks that have tried to bring inference to the Edge, the review uncovers significant issues and possible future directions for endorsing deep models as processing engines for real-time IoT. The study is directed to both researchers and industrialists to take on various applications to the Edge for better user experience.

Trace semantics and refinement patterns for real-time properties in event-B models

• Formal trace semantics and refinement semantics of timed Event-B models. • Prove infinite state trace simulation with Zorn's Lemma. • Formal definition of hiding operator of infinite traces. • Provide generic refinement semantics with proofs. Event-B is a formal method that utilizes a stepwise development approach for system-level modeling and analysis. We are interested in reasoning about real-time deadlines and delays between trigger and response events. There is existing work on treating these properties in Event-B but it lacks a semantic treatment in terms of trace behaviors. Because timing properties require fairness assumptions, we use infinite traces and develop conditions under which all infinite traces of a machine satisfy trigger-response and timing properties. We present refinement semantics of models whose behavior traces are infinite. In addition, we generalize our previous work by allowing a relation between concrete states and abstract states to simulate infinite state traces. Forward simulation, which is a proof technique for refinement, has been used to verify the consistency between different refinement levels regarding finite traces. Based on forward simulation, fairness assumptions, relative deadlock freedom, and conditional convergence are adopted as additional conditions that guarantee infinite trace refinement of timed models. The bounded retransmission protocol is used to illustrate the required proof obligations for timed traces.

MC-Safe

In a Vehicular Cyber Physical System (VCPS), ensuring the real-time delivery of safety messages is an important research problem for Vehicle to Vehicle (V2V) communication. Unfortunately, existing work relies only on one or two pre-selected control channels for safety message communication, which can result in poor packet delivery and potential accident when the vehicle density is high. If all the available channels can be dynamically utilized when the control channel is having severe contention, then safety messages can have a much better chance to meet their real-time deadlines. In this article, we propose MC-Safe, a multi-channel V2V communication framework that monitors all the available channels and dynamically selects the best one for safety message transmission. During normal driving, MC-Safe monitors periodic beacons sent by other vehicles and estimates the communication delay on all the channels. Upon the detection of a potential accident, MC-Safe leverages a novel channel negotiation scheme that allows all the involved vehicles to work collaboratively, in a distributed manner, for identifying a communication channel that meets the delay requirement. MC-safe also features a novel coordinator selection algorithm that minimizes the delay of channel negotiation. Once a channel is selected, all the involved vehicles switch to the same selected channel for real-time communication with the least amount of interference. Our evaluation results both in simulation and on a hardware testbed with scaled cars show that MC-Safe outperforms existing single-channel solutions and other well-designed multi-channel baselines by having a 23.4% lower packet delay on average compared with other well-designed channel selection baselines.

Synthesis of Parallel Synchronous Software

In typical embedded applications, the precise execution time of the program does not matter and it is sufficient to meet a real-time deadline. However, modern applications in information security have become much more time-sensitive due to the risk of timing side-channel leakage. The timing of such programs needs to be data-independent and precise. We describe a parallel synchronous software model, which executes as $N$ parallel threads on a processor with word-length $N$ . Each thread is a single-bit synchronous machine with precise, contention-free timing, while each of the $N$ threads still executes as an independent machine. The resulting software supports fine-grained parallel execution. In contrast to earlier work to obtain precise and repeatable timing in software, our solution does not require modifications to the processor architecture nor specialized instruction scheduling techniques. In addition, all threads run in parallel and without contention, which eliminates the problem of thread scheduling. We use hardware (HDL) semantics to describe a thread as a single-bit synchronous machine. Using logic synthesis and code generation, we derive a parallel synchronous implementation of this design. We illustrate the synchronous parallel programming model with practical examples from cryptography and other applications with precise timing requirements.

A Real-Time Chain and Variable Bulk Arrival and Variable Bulk Service (VBAVBS) Model with λF

This paper proposes a real-time chain and a novel embedded Markovian queueing model with variable bulk arrival (VBA) and variable bulk service (VBS) in order to establish and assure a theoretical foundation to design a blockchain-based real-time system with particular interest in Ethereum. Based on the proposed model, various performances are simulated in a numerical manner in order to validate the efficacy of the model by checking good agreements with the results against intuitive and typical expectations as a baseline. A demo of the proposed real-time chain is developed in this work by modifying the open source of Ethereum Geth 1.9.11. The work in this paper will provide both a theoretical foundation to design and optimize the performances of the proposed real-time chain, and ultimately address and resolve the performance bottleneck due to the conventional block-synchrony by employing an asynchrony by the real-time deadline to some extent.

AdViSED: Advanced Video SmokE Detection for Real-Time Measurements in Antifire Indoor and Outdoor Systems

This paper proposes a video-based smoke detection technique for early warning in antifire surveillance systems. The algorithm is developed to detect the smoke behavior in a restricted video surveillance environment, both indoor (e.g., railway carriage, bus wagon, industrial plant, or home/office) or outdoor (e.g., storage area or parking area). The proposed technique exploits a Kalman estimator, color analysis, image segmentation, blob labeling, geometrical features analysis, and M of N decisor, in order to extract an alarm signal within a strict real-time deadline. This new technique requires just a few seconds to detect fire smoke, and it is 15 times faster compared to the requirements of fire-alarm standards for industrial or transport systems, e.g., the EN50155 standard for onboard train fire-alarm systems. Indeed, the EN50155 considers a response time of at least 60 s for onboard systems. The proposed technique has been tested and compared with state-of-art systems using the open access Firesense dataset developed as an output of a European FP7 project, including several fire/smoke indoor and outdoor scenes. There is an improvement of all the detection metrics (recall, accuracy, F1 score, precision, etc.) when comparing Advanced Video SmokE Detection (AdViSED) with other video-based antifire works recently proposed in literature. The proposed technique is flexible in terms of input camera type and frame size and rate and has been implemented on a low-cost embedded platform to develop a distributed antifire system accessible via web browser.

Latency-Aware Power Management in Software-Defined Radios

Cloud computing provides benefits in terms of equipment consolidation and power savings from higher utilization for virtualizable software. Cellular communication software faces challenges in cloud computing platforms. BSs create a specific load profile that differs from traditional cloud service loads. Cellular communication system implementations have real-time deadlines with fixed, periodic latency requirements. In this paper, we assess the suitability of an unmodified Ubuntu Linux OS running on a commodity server to operate latency-critical software using a 4G LTE BS software-defined radio implementation. Scaling of the CPU clock frequency is shown to be feasible without excessive impact on the platform’s ability to meet the 4 ms processing delay requirement imposed by the LTE standard. Measurements show the relationship between the processor’s operating frequency and the number of missed subframe processing deadlines to be nonlinear. The results obtained also indicate that a high computational capacity does not suffice to ensure satisfactory operation since fronthaul processing overhead can limit achievable performance. Use of offload-capable network interface cards is studied as a potential remedy.

Heterogeneous Scheduling of Deep Neural Networks for Low-power Real-time Designs

Deep neural networks have become the readiest answer to a range of application challenges including image recognition, stock analysis, natural language processing, and biomedical applications such as seizure detection. All while outperforming prior leading solutions that relied heavily on hand-engineered techniques. However, deployment of these neural networks often requires high-computational and memory-intensive solutions. These requirements make it challenging to deploy Deep Neural Networks (DNNs) in embedded, real-time low-power applications where classic architectures, GPUs and CPUs, still impose significant power burden. Systems-on-Chip (SoC) with Field-programmable Gate Arrays (FPGAs) can be used to improve performance and allow more fine-grain control of resources than CPUs or GPUs, but it is difficult to find the optimal balance between hardware and software to improve DNN efficiency. In the current research literature there have been few proposed solutions to address optimizing hardware and software deployments of DNNs in embedded low-power systems. To address the computation resource restriction and low-power needs for deploying these networks, we describe and implement a domain-specific metric model for optimizing task deployment on differing platforms, hardware and software. Next, we propose a DNN hardware accelerator called Scalable Low-power Accelerator for real-time deep neural Networks (SCALENet) that includes multithreaded software workers. Finally, we propose a heterogeneous aware scheduler that uses the DNN-specific metric models and the SCALENet accelerator to allocate a task to a resource based on solving a numerical cost for a series of domain objectives. To demonstrate the applicability of our contribution, we deploy nine modern deep network architectures, each containing a different number of parameters within the context of two different neural network applications: image processing and biomedical seizure detection. Utilizing the metric modeling techniques integrated into the heterogeneous aware scheduler and the SCALENet accelerator, we demonstrate the ability to meet computational requirements, adapt to multiple architectures, and lower power by providing an optimized task to resource allocation. Our heterogeneous aware scheduler improves power saving by decreasing power consumption by 10% of the total system power, does not affect the accuracy of the networks, and still meets the real-time deadlines. We demonstrate the ability to achieve parity with or exceed the energy efficiency of NVIDIA GPUs when evaluated against Jetson TK1 with embedded GPU SoC and with a 4× power savings in a power envelope of 2.0W. When compared to existing FPGA-based accelerators, SCALENet’s accelerator and heterogeneous aware scheduler achieves a 4× improvement in energy efficiency.

A Real-Time Capable Dynamic Partial Reconfiguration System for an Application-Specific Soft-Core Processor

Modern FPGAs (Field Programmable Gate Arrays) are becoming increasingly important when it comes to embedded system development. Within these FPGAs, soft-core processors are often used to solve a wide range of different tasks. Soft-core processors are a cost-effective and time-efficient way to realize embedded systems. When using the full potential of FPGAs, it is possible to dynamically reconfigure parts of them during run time without the need to stop the device. This feature is called dynamic partial reconfiguration (DPR). If the DPR approach is to be applied in a real-time application-specific soft-core processor, an architecture must be created that ensures strict compliance with the real-time constraint at all times. In this paper, a novel method that addresses this problem is introduced, and its realization is described. In the first step, an application-specializable soft-core processor is presented that is capable of solving problems while adhering to hard real-time deadlines. This is achieved by the full design time analyzability of the soft-core processor. Its special architecture and other necessary features are discussed. Furthermore, a method for the optimized generation of partial bitstreams for the DPR as well as its practical implementation in a tool is presented. This tool is able to minimize given bitstreams with the help of a differential frame bitmap. Experiments that realize the DPR within the soft-core framework are presented, with respect to the need for hard real-time capability. Those experiments show a significant resource reduction of about 40% compared to a functionally equivalent non-DPR design.

<italic>RT-ByzCast</italic>: Byzantine-Resilient Real-Time Reliable Broadcast

Today's cyber-physical systems face various impediments to achieving their intended goals, namely, communication uncertainties and faults, relative to the increased integration of networked and wireless devices, hinder the synchronism needed to meet real-time deadlines. Moreover, being critical, these systems are also exposed to significant security threats. This threat combination increases the risk of physical damage. This paper addresses these problems by studying how to build the first real-time Byzantine reliable broadcast protocol (RTBRB) tolerating network uncertainties, faults, and attacks. Previous literature describes either real-time reliable broadcast protocols, or asynchronous (non real-time) Byzantine ones. We first prove that it is impossible to implement RTBRB using traditional distributed computing paradigms, e.g., where the error/failure detection mechanisms of processes are decoupled from the broadcast algorithm itself, even with the help of the most powerful failure detectors. We circumvent this impossibility by proposing RT-ByzCast , an algorithm based on aggregating digital signatures in a sliding time-window and on empowering processes with self-crashing capabilities to mask and bound losses. We show that RT-ByzCast (i) operates in real-time by proving that messages broadcast by correct processes are delivered within a known bounded delay, and (ii) is reliable by demonstrating that correct processes using our algorithm crash themselves with a negligible probability, even with message loss rates as high as 60 percent.

A layered IoT architecture for greenhouse monitoring and remote control

Wireless Sensor Networks have been often used in the context of Greenhouse architectures. In this paper, an architecture is proposed for two Greenhouses based on Networked Control Systems. This architecture is IoT-based and built on top of switched Ethernet and Wi-Fi. Some sensors in the proposed architecture require a one-second real-time deadline. Riverbed simulations prove that there is zero packet loss and no over-delayed packets. An important contribution of this work is the design of a channel allocation scheme that prevents interference in this relatively large Greenhouse system. Another contribution of this work is the introduction of fault tolerance at the controller level. If one controller fails in one of the Greenhouses, the other controller automatically takes over the entire operation of the two-Greenhouse system. Riverbed simulations again show that this fault-tolerant system does not suffer any packet loss or over-delayed packets. Continuous Time Markov Chains are then developed to calculate the reliability as well as the steady state availability of the two-Greenhouse system. The Coverage parameter is taken into account. Finally, a case study is presented to quantitatively assess the advantage of fault tolerance in terms of downtime reduction; this is expected to be attractive especially in developing countries.