Broker Middleware Research Articles

On the role of message broker middleware for many-task computing on a big-data platform

We have designed and implemented a new data processing framework called “Many-task computing On HAdoop” (MOHA) which aims to effectively support fine-grained many-task applications that can show another type of data-intensive workloads in the YARN-based Hadoop 2.0 platform. MOHA is developed as one of Hadoop YARN applications so that it can transparently co-host existing many-task computing (MTC) applications with other data processing workflows such as MapReduce in a single Hadoop cluster. In this paper, we investigate main characteristics of two well-known open-source message broker middleware systems (Apache ActiveMQ and Kafka) and their implications on a many-task management scheme in our MOHA framework. Through our extensive experiments with a real MTC application, we demonstrate and discuss trade-offs between parallelism and load balancing of data access patterns in message broker middleware systems for Many-Task Computing on Hadoop.

Spatio-temporal multicast grouping for content-based routing in vehicular networks: A distributed approach

Efficient dissemination of vehicular network applications is still a challenging objective in the area of research. Their unique characteristics impose difficulties in proposing a framework to solve the dissemination related problems. Event broker middleware provides asynchronous communication and extensive messaging in heterogeneous environments. This can be integrated with multicast routing in vehicular networks for efficient dissemination. Proposed approach aggregates content-based subscriptions in a compact data format using Binary Decision Diagram and applies Spatio-temporal Multicast Routing Protocol to extend this context from the middleware-tier to construct an optimized dynamic dissemination mesh. The dynamic multicast group is created by aggregating subscriptions, applying clustering. This work has compared the performance of K-Means and hierarchical clustering methods for clustering subscriptions to form multicast groups. Cooperation between middleware-tier and network components allows to represent fine-grained subscriptions. Simulation results support the efficiency of the proposed approach.

Impact of the Use of Object Request Broker Middleware for Inter-Component Communications in C6416 Digital Signal Processor Based Software Communications Architecture Radio Systems

Problem statement: This study presents an in-depth analysis of the performance of Software Communications Architecture (SCA) component-based waveform applications in terms of inter-component communications. The main limitation with SCA, in the context of embedded systems, is the additional cost introduced by the use of Object Request Broker (ORB) middleware. The ORB middleware handles the interaction between components and objects in SCA distributed environment. This interaction should be highly efficient, due to the real time nature of SCA systems and transparent to the application programmer. Approach: We can achieve high efficiency in SCA systems by enhancing the Inter-Process Communications (IPC) mechanisms in Operating systems (OS) micro kernels, while we achieve transparency through Interface Definition Language (IDL). Different encoding mechanisms like “External Data Representation (XDR), Network Data Representation (NDR) and Common Data Representation (CDR) facilitate inter-component communication transparently and efficiently”. Marshalling procedures format data from the local machine representation to common network representations. A most common encoding mechanism for Common Object Request Broker Architecture (CORBA) systems is CDR representation. Measurements have been performed with ORBExpress DSP as a CORBA distribution and Open Source SCA Implementation Embedded (OSSIE) for SCA implementation. In order to perform these measurements we proposed two metrics for profiling the ORB that are invocation and marshalling. In addition, we propose three elements of data types to evaluate the performance of ORB middleware that are, Basic, Array and Sequence data types. Results: The CORBA bus is really the part, which brings an overhead to the SCA radio systems. This overhead is due to method invocations that have been carried out by ORB middleware. Conclusion: Performance benchmarks of ORBExpress DSP middleware show that, although using CORBA for inter-component communications introduces delays and overheads, the overall effect can be reduced by sending packets of data instead of basic data type elements.

Hardware Middleware for Person Tracking on Embedded Distributed Smart Cameras

Tracking individuals is a prominent application in such domains like surveillance or smart environments. This paper provides a development of a multiple camera setup with jointed view that observes moving persons in a site. It focuses on a geometry-based approach to establish correspondence among different views. The expensive computational parts of the tracker are hardware accelerated via a novel system-on-chip (SoC) design. In conjunction with this vision application, a hardware object request broker (ORB) middleware is presented as the underlying communication system. The hardware ORB provides a hardware/software architecture to achieve real-time intercommunication among multiple smart cameras. Via a probing mechanism, a performance analysis is performed to measure network latencies, that is, time traversing the TCP/IP stack, in both software and hardware ORB approaches on the same smart camera platform. The empirical results show that using the proposed hardware ORB as client and server in separate smart camera nodes will considerably reduce the network latency up to 100 times compared to the software ORB.

FCS/nORB: A feedback control real-time scheduling service for embedded ORB middleware

Object Request Broker (ORB) middleware has shown promise in meeting the functional and real-time performance requirements of distributed real-time and embedded (DRE) systems. However, existing real-time ORB middleware standards such as RT-CORBA do not adequately address the challenges of (1) managing unpredictable workload, and (2) providing robust performance guarantees portably across different platforms. To overcome this limitation, we have developed software called FCS/nORB that integrates a Feedback Control real-time Scheduling (FCS) service with the nORB small-footprint real-time ORB designed for networked embedded systems. FCS/nORB features feedback control loops that provide real-time performance guarantees by automatically adjusting the rate of remote method invocations transparently to an application. FCS/nORB thus enables real-time applications to be truly portable in terms of real-time performance as well as functionality, without the need for hand tuning. This paper presents the design, implementation, and empirical evaluation of FCS/nORB. Our extensive experiments on a Linux testbed demonstrate that FCS/nORB can provide deadline miss ratio and utilization guarantees in the face of changes in platform and task execution times, while introducing only a small amount of overhead.

FC-ORB: A robust distributed real-time embedded middleware with end-to-end utilization control

A key challenge for distributed real-time and embedded (DRE) middleware is maintaining both system reliability and desired real-time performance in unpredictable environments where system workload and resources may fluctuate significantly. This paper presents FC-ORB, a real-time Object Request Broker (ORB) middleware that employs end-to-end utilization control to handle fluctuations in application workload and system resources. The contributions of this paper are three-fold. First, we present a novel utilization control service that enforces desired CPU utilization bounds on multiple processors by adapting the rates of end-to-end tasks within user-specified ranges. Second, we describe a set of middleware-level mechanisms designed to support end-to-end tasks and distributed multi-processor utilization control in a real-time ORB. Finally, we present extensive experimental results on a Linux testbed. Our results demonstrate that our middleware can maintain desired utilizations in face of uncertainties and variations in task execution times, resource contentions from external workloads, and permanent processor failure. FC-ORB demonstrates that the integration of utilization control, end-to-end scheduling, and fault-tolerance mechanisms in DRE middleware is a promising approach for enhancing the robustness of DRE applications in unpredictable environments.

ICALB: an integrated congestion avoidance and load balancing algorithm for distributed intelligent networks. Part II: Performance evaluation of ICALB

This paper evaluates the Integrated Congestion Avoidance and Load Balancing (ICALB) algorithm [ICALB: an integrated congestion avoidance and load balancing algorithm for distributed intelligent networks—Part I. Description of ICALB, 2002]. We test its performance by applying it in a simulated Distributed Intelligent Network (D-IN). We evaluate the algorithm ability to protect the service execution nodes from congestion, to maintain a high throughput, and to ensure a fair distribution of the resource's capacity among services taking into account service priorities and quality of service requirements. The algorithm's operation mainly exploits the advantages of Common Object Request Broker Architecture (CORBA) Object Request Broker (ORB) middleware that allow the invocation of operations on distributed objects without concern for object location, programming language, OS platform, communication protocols and interconnects, and hardware. Hence, all necessary operations of the algorithm are assumed to be executed over the CORBA ORB.

Electronic education system model

E-learning 1 1 Electronic learning (e-learning). efforts and experiments currently receive much attention across the globe. The availability of electronic and web-enabling technologies also dramatically influences the way we view the learning strategies of the future [ Kramer, B. J. (2000). Forming a federal virtual university through course broker middleware. In Proceedings: LearnTec 2000. Heidelberg, Germany, 2000. Hiltz, S. R. (1995). Teaching in a virtual classroom. In Proceedings: International conference on computer assisted instruction.(ICCAI'95), Taiwan, March 1995]. However, due to disappointing experiences in wide spread implementation of computers in schools [ Foshay, W. R. (1998). Education technology in schools and in business: a personal experience. Education Horizons, 66(4),154–157 ], many are already predicting the failure of web technologies for learning [ Rogers, A. (2000). The failure and the promise of technology in education. Global SchoolNet Foundation, 27 May 2000 (http://www.gsm.org/teacharticles/promise.html)]. It is indeed likely that e-learning, making use of technological advances such as the Internet, may also be dissatisfying and frustrating unless we design electronic educational models that can avoid potential complications. In this paper, we define and describe an electronic educational system model (EES model). The aim of this model is to assist the designers of different e-learning settings to plan and implement a specific learning situation, with the focus on the individual requirements and milieu of the learning group. The EES model is composed of four layers, each consisting of different objects (components) addressing issues specific to each layer. When constructing a learning situation, the planners, schedulers and facilitators come together with a clear view of their particular learning situation in mind. They then use the EES model to design their course layer by layer, including objects from each layer. Each object consists of one or more methods/strategies to be implemented in order to achieve the learning objectives of the course. This approach promises to increase the chances of successful and quality implementations [Cloete, E. (2000). Quality issues in system engineering affecting virtual distance learning systems. To appear in Proceedings. COMPSAC'2000. Taiwan, October 2000] with as few frustrations and disappointments as possible.

Software architectures for reducing priority inversion and non-determinism in real-time object request brokers

There is increasing demand to extend Object Request Broker (ORB) middleware to support distributed applications with stringent real-time requirements. However, conventional ORB implementations, such as CORBA ORBs, exhibit substantial priority inversion and non-determinism, which makes them unsuitable for applications with deterministic real-time requirements. This paper provides two contributions to the study and design of real-time ORB middleware. First, it illustrates empirically why conventional ORBs do not yet support real-time quality of service. Second, it evaluates connection and concurrency software architectures to identify strategies that reduce priority inversion and non-determinism in real-time CORBA ORBs. The results presented in this paper demonstrate the feasibility of using standard OO middleware like CORBA to support certain types of real-time applications over the Internet.

Supporting high-performance I/O in QoS-enabled ORB middleware

To be an effective platform for high-performance distributed applications, off-the-shelf Object Request Broker (ORB) middleware, such as CORBA, must preserve communication-layer quality of service (QoS) properties both vertically (i.e., network interface${}\leftrightarrow{}$application layer) and horizontally (i.e., end-to-end). However, conventional network interfaces, I/O subsystems, and middleware interoperability protocols are not well-suited for applications that possess stringent throughput, latency, and jitter requirements. It is essential, therefore, to develop vertically and horizontally integrated ORB endsystems that can be (1) configured flexibly to support high-performance network interfaces and I/O subsystems and (2) used transparently by performance-sensitive applications. This paper provides three contributions to research on high-performance I/O support for QoS-enabled ORB middleware. First, we outline the key research challenges faced by high-performance ORB endsystem developers. Second, we describe how our real-time I/O (RIO) subsystem and pluggable protocol framework enables ORB endsystems to preserve high-performance network interface QoS up to applications running on off-the-shelf hardware and software. Third, we illustrate empirically how highly optimized ORB middleware can be integrated with real-time I/O subsystem to reduce latency bounds on communication between high-priority clients without unduly penalizing low-priority and best-effort clients. Our results demonstrate how it is possible to develop ORB endsystems that are both highly flexible and highly efficient.