Event Processing Research Articles

Hypergraph-Based Asynchronous Event Processing for Moving Object Classification

Hypergraph-Based Asynchronous Event Processing for Moving Object Classification

Distributed mobile CEP for collaborative social computing

Smartphones have become highly personalized interfaces between people and the technology ecosystem around them. In this sense, they play a key role for a technology shift from the current Internet of Things to a future human-centric paradigm of an Internet of People, automatically adapting smart things and services to the preferences and context of their users. In this paper, we propose the use of Complex Event Processing (CEP) engines deployed in the users’ smartphones granting them context-awareness capabilities in order to react to external stimulus, and enabling them to interact both with smart things and services in the surroundings of the users. With that purpose, we have designed a communication architecture that interconnects CEP engines running on smartphones, providing a framework for building applications for Mobile-based Collaborative Social Computing (MCSC). For that, we make use of previous works of the authors with Digital Avatars, a framework which promotes the use of smartphones for inferring and sharing a unique digital avatar or virtual profile of each user. The resulting framework, which we have called Collaborative CEP, allows to implement complex interactions among users, and between them and the IoT, a common need in Collaborative Social Computing applications. We provide a proof of concept based on the implementation of a Cops and Robbers game to test the expressiveness and correct functioning of the framework, and we evaluate its performance and efficiency.

Rapid Estimation of Single-Station Earthquake Magnitudes with Machine Learning on a Global Scale

ABSTRACT The foundation of earthquake monitoring is the ability to rapidly detect, locate, and estimate the size of seismic sources. Earthquake magnitudes are particularly difficult to rapidly characterize because magnitude types are only applicable to specific magnitude ranges, and location errors propagate to substantial magnitude errors. We developed a method for rapid estimation of single-station earthquake magnitudes using raw three-component P waveforms observed at local to teleseismic distances, independent of prior size or location information. We used the MagNet regression model architecture (Mousavi and Beroza, 2020b), which combines convolutional and recurrent neural networks. We trained our model using ∼2.4 million P-phase arrivals labeled by the authoritative magnitude assigned by the U.S. Geological Survey. We tested input data parameters (e.g., window length) that could affect the performance of our model in near-real-time monitoring applications. At the longest waveform window length of 114 s, our model (Artificial Intelligence Magnitude [AIMag]) is accurate (median estimated magnitude within ±0.5 magnitude units from catalog magnitude) between M 2.3 and 7.6. However, magnitudes above M ∼7 are more underestimated as true magnitude increases. As the windows are shortened down to 1 s, the point at which higher magnitudes begin to be underestimated moves toward lower magnitudes, and the degree of underestimation increases. The over and underestimation of magnitudes for the smallest and largest earthquakes, respectively, are potentially related to the limited number of events in these ranges within the training data, as well as magnitude saturation effects related to not capturing the full source time function of large earthquakes. Importantly, AIMag can determine earthquake magnitudes with individual stations’ waveforms without instrument response correction or knowledge of an earthquake’s source-station distance. This work may enable monitoring agencies to more rapidly recognize large, potentially tsunamigenic global earthquakes from few stations, allowing for faster event processing and reporting. This is critical for timely warnings for seismic-related hazards.

A Conventional Approach Through Complex Event Processing in Intrusion Detection System

A Conventional Approach Through Complex Event Processing in Intrusion Detection System

Blockchain event ordering algorithm based on high-dimensional multi-objective differential evolution

Abstract Since the birth of Bitcoin, blockchain technology has been widely used in many fields, such as finance, supply chain management, smart contracts, etc.and has been highly valued for its decentralized and tamper-proof characteristics. With the expansion of application scenarios, blockchain faces inefficient data processing challenges, particularly in event sequencing and processing. Developing efficient event sequencing algorithms is a research priority to enhance the performance of blockchain systems. This study addresses the efficiency problem in blockchain event sequencing by introducing a high-dimensional multi-objective differential evolution strategy to improve blockchain technology’s performance in various fields. This paper uses the NSGA-III-WA algorithm to efficiently sort blockchain events by combining differential evolution strategies. Compared to the traditional sorting algorithm, this algorithm reduces latency by 30% and improves memory usage by about 20% when dealing with events of the same size, ensuring sorting accuracy. By optimizing the weight vector adjustment strategy, the convergence speed and distribution uniformity of the algorithm and the solution can be optimized. In this paper, the algorithm can effectively improve the efficiency and performance of blockchain event processing, which is critical in optimizing blockchain systems.

The Formation and Implementation of Ethical Norms for Artificial Intelligence in the Soil of the Rule of Law - Focusing on AI Governance

Abstract In the context of today’s fast-reading development of artificial intelligence (AI), the governance model supported by AI technology has become the focus of global attention. This study focuses on the AI governance model based on ethical norms in the construction of the rule of law, with the aim of providing research support for effective AI governance. In this paper, the ethical core and mechanism of AI governance in the soil of the rule of law are studied under the premise of the rule of law, and a risk governance identification model based on the DEMATEL-ISM model is constructed to analyze the risk factors of AI governance. Then, the ethical norms of AI governance are analyzed in depth, and the path of improving the ethical norms of AI governance is explored through the fsQCA method, and the AI ethical norms guidelines are constructed under the basic guideline of adhering to human-centeredness. Finally, the implementation of AI governance ethical norms is carried out, and under AI governance, the average correct rate of event classification and processing accuracy is above 85%, 207 resolved events are added every month, and the cumulative number of resolved events is increased by 2,486. At the same time, the risk factor risk can be seen, proving that the governance model under the participation of AI can effectively improve the efficiency of governance to promote the development and application of AI governance model provides a certain theoretical basis and reference value.

An obsolescence management framework for a defense industry company

Obsolescence issues in complex product lifecycles can cause disruptions in projects, leading to delays in project schedules, significant costs, and loss of prestige, particularly in defense companies. This article aims to address the obsolescence management issues of electronic components in complex product lifecycles in the defense industry with a new framework called the "Shell Structure”. The Shell Structure acts as a shield that safeguards and reinforces electronic components involved in every phase of the product lifecycle from obsolescence problems. The Shell Structure consists of five main processes: project management, research and development, manufacturing and services, supply chain management, and the new obsolescence management process. A new overarching obsolescence management process integrated into these main processes have been designed, specifically tailored for a large-scale defense industry company, using a process map example. The main obsolescence management process encompasses three subprocesses: the development of obsolescence management plan, the development of obsolescence management strategies, and the monitoring of obsolescence management plan. These processes interact in real-time, ensuring that electronic component data flows seamlessly in the context of digitalization utilizing Complex Event Processing System architecture. The obsolescence management-focused framework fosters interdisciplinary collaboration among the main processes of defense companies through the Shell Structure.

Commissioning of the ALICE readout software for LHC Run 3

ALICE (A Large Ion Collider Experiment) is a heavy-ion detector studying the physics of strongly interacting matter and the quarkgluon plasma at the CERN LHC (Large Hadron Collider). During the second long shut-down of the LHC, the ALICE detector was upgraded to cope with an interaction rate of 50 kHz in Pb-Pb collisions, producing in the online computing system (O2) a sustained input throughput of 3.5 TB/s. In the past years, the O2/FLP project built the new data-acquisition system capable of handling this load. It consists of 200 readout nodes, collecting the data transferred from over 8000 detector links to PCs memory by dedicated PCI boards. The readout software manages the hardware and software memory buffers used for DMA and inter-process communication. It initiates the data flow, performs on-the-fly consistency checks, formats the data, reports performance, and finally pushes the data to the local processing pipeline. The output is then sent by the data distribution software over 100Gb/s links to a dedicated event processing farm. The readout software modular design allowed to address the manifold needs faced during the prototyping, installation and commissioning phases, which proved essential from the lab tests to physics production, like file replay and recording, or online multi-threaded LZ4 compression. We will describe the hardware and software implementation of the O2 readout system, and review the challenges met during the commissioning and first months of operation with LHC collisions in 2022.

Video2Haptics: Converting Video Motion to Dynamic Haptic Feedback with Bio-Inspired Event Processing.

In cinematic VR applications, haptic feedback can significantly enhance the sense of reality and immersion for users. The increasing availability of emerging haptic devices opens up possibilities for future cinematic VR applications that allow users to receive haptic feedback while they are watching videos. However, automatically rendering haptic cues from real-time video content, particularly from video motion, is a technically challenging task. In this paper, we propose a novel framework called "Video2Haptics" that leverages the emerging bio-inspired event camera to capture event signals as a lightweight representation of video motion. We then propose efficient event-based visual processing methods to estimate force or intensity from video motion in the event domain, rather than the pixel domain. To demonstrate the application of Video2Haptics, we convert the estimated force or intensity to dynamic vibrotactile feedback on emerging haptic gloves, synchronized with the corresponding video motion. As a result, Video2Haptics allows users not only to view the video but also to perceive the video motion concurrently. Our experimental results show that the proposed event-based processing methods for force and intensity estimation are one to two orders of magnitude faster than conventional methods. Our user study results confirm that the proposed Video2Haptics framework can considerably enhance the users' video experience.

EPN2EOS Data Transfer System

EPN2EOS Data Transfer System

Modular toolsets for integrating HPC clusters in experiment control systems

New particle/nuclear physics experiments require a massive amount of computing power that is only achieved by using high performance clusters directly connected to the data acquisition systems and integrated into the online systems of the experiments. However, integrating an HPC cluster into the online system of an experiment means: Managing and synchronizing thousands of processes that handle the huge throughput. In this work, modular components that can be used to build and integrate such a HPC cluster in the experiment control systems (ECS) will be introduced. The Online Device Control library (ODC) [1] in combination with the Dynamic Deployment System (DDS) [2, 3] and FairMQ [4] message queuing library offers a sustainable solution for integrating HPC cluster controls into an ECS. DDS as part of the ALFA framework [5] is a toolset that automates and significantly simplifies a dynamic deployment of user-defined processes and their dependencies on any resource management system (RMS) using a given process graph (topology). ODC, in this architecture, is the tool to control and communicate with a topology of FairMQ processes using DDS. ODC is designed to act as a broker between a high level experiment control system and a low level task management system e.g.: DDS. In this work the architecture of both DDS and ODC will be discussed, as well as the design decisions taken based on the experience gained of using these tools in production by the ALICE experiment at CERN to deploy and control thousands of processes (tasks) on the Event Processing Nodes cluster (EPN) during Run3 as a part of the ALICE O2 software ecosystem [6].

Towards podio v1.0 - A first stable release of the EDM toolkit

A performant and easy-to-use event data model (EDM) is a key component of any HEP software stack. The podio EDM toolkit provides a user friendly way of generating such a performant implementation in C++ from a high level description in yaml format. Finalizing a few important developments, we are in the final stretches for release v1.0 of podio, a stable release with backward compatibility for datafiles written with podio from then on. We present an overview of the podio basics, and go into slighty more technical detail on the most important topics and developments. These include: schema evolution for generated EDMs, multithreading with podio generated EDMs, the implementation of them as well as the basics of I/O. Using EDM4hep, the common and shared EDM of the Key4hep project, we highlight a few of the smaller features in action as well as some lessons learned during the development of EDM4hep and podio. Finally, we show how podio has been integrated into the Gaudi based event processing framework that is used by Key4hep, before we conclude with a brief outlook on potential developments after v1.0.

Evaluating Performance Portability with the CMS Heterogeneous Pixel Reconstruction code

In the past years the landscape of tools for expressing parallel algorithms in a portable way across various compute accelerators has continued to evolve significantly. There are many technologies on the market that provide portability between CPU, GPUs from several vendors, and in some cases even FPGAs. These technologies include C++ libraries such as Alpaka and Kokkos, compiler directives such as OpenMP, the SYCL open specification that can be implemented as a library or in a compiler, and standard C++ where the compiler is solely responsible for the offloading. Given this developing landscape, users have to choose the technology that best fits their applications and constraints. For example, in the CMS experiment the experience so far in heterogeneous reconstruction algorithms suggests that the full application contains a large number of relatively short computational kernels and memory transfer operations. In this work we use a stand-alone version of the CMS heterogeneous pixel reconstruction code as a realistic use case of HEP reconstruction software that is capable of leveraging GPUs effectively. We summarize the experience of porting this code base from CUDA to Alpaka, Kokkos, SYCL, std::par, and OpenMP offloading. We compare the event processing throughput achieved by each version on NVIDIA and AMD GPUs as well as on a CPU, and compare those to what a native version of the code achieves on each platform.

The SMARTHEP European Training Network

The SMARTHEP European Training Network

Boosting RDataFrame performance with transparent bulk event processing

Boosting RDataFrame performance with transparent bulk event processing

Vectorization of CMSSW offline software

The CMS experiment has been utilizing vectorization, or SIMD, in parts of its data processing applications for over a decade. On x86 platforms the vectorization level is still SSE3. In the past attempts to use wider vector instruction sets such as AVX or AVX-512 have, in practice, not resulted in improvements in the overall event processing throughput, because the CPUs scale down their frequency when processing AVX instructions. In addition, a notable part of the global pool of CMS resources has been old systems either not supporting AVX, or where the CPU frequency downscaling impacts all cores of the CPU. CMS has nevertheless continued to vectorize more of its application code, and in this work we review profiling methods we have found effective to find out pieces of code that would benefit from vectorization, and techniques to transform those codes such that the GCC compiler is able to auto-vectorize those codes. The build system used for CMSSW, Scram, has also been enhanced to be able to build code for multiple CPU microarchitectures such that the shared libraries of desired microarchitecture level can be loaded based on the CPU of the system. This multi-microarchitecture setup is invisible to the workflow management system, which makes its deployment straightforward. We describe in detail how this multimicroarchitecture build is set up, and measure the impact of using wider vector units than SSE3 on the event processing throughput of CMS applications such as simulation and reconstruction on recent x86 CPUs

A dimensions aware evaluator for High Energy Physics applications

A dimensions aware evaluator for High Energy Physics applications

Framework for custom event sample augmentations for ATLAS analysis data

For HEP event processing, data is typically stored in column-wise synchronized containers, such as most prominently ROOT’s TTree, which have been used for several decades to store by now over 1 exabyte. These containers can combine row-wise association capabilities needed by most HEP event processing frameworks (e.g. Athena for ATLAS) with column-wise storage, which typically results in better compression and more efficient support for many analysis use-cases. One disadvantage is that these containers, TTree in the HEP use-case, require to contain the same attributes for each entry/row (representing events), which can make extending the list of attributes very costly in storage, even if those are only required for a small subsample of events. Since the initial design, the ATLAS software framework features powerful navigational infrastructure to allow storing custom data extensions for subsamples of events in separate, but synchronized containers. This allows adding event augmentations to ATLAS standard data products (such as DAOD-PHYS or PHYSLITE) avoiding duplication of those core data products, while limiting their size increase. For this functionality, the framework does not rely on any associations made by the I/O technology (i.e. ROOT), however it supports TTree friends and builds the associated index to allow for analysis outside of the ATLAS framework. A prototype based on the Long-Lived Particle search is implemented and preliminary results with this prototype will be presented. At this point, augmented data are stored within the same file as the core data. Storing them in separate files will be investigated in future, as this could provide more flexibility, e.g. certain sites may only want a subset of several augmentations or augmentations can be archived to tape once their analysis is complete.

TEMPORAL EVENTS PROCESSING MODELS IN FINITE STATE MACHINES

Context. The issue of a synthesizable finite state machine with temporal events processing using hardware description language pattern. The object of this study is external event processing in real-time systems.
 Objective. The goal of this work is to introduce methods to express external temporal events on finite state machine state diagrams and corresponding HDL patterns of such events processing in control systems.
 Method. The classification of external events in real-time systems is analyzed. A device class that changes its internal state depending on the temporal external events is introduced. A method to express these events on the temporal state diagram is introduced. Possible model behavior scenarios based on the external event duration are analyzed. A Verilog HDL external event processing pattern is introduced. The efficiency of the proposed model is proved by developing, verifying, and synthesis of a powersaving module in Xilinx ISE. The results and testing showed the model’s correctness.
 Results. External temporal events processing methods in real-time device models are proposed. The corresponding HDL pattern for the proposed model implementation is presented.
 Conclusions. The real-time systems with external temporal events automated synthesis problem has been solved. To solve this problem, a finite state machine model-based device using the Verilog language was developed and tested. The scientific novelty lies in the introduction a method to express temporal events on the state diagram of the finite state machine as well as in a HDL when implementing the proposed model on CPLD and FPGA.

A Platform for Holistic Embodied Models of Infant Cognition, and Its Use in a Model of Event Processing

A Platform for Holistic Embodied Models of Infant Cognition, and Its Use in a Model of Event Processing