Real-time Framework Research Articles

Grassmann manifold based framework for automated fall detection from a camera

In this paper, we propose a Grassmann manifold based novel, real-time framework for automated fall detection in an indoor environment using a single camera. Fall is an activity that is uncontrolled, unintentional, involuntary and can occur while a person is doing any daily living activity. This is especially true for the elderly and the sick, for whom a fall can lead to further complications that may cause irreversible damage to their health. Therefore, it is important to develop a non-intrusive, automated fall detection method such that an alert can be raised in case a fall occurs. We propose a Grassmann manifold based framework for fall detection from a single camera that is also capable of recognizing other daily living activities (DLA) such as walking, sitting, etc. We perform experiments using publicly available datasets and our experimental results show that the fall detection and recognition accuracy of our proposed framework is comparable with the state of the art.

Learning-based supervisory control of dual mode engine-based hybrid electric vehicle with reliance on multivariate trip information

The combination of electrification and advanced dedicated hybrid engines (DHEs) has been attached with promising prospects for the abatement of fuel consumption and emissions. To explore the potential of this solution, experimental data are used from a high-efficiency DHE operated under dual-mode combustion: the lean-burn spark-induced compression ignition (SICI) and the stoichiometric spark ignition (SI). This work endeavors to develop the integrated energy management and engine control system for such DHE-based plug-in hybrid electric vehicles (PHEVs), to this end, the synergy of artificial intelligent control and trip preview can be exploited to boost the control performance, with the engine combustion process being optimally controlled in transient. This paper presents a multistate deep reinforcement learning (M−DRL)-based energy management strategy (EMS) with a discrete–continuous hybrid action space to select optimal combustion mode while output continuous engine power demand, and its state space is expanded to incorporate the multivariate traffic and terrain information processed by a spatio-temporal data processing (STDP) framework in real time. Comprehensive experiments are carried out to validate the capability of the proposed method, based on a standard driving cycle and a real-world driving cycle in China with real-time traffic data recorded. The results indicate that the proposed M−DRL strategy avoids frequent switchover of engine operation modes as well as achieves more than 93% of the dynamic programming’s performance in terms of the fuel consumption and NOx emissions abatement.

A Real-Time Framework for Human Face Detection and Recognition in CCTV Images

This paper aims to develop a machine learning and deep learning-based real-time framework for detecting and recognizing human faces in closed-circuit television (CCTV) images. The traditional CCTV system needs a human for 24/7 monitoring, which is costly and insufficient. The automatic recognition system of faces in CCTV images with minimum human intervention and reduced cost can help many organizations, such as law enforcement, identifying the suspects, missing people, and people entering a restricted territory. However, image-based recognition has many issues, such as scaling, rotation, cluttered backgrounds, and variation in light intensity. This paper aims to develop a CCTV image-based human face recognition system using different techniques for feature extraction and face recognition. The proposed system includes image acquisition from CCTV, image preprocessing, face detection, localization, extraction from the acquired images, and recognition. We use two feature extraction algorithms, principal component analysis (PCA) and convolutional neural network (CNN). We use and compare the performance of the algorithms K-nearest neighbor (KNN), decision tree, random forest, and CNN. The recognition is done by applying these techniques to the dataset with more than 40K acquired real-time images at different settings such as light level, rotation, and scaling for simulation and performance evaluation. Finally, we recognized faces with a minimum computing time and an accuracy of more than 90%.

Smart IoT and Machine Learning-based Framework for Water Quality Assessment and Device Component Monitoring

Water is the most important natural element present on earth for humans, yet the availability of pure water is becoming scarce and decreasing. An increase in population and rise in temperatures are two major factors contributing to the water crisis worldwide. Desalinated, brackish water from the sea, lake, estuary, or underground aquifers is treated to maximize freshwater availability for human consumption. However, mismanagement of water storage, distribution, or quality leads to serious threats to human health and ecosystems. Sensors, embedded and smart devices in water plants require proactive monitoring for optimal performance. Traditional quality and device management require huge investments in time, manual efforts, labour, and resources. This research presents an IoT-based real-time framework to perform water quality management, monitor, and alert for taking actions based on contamination and toxic parameter levels, device and application performance as the first part of the proposed work. Machine learning models analyze water quality trends and device monitoring and management architecture. The results display that the proposed method manages water monitoring and accessing water parameters efficiently than other works.

Model predictive controller for a retrofitted heat exchanger temperature control laboratory experiment

This paper aims to demonstrate the practical aspects of process control theory for undergraduate students at the Department of Chemical Engineering at the University of Bahrain. Both, the ubiquitous proportional integral derivative (PID) as well as model predictive control (MPC) and their auxiliaries were designed and implemented in a real-time framework. The latter was realized through retrofitting an existing plate-and-frame heat exchanger unit that has been operated using an analog PID temperature controller. The upgraded control system consists of a personal computer (PC), low-cost interface using X-transposed-region (XTR) converter, national instruments USB 6008 data acquisition card, and LabVIEW software. LabVIEW control design and simulation modules were used to design and implement the PID and MPC controllers. The performance of the designed controllers was evaluated while controlling the outlet temperature of the retrofitted plate-and-frame heat exchanger. The distinguished feature of the MPC controller in handling input and output constraints was perceived in real-time. From a pedagogical point of view, realizing the theory of process control through practical implementation was substantial in enhancing the student’s learning and the instructor’s teaching experience

Strategic real time framework for healthcare using fuzzy C-means systems

Strategic real time framework for healthcare using fuzzy C-means systems

Co-optimization method of speed planning and energy management for fuel cell vehicles through signalized intersections

Network of vehicles technology can help vehicles obtain more traffic information during their journey. In this paper, a co-optimization method of speed planning and energy management is proposed for fuel cell vehicles (FCVs) while passing through multiple continuous signalized intersections. Based on the off-line dynamic programming (DP) algorithm and the signal phase and timing (SPaT), the constant-speed hydrogen consumption map and the weighted orientation graph of the FCV are established. Afterwards, a bi-level DP method is introduced to solve the speed planning problem online. In the meantime, a modified alternating direction method of multipliers (ADMM) algorithm, which includes a cyclic constraint inspection strategy to constrain the power change rate of fuel cell system (FCS), is used to solve the energy management problem. The simulation results demonstrate that the proposed method can generate an optimal speed reference trajectories for FCV through 4 and 8 signalized intersections in 3 s and 5 s respectively, which is practically implementable on real vehicles. Simulation results show that our proposed method is able to achieve similar fuel saving performance with off-line DP but a in real time framework, which is over 20% better than the improved intelligent driving model (IDM) with ADMM. • Co-optimization method of speed planning and energy management is investigated. • A bi-level DP method is presented to solve the speed planning problem. • The modified ADMM is applied to energy management system of fuel cell vehicles.

The Real-Time Framework of the Push-to-Talk (PTT) Synchronization Scheme for Distributed SAR

Distributed synthetic aperture radar (SAR) has various applications, including multi-angle imaging, SAR tomography, and interferometric SAR. However, these applications are affected by several factors, especially clock synchronization. Recently, an innovative push-to-talk (PTT) scheme was proposed to tackle the clock synchronization problem. As the PTT scheme can directly obtain the time and frequency deviations of the ultra-stable oscillators (USOs), it gives the possibility to achieve the clock synchronization of distributed SAR in real-time. Therefore, this paper builds a real-time framework of the PTT scheme. Compared with the previous clock synchronization module (CSM), an estimation module and a control module are added in this framework to fabricate an improved CSM (ICSM). The time and frequency deviations obtained by the ICSM are used to directly correct the GPS disciplined rubidium clocks. Based on the ICSM, some simulations are performed. The results indicate that the established real-time framework can achieve high precision time and frequency synchronization. In addition, we discuss the system implementation.

Performance and Features: Mitigating the Low-Rate TCP-Targeted DoS Attack via SDN

Software-Defined Networking (SDN) is an emerging network architecture. The decoupled data and control plane provides programmability for efficient network management. However, the centralized control mode of SDN also exposes unique vulnerabilities. Low-rate Denial of Service (LDoS) has a lower attack rate than ordinary DDoS attacks with the characteristics of periodicity and concealment, which is among one of the severe threats to SDN. In this paper, we propose a lightweight, real-time framework Performance and Features (P&F) to detect and mitigate LDoS attacks with SDN. We implement LDoS attacks in SDN, extract traffic features with OpenFlow, and classify the features into two categories. By analyzing the performance (P) of normal traffic under attack state, P&F determines whether LDoS attacks take effect based on machine learning. Meanwhile, P&F tries to locate attack sources and victims according to flow features (F) of LDoS attacks based on time-frequency analysis. According to detection and locating results, P&F sets corresponding mitigation schemes. Experimental results show that P&F has a high detection rate and low false positive rate for detecting LDoS attacks. P&F can deploy on controllers to achieve real-time attack detection and mitigation with low system cost, which can defend against LDoS attacks effectively.

High Accuracy Intelligent Real-Time Framework for Detecting Infant Drowning Based on Deep Learning

High Accuracy Intelligent Real-Time Framework for Detecting Infant Drowning Based on Deep Learning

Real-time user clickstream behavior analysis based on apache storm streaming

This paper presents an approach to analyzing consumers’ e-commerce site usage and browsing motifs through pattern mining and surfing behavior. User-generated clickstream is first stored in a client site browser. We build an ingestion pipeline to capture the high-velocity data stream from a client-side browser through Apache Storm, Kafka, and Cassandra. Given the consumer’s usage pattern, we uncover the user’s browsing intent through n-grams and Collocation methods. An innovative clustering technique is constructed through the Expectation-Maximization algorithm with Gaussian Mixture Model. We discuss a framework for predicting a user’s clicks based on the past click sequences through higher order Markov Chains. We developed our model on top of a big data Lambda Architecture which combines high throughput Hadoop batch setup with low latency real-time framework over a large distributed cluster. Based on this approach, we developed an experimental setup for an optimized Storm topology and enhanced Cassandra database latency to achieve real-time responses. The theoretical claims are corroborated with several evaluations in Microsoft Azure HDInsight Apache Storm deployment and in the Datastax distribution of Cassandra. The paper demonstrates that the proposed techniques help user experience optimization, building recently viewed products list, market-driven analyses, and allocation of website resources.

Drowsiness Classification for Internal Driving Situation Awareness on Mobile Platform

the sleeping driver is potentially more likely to cause an accident than the person who speeds up since the driver is the victim of sleepiness. Automobile industry researchers, including manufacturers, seek to solve this issue with various technical solutions that can avoid such a situation. This paper proposes an implementation of a lightweight method to detect driver's sleepiness using facial landmarks and head pose estimation based on neural network methodologies on a mobile device. We try to improve the accurateness by using face images that the camera detects and passes to CNN to identify sleepiness. Firstly, applied a behavioral landmark's sleepiness detection process. Then, an integrated Head Pose Estimation technique will strengthen the system's reliability. The preliminary findings of the tests demonstrate that with real-time capability, more than 86% identification accuracy can be reached in several real-world scenarios for all classes, including with glasses, without glasses, and light-dark background. This work aims to classify drowsiness, warn, and inform drivers, helping them to stop falling asleep at the wheel. The integrated CNN-based method is used to create a high accuracy and simple-to-use real-time driver drowsiness monitoring framework for embedded devices and Android phones

Joint face detection and Facial Landmark Localization using graph match and pseudo label

Face detection and Facial Landmark Localization (FLL) are not integrated well because training samples with annotations of both bounding box and facial landmarks are costly to get. This paper presents a real-time framework for joint face detection and FLL. We exploit the synergy between the two tasks, then design a fully convolutional network to predict the location of facial landmarks and face regions. Besides, we observe the cluster assumption in FLL and thus propose a progressively pseudo labeling training that not only eliminates the harmful effect caused by inaccurate/noisy annotations, but also makes full use of exact, inexact and coarse-grained labels. To complete the bottom-up face assembly after model inference, we proposed two graph matching algorithms without learnable parameters. The whole framework keeps end-to-end attribute. Extensive experiments show that our approaches achieve state-of-the-art level on face detection and FLL in the different datasets, both in accuracy and runtime.

Analysis and implementation of realtime stock prediction using reinforcement frameworks

Real time stock prediction is interesting research topic due to the risk involved with volatile scenarios. Modelling of the stocks by reducing the overestimation in ANN model, due to rapid fluctuations in the market guide fund managers risky decisions while building stock portfolio. This paper builds real time framework for stock prediction using deep reinforcement learning to buy, sell or hold the stocks. This paper models the transformed stock tick data and technical indicators using Transformed Deep-Q Learning. Our framework is cost reduced and transaction time optimized to get real time stock prediction using GPU and Memory containers. Stock predictor is architected using GRPC based clean architecture which has the benefits of easy updates, addition of new services with reduced integration costs. Data archive features of the cloud will give benefit of reduced cost of the new stock predictor framework.

Long-Range Augmented Reality with Dynamic Occlusion Rendering.

Proper occlusion based rendering is very important to achieve realism in all indoor and outdoor Augmented Reality (AR) applications. This paper addresses the problem of fast and accurate dynamic occlusion reasoning by real objects in the scene for large scale outdoor AR applications. Conceptually, proper occlusion reasoning requires an estimate of depth for every point in augmented scene which is technically hard to achieve for outdoor scenarios, especially in the presence of moving objects. We propose a method to detect and automatically infer the depth for real objects in the scene without explicit detailed scene modeling and depth sensing (e.g. without using sensors such as 3D-LiDAR). Specifically, we employ instance segmentation of color image data to detect real dynamic objects in the scene and use either a top-down terrain elevation model or deep learning based monocular depth estimation model to infer their metric distance from the camera for proper occlusion reasoning in real time. The realized solution is implemented in a low latency real-time framework for video-see-though AR and is directly extendable to optical-see-through AR. We minimize latency in depth reasoning and occlusion rendering by doing semantic object tracking and prediction in video frames.

SFPD: Simultaneous Face and Person Detection in Real-Time for Human-Robot Interaction.

Face and person detection are important tasks in computer vision, as they represent the first component in many recognition systems, such as face recognition, facial expression analysis, body pose estimation, face attribute detection, or human action recognition. Thereby, their detection rate and runtime are crucial for the performance of the overall system. In this paper, we combine both face and person detection in one framework with the goal of reaching a detection performance that is competitive to the state of the art of lightweight object-specific networks while maintaining real-time processing speed for both detection tasks together. In order to combine face and person detection in one network, we applied multi-task learning. The difficulty lies in the fact that no datasets are available that contain both face as well as person annotations. Since we did not have the resources to manually annotate the datasets, as it is very time-consuming and automatic generation of ground truths results in annotations of poor quality, we solve this issue algorithmically by applying a special training procedure and network architecture without the need of creating new labels. Our newly developed method called Simultaneous Face and Person Detection (SFPD) is able to detect persons and faces with 40 frames per second. Because of this good trade-off between detection performance and inference time, SFPD represents a useful and valuable real-time framework especially for a multitude of real-world applications such as, e.g., human–robot interaction.

MimicA: A Framework for Self-Learning Companion AI Behavior

We explore fully autonomous companion characters within the context of Real Time Strategy games. Non-player Characters that are controlled by Artificial Intelligence to some degree, have been a feature of Role Playing games for decades. But RTS games rarely have a player avatar, and thus no real companions. The universe of RTS games where both an avatar and a companion character exist is small. Most friendly RTS units are semi-autonomous at best, requiring player micromanagement of their behavior. We present MimicA, a real-time framework to govern AI companion behavior by modeling that of the current player. Built for the Unity engine, MimicA is a learn-by-demonstration framework that differs from existing practices in that the behavior is fully autonomous, does not rely on previous modeling exercises and is designed to be generalized and extensible. We analyze and discuss MimicA through a thirty person user study with our own demonstration game, Lord of Towers. We find that 22 out of 30 participants (73%) indicate they enjoyed the game, and this self-reported enjoyment was on par with “traditional tower defense games”. 63% agree that MimicA controlled NPCs are doing what the player would do while 20% disagree. Similarly, 53% realize the NPCs are learning from the player while 20% do not. We also show that NPC with underlying Decision Tree and Naive Bayes algorithms are better than KNN in making the player realize the learning nature of the NPC.

Predicting approximate seismic responses in multistory buildings from real-time earthquake source information, for earthquake early warning applications

Regional earthquake early warning (EEW) alerts and related risk-mitigation actions are often triggered when the expected value of a ground-motion intensity measure (IM), computed from real-time magnitude and source location estimates, exceeds a predefined critical IM threshold. However, the shaking experienced in mid- to high-rise buildings may be significantly different from that on the ground, which could lead to sub-optimal decision-making (i.e., increased occurrences of false and missed EEW alarms) with the aforementioned strategy. This study facilitates an important advancement in EEW decision-support, by developing empirical models that directly relate earthquake source parameters to resulting approximate responses in multistory buildings. The proposed models can leverage real-time earthquake information provided by a regional EEW system, to provide rapid predictions of structure-specific engineering demand parameters that can be used to more accurately determine whether or not an alert is triggered. We use a simplified continuum building model consisting of a flexural/shear beam combination and vary its parameters to capture a wide range of deformation modes in different building types. We analyse the approximate responses for the building model variations, using Italian accelerometric data and corresponding source parameter information from 54 earthquakes. The resulting empirical prediction equations are incorporated in a real-time Bayesian framework that can be used for building-specific EEW applications, such as (1) early warning of floor-shaking sensed by occupants; and (2) elevator control. Finally, we demonstrate the improvement in EEW alert accuracy that can be achieved using the proposed models.

An eye-tracking based robotic scrub nurse: proof of concept

BackgroundWithin surgery, assistive robotic devices (ARD) have reported improved patient outcomes. ARD can offer the surgical team a “third hand” to perform wider tasks and more degrees of motion in comparison with conventional laparoscopy. We test an eye-tracking based robotic scrub nurse (RSN) in a simulated operating room based on a novel real-time framework for theatre-wide 3D gaze localization in a mobile fashion.MethodsSurgeons performed segmental resection of pig colon and handsewn end-to-end anastomosis while wearing eye-tracking glasses (ETG) assisted by distributed RGB-D motion sensors. To select instruments, surgeons (ST) fixed their gaze on a screen, initiating the RSN to pick up and transfer the item. Comparison was made between the task with the assistance of a human scrub nurse (HSNt) versus the task with the assistance of robotic and human scrub nurse (R&HSNt). Task load (NASA-TLX), technology acceptance (Van der Laan’s), metric data on performance and team communication were measured.ResultsOverall, 10 ST participated. NASA-TLX feedback for ST on HSNt vs R&HSNt usage revealed no significant difference in mental, physical or temporal demands and no change in task performance. ST reported significantly higher frustration score with R&HSNt. Van der Laan’s scores showed positive usefulness and satisfaction scores in using the RSN. No significant difference in operating time was observed.ConclusionsWe report initial findings of our eye-tracking based RSN. This enables mobile, unrestricted hands-free human–robot interaction intra-operatively. Importantly, this platform is deemed non-inferior to HSNt and accepted by ST and HSN test users.

A Context Aware Job Information Real-Time Delivery Framework for Pervasive Environment

Context-aware systems are getting rapid popularity towards enabling a pervasive computing environment over hand-held mobile devices, wearable devices, and Tablets. These systems deliver services tailored to the specific needs and user’s context. The context-aware system services can utilize information about the user’s context to adapt services based on the user’s location. The prime aim of this research paper is to investigate and analyse the critical issues and challenges while delivering the job information to the job seekers when they migrate from one place to another in search of jobs. In addition, the research also tries to investigate the challenges faced by employers/ job providers when they search and recruit the right person for the right jobs. In this paper the issues and challenges investigated and analysed were timeliness of information dissemination, opportunity missing, fake/genuine information, anywhere, anytime accessibility over small handheld devices. After rigorous analysis of the investigated issues and challenges in the existing state of art employment/job information dissemination systems, a real-time context-aware employment information delivery system framework for the pervasive environment is proposed. The research parameters considered are Preference of Job seekers, Location of Jobs, Context, Qualification, CGPA, Experience, everywhere access at any time, and Identity. The paper used both the quantitative and qualitative approaches for proposing a research-based applied solution. Survey of job seekers, Interview of recruiters, and technical observation of the researcher are used for collecting the relevant facts to check the researchability of the issues and challenges. Protégé version 4.3 is used for modelling, Edraw-Max for designing the framework, Google Form for Survey, and Justin mind tool for Prototype design and testing. The framework and its Prototype was examined using a user acceptance test. The acceptance test indicated that 84.6% of target stakeholders accepted the proposed system framework as a new knowledge solution for the aforementioned issues and challenges in the employment sectors