Real-time Framework Research Articles

A Stethoscope for Drones: Transformers-Based Methods for UAVs Acoustic Anomaly Detection

Unmanned Aerial Vehicles and the increasing variety of their applications are raising in popularity. The growing number of UAVs, emphasizes the significance of drones’ reliability and robustness. Thus, there is a need for an efficient self-observing sensing mechanism to detect real-time anomalies in drone behavior. Previous works suggested prediction models from control theory, yet, they are complex by nature and hard to implement, while Deep Learning solutions are of great utility. In this paper, we propose a real-time framework to detect anomalies in drones by analyzing the sound emitted from them. For this purpose, we construct a hybrid Deep Learning based Transformer and a Convolutional Neural Network inspired by the well-known VGG architecture. Our approach is examined over a dataset that is collected from a single microphone set located on a micro drone in real-time. Our approach achieves an F1-score of 88.4% in detecting anomalies and outperforms the VGG-16 architecture. Moreover, the framework presented in this paper reduces the number of parameters of the well-known VGG-16 from 138M, into a shrunk version with 3.6M parameters only. Additionally, our real-time approach, results in a smaller number of parameters in the neural network, and yet yields high accuracy in anomaly detection in drones with an average inference time of 0.2 seconds per second. Moreover, with an earphone that weighs less than 100 grams on top of the UAV, our method is shown to be beneficial, even in extreme conditions such as a micro-size dataset that is composed of three hours of flight recordings. The presented self-observing method can be implemented by simply adding a microphone to drones and transmitting the captured audio for analysis to the remote control or performing it onboard the drone using a dedicated microcontroller.

NETWORK SIZE, SOCIAL ACCOMPANIMENT, AND GENDER: PROTECTIVE OR RISK FACTOR FOR REAL-TIME LONELINESS, AND FOR WHOM?

Abstract Large personal networks are a key marker of social integration that protect against loneliness. Yet how such social integration shapes loneliness on a momentary basis remains unclear. This study examines (1) the role of personal network size in shaping older adults’ momentary loneliness, (2) how momentary social accompaniment modifies this association (protective or risk factor), and (3) whether this moderation varies by gender. We use three waves of ecological momentary assessments (n = 12,251) and personal network data collected from 333 older adults as part of the Chicago Health and Activity in Real-Time (CHART). Findings from multilevel regression models suggest that older adults with large social networks experience greater intensity loneliness when momentarily alone compared to those with smaller social networks. This association is also more pronounced among men compared to women. Results support our deviation-from-baseline hypothesis, which is further supported by additional analyses that find a similar association between more frequent interaction with network ties and heightened loneliness when alone. Additional analyses provide insight into observed gender differences as well: women’s networks and momentary social company tend to be familial ties characterized by caregiving, compared to men’s ties, which are characterized as peers or companions. Here, moments of momentary social isolation may be a welcome reprieve from otherwise demanding social relationships (“tethering”). We conclude that extending the well-documented links between global measures of social integration and loneliness into a real-time framework offers novel insight into existing, high-level trends in loneliness among older adults.

A Bayesian generalised extreme value model to estimate real-time pedestrian crash risks at signalised intersections using artificial intelligence-based video analytics

Pedestrians represent a vulnerable road user group at signalised intersections. As such, properly estimating pedestrian crash risk at discrete short intervals is important for real-time safety management. This study proposes a novel real-time vehicle-pedestrian crash risk modelling framework for signalised intersections. At the core of this framework, a Bayesian Generalised Extreme Value modelling approach is employed to estimate crash risk in real-time from traffic conflicts captured by post encroachment time. A Block Maxima sampling approach, corresponding to a Generalised Extreme Value distribution, is used to identify pedestrian conflicts at the traffic signal cycle level. Several signal-level covariates are used to capture the time-varying heterogeneity of traffic extremes, and the crash risk of different signal cycles is also addressed within the Bayesian framework. The proposed framework is operationalised using a total of 144 hours of traffic movement video data from three signalised intersections in Queensland, Australia. To obtain signal cycle-level covariates, an automated covariate extraction algorithm is used that fuses three data sources (trajectory database from the video feed, traffic conflict database, and signal timing database) to obtain various covariates to explain time-varying crash risk across different cycles. Results show that the model provides a reasonable estimate of historical crash records at the study sites. Utilising the fitted generalised extreme value distribution, the proposed model provides real-time crash estimates at a signal cycle level and can differentiate between safe and risky signal cycles. The real-time crash risk model also helps understand the differential crash risk of pedestrians at a signalised intersection across different periods of the day. The findings of this study demonstrate the potential for the proposed real-time framework in estimating the vehicle-pedestrian crash risk at the signal cycle level, allowing proactive safety management and the development of real-time risk mitigation strategies for pedestrians.

FRCNN-Based Reinforcement Learning for Real-Time Vehicle Detection, Tracking and Geolocation from UAS

In the last few years, uncrewed aerial systems (UASs) have been broadly employed for many applications including urban traffic monitoring. However, in the detection, tracking, and geolocation of moving vehicles using UAVs there are problems to be encountered such as low-accuracy sensors, complex scenes, small object sizes, and motion-induced noises. To address these problems, this study presents an intelligent, self-optimised, real-time framework for automated vehicle detection, tracking, and geolocation in UAV-acquired images which enlist detection, location, and tracking features to improve the final decision. The noise is initially reduced by applying the proposed adaptive filtering, which makes the detection algorithm more versatile. Thereafter, in the detection step, top-hat and bottom-hat transformations are used, assisted by the Overlapped Segmentation-Based Morphological Operation (OSBMO). Following the detection phase, the background regions are obliterated through an analysis of the motion feature points of the obtained object regions using a method that is a conjugation between the Kanade–Lucas–Tomasi (KLT) trackers and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering. The procured object features are clustered into separate objects on the basis of their motion characteristics. Finally, the vehicle labels are designated to their corresponding cluster trajectories by employing an efficient reinforcement connecting algorithm. The policy-making possibilities of the reinforcement connecting algorithm are evaluated. The Fast Regional Convolutional Neural Network (Fast-RCNN) is designed and trained on a small collection of samples, then utilised for removing the wrong targets. The proposed framework was tested on videos acquired through various scenarios. The methodology illustrates its capacity through the automatic supervision of target vehicles in real-world trials, which demonstrates its potential applications in intelligent transport systems and other surveillance applications.

Estimating the level and direction of aggregated sound change of dialects in the northern Netherlands1

This article reports investigations into sound change at the community-level of Frisian and Low Saxon dialect groups in the north of the Netherlands, which differ in key factors influencing dialect decline. We combine phonetically transcribed corpora with dialectometric approaches that can quantify change among older male dialect speakers in a real-time framework. A multidimensional variant of the Levenshtein distance, combined with methods that induce realistic distances between sounds, is used to estimate how much dialect groups converged to and diverged from Standard Dutch between 1990 and 2010. Our analyses indicate that sound change is a slow process in this geographical area. The Frisian and North Low Saxon dialect groups seem to be most stable, while Westphalian Low Saxon varieties seem to be most prone to change. We offer possible explanations for our findings and discuss shortcomings of the data and approach in detail.

Wastewater-based epidemiology approach: The learning lessons from COVID-19 pandemic and the development of novel guidelines for future pandemics

Wastewater-based epidemiology (WBE) provides a comprehensive real-time framework of population attitude and health status. This approach is attracting the interest of medical community and health authorities to monitor the prevalence of a virus (such as the severe acute respiratory syndrome coronavirus 2, SARS-CoV-2) among a community. Indeed, WBE is currently fine-tuning as environmental surveillance tool for coronavirus disease 2019 (COVID-19) pandemic. After a bibliometric analysis conducted to discover the research trends in WBE field, this work aimed to side-by-side compare the conventional method based on clinical testing with WBE approach. Furthermore, novel guidelines were developed to apply the WBE approach to a pandemic. The growing interest on WBE approach for COVID-19 pandemic is demonstrated by looking at the sharp increase in scientific papers published in the last years and at the ongoing studies on viral quantification methods and analytical procedures. The side-by-side comparison highlighted the ability of WBE to identify the hot-spot areas faster than the conventional approach, reducing the costs (e.g., rational use of available resources) and the gatherings at medical centers. Contrary to clinical testing, WBE has the surveillance capacity for preventing the virus resurgence, including asymptomatic contribution, and ensuring the preservation of medical staff health by avoiding the exposure to the virus infection during clinical testing. As extensively reported, the time in collecting epidemiological data is crucial for establishing the prevention and mitigation measures that are essential for curbing a pandemic. The developed guidelines can help to build a WBE system useful to control any future pandemic.

Co-optimization of velocity planning and energy management for autonomous plug-in hybrid electric vehicles in urban driving scenarios

Co-optimization of vehicle velocity planning and powertrain control for plug-in hybrid electric vehicle (PHEV) can lead to an optimal energy saving with the help of vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) communications. In this study, a real-time hierarchical effective and efficient co-optimization control strategy is designed for automated and connected PHEV to co-optimize vehicle velocity and energy management in urban driving scenarios. In the upper layer, the external traffic disturbance and powertrain characteristics are integrated into velocity planning via a Gaussian process (GP) model and a desired acceleration, respectively. In the power allocation layer, a double delayed Q-learning (DDQL) algorithm is employed to instantaneously optimize the power allocation for powertrain system based on the planned velocity. The feasibility and energy-saving effect of the proposed co-optimization strategy is verified through a traffic-in-the-loop simulator under various urban driving scenarios. The simulation results demonstrate that the integration of traffic lights, powertrain characteristics and speed prediction of preceding vehicle into velocity planning of PHEV can make vehicle velocity smoother, so as to improve fuel economy, driving comfort and traffic efficiency. As coupled with DDQL algorithm, our proposed co-optimization strategy can reach 97.31% energy economy of typical DP-based strategy but in a real-time framework.

Bifurcating Cognitive Attention from Visual Concentration: Utilizing Cooperative Audiovisual Sensing for Demarcating Inattentive Online Meeting Participants

The profuse popularity of video conferencing has led to a simultaneous rise in the opportunity for the participants to multitask. Productive multitasking, such as taking notes, browsing for relevant information, etc., can help promote the cognitive attentiveness of participants. However, existing approaches of tagging inattentive participants solely based on their visual concentration on the meeting app fail to work in such instances. This paper proposes EmotiConf -- a novel real-time framework to monitor participants' attentiveness and a non-real-time framework for visual multitask detection without explicitly relying on their visual concentration. EmotiConf utilizes an unconventional observation where the emotional states of attentive participants, captured through their facial expressions, correlate and also correspond to the vocal expression of the speaker and the intent of the speech. Accordingly, EmotiConf develops a software wrapper to tag the inattentive participants while also characterizing visual multitasking instances performed by them. A thorough evaluation of EmotiConf confirms its usability with a high score of >80.

Identifying port congestion and evaluating its impact on maritime logistics

ABSTRACT Port congestion is an obstacle to smooth energy supply chain management. This research identifies and quantifies the economic implications of congestion in a real-time framework, in which the temporal and geospatial port congestion status is analysed using the vessel tracking information captured by satellites, the Automatic Identification System. A Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is proposed to automatically identify vessel clusters at ports and quantify port turnaround time, and thus real-time port congestion status. A framework is then outlined to analyze the economic implications of congestion for various stakeholders in the system. A case study is conducted on Indian LPG ports, where congestion frequently occurs. The analysis has important implications for the industrial participants engaged in energy transportation to assess the impact, and for policymakers in a better network design.

Timely Estimates of the Monthly Mexican Economic Activity

Abstract In this article, we present a new approach based on dynamic factor models (DFMs) to perform accurate nowcasts for the percentage annual variation of the Mexican Global Economic Activity Indicator (IGAE), the commonly used variable as an approximation of monthly GDP. The procedure exploits the contemporaneous relationship of the timely traditional macroeconomic time series and nontraditional variables as Google Trends with respect to the IGAE. We evaluate the performance of the approach in a pseudo real-time framework, which includes the pandemic of COVID-19, and conclude that the procedure obtains accurate estimates, for one and two-steps ahead, above all, given the use of Google Trends. Another contribution for economic nowcasting is that the approach allows to disentangle the key variables in the DFM by estimating the confidence interval for the factor loadings, hence allows to evaluate the statistical significance of the variables in the DFM. This approach is used in official statistics to obtain preliminary and accurate estimates for IGAE up to 40 days before the official data release.

Real-time plasma horizontal displacement estimator based on the neural network

Plasma displacement diagnostics is one of the basic diagnostics for evaluating the plasma operating state in tokamak devices. It is related to the plasma current centroid and generally obtained by integrating and calculating the induced voltage in the displacement coils. This induced voltage is derived from the magnetic field generated by the plasma. If the induced voltage is not solely due to the plasma magnetic field, there will be an unreal deviation in the plasma displacement. In J-TEXT, this phenomenon usually occurs when the Resonant Magnetic Perturbation (RMP) coils close to the displacement coils are energized. For long pulse tokamak devices, the source of the unreal deviation may be drifting integrators or radiation-induced voltages in the magnetic pickup coils, not only the RMP coils. To solve this problem, this paper proposes a real-time non-magnetic plasma horizontal displacement measurement method using a deep fully connected neural network. The soft-X ray (SXR) diagnostic data and the plasma displacement diagnostic data are used as the training data for the network. The training and testing results show that the network obtains a satisfactory fitting effect. On the other hand, to deploy this network for real-time diagnostics, this paper has designed and developed a plasma horizontal displacement estimator based on J-TEXT real-time framework (JRTF) in J-TEXT. The experimental results show that the real-time estimator is sufficient for real-time plasma displacement diagnostics. The feasibility of establishing real-time plasma displacement diagnostics based on the SXR diagnostic system has been verified.

A flatter curve affords hospitals greater time to prepare for a pandemic surge

This paper quantifies the benefits of flattening the curve (with a constant total patient load over the study period) on the risk of a hospital bed shortage in a pandemic. Using discrete-event simulation of patient care paths in hospitals, synthetic data that eliminates issues of confounding affects from the simultaneous occurrence of regional response actions and/or changes in resources, treatments or other situational circumstances, is produced for estimating hospital capacity for pandemic response. Results from systematically designed numerical experiments produced several findings. These include that the higher the acceleration in pandemic patient demand growth, the greater the impact of the intervention. Cutting this acceleration by 75% from the greatest studied rate created over four additional weeks to prepare for an 80% risk of running out of intensive care beds. Additionally, the greater the acceleration in growth, the fewer the days with a high risk of running out of beds, but the greater the total number of critical patients that could not be served with existing resources. Finally, the lower this acceleration, the fewer resources or modifications needed to cope with the surge, but the longer they are needed. The findings further show how hospitals can benefit from analytical tools that exploit digital health information to predict and plan for need levels and time to onset of these levels. These tools can be embedded within a real-time framework in which automated and early warnings can inform the selection of strategies for managing or coping with expected increases in demand for emergency hospital services.

(Digital Presentation) Cooperative Radial Corrosion- and Pressure-Assisted Stress Buildup in High Pressure Pipes

The corrosion in the pressurized pipelines catastrophic event, specifically in the light water reactors, ultimately leading to the failure and fracture. We develop a real-time framework for the accumulation of compressive stresses due to both corrosion (i.e. plastic) and external pressure (i.e. elastic) events. In this regard, we quantify the infiltration of the oxygen in the metal matrix within the curved boundary and establish the stoichiometrically equivalent resulted oxide thickness. Subsequently, we quantify the real-time augmentation in the volume due to oxide formation and we analytically calculate the resulted stress build-up. Combining the build-up stress with the original stress built-up due to external pressure provides the total imposed compressive stress into the material as the estimate for the onset of the failure. The developed analytical framework could be utilized for estimating the design parameters for safe operation of transport pipes, particularly in applications related to the high-pressure and highly-corrosive environments. Figure 1

Moving Object Detection in Dynamic Environment via Weighted Low-Rank Structured Sparse RPCA and Kalman Filtering

As a classical problem for computer vision, moving object detection (MOD) can be efficiently achieved by foreground and background separation. The Robust Principal Component Analysis (RPCA)-based method has been potentially utilized to solve the problem. However, the detection accuracy for RPCA-based method is limited for complex scenes with slow-motion. Besides, it is time consuming for the way to seek for background modeling based on solving a low-rank minimization problem, for which multiple frames of the videos are required as the input. Therefore, a real-time MOD framework (LSRPCA_KF) is proposed for the dynamic background, where a weighted low-rank and structured sparse RPCA algorithm is used to achieve background modeling for history data, while the online MOD is achieved by the background subtraction method and updated by the Kalman filter for every real time frame. Specifically, for the background model, a newly designed weight is incorporated to distinguish the significance of different singular values, and a structured sparse prior is added to penalty the spatial connection property of the moving object. Besides, the weighted low-rank and structured sparse RPCA model is efficiently solved by the Alternating Direction Method of Multipliers (ADMM) optimization algorithm. Experimental results demonstrated that better performance of our method with significantly reduced delay in processing and better detect the moving object has been achieved, especially for the dynamic background.

Application of ITER real-time framework in EAST radiation power calculation subsystem

To apply the RTF (real-time framework) of ITER in EAST control, the radiation power calculation subsystem independent of the PCS (plasma control system) was designed and implemented based on RTF. Four RTF function blocks are designed to accomplish data processing and radiation power calculation, namely: communication function block, data acquisition function block, radiation calculation function block, and data storage function block. In order to satisfy the needs of EAST long pulse operation, the real-time segmental data storage of MDSplus is applied, which complements the reading and writing function of RTF to MDSplus. And a GUI (graphics user interface) written in Python is integrated in the system for real-time data display and parameter configuration. The benchmark test using history data and the piggyback experiment were carried out, the test result verified the effectiveness of each function blocks and the availability of hardware devices. In the 2021 EAST operation campaign, the subsystem was successfully applied to calculate the radiation power for real-time feedback control in EAST PCS, which provided strong supports for divertor physics experiment and illustrated the first successful attempt of RTF on EAST.

Real-Time Framework for Monitoring Cyber Disruptions in Power Grids

This article presents a novel real-time attack monitoring framework to locate various attacks and identify postattack operating conditions (OCs) using enormous data measured by phasor measurement units (PMU's). Data Mining based online divisive-agglomerative time-series data stream clustering along with data reduction and classification techniques are employed to detect attacks and determine their location by monitoring a minimal number of measuring units. Attack severity and criticality indices are formulated to assess the severity of the attack and identify postattack critical elements in the system to prevent the attack resulting in a large failure. The proposed methodology is tested on IEEE 24 Bus reliability test system and 118 Bus test systems using simulated data considering attacks on different system elements under varying OCs. Additionally, the robustness and real-time applicability of the proposed attack detection methodology is verified, and its strengths and limitations are discussed.

Implementation Paper on Identify Citizens Receiving Multiple Benefits Like Pension Under Different Schemes of the Central and State

Abstract: We build an effective, realistic retirement income plan for an individual investor. We propose real time frameworks with such mechanism that investor give some inputs such as number of investment instruments, income, and length of the time period before retirement using Modern Portfolio theory. This aim to develop a retirement framework using fundamentals of Modern Portfolio Theory as per investor’s needs on asset allocation assuming investor’s risk appetite reduces as investor ages in life and worries for real retirement income planning by comparing different statistical models scenarios. In each of the Scenarios we have 3 changing probability profile scenarios to allow for flexibility to the investor to withdraw from the portfolio for personal needs with increasing probability, decreasing probability and uniform probability of withdrawal throughout the portfolio investment time horizon. The results clearly reveal that there is no one best model for different investors as each investor is different with different objective functions. Keywords: Retirement, Social Securit, Modern Portfolio theory

Top-Down System for Multi-Person 3D Absolute Pose Estimation from Monocular Videos.

Two-dimensional (2D) multi-person pose estimation and three-dimensional (3D) root-relative pose estimation from a monocular RGB camera have made significant progress recently. Yet, real-world applications require depth estimations and the ability to determine the distances between people in a scene. Therefore, it is necessary to recover the 3D absolute poses of several people. However, this is still a challenge when using cameras from single points of view. Furthermore, the previously proposed systems typically required a significant amount of resources and memory. To overcome these restrictions, we herein propose a real-time framework for multi-person 3D absolute pose estimation from a monocular camera, which integrates a human detector, a 2D pose estimator, a 3D root-relative pose reconstructor, and a root depth estimator in a top-down manner. The proposed system, called Root-GAST-Net, is based on modified versions of GAST-Net and RootNet networks. The efficiency of the proposed Root-GAST-Net system is demonstrated through quantitative and qualitative evaluations on two benchmark datasets, Human3.6M and MuPoTS-3D. On all evaluated metrics, our experimental results on the MuPoTS-3D dataset outperform the current state-of-the-art by a significant margin, and can run in real-time at 15 fps on the Nvidia GeForce GTX 1080.

Framework for near real-time forest inventory using multi source remote sensing data

AbstractForestry inventory update is a critical component of sustainable forest management, requiring both the spatially explicit identification of forest cover change and integration of sampled or modelled components like growth and regeneration. Contemporary inventory data demands are shifting, with an increased focus on accurate attribute estimation via the integration of advanced remote sensing data such as airborne laser scanning (ALS). Key challenges remain, however, on how to maintain and update these next-generation inventories as they age. Of particular interest is the identification of remotely sensed data that can be applied cost effectively, as well as establishing frameworks to integrate these data to update information on forest condition, predict future growth and yield, and integrate information that can guide forest management or silvicultural decisions such as thinning and harvesting prescriptions. The purpose of this article is to develop a conceptual framework for forestry inventory update, which is also known as the establishment of a ‘living inventory’. The proposed framework contains the critical components of an inventory update including inventory and growth monitoring, change detection and error propagation. In the framework, we build on existing applications of ALS-derived enhanced inventories and integrate them with data from satellite constellations of free and open, analysis-ready moderate spatial resolution imagery. Based on a review of the current literature, our approach fits trajectories to chronosequences of pixel-level spectral index values to detect change. When stand-replacing change is detected, corresponding values of cell-level inventory attributes are reset and re-established based on an assigned growth curve. In the case of non–stand-replacing disturbances, cell estimates are modified based on predictive models developed between the degree of observed spectral change and relative changes in the inventory attributes. We propose that additional fine-scale data can be collected over the disturbed area, from sources such as CubeSats or remotely piloted airborne systems, and attributes updated based on these data sources. Cells not identified as undergoing change are assumed unchanged with cell-level growth curves used to increment inventory attributes. We conclude by discussing the impact of error propagation on the prediction of forest inventory attributes through the proposed near real-time framework, computing needs and integration of other available remote sensing data.

Gradient Flow Evolution for 3D Fusion From a Single Depth Sensor

We present a novel real-time framework for non-rigid 3D reconstruction that is robust to noise, camera poses, and large deformation from a single depth camera. KinectFusion has achieved high-quality 3D object reconstructions in real-time by implicitly representing an object’s surface with a signed distance field (SDF) representation from a single depth camera. Many studies for incremental reconstruction have been presented since then, with the surface estimation improving over time. Previous works primarily focused on improving conventional SDF matching and deformation schemes. In contrast to these works, the proposed framework tackles the problem of temporal inconsistency caused by SDF approximation and fusion to manipulate SDFs and reconstruct a target more accurately over time. In our reconstruction pipeline, we introduce a refinement evolution method, where an erroneous SDF from a depth sensor is recovered more accurately in a few iterations by propagating erroneous SDF values from the surface. Reliable gradients of refined SDFs enable more accurate non-rigid tracking of a target object. Furthermore, we propose a level-set evolution for SDF fusion, enabling SDFs to be manipulated stably in the reconstruction pipeline over time. The proposed methods are fully parallelizable and can be executed in real-time. Qualitative and quantitative evaluations show that incorporating the refinement and fusion methods into the reconstruction pipeline improves 3D reconstruction accuracy and temporal reliability by avoiding cumulative errors over time. Evaluation results show that our pipeline results in more accurate reconstruction that is robust to noise and large motions, as well as outperforms previous state-of-the-art reconstruction methods.