Event Recognition Research Articles

Deep Learning Approach for Human Action Recognition Using a Time Saliency Map Based on Motion Features Considering Camera Movement and Shot in Video Image Sequences

In this article, a hierarchical method for action recognition based on temporal and spatial features is proposed. In current HAR methods, camera movement, sensor movement, sudden scene changes, and scene movement can increase motion feature errors and decrease accuracy. Another important aspect to take into account in a HAR method is the required computational cost. The proposed method provides a preprocessing step to address these challenges. As a preprocessing step, the method uses optical flow to detect camera movements and shots in input video image sequences. In the temporal processing block, the optical flow technique is combined with the absolute value of frame differences to obtain a time saliency map. The detection of shots, cancellation of camera movement, and the building of a time saliency map minimise movement detection errors. The time saliency map is then passed to the spatial processing block to segment the moving persons and/or objects in the scene. Because the search region for spatial processing is limited based on the temporal processing results, the computations in the spatial domain are drastically reduced. In the spatial processing block, the scene foreground is extracted in three steps: silhouette extraction, active contour segmentation, and colour segmentation. Key points are selected at the borders of the segmented foreground. The last used features are the intensity and angle of the optical flow of detected key points. Using key point features for action detection reduces the computational cost of the classification step and the required training time. Finally, the features are submitted to a Recurrent Neural Network (RNN) to recognise the involved action. The proposed method was tested using four well-known action datasets: KTH, Weizmann, HMDB51, and UCF101 datasets and its efficiency was evaluated. Since the proposed approach segments salient objects based on motion, edges, and colour features, it can be added as a preprocessing step to most current HAR systems to improve performance.

Safety of cardiopulmonary exercise testing in a large contemporary cohort of adults with congenital heart disease: implications on levels of test supervision

Abstract Background Exercise intolerance is common in adults with congenital heart disease (ACHD). Cardiopulmonary exercise testing (CPET) is used routinely as an objective assessment of the exercise capacity, the severity and mechanisms of exercise intolerance, and the effect of intervention. Little is known on the safety of CPET in ACHD, resulting in the American Heart Association recommending physician supervision of all patients with intracardiac shunts. Purpose To assess the safety of CPET in ACHD. Methods We retrospectively examined 9309 CPETs performed on ACHD patients in a tertiary referral Centre over 20 years to document major or minor events. All tests were performed in-hospital, with an emergency alarm system on site, and supervised routinely by 2 clinical exercise physiologists. Major events were classified as death within 48 hours and cardiac arrest, syncope, sustained/symptomatic ventricular tachycardia (VT) and sustained/symptomatic supraventricular tachycardia (SVT) during the test. Minor events were classified as non-sustained/asymptomatic VT, non-sustained/asymptomatic SVT, bradyarrhythmia, chest pain and significant ST segment ECG changes, new onset of left or right bundle branch block). Results During the study period, 9309 CPETs were performed on 4941 ACHD patients (age 33.0±14.7 years, 52.3% male), of varying complexity by Bethesda classification (18.3% simple, 51.2% moderate, 27.4% great) Figure 1. A major event occurred in 56 (0.6%) tests and a minor event in 870 (9.3%) tests. There were no deaths within 48 hours, nor cardiac arrests during the test. There were 6 (0.1%) cases of sustained or symptomatic VT, 4 (<0.1%) syncopal episodes and 46 (0.5%) instances of sustained or symptomatic SVT. A total of 31 (0.3%) episodes of bradyarrhythmia, 377 (4%) significant ECG changes and 267 (2.8%) episodes of chest pain were recorded. Moreover, there were 107 (1.2%) cases of non-sustained or asymptomatic VT and 88 (1%) cases of non-sustained or asymptomatic SVT. Patient fatigue (3701, 40%), shortness of breath (2246, 24.1%) and leg discomfort (1286, 13.8%) were common reasons for stopping the test, whereas only 95 (1%) tests were stopped due to chest pain. Conclusions Our data on a large ACHD population undergoing CPET demonstrate a potential risk for arrhythmic complications during exercise and recovery, mostly due to sustained or symptomatic SVT, while the risk of all the other major cardiovascular events is low. Our data support the use of CPET in routine ACHD practice, with quick recognition and management of potential adverse events in a controlled environment by appropriately trained cardiac physiologists.

Spatio-temporal human action localization in indoor surveillances

Spatio-temporal action localization is a crucial and challenging task in the field of video understanding. Existing benchmarks for spatio-temporal action detection are limited by factors such as incomplete annotations, high-level non-universal actions, and uncommon scenarios. To address these limitations and facilitate research in real-world security applications, we introduce a novel human-centric dataset for spatio-temporal localization of atomic actions in indoor surveillance settings, termed as HIA (Human-centric Indoor Actions). The HIA dataset is constructed by selecting 30 atomic action classes, compiling 100 surveillance videos, and annotating 219,225 frames with 370,937 bounding boxes. The primary characteristics of HIA include (1) accurate spatio-temporal annotations for atomic actions, (2) human-centric annotations at the frame level, (3) temporal linking of persons across discontinuous tracks, and (4) utilization of indoor surveillance videos. Our HIA, with its realistic settings in indoor surveillance scenes and comprehensive annotations, presents a valuable and novel challenge to the spatio-temporal action localization domain. To establish a benchmark, we evaluate various methods and provide an in-depth analysis of the HIA dataset. The HIA dataset will be made available soon, and we anticipate that it will serve as a standard and practical benchmark for the research community.

Detection of Unauthorized Actions in Networks Using Wavelet Analysis

Signal processing techniques are used to analyze and detect network anomalies because of their ability to detect new and unknown intrusions. The paper proposes a method of modeling network signals for the detection of network anomalies, which combines wavelet approximation and the theory of system identification. To characterize the behavior of network traffic, fifteen functions are provided, which are used as input signals within the system. At the same time, it is assumed that security violations within the network can be detected by checking abnormal patterns of system functioning according to audit data.
 Despite the fact that machine learning methods have achieved significant results in detecting network anomalies, they still face the difficulty of using the implemented algorithms, in the presence of differences in the behavior of the training data and test data, which in turn leads to inefficient performance of the algorithms. This effect is exacerbated by the limitation of algorithms to detect previously unknown types of attacks due to the large number of false positives.
 The paper develops a new method of modeling network signals for detecting anomalies in networks using wavelet analysis. In particular, the general architecture of the approach consists of three components: feature analysis, modeling of normal network traffic based on wavelet approximation and prediction using ARX model, intrusion or non-intrusion decision making
 The result is evaluated using the DARPA intrusion detection dataset, which performs a comprehensive analysis of the intrusions in the dataset. Evaluation results show that this approach provides a high level of detection of both instances and types of attacks.

A computer vision and residual neural network (ResNet) combined method for automated and accurate yeast replicative aging analysis of high-throughput microfluidic single-cell images

With the rapid development of microfluidic platforms in high-throughput single-cell culturing, laborious operation to manipulate massive budding yeast cells (Saccharomyces cerevisiae) in replicative aging studies has been greatly simplified and automated. As a result, large datasets of microscopy images bring challenges to fast and accurately determine yeast replicative lifespan (RLS), which is the most important parameter to study cell aging. Based on our microfluidic diploid yeast long-term culturing (DYLC) chip that features 1100 traps to immobilize single cells and record their proliferation and aging via time-lapse imaging, herein, a dedicated algorithm combined with computer vision and residual neural network (ResNet) was presented to efficiently process tremendous micrographs in a high-throughput and automated manner. The image-processing algorithm includes following pivotal steps: (i) segmenting multi-trap micrographs into time-lapse single-trap sub-images, (ii) labeling 8 yeast budding features and training the 18-layer ResNet, (iii) converting the ResNet predictions in analog values into digital signals, (iv) recognizing cell dynamic events, and (v) determining yeast RLS and budding time interval (BTI) ultimately. The ResNet algorithm achieved high F1 scores (over 92%) demonstrating the effectiveness and accuracy in the recognition of yeast budding events, such as bud appearance, daughter dissection and cell death. Therefore, the results conduct that similar deep learning algorithms could be tailored to analyze high-throughput microscopy images and extract multiple cell behaviors in microfluidic single-cell analysis.

A spatial and temporal signal fusion based intelligent event recognition method for buried fiber distributed sensing system

Due to the characteristics of high sensitivity, fast response speed and multi-point monitoring, Phase-sensitive optical time-domain reflectometry (Φ-OTDR) has attracted attention in the field of perimeter security. However, the intrusion signals are susceptible to interference by ambient signals and difficult to be recognized. In the field of the vibration event recognition of Φ-OTDR system, the deep-learning based methods achieve great recognition ability. However, the previous works are mostly built on single signal source like temporal signal and may not completely adapt to different environment. In this work, a novel phenomenon is reported that a specific variation pattern of light intensity, which is related to the type of vibration source, is hided in the backscattering traces in spatial domain. Inspired by this, a lightweight model and data composition method is proposed to fuse spatial information with temporal correlation information based on end-to-end CNN-LSTM combined model and bicubic scaling. The experiment is conducted on a portable computer (a Nvidia GPU RTX 2080 with 2944 compute unified device architecture cores) with 8 event types and shows that this method can achieve 95.56% validation accuracy through less than 6 min training. Compared with previous method trained by single image structure signal, this lightweight work can achieve higher validation accuracy faster.

Intelligent water perimeter security event recognition based on NAM-MAE and distributed optic fiber acoustic sensing system.

Distributed optical acoustic sensing (DAS) based on phase-sensitive optical time-domain reflectometry can realize the distributed monitoring of multi-point disturbances along an optical fiber, thus making it suitable for water perimeter security applications. However, owing to the complex environment and the production of various noises by the system, continuous and effective recognition of disturbance signals becomes difficult. In this study, we propose a Noise Adaptive Mask-Masked Autoencoders (NAM-MAE) algorithm based on the novel mask mode of a Masked Autoencoders (MAE) and applies it to the intelligent event recognition in DAS. In this method, fewer but more accurate features are fed into the deep learning model for recognition by directly shielding the noise. Taking the fading noise generated by the system as an example, data on water perimeter security events collected in DAS underwater acoustic experiments are used. The NAM-MAE is compared with other models. The results indicate higher training accuracy and higher convergence speed of NAM-MAE than other models. Further, the final test accuracy reaches 96.6134%. It can be demonstrated that the proposed method has feasibility and superiority.

Vehicular social aspects in data dissemination of urban critical events with QoS and QoE for supporting emergency response

AbstractCritical urban events impact the city agenda, like sustainable urban mobility, safe transportation, and efficient governance, thus leading many times to social disturbance, environmental degradation, and economic and human losses. Such events demand efficient dissemination of their data to the authorities to take rapid and effective actions to minimize their effects. The Social Internet of Vehicles (SIoV) paradigm can aid in monitoring such events in time and space. Vehicles are suitable for monitoring random events in space and time due to their ubiquity. Simultaneously, social relationships between vehicle users establish local communities where social intelligence can help disseminate data collected by VANET to support public management in event recognition. Thus, this article proposes a distributed and robust QoS/QoE aware, social‐based data dissemination solution for critical urban events and investigates how social parameters influence the quality of data dissemination. The impact of social influence, social activity, and friendship strategies was analyzed by simulation. The results show the gains in QoS when using social activity and in QoE subjective metrics when employing social influence.

Language Model-Based Player Goal Recognition in Open World Digital Games

Devising models that reliably recognize player goals is a key challenge in creating player-adaptive games. Player goal recognition is the task of automatically recognizing the intent of a player from a sequence of observed player actions in a game environment. In open-world digital games, players often undertake suboptimal and varied sequences of actions to achieve goals, and the high degree of freedom afforded to players makes it challenging to identify sequential patterns that lead toward specific goals. To address these issues, we present a player goal recognition framework that utilizes a fine-tuned T5 language model, which incorporates our novel attention mechanism called Temporal Contrary Attention (TCA). The T5 language model enables the framework to exploit correlations between observations through non-sequential self-attention within input sequences, while TCA enables the framework to learn to eliminate goal hypotheses by considering counterevidence within a temporal window. We evaluate our approach using game trace data collected from 144 players' interactions with an open-world educational game. Specifically, we investigate the predictive capacity of our approach to recognize player goals as well as player plans represented as abstract actions. Results show that our approach outperforms non-linguistic machine learning approaches as well as T5 without TCA. We discuss the implications of these findings for the design and development of player goal recognition models to create player-adaptive games.

Real-Time Online Goal Recognition in Continuous Domains via Deep Reinforcement Learning.

The problem of goal recognition involves inferring the high-level task goals of an agent based on observations of its behavior in an environment. Current methods for achieving this task rely on offline comparison inference of observed behavior in discrete environments, which presents several challenges. First, accurately modeling the behavior of the observed agent requires significant computational resources. Second, continuous simulation environments cannot be accurately recognized using existing methods. Finally, real-time computing power is required to infer the likelihood of each potential goal. In this paper, we propose an advanced and efficient real-time online goal recognition algorithm based on deep reinforcement learning in continuous domains. By leveraging the offline modeling of the observed agent's behavior with deep reinforcement learning, our algorithm achieves real-time goal recognition. We evaluate the algorithm's online goal recognition accuracy and stability in continuous simulation environments under communication constraints.

Research on multimedia intelligent algorithm based on recognition and detection of table tennis ball striking action

Abstract There are many different types of batting motions in table tennis, and the accurate identification of these motion patterns is of great importance for the analysis of batting motions. In this paper, firstly, we explore the efficient representation of the batting motion in the time domain by reasonably placing the sensors and processing the collected data with respect to the characteristics of the batting motion. Secondly, based on the SVM algorithm, in real-world applications, the issue of creating complicated hypersurfaces is resolved using the kernel function technique., so that the simple hypersurfaces in the original space can achieve satisfactory classification results. Finally, a table tennis batting action recognition algorithm based on the SVM algorithm is calculated. The algorithm can efficiently extract the features of batting actions in temporal order and finally accomplishes an average recognition correctness of 93.58%, which is 4.155% more accurate than decision trees, 5.255% more accurate than random forests, and 3.305% more accurate than integrated classifiers. The best performance among the four classifiers indicates that the model has the best performance. This study demonstrates that the SVM algorithm model can quickly and effectively classify and identify action signals for the batting action during table tennis training.

Human Norovirus Detection: How Much Are We Prepared?

Norovirus (NoV) is known to be the second nonbacterial enteric pathogen after rotavirus that causes acute gastroenteritis. They can be spread from person to person through fecal-oral routes. Infection can lead to severe diarrhea, causing stomach pain, vomiting, and nausea. Rapid detection of NoV can control huge economic and productive losses. Genotyping various emerging NoV strains is important to compare the severity among different strains. Conventional immunological and molecular methods have evolved and contributed to developing detection techniques. Immunological (enzyme-linked immunosorbent assay) and molecular detection (reverse transcriptase polymerase chain reaction [RT-PCR], RT-quantitative PCR, loop-mediated isothermal amplification, nucleic acid sequence-based alignment, recombinase polymerase amplification) methods have been mainly used. The development of biosensors using aptasensor, affinity peptides, nanoparticles, microfluidics, and so on, are currently the most researched topics. The availability of next-generation sequencing technologies has greatly influenced the diagnosis of NoV. The complementation of advanced technologies is helpful in identification of new variants. In this study, techniques that are useful in detecting NoV are discussed. This review has investigated the availability of recent methods used in the detection, present status, and futuristic plan of action in case of outbreak and pandemic.

Cyclic Action Graphs for goal recognition problems with inaccurately initialised fluents

Goal recognisers attempt to infer an agent’s intentions from a sequence of observed actions. This is an important component of intelligent systems that aim to assist or thwart actors; however, there are many challenges to overcome. For example, the initial state of the environment could be partially unknown, and agents can act suboptimally and observations could be missing. Approaches that adapt classical planning techniques to goal recognition have previously been proposed, but, generally, they assume the initial world state is accurately defined. In this paper, a state is inaccurate if any fluent’s value is unknown or incorrect. Our aim is to develop a goal recognition approach that is as accurate as the current state-of-the-art algorithms and whose accuracy does not deteriorate when the initial state is inaccurately defined. To cope with this complication, we propose solving goal recognition problems by means of an Action Graph. An Action Graph models the dependencies, i.e. order constraints, between all actions rather than just actions within a plan. Leaf nodes correspond to actions and are connected to their dependencies via operator nodes. After generating an Action Graph, the graph’s nodes are labelled with their distance from each hypothesis goal. This distance is based on the number and type of nodes traversed to reach the node in question from an action node that results in the goal state being reached. For each observation, the goal probabilities are then updated based on either the distance the observed action’s node is from each goal or the change in distance. Our experimental results, for 15 different domains, demonstrate that our approach is robust to inaccuracies within the defined initial state.

Analisis Penerapan Akuntansi Isak 35 Pada Organisasi Keagamaan (Studi Kasus 5 Masjid Di Kabupaten Labuhanbatu Selatan)

This research aims to analyze the application of ISAK 35 accounting standards in religious organizations by taking case studies of five mosques in South Labuhanbatu Regency. ISAK 35 is an accounting standard that regulates the recognition, measurement, and placement of transactions and events in hazardous organizations. In the context of religious organizations, namely mosques, the implementation of ISAK 35 has an important importance in ensuring transparency and accountability in financial management. This research uses a qualitative approach with a case study method. Primary data was collected through in-depth interviews with mosque administrators and related financial staff. Secondary data was obtained through literature and documentation studies related to the implementation of ISAK 35 in religious organizations. The research results show that five mosques located in South Labuhanbatu Regency still have not implemented the ISAK 35 standard in preparing their financial reports. This shows that there are deficiencies in the presentation and delivery of financial information in accordance with applicable accounting standards. This research provides an overview of the importance of implementing ISAK 35 standards in producing more transparent, accurate and trustworthy financial reports for mosques in the region. These factors include limited knowledge and understanding of ISAK 35 by mosque administrators, a lack of human resources skilled in accounting, and a lack of support from external parties such as local governments or financial supervisory bodies.

Human Activity Recognition with Smartwatch Data by using Mahalanobis Distance-Based Outlier Detection and Ensemble Learning Methods

Recognition of human activities is part of smart healthcare applications. In this context, the detection of human actions with high accuracy has been a field that has been working for many years. With the increase in the usage of smart devices, smartphones and smartwatches have become the constant equipment of these studies thanks to their internal sensors. Sometimes abnormal data are included in data sets due to the way the data were collected and for reasons arising from the sensors. For this reason, it becomes important to detect outlier data. In this study, step counter and heart rate sensors were used in addition to an accelerometer and gyroscope in order to detect human activities. Afterward, the outliers were detected and cleared with a Mahalanobis distance-based approach. With the aim of achieving a better classification performance, machine learning methods were used by strengthening them with ensemble learning methods. The obtained results showed that step counter, heart rate sensors, and ensemble learning methods positively affect the success of the classification. In addition, it was found that the Mahalanobis distance-based outlier detection method increased the classification accuracy significantly.

NURSE’S ROLE IN THE CONTROL OF ARTERIAL HYPERTENSION AND MORBIMORTALITY FACTORS IN THE ELDERLY

Arterial hypertension is a non-communicable, asymptomatic chronic disease that is entirely linked to the high rates of morbidity and mortality that affect the quality of life of the elderly. We seek to understand how arterial hypertension can be treated to reduce the number of deaths in elderly people and describe the role of the nursing team in controlling arterial hypertension. This is an integrative review established by Cooper, the searches took place in the Virtual Health Library and Pubmed, using the descriptors “hypertension”, “nurse”, “prevention”, “mortality associated with each other by the Boolean operator “and”, in the period from 2016 to 2021, resulting in the initial search of 972 articles that after applying the selection criteria and reading in full, 38 remained that made up the research corpus. The socio-environmental and genetic conditions, which when related to senescence and unhealthy life habits, imply the triggering of hypertension, a health problem with loss of quality of life and increased morbidity and mortality. Nursing professionals who work in primary care are the crucial link in the early detection and development of actions aimed at health care, aiming to control the blood pressure levels of patients affected by AH, disseminating information aimed at health education.

Impulsive Aggression Break, Based on Early Recognition Using Spatiotemporal Features

The study of human behaviors aims to gain a deeper perception of stimuli that control decision making. To describe, explain, predict, and control behavior, human behavior can be classified as either non-aggressive or anomalous behavior. Anomalous behavior is any unusual activity; impulsive aggressive, or violent behaviors are the most harmful. The detection of such behaviors at the initial spark is critical for guiding public safety decisions and a key to its security. This paper proposes an automatic aggressive-event recognition method based on effective feature representation and analysis. The proposed approach depends on a spatiotemporal discriminative feature that combines histograms of oriented gradients and dense optical flow features. In addition, the principal component analysis (PCA) and linear discriminant analysis (LDA) techniques are used for complexity reduction. The performance of the proposed approach is analyzed on three datasets: Hockey-Fight (HF), Stony Brook University (SBU)-Kinect, and Movie-Fight (MF), with accuracy rates of 96.5%, 97.8%, and 99.6%, respectively. Also, this paper assesses and contrasts the feature engineering and learned features for impulsive aggressive event recognition. Experiments show promising results of the proposed method compared to the state of the art. The implementation of the proposed work is available here.

An Intelligent Framework for Log Anomaly Detection Based on Log Template Extraction

Log anomaly detection holds great significance in computer systems and network security. A large amount of log data is generated in the background of various information systems and equipment, so automated methods are required to identify abnormal behavior that may indicate security threats or system malfunctions. The traditional anomaly detection methods usually rely on manual statistical discovery, or match by regular expression which are complex and time-consuming. To prevent system failures, minimize troubleshooting time, and reduce service interruptions, a log template-based anomaly detection method has been proposed in this context. This approach leverages log template extraction, log clustering, and classification technology to timely detect abnormal events within the information system. The effectiveness of this method has been thoroughly tested and compared against traditional log anomaly detection systems. The results demonstrate improvements in log analysis depth, event recognition accuracy, and overall efficiency.

ActionMixer: Temporal action detection with Optimal Action Segment Assignment and mixers

In this paper, we propose a novel method for dynamic label assignment in temporal action detection (TAD) called Optimal Action Segment Assignment (OASA). The OASA method converts label assignment into an optimal transportation problem by computing the cost matrix between predicted temporal action segments and groundtruths. The unit transportation cost between a predicted temporal segment and a groundtruth pair is defined as the weighted summation of action classification loss and temporal localization loss. Additionally, we deploy Adaptive Estimation of Candidate Segment Number (AE-CSN) to adaptively determine the number of positive samples for each groundtruth. After formulation, the label assignment problem is converted to find a global optimal assignment plan by minimizing the cost. Therefore, OASA eliminates the need for manually designed prior parameters, which exist in fixed label assignment methods, and improves the generalization of the algorithm between different datasets. To evaluate OASA, we also introduce a simple anchor-free temporal action detector called ActionMixer. It consists of two components: Temporal Mixer and Channel Mixer. The Temporal Mixer employs depth-wise convolution layers with large kernels to capture temporal information, while the Channel Mixer mixes and extracts features across the channel dimension. Extensive experiments conducted on the THUMOS-14, ActivityNet-1.3, and EPIC-Kitchens-100 datasets show that ActionMixer equipped with OASA achieves state-of-the-art performance, surpassing other advanced temporal action detection methods.

Transmission Timing Control among Both Aperiodic and Periodic Flows for Reliable Transfer by Restricted Packet Loss and within Permissible Delay in Wireless Sensor Networks

Nowadays, in various wireless sensor networks, both aperiodically generated packets like event detections and periodically generated ones for environmental, machinery, vital monitoring, etc. are transferred. Thus, packet loss caused by collision should be suppressed among aperiodic and periodic packets. In addition, some packets for wireless applications such as factory IoT must be transferred within permissible end-to-end delays, in addition to improving packet loss. In this paper, we propose transmission timing control of both aperiodic and periodic packets at an upper layer of medium access control (MAC). First, to suppress packet loss caused by collision, transmission timings of aperiodic and periodic packets are distributed on the time axis. Then, transmission timings of delay-bounded packets with permissible delays are assigned within the bounded periods so that transfer within their permissible delays is possible to maximally satisfy their permissible delays. Such control at an upper layer has advantages of no modification to the MAC layer standardized by IEEE 802.11, 802.15.4, etc. and low sensor node cost, whereas existing approaches at the MAC layer rely on MAC modifications and time synchronization among all sensor nodes. Performance evaluation verifies that the proposed transmission timing control improves packet loss rate regardless of the presence or absence of packet's periodicity and permissible delay, and restricts average transfer delay of delay-bounded packets within their permissible delays comparably to a greedy approach that transmits delay-bounded packets to the MAC layer immediately when they are generated at an upper layer.