Intelligent Video Systems Research Articles

Rethinking prediction-based video anomaly detection from local–global normality perspective

Video anomaly detection (VAD) has been intensively studied for years because of its potential applications in intelligent video systems. Prediction-based VAD methods have gained a lot of attention in recent years due to their simplicity and effectiveness, which learn normality through a frame prediction tasks. However, we find that they often consider only local or global normality in the temporal dimension thus are difficult to generalize for various scenarios. Specifically, some of them focus on learning local spatiotemporal representations from consecutive frames to enhance the representation for normal events while the other methods are devoted to memorizing prototypical normal patterns of whole training videos to weaken the generalization for anomalies. In this paper, we rethink these two types of methods and find that the former methods are difficult to adapt to simple scenarios due to powerful representation allows these methods to represent some anomalies and causes miss detection while the latter struggle with complex scenarios due to the limitations for diverse normal patterns and causes false alarm. To verify our discoveries, we design a two-branch model, Local–Global Normality Network (LGN-Net), to simultaneously learn local and global normality. We verified the effect of local and global normality on the VAD model by ablation study and find that the model can be more adaptive to different scenarios when accounting for both normality. The code is available online: https://github.com/Myzhao1999/LGN-Net.

Human Activity Recognition using Machine Learning

Working information and classification is one of the most important problems in computer science. Recognizing and identifying actions or tasks performed by humans is the main goal of intelligent video systems. Human activity is used in many applications, from human-machine interaction to surveillance, security and healthcare. Despite continuous efforts, working knowledge in a limitless field is still a difficult task and faces many challenges. In this article, we focus on some of the current research articles on various cognitive functions. This project includes three popular methods to define projects: vision-based (using estimates), practical devices, and smartphones. We will also discuss some advantages and disadvantages of the above methods and give a brief comparison of their accuracy. The results will also show how the visualization method has become a popular method for HAR research today.

UGV-awareness task placement in edge-cloud based urban intelligent video systems

UGV-awareness task placement in edge-cloud based urban intelligent video systems

RISIR: Rapid Infrared Spectral Imaging Restoration Model for Industrial Material Detection in Intelligent Video Systems

Material detection by industrial infrared imaging spectrometer (IRIS) is a key technique in multiple industrial applications, including garbage collection, material analysis, and robot vision. However, IRIS often suffers from overlapped bands and random noises, which limit the precision of subsequent processing. In this article, we propose a novel Gabor transform-based mid-wave infrared (MWIR) spectrum restoration model by successfully exploring the intrinsic structure of the clean MWIR spectrum from the degraded one. First, a total variation-regularized Gabor coefficient adjustment descriptor is designed and incorporated into the spectrum restoration model. In addition to adjusting the Gabor coefficient distribution by total variation-norm, the L2-norm of gradient is leveraged to regulate the smoothness of the instrument degradation function. Then, the proposed model is inferred using an efficient optimization approach based on split Bregman iteration method and alternating minimization algorithm. Quantitative and qualitative experimental results demonstrate that the proposed model favorably outperforms the state-of-the-art approaches. The restored high-resolution MWIR spectrum can be used to rapidly detect different materials in intelligent video systems.

Dynamic Queries through Augmented Reality for Intelligent Video Systems

Artificial vision system applications have generated significant interest as they allow information to be obtained through one or several of the cameras that can be found in daily life in many places, such as parks, avenues, squares, houses, etc. When the aim is to obtain information from large areas, it can become complicated if it is necessary to track an object of interest, such as people or vehicles, due to the vision space that a single camera can cover; this opens the way to distributed zone monitoring systems made up of a set of cameras that aim to cover a larger area. Distributed zone monitoring systems add great versatility, becoming more complex in terms of the complexity of information analysis, communication, interoperability, and heterogeneity in the interpretation of information. In the literature, the development of distributed schemes has focused on representing data communication and sharing challenges. Currently, there are no specific criteria for information exchange and analysis in a distributed system; hence, different models and architectures have been proposed. In this work, the authors present a framework to provide homogeneity in a distributed monitoring system. The information is obtained from different cameras, where a global reference system is defined for generated trajectories, which are mapped independently of the model used to obtain the dynamics of the movement of people within the vision area of a distributed system, thus allowing for its use in works where there is a large amount of information from heterogeneous sources. Furthermore, we propose a novel similarity metric that allows for information queries from heterogeneous sources. Finally, to evaluate the proposed performance, the authors developed several distributed query applications in an augmented reality system based on realistic environments and historical data retrieval using a client–server model.

An integrated and real-time social distancing, mask detection, and facial temperature video measurement system for pandemic monitoring

This paper presents a new Edge-AI algorithm for real-time and multi-feature (social distancing, mask detection, and facial temperature) measurement to minimize the spread of COVID-19 among individuals. COVID-19 has extenuated the need for an intelligent surveillance video system that can monitor the status of social distancing, mask detection, and measure the temperature of faces simultaneously using deep learning (DL) models. In this research, we utilized the fusion of three different YOLOv4-tiny object detectors for each task of the integrated system. This DL model is used for object detection and targeted for real-time applications. The proposed models have been trained for different data sets, which include people detection, mask detection, and facial detection for measuring the temperature, and evaluated on these existing data sets. Thermal and visible cameras have been used for the proposed approach. The thermal camera is used for social distancing and facial temperature measurement, while a visible camera is used for mask detection. The proposed method has been executed on NVIDIA platforms to assess algorithmic performance. For evaluation of the trained models, accuracy, recall, and precision have been measured. We obtained promising results for real-time detection for human recognition. Different couples of thermal and visible cameras and different NVIDIA edge platforms have been adopted to explore solutions with different trade-offs between cost and performance. The multi-feature algorithm is designed to monitor the individuals continuously in the targeted environments, thus reducing the impact of COVID-19 spread.

A Modular Architecture of Command-and-Control Software in Multi-Sensor Systems Devoted to Public Security

Preventing terrorist attacks at soft targets has become a priority for our society. The realization of sensor systems for automatic threat detection in crowded spaces, such as airports and metro stations, is challenged by the limited sensing coverage capability of the devices in place due to the variety of dangerous materials, to the scanning rate of the devices, and to the detection area covered. In this context, effectiveness of the physical configuration of the system based on the detectors used, the coordination of the sensor data collection, and the real time data analysis for threat identification and localization to enable timely reactions by the security guards are essential requirements for such integrated sensor-based applications. This paper describes a modular distributed architecture of a command-and-control software, which is independent from the specific detectors and where sensor data fusion is supported by two intelligent video systems. Furthermore, the system installation can be replicated at different locations of a public space. Person tracking and later re-identification in a separate area, and tracking hand-over between different video components, provide the command-and-control with localization information of threats to timely activate alarm management and support the activity of subsequent detectors. The architecture has been implemented for the NATO-funded DEXTER program and has been successfully tested in a big city trial at a metro station in Rome both when integrated with two real detectors of weapons and explosives and as a stand-alone system. The discussion focuses on the software functions of the command-and-control and on the flexibility and re-use of the system in wider settings.

An Intelligent System for Detecting Abnormal Behavior in Students Based on the Human Skeleton and Deep Learning.

With the use of an intelligent video system, this research provides a method for detecting abnormal behavior based on the human skeleton and deep learning. To begin with, the spatiotemporal features of human bones are extracted through iterative training using the OpenPose deep learning network and the redundant information of human bone facial features is reduced in the feature extraction process, effectively reducing the time it takes to identify and analyze abnormal behavior. The collected human skeleton features are then classified using a graph convolution neural network to reduce the computational complexity of the behavior identification algorithm, and the sliding window voting method is used to further improve the accuracy of the behavior classification in practical application, resulting in the diagnosis and classification of abnormal behavior of students under video surveillance. Finally, using the self-built student trajectory data set and the INRIA data set, simulation analysis is performed, and the practicality and superiority of the proposed method for abnormal behavior detection is confirmed by comparing it to the existing abnormal behavior recognition methods. The proposed method for detecting anomalous behavior in a self-built database and INRIA data set has a high accuracy of more than 99.50 percent and a high processing efficiency rate.

An Efficient Unusual Event Tracking in Video Sequence using Block Shift Feature Algorithm

The area of video technology is rapidly growing owing to advancements in intelligent video systems in sensor operations, higher bandwidth capacity, storage, and high-resolution displays. This led to the proliferation of video-based computing modeling to perform specific tasks on video sequences to gain more insight from the data. Visual tracking of events is a core component in video visual surveillance systems that classify and track moving objects to describe their behavioral aspects. The prime motive behind intelligent video systems is to perform efficient video analytics to meet the specific requirements of the user/use-cases. It involves a self-directed paradigm to understand event sequences, reducing the computational burden of characterizing the activities. The study incorporates a block-shift feature algorithm and introduces a novel computational research method for unusual event tracking in video sequences. The formulated approach employs a framework combining operational blocks to compute sequential operations such as block-matching from the dictionary of motion estimations. Before applying the learning model, the subsequent analysis procedure adds feature lexicon and dominant attributes to make the execution computationally efficient. Further, it uses a sparse-non negative factorization approach to organize the informative details into k possible finite clusters. The event detection outcome from the training datasets of video sequences shows better experimental results than the traditional highly cited related approach of unusual object detection and tracking.

Task placement for crowd recognition in edge-cloud based urban intelligent video systems

Task placement for crowd recognition in edge-cloud based urban intelligent video systems

RETRACTED ARTICLE: Simulation of English feature recognition based on machine learning and artificial intelligence technology

Computer vision includes several subjects, such as machine learning, image processing and model recognition, and replaces some human work by analyzing and extracting a large amount of video data and analyzing video content information. With the continuous improvement and development of educational video system and artificial intelligence technology, an intelligent educational video system based on real-time recognition technology is being developed rapidly. To determine and analyze the behavior of teachers and students by using behavior recognition algorithms based on continuous video images, classroom scene video can reflect students' participation in classroom teaching, and evaluate teachers' attractiveness to students through students' academic performance. Including their academic performance. Through the analysis of the behavior of teachers and students in the classroom, we can effectively find the intersection of students and teachers in the classroom. Through the interaction and interest between students, we can analyze the influence of teachers' style and interaction on students in teaching. Students, educational reform, improving the quality of education and training highly skilled personnel to achieve a positive direction.

An Accurate Matching Method for Projecting Vector Data into Surveillance Video to Monitor and Protect Cultivated Land

The integration of intelligent video surveillance and GIS (geograhical information system) data provides a new opportunity for monitoring and protecting cultivated land. For a GIS-based video monitoring system, the prerequisite is to align the GIS data with video image. However, existing methods or systems have their own shortcomings when implemented in monitoring cultivated land. To address this problem, this paper aims to propose an accurate matching method for projecting vector data into surveillance video, considering the topographic characteristics of cultivated land in plain area. Once an adequate number of control points are identified from 2D (two-dimensional) GIS data and the selected reference video image, the alignment of 2D GIS data and PTZ (pan-tilt-zoom) video frames can be realized by automatic feature matching method. Based on the alignment results, we can easily identify the occurrence of farmland destruction by visually inspecting the image content covering the 2D vector area. Furthermore, a prototype of intelligent surveillance video system for cultivated land is constructed and several experiments are conducted to validate the proposed approach. Experimental results show that the proposed alignment methods can achieve a high accuracy and satisfy the requirements of cultivated land monitoring.

Guest Editorial: Special Section on Latest Advances on Industrial Intelligent Video Systems and Analytics

Guest Editorial: Special Section on Latest Advances on Industrial Intelligent Video Systems and Analytics

An efficient method to resolve intraclass variability using highly refined HOG description model for human action recognition

SummaryRecognizing the human action is an interesting field and promising area of research due to its importance in various applications. HAR system seeks to interpret the actions being accomplished by the human within the video sequence instinctively and tagging their actions for the primary need of an intelligent video system. Recently, the research attention becomes more focused on recognizing the human actions in unrestrained videos, as the variations in scale, illumination, etc, are often will be the case for performance degradation. Hence, we introduced a novel refined gradient model to normalize the HOG descriptor for the scale‐invariant and appearance modeling. In addition, as different subjects can perform the same action in a different manner. Their different appearance or style variation leads to errors. To resolve this Intraclass variability of actions a discriminative approach has been proposed to explore action attributes. Thus, this system unites these two to reduce the discrimination error. This new approach gives near‐perfect ways for reducing overall dense trajectory displacements with significant accuracy enhancements. The complete action recognition system was evaluated on a number of test videos from real‐world data sets and compared against state‐of‐the‐art methods.

Directions of Development of Intelligent Real Time Video Systems

Real time video systems play a significant role in many fields of science and technology. The range of their applications is constantly increasing together with requirements to them, especially it concerns to real time video systems with the feedbacks. Conventional fundamentals and principles of real-time video systems construction are extremely redundant and do not take into consideration the peculiarities of real time processing and tasks, therefore they do not meet the system requirements neither in technical plan nor in informational and methodical one. Therefore, the purpose of this research is to increase responsiveness, productivity and effectiveness of real time video systems with a feedback during the operation with the high-speed objects and dynamic processes. The human visual analyzer is considered as a prototype for the construction of intelligent real time video systems. Fundamental functions, structural and physical peculiarities of adaptation and processes taking place in a visual analyzer relating to the information processing, are considered. High selectivity of information perception and wide parallelism of information processing on the retinal neuron layers and on the higher brain levels are most important peculiarities of a visual analyzer for systems with the feedback. The paper considers two directions of development of intelligent real time video systems. First direction based on increasing intellectuality of video systems at the cost of development of new information and dynamic models for video information perception processes, principles of control and reading parameters of video information from the sensor, adapting them to the requirements of concrete task, and combining of input processes with data processing. Second direction is associated with the development of new architectures for parallel perception and level-based processing of information directly on a video sensor matrix. The principles of annular and linear structures on the neurons layers, of close-range interaction and specialization of layers, are used to simplify the neuron network.

DrugCam®—An intelligent video camera system to make safe cytotoxic drug preparations

DrugCam® is a new approach to control the chemotherapy preparations with an intelligent video system that enables automatic verification during the critical stages of preparations combined with an a posteriori control with partial or total visualization of the video recording of preparations. The assessment was about the recognizing of anticancer drug vials (qualitative analysis) and syringe volumes (quantitative analysis). The qualitative analysis was conducted for a total of 120 vials with sensitivity of 100% for 84.2% of the vials and at least 97% for all the vials tested. Accuracy was at least 98.5% for all vials. The quantitative analysis was assessed by detecting 10 measures of each graduation for syringes. The identification error rate was 2.1% (244/11,640) i.e. almost 94% to the next graduation. Only 3% (35/1164) of the graduations tested, i.e. 23/35 for volume <0.13ml of 1ml syringes, presented a volume error outside the admissible limit of ±5% of a confidence band constructed for the estimated linear regression line for each syringe. In addition to the vial detection model, barcodes can also read when they are present on vials. DrugCam® offers an innovative approach for controlling chemotherapy preparations and constitutes an optimized application of telepharmacy.

English

With the fast development of video hardware and software in recent years, intelligent video systems have been widely used in industry, transportation, security, etc. at the same time, a lot of biometric technologies, comprising automated methods for uniquely recognizing people based on their physical or behavioural traits, such as face, fingerprint, palm print, finger-knuckle-print, gait, are also based on video or image analysis. Iris recognition is emerging as one of the important methods of biometrics-based identification systems. Several crucial factors Gait Keystroke pattern Signature of iris biometrics include rich and unique textures, non-invasiveness, and stability of iris pattern throughout the person’s lifetime, public acceptance, and availability of user friendly capturing devices. These factors have attracted the researchers to work in this evolving field over the past decade. The iris recognition consists of iris localization, normalization, encoding and comparison. In this paper, segmentation of periocular features and the encoding part of iris recognition is analysed.

Cloud-assisted analysis for energy efficiency in intelligent video systems

The prevalence of intelligent video systems such as urban video surveillance or Google Glass, is gradually changing our daily life. This type of systems applies online analysis on video streams for...

The Design of Intelligent Video System Based on DM8168

With the development of the times and the increasing mobility of the population, the situation of public environmental safety is becoming more and more serious. So an intelligent video monitoring system is designed in order to better carry out constantly effective monitoring for the public environment. In this paper, the IT Company DM8168 as the core of hardware designs an intelligent video surveillance system to realize the function of video capture, encoding, transmission, storage and so on. Besides, it will carry out motion detection in the aspect of intelligence. The main tasks can be completed as follows, first, architecture and front-end network camera hardware design of intelligent video system requires single-stream network. Second, the design of intelligent video system software, which is to complete the overall software design flow chart. Clients using the media player on the PC front-end network camera based on Davinci platform development. Third, algorithm design of intelligent software system is to study of the moving object detection algorithm and provide the algorithm effect diagram.

Intelligent Video Systems and Analytics: A Survey

Recent technology and market trends have demanded the significant need for feasible solutions to video/camera systems and analytics. This paper provides a comprehensive account on theory and application of intelligent video systems and analytics. It highlights the video system architectures, tasks, and related analytic methods. It clearly demonstrates that the importance of the role that intelligent video systems and analytics play can be found in a variety of domains such as transportation and surveillance. Research directions are outlined with a focus on what is essential to achieve the goals of intelligent video systems and analytics.