Video Sensor Research Articles

Periodic Physical Activity Information Segmentation, Counting and Recognition From Video

The research on complex human body motion including sports and workout activity recognition is a major challenge and long-lasting problem for the computer vision community. Recent development in deep learning algorithms to track people’s workout activity characteristics based on video sensors can be used to infer the human body pose for further analysis. Specifically, tracking complex body movements while performing multi-pose physical exercise helps individuals provide fine granularity feedback including activity repetition counting and activity recognition. Therefore, this research proposes a system that provides a repetition counter and activity recognition of physical exercise from video frames (extracted 3D human skeleton using VIBE) based on the deep semantic features and repetitive segmentation algorithm. The proposed system locates both ends of the activity’s action and segments the activity into multiple unit actions which improves activity recognition, time intervals, # of sets, and other quantitative values of activity. The proposed system is evaluated on the physical activities dataset named “NOL-18 Exercise” through extensive experiments. The proposed system results show that the accuracy of the repetitive action segmentation is 96.27% with 0.23% time error, and action recognition reaches 99.06%. The system can be employed to fitness or rehabilitation centers and used for treating patients.

Progressive Transfer Learning-based Deep Q Network for DDOS Defence in WSN

In The Wireless Multimedia Sensor Network (WNSMs) have achieved popularity among diverse communities as a result of technological breakthroughs in sensor and current gadgets. By utilising portable technologies, it achieves solid and significant results in wireless communication, media transfer, and digital transmission. Sensor nodes have been used in agriculture and industry to detect characteristics such as temperature, moisture content, and other environmental conditions in recent decades. WNSMs have also made apps easier to use by giving devices self-governing access to send and process data connected with appropriate audio and video information. Many video sensor network studies focus on lowering power consumption and increasing transmission capacity, but the main demand is data reliability. Because of the obstacles in the sensor nodes, WMSN is subjected to a variety of attacks, including Denial of Service (DoS) attacks. Deep Convolutional Neural Network is designed with the state-action relationship mapping which is used to identify the DDOS Attackers present in the Wireless Sensor Networks for Smart Agriculture. The Proposed work it performs the data collection about the traffic conditions and identifies the deviation between the network conditions such as packet loss due to network congestion and the presence of attackers in the network. It reduces the attacker detection delay and improves the detection accuracy. In order to protect the network against DoS assaults, an improved machine learning technique must be offered. An efficient Deep Neural Network approach is provided for detecting DoS in WMSN. The required parameters are selected using an adaptive particle swarm optimization technique. The ratio of packet transmission, energy consumption, latency, network length, and throughput will be used to evaluate the approach’s efficiency.

MAP-Inspired Deep Unfolding Network for Distributed Compressive Video Sensing

Due to the lack of probabilistic inference guidance, the network structure of current deep learning-based Distributed Compressive Video Sensing (DCVS) algorithms cannot efficiently exploit frame measurements, inter-frame correlations, and intra-frame correlations. In this paper, we infer the optimization equation for DCVS reconstruction according to the maximum a posteriori estimation and propose a Deep Unfolding Multi-Hypothesis Aggregation Network (DUMHAN) to obtain high-quality reconstruction frames. In each stage of the DUMHAN, three modules are designed to implement the three update steps in the inferred optimization equation, which correspond to the three information flows, namely measurement, intra-frame, and inter-frame information flows. In particular, a Multi-Hypothesis Aggregation (MHA) module is developed to enhance the utilization of inter-frame and intra-frame correlation by generating and fusing multiple hypothesis sets. Moreover, we propose the stage-by-stage optical flow update approach in DUMHAN to improve the accuracy of both intra-frame and inter-frame correlation modeling. The experiments demonstrate that the proposed DUMHAN achieves state-of-the-art performance.

Person Tracking by Fusing Posture Data From UAV Video and Wearable Sensors

In this article, a novel framework that fuses the posture data taken by a drone (or unmanned aerial vehicle, UAV) camera and the wearable sensors data recorded by smartwatches is proposed. The framework is designed for continuously tracking persons in a drone view by analyzing location-independent human posture features and correctly tagging smartwatch identities (IDs) and personal profiles to video human objects, thus conquering the former work in requiring ground markers. Person detection, ID assignment, and pose estimation are integrated into our framework to obtain recognized human postures. These recognized postures are then paired with those from the wearable sensors. Through fusing common postures, such as standing, walking, jumping, and falling down, person tracking accuracy by UAV up to 95.36% can be attained in our testing scenarios.

Reduction of Video Data at Translation of a Registered Object Relative to Video Sensors Based on the Eigenbasis of Interpretation Model

Reduction of Video Data at Translation of a Registered Object Relative to Video Sensors Based on the Eigenbasis of Interpretation Model

RSSI Analysis on CSS Modulation in the 433 MHz Frequency Band Using Lora in Flood Sensor

Long Range (LoRa) low power is a transmission technology capable of connecting multiple sensors to the Internet of Things in the future. LoRa uses Chirp Spread Spectrum (CSS) frequency modulation which promises to achieve remote connectivity at very low power. More specifically, this paper focuses on the analysis of Receive Signal Strength Indicator (RSSI) on CSS modulation on flood sensors. This paper presents a detailed model of using LoRa on flood sensors, showing that LoRa is capable of transmitting sensor data over long distances with perfect RSSI values. The results showed the potential for a close communication range with a narrow band network at a frequency of 433 MHz and an average RSSI of -50 dB at a maximum distance of 600 meters without obstruction. Although LoRa communication uses CSS modulation and has good RSSI values, in long-distance communication with data in the form of video or large sensor data, this toughness is not sufficient due to long distances and large bandwidth usage, while in this study LoRa type Ai Thinker Ra- 02 SX 1278 can only be set up to 125 KHz according to the datasheet. The disadvantages of using large sensor data over long distances and a larger footprint make CSS vulnerable to other, possibly larger, resources.

Artificialintelligence‐basedremote monitoring system for automated anomaly detection in wireless sensor networks

SummaryIn recent decades, wireless sensor networks (WSNs) are employed in the remote monitoring system which experiences a huge delay due to the excessive time consumption by the image processing techniques. The existing video surveillance system is unable to detect the anomaly accurately and falls short in classifying the routine activity and the anomaly activities. Therefore, an efficient anomaly detection model is necessitated for identifying the anomaly behavior of malicious nodes in the WSN environment. This research work introduces a novel artificial intelligence (AI)‐based remote monitoring system for WSN. Primarily, the online video frames are retrieved from the wireless video sensors where the frames are generated by segmenting the entire stream of video as a constant‐sized structural frame. The sequence/stream of video segments is fed into the proposed AI framework to detect anomaly activities. It has multiple levels of hidden layers to compare the features of the input captured image with the preloaded reference image. Furthermore, a 5‐fold cross‐validation approach has been inspected to validate the efficacy of the suggested AI model. The performance results evident that the proposed AI system acquires a better prediction accuracy of 98.2% and precision of 97.3% than the existing monitoring systems.

Retraction Note: QoS and QoE Enhanced Resource Allocation for Wireless Video Sensor Networks Using Hybrid Optimization Algorithm

Retraction Note: QoS and QoE Enhanced Resource Allocation for Wireless Video Sensor Networks Using Hybrid Optimization Algorithm

Accuracy of gait and posture classification using movement sensors in individuals with mobility impairment after stroke.

Background: Stroke leads to motor impairment which reduces physical activity, negatively affects social participation, and increases the risk of secondary cardiovascular events. Continuous monitoring of physical activity with motion sensors is promising to allow the prescription of tailored treatments in a timely manner. Accurate classification of gait activities and body posture is necessary to extract actionable information for outcome measures from unstructured motion data. We here develop and validate a solution for various sensor configurations specifically for a stroke population. Methods: Video and movement sensor data (locations: wrists, ankles, and chest) were collected from fourteen stroke survivors with motor impairment who performed real-life activities in their home environment. Video data were labeled for five classes of gait and body postures and three classes of transitions that served as ground truth. We trained support vector machine (SVM), logistic regression (LR), and k-nearest neighbor (kNN) models to identify gait bouts only or gait and posture. Model performance was assessed by the nested leave-one-subject-out protocol and compared across five different sensor placement configurations. Results: Our method achieved very good performance when predicting real-life gait versus non-gait (Gait classification) with an accuracy between 85% and 93% across sensor configurations, using SVM and LR modeling. On the much more challenging task of discriminating between the body postures lying, sitting, and standing as well as walking, and stair ascent/descent (Gait and postures classification), our method achieves accuracies between 80% and 86% with at least one ankle and wrist sensor attached unilaterally. The Gait and postures classification performance between SVM and LR was equivalent but superior to kNN. Conclusion: This work presents a comparison of performance when classifying Gait and body postures in post-stroke individuals with different sensor configurations, which provide options for subsequent outcome evaluation. We achieved accurate classification of gait and postures performed in a real-life setting by individuals with a wide range of motor impairments due to stroke. This validated classifier will hopefully prove a useful resource to researchers and clinicians in the increasingly important field of digital health in the form of remote movement monitoring using motion sensors.

SCVS: On AI and Edge Clouds Enabled Privacy-preserved Smart-city Video Surveillance Services

Video surveillance systems are increasingly becoming common in many private and public campuses, city buildings, and facilities. They provide many useful smart campus/city monitoring and management services based on data captured from video sensors. However, the video surveillance services may also breach personally identifiable information, especially human face images being monitored; therefore, it may potentially violate the privacy of human subjects involved. To address this privacy issue, we introduced a large-scale distributed video surveillance service model, called Smart-city Video Surveillance (SCVS). SCVS is a video surveillance data collection and processing platform to identify important events, monitor, protect, and make decisions for smart campus/city applications. In this article, the specific research focus is on how to identify and anonymize human faces in a distributed edge cloud computing infrastructure. To preserve the privacy of data during video anonymization, SCVS utilizes a two-step approach: (i) parameter server-based distributed machine learning solution, which ensures that edge nodes can exchange parameters for machine learning-based training. Since the dataset is not located on a centralized location, the data privacy and ownership are protected and preserved. (ii) To improve the machine learning model’s accuracy, we presented an asynchronous training approach to protect data and model privacy for both data owners and data users, respectively. SCVS adopts an in-memory encryption approach, where edge computing nodes collect and process data in the memory of edge nodes in encrypted form. This approach can effectively prevent honest but curious attacks. The performance evaluation shows the presented privacy protection platform is efficient and effective compared to traditional centralized computing models as presented in Section 5 .

Real-time snowy weather detection based on machine vision and vehicle kinematics: A non-parametric data fusion analysis protocol

Introduction: This study introduces a new analysis protocol for detecting real-time snowy weather conditions on freeways by utilizing trajectory-level data extracted from the Second Strategic Highway Research Program (SHRP2) Naturalistic Driving Study (NDS) dataset. The data include parameters reduced from a real-time image feature extraction technique, time series data collected from external sensors, and CANbus data collected by the NDS ego-vehicles. To provide flexibility in winter maintenance, two segmentation types of one-minute and one-mile segments were used to sample snowy trips and their matched clear weather trips. Method: In this study, four non-parametric models were developed using six data assemblies to detect snowy weather on freeways. The data assemblies are arranged based on three data sources, including image database extracted from an in-vehicle video camera, sensors, and CANbus data, to examine the effectiveness of snow detection models for different data types considering real-time availability of data. Results: Overall, the developed models successfully detected snowy weather on freeways with an accuracy ranging between 76% to 89%. Results indicated that high accuracy of estimating snowy weather can be accomplished using the data fusion between external sensors data and texture parameters of images, without accessing to CANbus data. Practical Applications: Practical applications can be driven with respect to the time or distance coordinates, using different data fusion assemblies, and data availability. The study proves the importance of employing vehicles as weather sensors in the Connected Vehicles (CV) applications and Variable Speed Limit (VSL) to improve traffic safety on freeways.

Occluded Pedestrian-Attribute Recognition for Video Sensors Using Group Sparsity.

Pedestrians are often obstructed by other objects or people in real-world vision sensors. These obstacles make pedestrian-attribute recognition (PAR) difficult; hence, occlusion processing for visual sensing is a key issue in PAR. To address this problem, we first formulate the identification of non-occluded frames as temporal attention based on the sparsity of a crowded video. In other words, a model for PAR is guided to prevent paying attention to the occluded frame. However, we deduced that this approach cannot include a correlation between attributes when occlusion occurs. For example, “boots” and “shoe color” cannot be recognized simultaneously when the foot is invisible. To address the uncorrelated attention issue, we propose a novel temporal-attention module based on group sparsity. Group sparsity is applied across attention weights in correlated attributes. Accordingly, physically-adjacent pedestrian attributes are grouped, and the attention weights of a group are forced to focus on the same frames. Experimental results indicate that the proposed method achieved 1.18% and 6.21% higher -scores than the advanced baseline method on the occlusion samples in DukeMTMC-VideoReID and MARS video-based PAR datasets, respectively.

Multi-objective optimization for coverage aware energy consumption in wireless 3D video sensor network

This paper addresses the problem of minimizing energy consumption as well as maximizing the coverage area for 3D video sensor nodes deployed over a 2D target area in a wireless video sensor network. Reducing the number of active video sensor nodes helps decrease energy consumption, but decrease the coverage area as well as connectivity in the network. The problem is solved initially by a multi-objective integer linear programming based approach (APP_7) which provides an optimal solution but leads to intractability for a large-sized network due to the non-deterministic polynomial time-hard complexity of the problem. This fact motivates us to design a non-dominated sorting genetic algorithm-II based heuristic (APP_8), which generates a near-optimal solution and that is substantiated by comparing the simulation results of APP_7 with APP_8, and two existing greedy heuristics (EX_1, EX_11) for small-sized problem. Simulation experiments are conducted also to compare the results of APP_8 with EX_1 and EX_11 for large-sized network. Between two heuristics, EX_11/(EX_1) is better in terms of energy consumption/(area coverage). It has been observed that for the large-sized network (i.e. node density 100 on the target area of size 75 × 75 square meters) APP_8 is able to reduce energy consumption by 47.76% from EX_11, whereas area coverage is decreased by 0.51% from EX_1, which confirms the acceptability of APP_8 as the proper heuristic in such a scenario.

Enhancing motion tracking accuracy of a low-cost 3D video sensor using a biomechanical model, sensor fusion, and deeplearning.

Low-cost 3D video sensors equipped with routines for extracting skeleton data facilitate the widespread use of virtual reality (VR) for rehabilitation. However, the accuracy of the extracted skeleton data is often limited. Accuracy can be improved using a motion tracker, e.g., using a recurrent neural network (RNN). Yet, training an RNN requires a considerable amount of relevant and accurate training data. Training databases can be obtained using gold-standard motion tracking sensors. This limits the use of the RNN trackers in environments and tasks that lack accessibility to gold-standard sensors. Digital goniometers are typically cheaper, more portable, and simpler to use than gold-standard motion tracking sensors. The current work suggests a method for generating accurate skeleton data suitable for training an RNN motion tracker based on the offline fusion of a Kinect 3D video sensor and an electronic goniometer. The fusion applies nonlinear constraint optimization, where the constraints are based on an advanced shoulder-centered kinematic model of the arm. The model builds on the representation of the arm as a triangle (the arm triangle). The shoulder-centered representation of the arm triangle motion simplifies constraint representation and consequently the optimization problem. To test the performance of the offline fusion and the RNN trained using the optimized data, arm motion of eight participants was recorded using a Kinect sensor, an electronic goniometer, and, for comparison, a passive-marker-based motion tracker. The data generated by fusing the Kinect and goniometer recordings were used for training two long short-term memory (LSTM) RNNs. The input to one RNN included both the Kinect and the goniometer data, and the input to the second RNN included only Kinect data. The performance of the networks was compared to the performance of a tracker based on a Kalman filter and to the raw Kinect measurements. The accuracy of the fused data was high, and it considerably improved data accuracy. The accuracy for both trackers was high, and both were more accurate than the Kalman filter tracker and the raw Kinect measurements. The developed methods are suitable for integration with immersive VR rehabilitation systems in the clinic and the home environments.

Optimized Hand Gesture Based Home Automation for Feebles

Abstract: Home automation innovation is utilized to plan and actualize inaccessible, vitality proficient and versatile keen homes with the essential highlights that guarantee inhabitant consolation and security. A savvy domestic robotization framework that mechanizes fans, lights, and other apparatuses for individuals with disabilities, the elderly, and out of commission patients utilizing hand signals. The framework comprises networks (sensors and gadget actuators and cameras that portray a controller that communicates with a hand-off that speaks to the client interface and an Arduino microcontroller). Smart homes are cost effective since they utilize video handling frameworks and sensors. Send and get commands utilizing the MQTT server. It employs a remote WIFI module named Esp8266 NodeMCU to supply smooth and hassle-free WIFI use.

Dynamic Synchronization Modeling and Simulation of Video Sensing Nodes Based on Internet of Things

In order to solve the dynamic synchronization problem of IoT video perception, the authors propose an abstract model DSAM based on IoT video perception. This method firstly establishes the abstract model DSAM of IoT video perception based on the π network and then analyzes the state evolution, model transition, and dynamic interaction of the model; finally, the model DSAM is used to analyze and simulate the example. The result shows that when the model DSAM is tested and verified and when the number of buffer media is 50, 100, and 200, the minimum number of underflows of the DSAM model is 2 and the minimum video data packet loss rate is 1.02; the results are better than those of the MTFSC and 3TFSC models; in addition, without synchronization control, the synchronization rate between video frames is lower than that with synchronization control. The test results show that the model proposed and established by the authors can correctly handle the dynamic synchronization of IoT video perception and has certain practical value.

Multimodal Assessment of Schizophrenia and Depression Utilizing Video, Acoustic, Locomotor, Electroencephalographic, and Heart Rate Technology: Protocol for an Observational Study.

BackgroundCurrent standards of psychiatric assessment and diagnostic evaluation rely primarily on the clinical subjective interpretation of a patient’s outward manifestations of their internal state. While psychometric tools can help to evaluate these behaviors more systematically, the tools still rely on the clinician’s interpretation of what are frequently nuanced speech and behavior patterns. With advances in computing power, increased availability of clinical data, and improving resolution of recording and sensor hardware (including acoustic, video, accelerometer, infrared, and other modalities), researchers have begun to demonstrate the feasibility of cutting-edge technologies in aiding the assessment of psychiatric disorders.ObjectiveWe present a research protocol that utilizes facial expression, eye gaze, voice and speech, locomotor, heart rate, and electroencephalography monitoring to assess schizophrenia symptoms and to distinguish patients with schizophrenia from those with other psychiatric disorders and control subjects.MethodsWe plan to recruit three outpatient groups: (1) 50 patients with schizophrenia, (2) 50 patients with unipolar major depressive disorder, and (3) 50 individuals with no psychiatric history. Using an internally developed semistructured interview, psychometrically validated clinical outcome measures, and a multimodal sensing system utilizing video, acoustic, actigraphic, heart rate, and electroencephalographic sensors, we aim to evaluate the system’s capacity in classifying subjects (schizophrenia, depression, or control), to evaluate the system’s sensitivity to within-group symptom severity, and to determine if such a system can further classify variations in disorder subtypes.ResultsData collection began in July 2020 and is expected to continue through December 2022.ConclusionsIf successful, this study will help advance current progress in developing state-of-the-art technology to aid clinical psychiatric assessment and treatment. If our findings suggest that these technologies are capable of resolving diagnoses and symptoms to the level of current psychometric testing and clinician judgment, we would be among the first to develop a system that can eventually be used by clinicians to more objectively diagnose and assess schizophrenia and depression with the possibility of less risk of bias. Such a tool has the potential to improve accessibility to care; to aid clinicians in objectively evaluating diagnoses, severity of symptoms, and treatment efficacy through time; and to reduce treatment-related morbidity.International Registered Report Identifier (IRRID)DERR1-10.2196/36417

Modern Clothing Design Based on Human 3D Somatosensory Technology

In the era of e-commerce, online clothing sales have grown rapidly, but the differences in clothing size and style details have brought serious problems to consumers’ purchase. At present, as far as online clothing sales are concerned, the most prominent problem is that consumers cannot try on clothes online as they do in the real environment, let alone touch and feel the texture of clothing fabrics. This has seriously affected and restricted the development of online clothing sales. The after-sales service cannot keep up, which is mainly reflected in the consumer return and replacement which is not convenient, and the purchased clothing cannot be received in time and other aspects. 3D garment customization is a hot area of research at present, and there is a wide range of needs to implement a software that can be customized for users. As a depth sensing technology, it strengthens the ability of video sensors for target recognition and brings new advances in sensing and processing deep vision tasks. Aiming at this problem, this paper proposes a modern clothing design scheme based on human 3D somatosensory technology. The designed scheme focuses on the acquisition of human size data and the personalized combination process of clothing components in the process of clothing design. In the scene where the human body changes greatly or the human body moves rapidly, the real-time limb coverage of the clothing fabric is higher, and the posture matching degree is higher. Applicable clothing types and human body pose types are more abundant, with a higher sense of reality. The clothing perception model fused with profile and girth features can match the dimensional changes of key parts of the human body. Through personalized clothing design, the combination of clothing parts is used to provide more choices of clothing styles, colors, and sizes. Using this solution, the time of clothing design can be greatly shortened, and the user’s satisfaction with the clothing design can be improved.

Can the MIT App Inventor® application be integrated into soil protection strategies?

This paper starts from the definition, descriptive characterization, and analysis of the MIT App Inventor® platform, according to the users' perceptions, to emphasize the malleability of the platform to the digital consumer society demands. The analysis of how the App Inventor® can be implemented in soil protection strategies is done in conjunction with the configuration of a digital field agenda. The analysis was conducted in the Office of Informatics, from October 2020 to December 2021, and involved identifying and scoring the elements that can make App Inventor® used to create specific soil protection apps. The analysis methodology included the main components of mobile devices that may be related to the acquisition and processing of field data (camera, video, navigation, map, and other sensors). As a result of the configuration and pre-testing of the application, it was observed that the App Inventor® can be used successfully in soils monitoring and protection.

Recognition of Geometric Figures and Determination of Their Characteristics by Means of Computer Vision

Introduction. Many computer vision applications often use procedures for recognizing various shapes and estimating their dimensional characteristics. The entire pipeline of such processing consists of several stages, each of which has no clearly defined boundaries. However, it can be divided into low, medium, and high-level processes. Low-level processes only deal with primitive operations such as preprocessing to reduce noise, enhance contrast, or sharpen images. The processes of this level are characterized by the fact that there are images at the input and output. Image processing at the middle level covers tasks such as segmentation, description of objects, and their compression into a form convenient for computer processing. Middle-level processes are characterized by the presence of images only at the input, and only signs and attributes extracted from images are received at the output. High-level processing involves “understanding” a set of recognized objects and recognizing their interactions. Using the example of the developed software models for recognizing figures and estimating their characteristics, it is shown that the image processing process is reduced to transforming spatial image data into metadata, compressing the amount of information, which leads to a significant increase in the importance of data. This indicates that at the input of the middle level, the image should be as informative as possible (with high contrast, no noise, artifacts, etc.) because after the transformation of the spatial image data into metadata, no further the procedures are not able to correct the data obtained by the video sensors in the direction of improving or increasing the information content. Recognition of figures in an image can be realized quite efficiently through the use of the procedure for determining the contours of figures. To do this, you need to determine the boundaries of objects and localize them in the image, often the first step for procedures such as separating objects from the background, image segmentation, detection and recognition of various objects, etc. The purpose of the article is to study the image processing pipeline from the moment of image fixation to the recognition of a certain set of figures (for example, geometric shapes, such as a triangle, quadrilateral, etc.) in an image, the development of software models for recognizing figures in an image, determining the center of mass figures by means of computer vision. Results. We proposed and tested some variants of nonlinear estimating problem. The properties of such problems depend on value of regulating parameter. The dependence of estimation on value of parameter was studied. It was defined a range for parameter's value for which estimating problem gives adequate result for initial task. Numerical examples show how much volume of calculations reduces when using a dynamic branching tree. Conclusions. The results obtained can be used in many applications of computer vision, for example, counting objects in a scene, estimating their parameters, estimating the distance between objects in a scene, etc. Keywords: contour, segmentation, image binarization, computer vision, histogram.