Camera Nodes Research Articles

IoT Video Delivery Optimization through Machine Learning-Based Frame Resolution Adjustment

Providing acceptable video quality in the Internet of Things (IoT) implementation poses a significant challenge, mainly when the application is performed on low-cost and low-power devices. This research focuses on developing a frame resolution adjustment system that maintains the frame rate value of video delivery in wireless IoT environments with resource-constrained devices. Consistent frame rates prevent motion lag and data loss, improving user experience. The system works by predicting the upcoming throughput values using machine learning methods to adjust the sensing parameter, which is the resolution of the video frame to be captured by camera nodes. Hence, the proposed system is equipped with a file size estimator to estimate the size of the next video frame and then adjust the resolution in accordance with the throughput prediction. In this research, we conducted extensive experiments to evaluate the accuracy of the file size estimator and the throughput prediction. The experiment generated a dataset to evaluate throughput prediction and file size estimator model. The evaluation results for the file size estimator showed a mean absolute percentage error (MAPE) of 6.73% in the experiment using 317 frames with video resolutions between 72p and 720p. Experiments were also conducted to compare several machine learning methods for predicting throughput values. Compared to long short-term memory (LSTM) and autoregressive integrated moving average (ARIMA), simple exponential smoothing (SES) outperforms the others with the lowest root mean squared error (RMSE) and mean absolute error (MAE) values. Building upon these findings, we implemented the frame resolution adjustment system using SES as the method for predicting the upcoming throughput values. Finally, we demonstrated that the proposed system can maintain the frame rate according to the threshold set by the system while the resolution is being maximized, thereby addressing the challenges of maintaining video quality in resource-constrained IoT environments.

Aligning security and energy-efficiency using change detection and partial encryption for wireless camera networks

Wireless camera networks (WCNs) have seen widespread deployment in recent years, leading to an upsurge in monitoring and surveillance systems aimed at detecting foreign objects and changes within remote environments. However, wireless communication remains susceptible to cybersecurity threats, enabling eavesdroppers to access and manipulate confidential multimedia data effortlessly. Moreover, wireless cameras are low-power devices, encounter challenges as many state-of-the-art schemes adopt intricate security mechanisms, resulting in elevated costs, latency, and power consumption. This significantly compromises the stability of the node power and network lifetime and limits the applicability of security schemes in real-time implementations. This study addresses these issues by presenting DyimEnc, a dynamic image encryption scheme, to establish secure image transmission within wireless camera networks. The proposed scheme dynamically implements a security, complexity, and power consumption trade-off based on the currently available resources in each camera node. In the DyimEnc scheme, an effective in-node change-detection method is used to detect changes in the captured images and reduce the false transmission of images across the network. Additionally, a dynamic lightweight partial-encryption method is proposed to encrypt the edge maps of sensitive images that include changes. Moreover, a flexible key-management method is proposed for image encryption and transmission. Simulation results underscore DyimEnc's substantial impact, revealing significant reductions in power consumption, encryption cost, and transmission latency. It effectively achieves an average throughput of 254.9 KB/s and a remarkable 48 % decrease in network power consumption, showcasing a substantial 59 % improvement compared to a related study. Security analysis proved the robustness of our scheme in resisting attacks and satisfying the security requirements of WCNs.

WSN-IoT Forecast: Wireless Sensor Network Throughput Prediction Framework in Multimedia Internet of Things

Accurate throughput predictions can significantly improve the quality of experience (QoE), where QoE denotes a network’s capacity to provide satisfactory service. By increasing the results of good throughput predictions, the best strategy can be planned for managing data transmission networks with the aim of better and faster data transmission, thereby increasing QoE. Consequently, this paper investigates how to predict the throughput of wireless sensor networks utilizing multimedia data. First, we conducted a comparative analysis of relevant prior research on the topic of throughput prediction in Multimedia Internet of Things (Multimedia IoT). We developed a throughput prediction framework for wireless sensor networks based on what we learned from these studies using machine learning. The Throughput Prediction Framework identifies historical throughput data and employs these traits to predict throughput. In the final phase, multiple camera nodes and local servers are utilized to test a framework for throughput prediction. Our analysis demonstrates that WSN-IoT predictions are quite precise. For a 1-second time breakdown, the average absolute percentage error for all investigated scenarios ranges from 1 to 8 percent.

Redundant node identification using Siamese neural network to enhance the availability in visual sensor network

SummaryMany monitoring applications related to surveillance, tracking and multipurpose visual monitoring have taken into consideration the use of wireless visual sensor networks. When sensors are deployed over a monitored field that could potentially damage the monitoring capability and availability in the visual sensor network. In order to overcome these issues, faulty nodes are identified and replaced by redundant node based on Siamese network in this research. Initially, camera nodes are randomly deployed in the visual sensor network, and the data are received from the network through gateway. To identify the redundant nodes, initially, the frames are divided into equivalent time slot, and then, Siamese network is utilized to identify the redundant nodes in a network. Siamese neural network is type of convolutional neural network that is utilized to recognize the similar images in the network. After that, faulty nodes are identified based on some parameters such as entropy, energy, transmission delay and network coverage. If the average energy, entropy, transmission delay and network coverage are below the threshold value, then the node is identified as faulty node. Finally, replace the faulty node with redundant node to enhance the availability in the visual sensor network for critical monitoring applications. The simulation analysis shows that the developed approach takes 772 s to identify redundant node and take 27 s to identify fake nodes and the developed method is executed at 1308 s. Thus, this prediction model helps to improve the coverage quality of target‐based monitoring in order to achieve availability.

Real-time distributed video analytics for privacy-aware person search

In this work, a novel distributed privacy-aware person search (PAPS) model has been proposed which circumvents the privacy risks. An intelligent IoT surveillance system has been designed to integrate the PAPS model for real-time distributed privacy-aware person search from surveillance videos. An important aspect of the intelligent surveillance system, particularly person search, is the visual feedback at the output, with ranked results of person images at the user-end. Therefore, even if edge processing is performed, there is still a need to store and transmit the cropped person images to the cloud server for displaying the results at the user-end. However, storing or transmission of videos/images to cloud-servers leads to privacy issues. The proposed PAPS model eliminates the need to store or transmit the images/videos while performing person search, thereby addressing the privacy concerns. The proposed system is easily scalable to incorporate more camera nodes to enhance the surveillance coverage as majority of the processing is performed at the edge servers, with a small amount of fog-processing. A very minimal amount of cloud-processing is performed only when a query is raised at the user-end. Only the processed and encoded data is transmitted across the edge, fog and the cloud servers, which protects privacy and significantly reduces bandwidth costs. Further, a new evaluation criterion, Person Capacity, has been proposed to evaluate the feasibility of an edge-based system to be deployed at crowded locations. The performance evaluation of our system, on our own video dataset, as well as the PRW, and CUHK-SYSU dataset for person search demonstrates that the proposed system achieves state-of-the-art or competitive performance while performing in real-time for practical scenarios.

Multi-Camera Extrinsic Calibration for Real-Time Tracking in Large Outdoor Environments

Calibrating intrinsic and extrinsic camera parameters is a fundamental problem that is a preliminary task for a wide variety of applications, from robotics to computer vision to surveillance and industrial tasks. With the advent of Internet of Things (IoT) technology and edge computing capabilities, the ability to track motion activities in large outdoor areas has become feasible. The proposed work presents a network of IoT camera nodes and a dissertation on two possible approaches for automatically estimating their poses. One approach follows the Structure from Motion (SfM) pipeline, while the other is marker-based. Both methods exploit the correspondence of features detected by cameras on synchronized frames. A preliminary indoor experiment was conducted to assess the performance of the two methods compared to ground truth measurements, employing a commercial tracking system of millimetric precision. Outdoor experiments directly compared the two approaches on a larger setup. The results show that the proposed SfM pipeline more accurately estimates the pose of the cameras. In addition, in the indoor setup, the same methods were used for a tracking application to show a practical use case.

A distributed scheme for energy-efficient event-based target recognition using Internet of Multimedia Things

The availability of low-cost embedded devices for multimedia sensing has encouraged their integration with low-power wireless sensors to create systems that enable advanced services and applications referred to as the Internet of Multimedia Things. Image-based sensing applications are challenged by energy efficiency and resource availability. Mainly, image sensing and transmission in Internet of Multimedia Things severely deplete the sensor energy and overflow the network bandwidth with redundant data. Some solutions presented in the literature, such as image compression, do not efficiently solve this problem because of the algorithms’ computational complexities. Thus, detecting the event of interest locally before the communication using shape-based descriptors would avoid useless data transmission and would extend the network lifetime. In this article, we propose a new approach of distributed event-based sensing scheme over a set of nodes forming a processing cluster to balance the processing load. This approach is intended to reduce per-node energy consumption in one sensing cycle. The conducted experiments show that our novel method based on the general Fourier descriptor decreases the energy consumption in the camera node to only 2.4 mJ, which corresponds to 75.32% of energy-saving compared to the centralized approach, promising to prolong the network lifetime significantly. In addition, the scheme achieved more than 95% accuracy in target recognition.

Smart Surveillance System Real-Time Multi-Person Multi-Camera Tracking at the Edge

Abstract: Nowadays, new Artificial Intelligence (AI) and Deep Learning based processing methods are replacing traditional computer vision algorithms. On the other hand, the rise of the Internet of Things (IoT) and edge computing, has led to many research works that propose distributed video-surveillance systems based on this notion. Usually, the advanced systems process massive volumes of data in different computing facilities. Instead, this paper presents a system that incorporates AI algorithms into low-power embedded devices. The computer vision technique, which is commonly used in surveillance applications, is designed to identify, count, and monitor people's movements in the area. A distributed camera system is required for this application. The proposed AI system detects people in the monitored area using a MobileNet-SSD architecture. This algorithm can keep track of people in the surveillance providing the number of people present in the frame. The proposed framework is both privacy-aware and scalable supporting a processing pipeline on the edge consisting of person detection, tracking and robust person re-identification. The expected results show the usefulness of deploying this smart camera node throughout a distributed surveillance system. Keywords: Edge Analytics, Person detection, Person re-identification, Deep learning, embedded systems, artificial intelligence

Edge Intelligence for Rendering Green Camera-Network-as-a-Service

This work proposes the Green Camera-Network-as-a-Service (G-CNaaS) architecture, which provides on-demand camera networks to multiple end-users simultaneously while utilizing minimal energy. G-CNaaS simultaneously reduces the carbon footprint and eliminates the single application-centric approach of traditional camera networks (TCNs) by enabling each camera to participate in multiple Virtual-Camera-Networks (VCNs) and selecting an optimal set of cameras for each VCN. We couple each camera node in every VCNs with a learning model suitable for the requested application. We assign an intelligent edge device to each VCN to analyze time-sensitive events. We introduce the camera selection factor by leveraging the properties of cameras: 1) field-of-view (FoV); 2) angular-distance; 3) observation range; and 4) residual energy to select the optimal camera set. The results of the extensive simulation of the G-CNaaS architecture show that it excels in performance concerning attributes such as the average lifetime, fair distribution of the work among the camera owners, and cost-effectiveness compared to the TCNs. We observe that the expenditure of a user using the TCN varies by 88.7%, while in the case of G-CNaaS, the expenditure varies by 10.28% with the increase in time from 1–60 months. On the other hand, the average energy consumed increases by 59.88% and 99.5% in the presence of 10 and 20 camera sensor owners.

Smart Video Surveillance System Based on Edge Computing.

New processing methods based on artificial intelligence (AI) and deep learning are replacing traditional computer vision algorithms. The more advanced systems can process huge amounts of data in large computing facilities. In contrast, this paper presents a smart video surveillance system executing AI algorithms in low power consumption embedded devices. The computer vision algorithm, typical for surveillance applications, aims to detect, count and track people’s movements in the area. This application requires a distributed smart camera system. The proposed AI application allows detecting people in the surveillance area using a MobileNet-SSD architecture. In addition, using a robust Kalman filter bank, the algorithm can keep track of people in the video also providing people counting information. The detection results are excellent considering the constraints imposed on the process. The selected architecture for the edge node is based on a UpSquared2 device that includes a vision processor unit (VPU) capable of accelerating the AI CNN inference. The results section provides information about the image processing time when multiple video cameras are connected to the same edge node, people detection precision and recall curves, and the energy consumption of the system. The discussion of results shows the usefulness of deploying this smart camera node throughout a distributed surveillance system.

Real-Time In-Network Image Compression via Distributed Dictionary Learning

Multi-camera networks are increasingly becoming pervasive in many monitoring and surveillance applications, and have attracted much attention in distributed systems with collaborative, real-time decision-making capabilities. While in-network data compression brings significant energy savings in camera nodes, signal representation using sparse approximations and overcomplete dictionaries have been shown to outperform traditional compression methods. In this work, an end-to-end and real-time solution is designed and implemented to enable energy-efficient and robust dictionary learning in distributed camera networks by leveraging the spatial correlation of the collected multimedia data. Traditional distributed dictionary learning relies on consensus-building algorithms, which involve communicating with neighboring nodes until convergence is achieved. Existing methods, however, do not exploit spatial correlations in camera networks for improved energy efficiency. In contrast, low-computational-complexity metrics are employed in this work to quantify and exploit the spatial correlation across camera nodes in a wireless network for efficient distributed dictionary learning and in-network image compression. The performance of the proposed approach is validated through extensive simulations on public datasets as well as via real-world experiments on a testbed composed of Raspberry Pi nodes.

IOT Based Home Security And Automation Using Google Assistant

Internet of Things (IOT) is an important technology that affects the internet and communication technology. IOT allows people and objects to be connected anytime, anywhere, in any way/shape and function. Earth's mechanical systems have become very popular in the last few decades and improved living conditions. This project provides the structure and model of an automated earth system. The most important feature is the Raspberry pi, which has an internal Wi-Fi module that allows you to control your device over the Internet. It supports a variety of earth tools, such as energy management systems. Besides, the application is built where the user can control the device using Google Assistant. The IOT project also aims to build a wireless home protection system that uses the ESP32 camera Node MCU module to send alerts to owners if anyone takes a picture of the door and sends it to the owner.

Target Tracking in Wireless Visual Sensor Networks: Challenges, Steps, and Metrics of Evaluation

In recent years, wireless sensor networks have been used in a wide range of applications such as smart cities, military, and environmental monitoring. Target tracking is one of the most interesting applications in this area of research, which mainly consists of detecting the targets that move in the area of interest and monitoring their motions. However, tracking a target using visual sensors is different and more difficult than that of scalar sensors due to the special characteristics of visual sensors, such as their directional limited field of view, and the nature and amount of the sensed data. In this paper, we first present the challenges of detection and target tracking in wireless visual sensor networks, then we propose a scheme that describes the basic steps of target tracking in these networks, we focus then on the tracking across camera nodes by presenting some metrics that can be considered when designing and evaluating this type of tracking approaches.

A New System for Real-time Video Surveillance in Smart Cities Based on Wireless Visual Sensor Networks and Fog Computing

Nowadays, public security is becoming an increasingly serious issue in our society and its requirements have been extended from urban centers to all remote areas. Therefore, surveillance and security cameras are being deployed worldwide. Wireless Visual Sensor Networks nodes can be employed as camera nodes to monitor in the city without the need for any cables installation. However, these cameras are constrained in processing, memory, and energy resources. Also, they generate a massive amount of data that must be analyzed in real-time to ensure public safety and deal with emergency situations. As a result, data processing, information fusion, and decision making have to be executed on-site (near to the data collection location). Besides, surveillance cameras are directional sensors, which makes the coverage problem another issue to deal with. Therefore, we present a new system for real-time video surveillance in a smart city, in which transportations equipped with camera nodes are used as the mobile part of the system and an architecture based on fog computing and wireless visual sensor networks is adopted. Furthermore, we propose an approach for selecting the camera nodes that will participate in the tracking process and we simulated three different use cases to test the effectiveness of our system in terms of target detection. The simulation results show that our system is a promising solution for smart city surveillance applications.

Fast Sample Adaptive Offset Jointly Based on HOG Features and Depth Information for VVC in Visual Sensor Networks.

Visual sensor networks (VSNs) can be widely used in multimedia, security monitoring, network camera, industrial detection, and other fields. However, with the development of new communication technology and the increase of the number of camera nodes in VSN, transmitting and compressing the huge amounts of video and image data generated by video and image sensors has become a major challenge. The next-generation video coding standard—versatile video coding (VVC), can effectively compress the visual data, but the higher compression rate is at the cost of heavy computational complexity. Therefore, it is vital to reduce the coding complexity for the VVC encoder to be used in VSNs. In this paper, we propose a sample adaptive offset (SAO) acceleration method by jointly considering the histogram of oriented gradient (HOG) features and the depth information for VVC, which reduces the computational complexity in VSNs. Specifically, first, the offset mode selection (select band offset (BO) mode or edge offset (EO) mode) is simplified by utilizing the partition depth of coding tree unit (CTU). Then, for EO mode, the directional pattern selection is simplified by using HOG features and support vector machine (SVM). Finally, experimental results show that the proposed method averagely saves 67.79% of SAO encoding time only with 0.52% BD-rate degradation compared to the state-of-the-art method in VVC reference software (VTM 5.0) for VSNs.

Steganography-assisted secure localization of smart devices in internet of multimedia things (IoMT)

Steganography is a secure way of communicating secret visual data/information from one place to another via multimedia carriers. Digital steganography, spatial and frequency domain has taken several methods over the past two decades. In this work, we present steganography-assisted secure localization of smart devices for Visual Sensor Network (VSN). The proposed system is based on achromatic-component (Y-Plane) of the YCbCr color space and maximum likelihood estimation algorithm (MLEA) in the spatial domain. Data/information security via multimedia carriers is the main concern in this technological era, if secret information is captured by a Visual Sensor network, (VSN) then the capture information in the case to send from one place to another needs fundamental security to keep the information confidential. In the proposed steganography-assisted Visual Sensor Network (VSN) the camera nodes are intelligent to process the image data and to extract the important information and the rich-description are provided to the system’s user for further action. In the proposed technique first, the input image is rotated on 90 degrees and transformed into YCbCr color space. Furthermore, the secret data is separated into different blocks of the same size based on key giving sub-blocks in Y-Plane, secret information is then encrypted using an MLE algorithm (MLEA) and the corresponding blocks of secret data is embedded in the sub-image of Y-Plane. The experimental results authenticate that the proposed technique does not only boost the visual quality of stegno-images but also delivers good imperceptibility, robustness, and security as compared to state-of-the-art approaches.

Right-of-Way Monitoring Camera Storage Energy Around High Voltage Power Transmission Using Hybrid Energy Harvesting-Mfield, Efield to Super Capacitor Batteries Back-Up Charger

Magnetic and electric fields are the loss of the field that occur continually around high voltage transmission lines in the power systems. The current and voltage are significant variables for generating magnetic and electric fields respectively. This research is interested in the development of these fields to store harvested energy and the correct way of using camera node monitoring. In this work, the authors purpose energy harvesting from magnetic and electric fields, with a case study based on 230 kV and 500 kV in the Thai power transmission system. Two approaches are used for the hybrid harvester, with the first method using the number of coil turns in each dielectric physical model, such as air, metal, or ferrite dielectric to harvest the magnetic field from the power line. The number of coils uses N-turn, 250 N, 600 N, and 1000 N, with a dielectric distance of 4, 6, and 10 cm is used. The second approach is electric field harvest, using a varied electrode plate from ground to an electric conductor, with three plates used in the experimental model. At the investigational stage, the common plate is varied from the nearest conductor to the ground to analyses data results. After that, two sources of energy are taken to the hybrid storage harvester using the rectifier circuit and stored in a super capacitor. The energy from the backup super capacitor is used to supply to the right-of-way camera monitoring or other low energy source devices. This study result show that energy harvesting from the field harvest varied according to the number of coil turns, distance of the dielectric, size of the plate, and distance from the high voltage conductor. The voltage from field can be stored to super capacitor from 0 V to approximately 0.8 V in 180 min for a magnetic field yield. Furthermore, the electric field harvesting can charge the supper capacitor from 0 to 15 V in 180 min. The research therefore combines two approaches for charging in the battery backup using low energy devices, referred to as a hybrid harvesting approach. This research is advantageous for right-of-way camera monitoring supply on high voltage–power transmission lines.

A Distributed Image Compression Scheme for Energy Harvesting Wireless Multimedia Sensor Networks.

As an emerging technology, edge computing will enable traditional sensor networks to be effective and motivate a series of new applications. Meanwhile, limited battery power directly affects the performance and survival time of sensor networks. As an extension application for traditional sensor networks, the energy consumption of Wireless Multimedia Sensor Networks (WMSNs) is more prominent. For the image compression and transmission in WMSNs, consider using solar energy as the replenishment of node energy; a distributed image compression scheme based on solar energy harvesting is proposed. Two level clustering management is adopted. The camera node-normal node cluster enables camera nodes to gather and send collected raw images to the corresponding normal nodes for compression, and the normal node cluster enables the normal nodes to send the compressed images to the corresponding cluster head node. The re-clustering and dynamic adjustment methods for normal nodes are proposed to adjust adaptively the operation mode in the working chain. Simulation results show that the proposed distributed image compression scheme can effectively balance the energy consumption of the network. Compared with the existing image transmission schemes, the proposed scheme can transmit more and higher quality images and ensure the survival of the network.

Communication and computation inter-effects in people counting using intelligence partitioning

The rapid development of the Internet of Things is affecting the requirements towards wireless vision sensor networks (WVSN). Future smart camera architectures require battery-operated devices to facilitate deployment for scenarios such as industrial monitoring, environmental monitoring and smart city, consequently imposing constraints on the node energy consumption. This paper provides an analysis of the inter-effects between computation and communication energy for a smart camera node. Based on a people counting scenario, we evaluate the trade-off for the node energy consumption with different processing configurations of the image processing tasks, and several communication technologies. The results indicate that the optimal partition between the smart camera node and remote processing is with background modelling, segmentation, morphology and binary compression implemented in the smart camera, supported by Bluetooth Low Energy (BLE) version 5 technologies. The comparative assessment of these results with other implementation scenarios underlines the energy efficiency of this approach. This work changes pre-conceptions regarding design space exploration in WVSN, motivating further investigation regarding the inclusion of intermediate processing layers between the node and the cloud to interlace low-power configurations of communication and processing architectures.

Wireless Visual Sensor Networks Energy Optimization Based on New Entropy Model

Last advances in low power sensors has led to the development of wireless visual sensor networks. These networks comparing to traditional wireless sensor networks provides valuable visual information, therefor are suitable for surveillance and control applications. One of the important challenges in wireless visual sensor networks is energy consumption. Therefore, in this paper we address the problem of energy optimization in wireless visual sensor networks. The entropy of the captured images in each camera node sensor as a quality criteria is used. For realization of this goal, we introduced a new formula for expressing the entropy of captured image in each camera node. This model is a function of the distance between camera node and target and the angel between the main view line of camera and the target. Then, we introduced a new formula which expresses the relation between the image entropy and the number of bits required for displaying image pixel value. In next step, we proposed node selection algorithm based on entropy. For proposing our algorithm, first optimization of energy consumption is formulated, then the problem is solved as a convex problem. We used from CVX library in MATLAB and Log-Barrier Method for solving our problem and shown the simulation result of them. Finally, we compare our proposed algorithm with MDT benchmark algorithm, CVX and Log-Barrier algorithms. Also we compare the sensitivity to error for both algorithms.