IoT Datasets Research Articles

From Network Inference Errors to Utility Suboptimality: How Much Is the Impact?

Accurate and timely network inference, known as network tomography, is a vital ingredient in efficient network operation, supporting more sophisticated and ambitious traffic optimization algorithms. In practice, however, precise knowledge of the dynamic internal network bottlenecks/states can be impossible to obtain under restricted network visibility on network edge where only end-to-end measurements are available without cooperation of internal network components. In this paper, we acknowledge that network tomography on the edge is often imperfect, leading to dynamic, time-varying inference errors that could change from iteration to iteration on the same time-scale as distributed network optimization algorithms. We quantify the impact of such imperfect bottleneck/state inference on algorithm convergence and optimality. In particular, we show that under arbitrary, bounded inference errors (belonging to three common classes including absent and incorrect bottlenecks, and inaccurate capacity), the solution of the distributed optimization algorithm still converges to a bounded neighborhood of the optimal solution. The resulted optimality gap is quantified in closed form and shown to be proportional to average inference errors. These results are evaluated using extensive network simulations and on real-world IoT data sets. The work provide a theoretical support for understanding the impact of imperfect inference on distributed network optimization.

Blockchain of Resource-Efficient Anonymity Protection with Watermarking for IoT Big Data Market

According to the ever-growing supply and demand of IoT content, IoT big data in diversified applications are deemed a valuable asset by private and public sectors. Their privacy protection has been a hot research topic. Inspired by previous work on bounded-error-pruned IoT content market, we observe that the anonymity protection with robust watermarking can be developed by further pruning data for better resource-efficient IoT big data without violating the required quality of sensor service or quality of decision-making. In this paper, resource-efficient anonymity protection with watermarking is thus proposed for data consumers and owners of IoT big data market via blockchain. Our proposed scheme can provide the IoT data with privacy protections of both anonymity and ownership in IoT big data market with resource efficiency. The experiments of four different-type IoT datasets with different settings included bounded-errors, sub-stream sizes, watermark lengths, and ratios of data tampering. The performance results demonstrated that our proposed scheme can provide data owners and consumers with ownership and anonymity via watermarking the IoT big data streams for lossless compressibility. Meanwhile, the developed DApp with our proposed scheme on the Ethereum blockchain can help data owners freely share and trade with consumers in convenience with availability, reliability, and security without mutual trust.

Deep learning approach to solve image retrieval issues associated with IOT sensors

Deep learning is particularly well suited for that kind of circumstance, but ensuring confidentiality and safety has turned into a major problem in IoT management. A principal component analysis (PCA) is included in this specific research to identify and extract features more effectively. Additionally, the primary purpose of this study project is to compile an in-depth survey mostly on various IoT installations, and concerns regarding security and privacy with something like a rapid percentage of detection. The achievement of a higher detection frequency in IoT image classification is another main objective of this research work. Mostly on IoT datasets, the CNN methodology was trained and validated for effectiveness by using a wide range of approaches. Investigating the use of deep learning with IoT capturing images might be the initial phase. Furthermore, the value of such a deep learning approach is mostly assessed for improving the suitability of image identification with continuous testing reliability whenever it pertains to IoT image registration. An image identification approach that provides a range of acceptable criteria summarizes the study findings on such use of deep learning inside the IoT platform.

ProvNet-IoT: Provenance based network layer forensics in Internet of Things

Internet of Things is rapidly changing the human lives to bring convenience in domestic, public and industrial environments spanning across multiple application domains. At the same time, increasing security attacks on these networks raised alarms for timely response by forensic investigators to avoid severe consequences of the attacks. Major network forensic approaches proposed so far for IoT are based on recording and analyzing the network traffic to produce suitable evidences. One of the greatest challenges in this process is the identification and correlation of suitable artifacts among volumes of network packets to reconstruct the attack scenarios during forensic investigation. To address this challenge, we propose ProvNet-IoT, a novel provenance based forensic model for investigating network level attacks in IoT environment. The interactions between different nodes at network layer are depicted using information, functional, and event modeling techniques. We use progressive network provenance to explain different events pertaining to various attack scenarios and to provide forensically sound evidences. ProvNet-IoT is validated using two publicly available labeled IoT datasets with a corpus of different attacks. Experimental results showed the benchmark performance of ProvNet-IoT in identifying selective artifacts to produce reliable evidences during forensic investigation.

Improving Massive Access to IoT Gateways

IoT networks handle incoming packets from large numbers of IoT Devices (IoTDs) to IoT Gateways. This can lead to the IoT Massive Access Problem that causes buffer overflow, large end-to-end delays and missed deadlines. This paper analyzes a novel traffic shaping method named the Quasi-Deterministic Traffic Policy (QDTP) that mitigates this problem by shaping the incoming traffic without increasing the end-to-end delay or dropping packets. Using queueing theoretic techniques and extensive data driven simulations with real IoT datasets, the value of QDTP is shown as a means to considerably reduce congestion at the Gateway, and significantly improve the IoT network’s overall performance.

Dynamic Ransomware Detection for Windows Platform Using Machine Learning Classifiers

In this world of growing technological advancements, ransomware attacks are also on the rise. This threat often affects the finance of individuals, organizations, and financial sectors. In order to effectively detect and block these ransomware threats, the dynamic analysis strategy was proposed and carried out as the approach of this research. This paper aims to detect ransomware attacks with dynamic analysis and classify the attacks using various machine learning classifiers namely: Random Forest, Naïve Bayes, J48, Decision Table and Hoeffding Tree. The TON IoT Datasets from the University of New South Wales' (UNSW) were used to capture ransomware attack features on Windows 7. During the experiment, a testbed was configured with numerous virtual Windows 7 machines and a single attacker host to carry out the ransomware attack. A total of 77 classification features are selected based on the changes before and after the attack. Random Forest and J48 classifiers outperformed other classifiers with the highest accuracy results of 99.74%. The confusion matrix highlights that both Random Forest and J48 classifiers are able to accurately classify the ransomware attacks with the AUC value of 0.997 respectively.  Our experimental result also suggests that dynamic analysis with machine learning classifier is an effective solution to detect ransomware with the accuracy percentage exceeds 98%.

Towards an Explainable Universal Feature Set for IoT Intrusion Detection.

As IoT devices’ adoption grows rapidly, security plays an important role in our daily lives. As part of the effort to counter these security threats in recent years, many IoT intrusion detection datasets were presented, such as TON_IoT, BoT-IoT, and Aposemat IoT-23. These datasets were used to build many machine learning-based IoT intrusion detection models. In this research, we present an explainable and efficient method for selecting the most effective universal features from IoT intrusion detection datasets that can help in producing highly-accurate and efficient machine learning-based intrusion detection systems. The proposed method was applied to TON_IoT, Aposemat IoT-23, and IoT-ID datasets and resulted in the selection of six universal network-flow features. The proposed method was tested and produced a high accuracy of 99.62% with a prediction time reduced by up to 70%. To provide better insight into the operation of the classifier, a Shapley additive explanation was used to explain the selected features and to prove the alignment of the explanation with current attack techniques.

Machine learning-based early detection of IoT botnets using network-edge traffic

In this work, we present an IoT botnet detection solution, EDIMA, consisting of a set of lightweight modules designed to be deployed at the edge gateway installed in home networks with the remaining modules expected to be implemented on cloud servers. EDIMA targets early detection of IoT botnets prior to the launch of an attack and includes a novel two-stage Machine Learning (ML)-based detector developed specifically for IoT bot detection at the edge gateway. The ML-based bot detector first employs supervised ML algorithms for aggregate traffic classification and subsequently Autocorrelation Function (ACF)-based tests to detect individual bots. The EDIMA architecture also comprises a malware traffic database, a policy engine, a feature extractor and a traffic parser. Performance evaluation results using our testbed setup with real-world IoT malware traffic as well as other public IoT datasets show that EDIMA achieves high bot scanning and bot-CnC traffic detection accuracies with very low false positive rates. The detection performance is also shown to be robust to an increase in the number of IoT devices connected to the edge gateway where EDIMA is deployed. Further, the runtime performance analysis of a Python implementation of EDIMA deployed on a Raspberry Pi reveals low bot detection delays and low RAM consumption. EDIMA is also shown to outperform existing detection techniques for bot scanning traffic and bot-CnC server communication.

Towards a Hybrid Machine Learning Model for Intelligent Cyber Threat Identification in Smart City Environments

The concept of a smart city requires the integration of information and communication technologies and devices over a network for the better provision of services to citizens. As a result, the quality of living is improved by continuous analyses of data to improve service delivery by governments and other organizations. Due to the presence of extensive devices and data flow over networks, the probability of cyber attacks and intrusion detection has increased. The monitoring of this huge amount of data traffic is very difficult, though machine learning algorithms have huge potential to support this task. In this study, we compared different machine learning models used for cyber threat classification. Our comparison was focused on the analyzed cyber threats, algorithms, and performance of these models. We have identified that real-time classification, accuracy, and false-positive rates are still the major issues in the performance of existing models. Accordingly, we have proposed a hybrid deep learning (DL) model for cyber threat intelligence (CTI) to improve threat classification performance. Our model was based on a convolutional neural network (CNN) and quasi-recurrent neural network (QRNN). The use of QRNN not only resulted in improved accuracy but also enabled real-time classification. The model was tested on BoT-IoT and TON_IoT datasets, and the results showed that the proposed model outperformed the other models. Due to this improved performance, we emphasize that the application of this model in the real-time environment of a smart system network will help in reducing threats in a reasonable time.

Performance analysis of machine learning models for intrusion detection system using Gini Impurity-based Weighted Random Forest (GIWRF) feature selection technique

To protect the network, resources, and sensitive data, the intrusion detection system (IDS) has become a fundamental component of organizations that prevents cybercriminal activities. Several approaches have been introduced and implemented to thwart malicious activities so far. Due to the effectiveness of machine learning (ML) methods, the proposed approach applied several ML models for the intrusion detection system. In order to evaluate the performance of models, UNSW-NB 15 and Network TON_IoT datasets were used for offline analysis. Both datasets are comparatively newer than the NSL-KDD dataset to represent modern-day attacks. However, the performance analysis was carried out by training and testing the Decision Tree (DT), Gradient Boosting Tree (GBT), Multilayer Perceptron (MLP), AdaBoost, Long-Short Term Memory (LSTM), and Gated Recurrent Unit (GRU) for the binary classification task. As the performance of IDS deteriorates with a high dimensional feature vector, an optimum set of features was selected through a Gini Impurity-based Weighted Random Forest (GIWRF) model as the embedded feature selection technique. This technique employed Gini impurity as the splitting criterion of trees and adjusted the weights for two different classes of the imbalanced data to make the learning algorithm understand the class distribution. Based upon the importance score, 20 features were selected from UNSW-NB 15 and 10 features from the Network TON_IoT dataset. The experimental result revealed that DT performed well with the feature selection technique than other trained models of this experiment. Moreover, the proposed GIWRF-DT outperformed other existing methods surveyed in the literature in terms of the F1 score.

A Novel Optimization of Cloud Instances with Inventory Theory Applied on Real Time IoT Data of Stochastic Nature

A Horizontal scaling is a Cloud architectural strategy by which the number of nodes or computers increased to meet the demand of continuously increasing workload. The cost of compute instances increases with increased workload & the research is aimed to bring an optimization of the reserved Cloud instances using principles of Inventory theory applied to IoT datasets with variable stochastic nature. With a structured solution architecture laid down for the business problem to understand the checkpoints of compute instances – the range of approximate reserved compute instances have been optimized & pinpointed by analysing the probability distribution curves of the IoT datasets. The Inventory theory applied to the distribution curves of the data provides the optimized number of compute instances required taking the range prescribed from the solution architecture. The solution would help Cloud solution architects & Project sponsors in planning the compute power required in AWS® Cloud platform in any business situation where ingestion & processing data of stochastic nature is a business need.

Construction of subsystem determination of attacks in cyberphysical systems by neural network methods

In this research work analyzes and compares existing methods for describing data from cyberphysical systems, methods for detecting network attacks targeting cyberphysical systems, analyzes fundamental approaches and solutions in the field of cyberphysical systems security, and makes recommendations for supplementing existing approaches using new algorithms. The considered application of the neuroevolutionary algorithm of NeuroEvolution of Augmenting Topology using a hypercube for the analysis of multivariate time series describing the state of cyberphysical systems in order to identify abnormal conditions. After the modification, the algorithm allows almost completely configuring the target neural network without user intervention according to the specified parameters, including additionally creating intermediate network layers that were previously unavailable in the primary version of the algorithm. The method is verified on the TON_IOT DATASETS dataset. The system topology is the structure of the Internet of Things. The data are relevant, verified and correct, which allows them to be used for analysis and assessment of the accuracy of the approach under consideration. The obtained overall accuracy, proximity of solutions, values of False Positive Rate and False Negative Rate indicate the lack of retraining of the model and the high reliability of this method for detecting attacks in cyberphysical systems

IoT Dataset Validation Using Machine Learning Techniques for Traffic Anomaly Detection

With advancements in engineering and science, the application of smart systems is increasing, generating a faster growth of the IoT network traffic. The limitations due to IoT restricted power and computing devices also raise concerns about security vulnerabilities. Machine learning-based techniques have recently gained credibility in a successful application for the detection of network anomalies, including IoT networks. However, machine learning techniques cannot work without representative data. Given the scarcity of IoT datasets, the DAD emerged as an instrument for knowing the behavior of dedicated IoT-MQTT networks. This paper aims to validate the DAD dataset by applying Logistic Regression, Naive Bayes, Random Forest, AdaBoost, and Support Vector Machine to detect traffic anomalies in IoT. To obtain the best results, techniques for handling unbalanced data, feature selection, and grid search for hyperparameter optimization have been used. The experimental results show that the proposed dataset can achieve a high detection rate in all the experiments, providing the best mean accuracy of 0.99 for the tree-based models, with a low false-positive rate, ensuring effective anomaly detection.

Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach

The advances in deep neural networks (DNN) have significantly enhanced real-time detection of anomalous data in IoT applications. However, the complexity-accuracy-delay dilemma persists: Complex DNN models offer higher accuracy, but typical IoT devices can barely afford the computation load, and the remedy of offloading the load to the cloud incurs long delay. In this article, we address this challenge by proposing an adaptive anomaly detection scheme with hierarchical edge computing (HEC). Specifically, we first construct multiple anomaly detection DNN models with increasing complexity and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network . We also incorporate a parallelism policy training method to accelerate the training process by taking advantage of distributed models. We build an HEC testbed using real IoT devices and implement and evaluate our contextual-bandit approach with both univariate and multivariate IoT datasets. In comparison with both baseline and state-of-the-art schemes, our adaptive approach strikes the best accuracy-delay tradeoff on the univariate dataset and achieves the best accuracy and F1-score on the multivariate dataset with only negligibly longer delay than the best (but inflexible) scheme.

CBO-IE: A Data Mining Approach for Healthcare IoT Dataset Using Chaotic Biogeography-Based Optimization and Information Entropy

Data mining is mostly utilized for a huge variety of applications in several fields like education, medical, surveillance, and industries. The clustering is an important method of data mining, in which data elements are divided into groups (clusters) to provide better quality data analysis. The Biogeography-Based Optimization (BO) is the latest metaheuristic approach, which is applied to resolve several complex optimization problems. Here, a Chaotic Biogeography-Based Optimization approach using Information Entropy (CBO-IE) is implemented to perform clustering over healthcare IoT datasets. The main objective of CBO-IE is to provide proficient and precise data point distribution in datasets by using Information Entropy concepts and to initialize the population by using chaos theory. Both Information Entropy and chaos theory are facilitated to improve the convergence speed of BO in global search area for selecting the cluster heads and cluster members more accurately. The CBO-IE is implemented to a MATLAB 2021a tool over eight healthcare IoT datasets, and the results illustrate the superior performance of CBO-IE based on F-Measure, intracluster distance, running time complexity, purity index, statistical analysis, root mean square error, accuracy, and standard deviation as compared to previous techniques of clustering like K-Means, GA, PSO, ALO, and BO approaches.

Unbalanced abnormal traffic detection based on improved Res-BIGRU and integrated dynamic ELM optimization

Problems such as a vanishing gradient and overfitting will occur when a recurrent neural network (RNN) is exploited to detect abnormal network traffic. In addition, some network traffic is unbalanced, which leads to low detection accuracy. Therefore, an unbalanced abnormal traffic detection method has been proposed. It is composed of the improved bidirectional residual gated recurrent unit (Res-BIGRU) and integrated dynamic extreme learning machine (IDELM). First, the candidate hidden state activation function of the GRU is changed into an unsaturated activation function. The residual connection is used to avoid the vanishing gradient. The purpose of alleviating network degradation is achieved, and the traffic features extracted are better. Second, an IDELM is proposed to solve the unbalanced classification. The minority samples are generated by the IDELM model. The set model in game theory is used to compute the combined weight, which improves the fitting effect. Third, two IDELMs are used to update the final classification results. Fourth, four network datasets and IoT datasets are used to verify the performance. The average accuracy on four network datasets is 91.11% when samples are unbalanced. Furthermore, it can be concluded that the improved Res-BIGRU and IDELM strategy is effective. Better classification results can be achieved when network traffic is unbalanced. In particular, the performance is better in unbalanced NSL-KDD datasets. The index values obtained are the best compared with other methods. It is also suitable for intrusion detection of the Internet of Things, which has good performance. The further advantage lies in that the robustness is better when there are other sample interferences.

A new distributed architecture for evaluating AI-based security systems at the edge: Network TON_IoT datasets

While there has been a significant interest in understanding the cyber threat landscape of Internet of Things (IoT) networks, and the design of Artificial Intelligence (AI)-based security approaches, there is a lack of distributed architecture led to generating heterogeneous datasets that contain the actual behaviors of real-world IoT networks and complex cyber threat scenarios to evaluate the credibility of the new systems. This paper presents a novel testbed architecture of IoT network which can be used to evaluate Artificial Intelligence (AI)-based security applications. The platform NSX vCloud NFV was employed to facilitate the execution of Software-Defined Network (SDN), Network Function Virtualization (NFV) and Service Orchestration (SO) to offer dynamic testbed networks, which allow the interaction of edge, fog and cloud tiers. While deploying the architecture, real-world normal and attack scenarios are executed to collect labeled datasets. The generated datasets are named ‘TON_IoT’, as they comprise heterogeneous data sources collected from telemetry datasets of IoT services, Windows and Linux-based datasets, and datasets of network traffic. The TON_IoT network dataset is validated using four machine learning-based intrusion detection algorithms of Gradient Boosting Machine, Random Forest, Naive Bayes, and Deep Neural Networks, revealing a high performance of detection accuracy using the set of training and testing. A comparative summary of the TON_IoT network dataset and other competing network datasets demonstrates its diverse legitimate and anomalous patterns that can be used to better validate new AI-based security solutions. The architecture and datasets can be publicly accessed from TON_IOT Datasets (2020).

Toward Distributed, Global, Deep Learning Using IoT Devices

Deep learning (DL) using large scale, high-quality IoT datasets can be computationally expensive. Utilizing such datasets to produce a problem-solving model within a reasonable time frame requires a scalable distributed training platform/system. We present a novel approach where to train one DL model on the hardware of thousands of mid-sized IoT devices across the world, rather than the use of GPU cluster available within a data center. We analyze the scalability and model convergence of the subsequently generated model, identify three bottlenecks that are: high computational operations, time consuming dataset loading I/O, and the slow exchange of model gradients. To highlight research challenges for globally distributed DL training and classification, we consider a case study from the video data processing domain. A need for a two-step deep compression method, which increases the training speed and scalability of DL training processing, is also outlined. Our initial experimental validation shows that the proposed method is able to improve the tolerance of the distributed training process to varying internet bandwidth, latency, and Quality of Service metrics.

DeepStream: Autoencoder-based stream temporal clustering and anomaly detection

The increasing number of IoT devices in “smart” environments, such as homes, offices, and cities, produce seemingly endless data streams and drive many daily decisions. Consequently, there is growing interest in identifying contextual information from sensor data to facilitate the performance of various tasks, e.g., traffic management, cyber attack detection, and healthcare monitoring. The correct identification of contexts in data streams is helpful for many tasks, for example, it can assist in providing high-quality recommendations to end users and in reporting anomalous behavior based on the detection of unusual contexts. This paper presents DeepStream, a novel data stream temporal clustering algorithm that dynamically detects sequential and overlapping clusters. DeepStream is tuned to classify contextual information in real time and is capable of coping with a high-dimensional feature space. DeepStream utilizes stacked autoencoders to reduce the dimensionality of unbounded data streams and for cluster representation. This method detects contextual behavior and captures nonlinear relations of the input data, giving it an advantage over existing methods that rely on PCA. We evaluated DeepStream empirically using four sensor and IoT datasets and compared it to five state-of-the-art stream clustering algorithms. Our evaluation shows that DeepStream outperforms all of these algorithms. Our evaluation also demonstrates how DeepStream’s improved clustering performance results in improved detection of anomalous data.

Evaluation of efficiency of Database Systems Used for IoT Applications-Review

The data put away in IoT databases increments as the IoT applications stretch out all through smart city appliances, industry, and agriculture. New database systems should handle tremendous measures of tactile and actuator data progressively or intuitively. Confronting this first rush of IoT unrest, database management step by step to acquire a piece of the overall industry, grow new capacities and endeavor to defeat past discharges' weaknesses while giving highlights to the IoT. There are two well-known database types: The Relational Database Management Systems and NoSQL databases, with NoSQL making progress on IoT data storage. With regards to this paper, these two sorts are inspected. Zeroing in on open source databases, the creators investigate IoT data sets and represent an answer to the inquiry that one performs better than the next.