Time Window Size Research Articles

Training Spiking Neural Networks with Accumulated Spiking Flow

The fast development of neuromorphic hardwares promotes Spiking Neural Networks (SNNs) to a thrilling research avenue. Current SNNs, though much efficient, are less effective compared with leading Artificial Neural Networks (ANNs) especially in supervised learning tasks. Recent efforts further demonstrate the potential of SNNs in supervised learning by introducing approximated backpropagation (BP) methods. To deal with the non-differentiable spike function in SNNs, these BP methods utilize information from the spatio-temporal domain to adjust the model parameters. With the increasing of time window and network size, the computational complexity of spatio-temporal backpropagation augments dramatically. In this paper, we propose a new backpropagation method for SNNs based on the accumulated spiking flow (ASF), i.e. ASF-BP. In the proposed ASF-BP method, updating parameters does not rely on the spike train of spiking neurons but leverage accumulated inputs and outputs of spiking neurons over the time window, which reduces the BP complexity significantly. We further present an adaptive linear estimation model to approach the dynamic characteristics of spiking neurons statistically. Experimental results demonstrate that with our proposed ASF-BP method, light-weight convolutional SNNs achieve superior performances compared with other spike-based BP methods on both non-neuromorphic (MNIST, CIFAR10) and neuromorphic (CIFAR10-DVS) datasets. The code is available at https://github.com/neural-lab/ASF-BP.

Multi-step influenza outbreak forecasting using deep LSTM network and genetic algorithm

Influenza epidemic is a serious public health problem that has attracted worldwide attention due to cause cases of severe illness, an enormous economic burden, and even deaths worldwide each year. Forecasting influenza outbreak in advance has great significance on influenza-like illness (ILI) prevention and healthcare management. Existing research approaches based on traditional statistical and machine learning have failed to select superior features that detect sophisticated and non-linear characteristics of influenza epidemic sequential data. In this paper, it is introduced a hybrid method that combines long short-term memory (LSTM) neural network and genetic algorithm (GA) for multi-step influenza outbreak forecasting problems. LSTM model is employed to overcome the complexity and nonlinearity issues in an influenza prediction. In order to enhance the efficiency and performance of the neural network, the genetic algorithm is used to obtain the epoch size of the network, the number of LSTM layers, the size of units in each LSTM layer, and the time window size simultaneously. For comparison purposes, it is chosen the weekly data of influenza-like illness (ILI), also known as influenza or other similar illness showing flu-like symptoms, in the USA collected by the Centers for Disease Control and Prevention (CDC). The experimental results demonstrated that the presented hybrid model outperforms other highly developed machine learning approaches, a statistical model, and a fully-connected neural network considering different performance metrics during peak periods.

Session-based news recommendations using SimRank on multi-modal graphs

Recommender systems are among the most widespread applications of artificial intelligence techniques. For instance, news recommender systems serve users in managing the overload of information they come across when accessing news portals. Obviously, in the news domain time-awareness of recommendation approaches are crucial. However, most of these approaches missed to consider user sessions, which group the items that a user interacted with. In this paper, we study the problem of session-based recommendations by running SimRank on time-evolving heterogeneous graphs. In particular, we construct a dynamic heterogeneous multi-partite graph and adjust SimRank to run on it by using different (i) sliding time window sizes, (ii) sub-graphs used for model learning and (iii) sequential article weighting strategies. We evaluate our algorithms on two real-life datasets, and we show that our method outperforms other state-of-the-art methods in terms of accuracy and diversity. The significance and impact of this work is important because it introduces to the research community of expert and intelligent systems, for the first time, a stream-based version of SimRank algorithm, which is able to run over time-evolving graphs.

Comparison of machine learning methods in sEMG signal processing for shoulder motion recognition

Machine learning (ML) methods have been previously applied and compared in pattern recognition of hand and elbow motions based on surface electromyographic (sEMG) signals. However, there are only a few studies that have investigated the ML methods for shoulder motion pattern recognition. This study compared the efficiency of ML algorithms, including support vector machine (SVM), logistic regression (LR), and artificial neural network (ANN) in processing sEMG signals for shoulder motion pattern recognition. This study also investigated the the effects of sliding time window epoch on the recognition accuracy. Eighteen healthy subjects were recruited for this study, their EMG signals were collected from twelve muscles during performing activities of daily living (ADL) motions including drinking, pushing forward/pulling backward, and abduction/adduction. The 80 % of recoded sEMG datasets were used for model training to build the ML models and 20 % were used for model validation and determination of the accuracy of ML algorithms in motion pattern recognition. The influence of sliding time window sizes was studied for algorithm optimization. Statistical analysis was performed to determine the difference in the accuracy of ML methods. Results showed that there was a significant difference among the three machine learning methods and different sliding time window sizes. There was not a significant difference in overlapping time. The highest accuracy was 97.41 ± 1.8 % using the SVM method with a sliding time window of 270 ms. Machine learning techniques provided a quick approach for shoulder motion pattern recognition. The better classifier for pattern recognition of shoulder motion was SVM.

A hybrid deep neural network based on multi-time window convolutional bidirectional LSTM for civil aircraft APU hazard identification

Safety is one of the important topics in the field of civil aviation. Auxiliary Power Unit (APU) is one of important components in aircraft, which provides electrical power and compressed air for aircraft. The hazards in APU are prone to cause economic losses and even casualties. So, actively identifying the hazards in APU before an accident occurs is necessary. In this paper, a Hybrid Deep Neural Network (HDNN) based on multi-time window convolutional neural network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) neural network is proposed for active hazard identification of APU in civil aircraft. In order to identify the risks caused by different types of failures, the proposed HDNN simultaneously integrates three CNN-BiLSTM basic models with different time window sizes in parallel by using a fully connected neural network. The CNN-BiLSTM basic model can automatically extract features representing the system state from the input data and learn the time information of irregular trends in the time series data. Nine benchmark models are compared with the proposed HDNN. The comparison results show that the proposed HDNN has the highest identification accuracy. The HDNN has the most stable identification performance for data with imbalanced samples.

Reliability Characterization and Failure Prediction of 3D TLC SSDs in Large-Scale Storage Systems

3D triple-level cell (TLC) NAND flash based solid state drive (SSD) is gradually becoming the dominant storage media in large-scale storage systems due to high storage density and low cost-per-bit. It ranks one of the top replaced hardware components in systems and their enormous amount also indirectly increases the failure probability, resulting in irreversible data loss disaster and service unavailability. This paper for the first time investigates system-level 3D TLC SSDs to characterize reliability and sub-health status based on field Self-Monitoring, Analysis and Reporting Technology (SMART) data, and predict impending failure proactively. We explore real-world datasets and derive some findings for each selected attribute in predetermined categories, contributing to the following feature selection and enhancing the interpretability of prediction models. Moreover, various machine learning models are trained to predict failures ahead of time, and experimental results show that random forest model can achieve 0.636 f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -score and 0.662 MCC for a 7-day prediction horizon, and 42.5% true positive rate (TPR) with 0.00% false positive rate (FPR). Different time window sizes, training set fractions and ratios of negative to positive are analyzed as well.

Energy Minimization Algorithm for Estimation of Clock Skew and Reception Window Selection in Wireless Networks.

The synchronization of time between devices is one of the more important and challenging problems in wireless networks. We discuss the problem of maximization of the probability of receiving a message from a device using a limited listening time window to minimize energy utilization. We propose a solution to two important problems in wireless networks of battery-powered devices: a method of establishing a connection with a device that has been disconnected from the system for a long time and developed unknown skew and also two approaches to follow-up clock synchronization using the confidence interval method. We start with the analysis of measurements of clock skew. The algorithms are evaluated using extensive simulations and we discuss the selection of parameters balancing between minimizing the energy utilization and maximizing the probability of reception of the message. We show that the selection of a time window of growing size requires less energy to receive a packet than using the same size of time window repeated multiple times. The shifting of reception windows can further decrease the energy cost if lower packet reception probability is acceptable. We also propose and evaluate an algorithm scaling the reception window size to the interval between the packet transmission.

An Exploration of Online Missing Value Imputation in Non-stationary Data Stream

Missing value imputation (MVI) is an important data preprocessing technique. In previous decades, MVI technique has been widely studied as well as most MVI approaches have been proposed by means of either statistics or machine learning techniques. However, all previous methods only focus on the static data, but ignore the imputation for the dynamic online data. It is intuitionistic that the imputation errors may be significantly increased when there exists concept drifts in the data stream. In this paper, we investigate the impact of adopting the conventional MVI methods in non-stationary data stream. Meanwhile, two slide time window-based strategies are proposed to alleviate this impact, where one is the plain average strategy, and the other is the logarithmic weighted average strategy that gradually adds the weights of instances along the time axis. Combining with the proposed strategies, three popular MVI techniques, mean imputation (MI), KNN imputation (KNNI) and the Bayesian principal component analysis imputation (BPCAI) are adopted, to indicate the effect of the strategies are irrelevant to the specific MVI technique. The experimental results on three different types' concept drift synthetic data sets and two real-world drifting data sets have presented the effectiveness and feasibility of the proposed strategies. Moreover, the impact of time window size has also been investigated for guiding the parameter settings in future practical applications.

Towards the design of real-time autonomous IoT NIDS

Classic security methods become less effective against the Internet of Things (IoT) cyber-attacks, such as cryptography. An urgent need for real-time and lightweight detection of cyber-attacks is needed to secure IoT networks. This demand is achieved by a reliable and efficient intrusion detection system (IDS) that can meet IoT environments' high scalability and dynamicity. Herein, we analyzed the traffic and features of commonly used and recently published datasets for IoT networks. Furthermore, we proposed an ensemble feature selection method. Also, we studied the effects of traffic heterogeneity levels and time-window size on several classification methods to justify the detection model selection. Regarding the BotNet-IoT dataset, we noticed that few features play a critical role in IDS performance, and larger time-windows were slightly better than the shorter time-windows. Furthermore, we found that PCA classifier performance was significantly affected by traffic heterogeneity. On the other hand, the Boosted Tree showed the best and the most stable performance among all the considered classification methods.

High Fidelity Physics Simulation-Based Convolutional Neural Network for Automotive Radar Target Classification Using Micro-Doppler

Detection and classification of vulnerable road users (VRUs) such as pedestrians and cyclists is a key requirement for the realization of fully autonomous vehicles. Radar-based classification of VRUs can be achieved by exploiting differences in the micro-Doppler signatures associated with VRUs. Specifically, machine learning (ML) algorithms can be trained to classify VRUs using the spectral content of radar signals. The performance of these models depends on the quality and quantity of the data used during the training process. Currently, data collection is typically done through measurements or low fidelity physics, primitive-based simulations. The feasibility of carrying out measurements to collect training data is typically limited by the vast amounts of data required and practicality issues when using VRUs like animals. In this paper, we present a computationally efficient, high fidelity physics-based simulation workflow that can be used to obtain a large quantity of spectrograms from the micro-Doppler signatures of VRUs. The simulations are conducted on full-scale VRU models with a 77 GHz, frequency-modulated continuous-wave (FMCW) radar sensor model. Here, we collect the spectrograms of 4 targets; car, pedestrian, cyclist and dog at different speeds and angles-of-arrival. This data is then used to train a 5-layer convolutional neural network (CNN) that achieves nearly 100% classification accuracy after 5 epochs. Studies are conducted to investigate the impact of training data size, velocity and observation time window size on the accuracy of the CNN. Results from this study demonstrate how an accuracy of 95% can be realized using spectrograms obtained over a 0.2 s time window.

Experimental Validation of Link Quality Prediction Using Exact Self-Status of Mobility Robots in Wireless LAN Systems

Autonomous mobility machines, such as self-driving cars, transportation robots, and automated construction machines, are promising to support or enrich human lives. To further improve such machines, they will be connected to the network via wireless links to be managed, monitored, or remotely operated. The autonomous mobility machines must have self-status based on their positioning system to safely conduct their operations without colliding with other objects. The self-status is not only essential for machine operation but also it is valuable for wireless link quality management. This paper presents self-status-based wireless link quality prediction and evaluates its performance by using a prototype mobility robot combined with a wireless LAN system. The developed robot has functions to measure the throughput and receive signal strength indication and obtain self-status details such as location, direction, and odometry data. Prediction performance is evaluated in offline processing by using the dataset gathered in an indoor experiment. The experiments clarified that, in the 5.6 GHz band, link quality prediction using self-status of the robot forecasted the throughput several seconds into the future, and the prediction accuracies were investigated as dependent on time window size of the target throughput, bandwidth, and frequency gap.

Driver identification using 1D convolutional neural networks with vehicular CAN signals

This study proposes a deep learning framework for driver identity identification by extracting information from the vehicular controller area network (CAN) bus signals. First, naturalistic driving data of 20 drivers were collected under a fixed testing route with different road types and different traffic conditions. Then, a one-dimensional convolutional neural network was constructed for driver identification, which consists of two convolutional-pooling layers, a fully connected layer, and a SoftMax layer. Model optimisation algorithms were applied to improve accuracy and speed up the training process. Also, the model parameters were optimised by evaluating their influences on the model results. Furthermore, the performance of the proposed algorithm was compared with that of the K-nearest neighbour, support vector machine, multi-layer perceptron, and long short-term memory model. The authors used the M a c r o F 1 score as an evaluation criterion and the identification score of the authors' proposed model reaches 99.10% under 20 testing subjects where the data time window size is one second and the sample data overlap is 80%. The results show that the model's performance is significantly better than the other algorithms, which can effectively identify driver identities with stability and robustness.

Inclusion of features derived from a mixture of time window sizes improved classification accuracy of machine learning algorithms for sheep grazing behaviours

Inertial motion sensors located on the animal have been used to study the behaviour of ruminant livestock. The time window size of segmented signal data can significantly affect the classification accuracy of animal behaviours. To date, there have been no studies evaluating the impact of a mixture of time window size features on the accuracy of animal behaviour classification. In this study, data was collected from accelerometers attached to the neck of 17 Merino sheep over a period of two days. We also recorded a ground truth dataset of behaviour recordings (grazing, ruminating, walking, and standing) over the same time period, We then investigated the ability of three machine learning (ML) approaches, Random Forest (RF), Support Vector Machine (SVM) and linear discriminant analysis (LDA), to accurately classify sheep behaviour. Our results clearly show that simultaneous inclusion of features derived from time windows of mixed sizes, ranging from 2 to 15 s, significantly improved the behaviour classification accuracy, in comparison to those determined from a single unique time window size. Of the three ML methods applied here, the RF approach yielded the best results. Together our results show that including features obtained from mixed window sizes improved the classification accuracy of sheep behaviours.

Using temporal convolution network for remaining useful lifetime prediction

AbstractBecause Convolutional Neural Network (CNN) can extract spatial feature, while Long Short‐Term Memory (LSTM) can learn temporal features, many methods combining CNN with LSTM are proposed for remaining useful lifetime prediction. In practice, it is better to learn temporal features from long history sequence data because of the slow inherently long‐term degradation process. However, LSTM is less efficient in processing long history sequence. To solve this problem, in this work, a temporal convolution network combining causal filters with dilated convolutions is used to expand the receptive field length of network. The network structure can be fixed through three key parameters, and the size of time window adopted for time sequence processing is the same as the receptive field length. These two characteristics allow the network to easily be applied for engineering purposes. The method is tested and evaluated using two well‐known datasets, namely the “Turbofan Engine Degradation Simulation Dataset C‐MAPSS” and “Milling Dataset.” The performance analysis shows that the proposed method outperforms more classical methods in terms of prediction accuracy.

Improved long‐term time‐series predictions of total blood use data from England

Red blood cells are essential for modern medicine but managing their collection and supply to cope with fluctuating demands represents a major challenge. As deterministic models based on predicted population changes have been problematic, there remains a need for more precise and reliable prediction of use. Here, we develop three new time-series methods to predict red cell use 4 to 52 weeks ahead. From daily aggregates of red blood cell (RBC) units issued from 2005 to 2011 from the NHS Blood and Transplant, we generated a new set of non-overlapping weekly data by summing the daily data over 7 days and derived the average blood use per week over 4-week and 52-week periods. We used three new methods for linear prediction of blood use by computing the coefficients using Minimum Mean Squared Error (MMSE) algorithm. We optimized the time-window size, order of the prediction, and order of the polynomial fit for our data set. By exploiting the annual periodicity of the data, we achieved significant improvements in long-term predictions, as well as modest improvements in short-term predictions. The new methods predicted mean RBC use with a standard deviation of the percentage error of 2.5% for 4 weeks ahead and 3.4% for 52 weeks ahead. This paradigm allows short- and long-term prediction of RBC use and could provide reliable and precise prediction up to 52 weeks ahead to improve the efficiency of blood services and sufficiency of blood supply with reduced costs.

Genetic-Algorithm-Optimized Sequential Model for Water Temperature Prediction

Advances in establishing real-time river water quality monitoring networks combined with novel artificial intelligence techniques for more accurate forecasting is at the forefront of urban water management. The preservation and improvement of the quality of our impaired urban streams are at the core of the global challenge of ensuring water sustainability. This work adopted a genetic-algorithm (GA)-optimized long short-term memory (LSTM) technique to predict river water temperature (WT) as a key indicator of the health state of the aquatic habitat, where its modeling is crucial for effective urban water quality management. To our knowledge, this is the first attempt to adopt a GA-LSTM to predict the WT in urban rivers. In recent research trends, large volumes of real-time water quality data, including water temperature, conductivity, pH, and turbidity, are constantly being collected. Specifically, in the field of water quality management, this provides countless opportunities for understanding water quality impairment and forecasting, and to develop models for aquatic habitat assessment purposes. The main objective of this research was to develop a reliable and simple urban river water temperature forecasting tool using advanced machine learning methods that can be used in conjunction with a real-time network of water quality monitoring stations for proactive water quality management. We proposed a hybrid time series regression model for WT forecasting. This hybrid approach was applied to solve problems regarding the time window size and architectural factors (number of units) of the LSTM network. We have chosen an hourly water temperature record collected over 5 years as the input. Furthermore, to check its robustness, a recurrent neural network (RNN) was also tested as a benchmark model and the performances were compared. The experimental results revealed that the hybrid model of the GA-LSTM network outperformed the RNN and the basic problem of determining the optimal time window and number of units of the memory cell was solved. This research concluded that the GA-LSTM can be used as an advanced deep learning technique for time series analysis.

Evaluation of the size of time windows for the travelling salesman problem in delivery operations

A great challenge in operational research is to apply time-efficient algorithms to find the optimal solutions to the travelling salesman problem (TSP) and its many variations. The TSP with time windows (TSPTW) arises due to intense pressure for business to improve customer service. As online shopping becomes more popular, customer satisfaction increases if customers can decide when their orders are delivered to them. Customers may choose a time window, which is defined by an earliest delivery time and a latest delivery time, during which the package is delivered. Delivering packages to multiple customers is a typical TSPTW. One main challenge for a delivery business is to determine the size of the time window (i.e., the difference between the latest and earliest delivery times), which affects delivery cost and customer satisfaction. Although many previous studies investigated the TSPTW, none of those focused on the size of time windows. This study is the first that experiments with different time window sizes and determines their impact on tour duration, customer satisfaction, and solution time of the optimal delivery routes. The experiment results show that increasing the size of the time window decreases tour duration and customer satisfaction and increases solution time. Decreasing the size of the time window increases tour duration and customer satisfaction and decreases solution time. A small solution time is necessary for the scheduling of deliveries to many customers. A large solution time prevents a delivery business from delivering packages using optimal routes, which increases delivery cost and decreases customer satisfaction. The results of this study indicate that a general guideline for business is to allow customers to choose a time window size that is within the cost limit but is sufficiently small to maximize customer satisfaction and optimize delivery routes.

Person Independent Recognition of Head Gestures from Parametrised and Raw Signals Recorded from Inertial Measurement Unit

Numerous applications of human–machine interfaces, e.g., dedicated to persons with disabilities, require contactless handling of devices or systems. The purpose of this research is to develop a hands-free head-gesture-controlled interface that can support persons with disabilities to communicate with other people and devices, e.g., the paralyzed to signal messages or the visually impaired to handle travel aids. The hardware of the interface consists of a small stereovision rig with a built-in inertial measurement unit (IMU). The device is to be positioned on a user’s forehead. Two approaches to recognize head movements were considered. In the first approach, for various time window sizes of the signals recorded from a three-axis accelerometer and a three-axis gyroscope, statistical parameters were calculated such as: average, minimum and maximum amplitude, standard deviation, kurtosis, correlation coefficient, and signal energy. For the second approach, the focus was put onto direct analysis of signal samples recorded from the IMU. In both approaches, the accuracies of 16 different data classifiers for distinguishing the head movements: pitch, roll, yaw, and immobility were evaluated. The recordings of head gestures were collected from 65 individuals. The best results for the testing data were obtained for the non-parametric approach, i.e., direct classification of unprocessed samples of IMU signals for Support Vector Machine (SVM) classifier (95% correct recognitions). Slightly worse results, in this approach, were obtained for the random forests classifier (93%). The achieved high recognition rates of the head gestures suggest that a person with physical or sensory disability can efficiently communicate with other people or manage applications using simple head gesture sequences.

Creating Grocery Delivery Hubs for Food Deserts at Local Convenience Stores via Spatial and Temporal Consolidation

For many socioeconomically disadvantaged customers living in food deserts, the high costs and minimum order size requirements make attended grocery deliveries financially non-viable, although it has potential to provide healthy foods to the food insecure population. This paper proposes consolidating customer orders and delivering to a neighborhood convenience store instead of home delivery. We employ an optimization framework involving the minimum cost set covering and the capacitated vehicle routing problems. Our experimental studies in three counties in the U.S. suggest that by spatial and temporal consolidation of orders, the deliverer can remove minimum order-size requirements and reduce the delivery costs, depending on various factors, compared to attended home-delivery. We find the number and size of time windows for home delivery to be the most important factor in achieving temporal consolidation benefits. Other significant factors in achieving spatial consolidation include the capacity of delivery vehicles, the number of depots, and the number of customer orders. We also find that the number of partner convenience stores and the walkable distance parameter of the model, have a significant impact on the number of accepted orders, i.e., the service level provided by the deliverer. The findings of this study imply consolidated grocery delivery as a viable solution to improve fresh food access in food deserts. In light of the recent global pandemic, and its exacerbating effects on food insecurity, the innovative solution proposed in this paper is even more relevant and timely.

Session-aware news recommendations using random walks on time-evolving heterogeneous information networks

Traditional news media Web sites usually provide generic recommendations that are not personalized to the preferences of their users. Typically, news recommendation algorithms mainly rely on the long-term preferences of users and do not adjust their model to the continuous stream of short-lived incoming stories to capture short-term intentions revealed by users’ sessions. In this paper, we therefore study the problem of session-aware recommendations by running random walks on dynamic heterogeneous graphs. Concretely, we construct a heterogeneous information network consisting of users, news articles, news categories, locations and sessions. By using different (1) sliding time window sizes, (2) sub-graphs for model learning, (3) sequential article weighting strategies and (4) more diversified random walks, we perform recommendations in a second step. Our algorithm proposal is evaluated on three real-life data sets, and we demonstrate that our method outperforms state-of-the-art methods by delivering more accurate and diversified recommendations.