Sequential Datasets Research Articles

Quantitative logging data clustering with hidden Markov model to assist log unit classification

Revealing subsurface structures is a fundamental task in geophysical and geological studies. Logging data are usually acquired through drilling projects, which constrain the subsurface structure, and together with the description of drill core samples, are used to distinguish geological units. Clustering is useful for interpreting logging data and making log unit classification and is usually performed by manual inspection of the data. However, the validity of clustering results with such subjective criteria may be questionable. This study proposed the application of a statistical clustering method, the hidden Markov model, to conduct unsupervised clustering of logging data. As logging data are aligned along the drilled hole, they and the geological structure hidden behind such sequential datasets can be regarded as observables and hidden states in the hidden Markov model. When log unit classification is manually conducted, depth dependency of logging data is usually focused. Therefore, we included depth information as observables to explicitly represent depth dependency of logging data. The model was applied to the following geological settings: the accretionary prism at the Nankai Trough, the onshore fault zone at the Kii Peninsula (southwest Japan), and the forearc basin at the Japan Trench. The optimum number of clusters were searched using a quantitative index. The clustering results using the hidden Markov model were consistent with previously reported classifications or lithological descriptions; however, our method allowed a more detailed division of logging data, which is useful to interpret geological structures, such as a fault or a fault zone. Therefore, the use of the hidden Markov model enabled us to clarify assumptions quantitatively and conduct clustering consistently for the entire depth range, even for different geological sites. The proposed method is expected to have wider applicability and extensibility for other types of data, including geochemical and structural geological data.Graphical

SyntEO: Synthetic dataset generation for earth observation and deep learning – Demonstrated for offshore wind farm detection

With the emergence of deep learning in the last years, new opportunities arose in Earth observation research. Nevertheless, they also brought with them new challenges. The data-hungry training processes of deep learning models demand large, resource expensive, annotated datasets and partly replaced knowledge-driven approaches so that model behaviour and the final prediction process became a black box. The proposed SyntEO approach enables Earth observation researchers to automatically generate large deep learning ready datasets by merging existing and procedural data. SyntEO does this by including expert knowledge in the data generation process in a highly structured manner to control the automatic image and label generation by employing an ontology. In this way, fully controllable experiment environments are set up, which support insights in the model training on the synthetic datasets. Thus, SyntEO makes the learning process approachable, which is an important cornerstone for explainable machine learning. We demonstrate the SyntEO approach by predicting offshore wind farms in Sentinel-1 images on two of the worlds largest offshore wind energy production sites. The largest generated dataset has 90,000 training examples. A basic convolutional neural network for object detection, that is only trained on this synthetic data, confidently detects offshore wind farms by minimising false detections in challenging environments. In addition, four sequential datasets are generated, demonstrating how the SyntEO approach can precisely define the dataset structure and influence the training process. SyntEO is thus a hybrid approach that creates an interface between expert knowledge and data-driven image analysis.

Dual attention-based sequential auto-encoder for Covid-19 outbreak forecasting: A case study in Vietnam

For preventing the outbreaks of Covid-19 infection in different countries, many organizations and governments have extensively studied and applied different kinds of quarantine isolation policies, medical treatments as well as organized massive/fast vaccination strategy for over-18 citizens. There are several valuable lessons have been achieved in different countries this Covid-19 battle. These studies have presented the usefulness of prompt actions in testing, isolating confirmed infectious cases from community as well as social resource planning/optimization through data-driven anticipation. In recent times, many studies have demonstrated the effectiveness of short/long-term forecasting in number of new Covid-19 cases in forms of time-series data. These predictions have directly supported to effectively optimize the available healthcare resources as well as imposing suitable policies for slowing down the Covid-19 spreads, especially in high-populated cities/regions/nations. There are several progresses of deep neural architectures, such as recurrent neural network (RNN) have demonstrated significant improvements in analyzing and learning the time-series datasets for conducting better predictions. However, most of recent RNN-based techniques are considered as unable to handle chaotic/non-smooth sequential datasets. The consecutive disturbances and lagged observations from chaotic time-series dataset like as routine Covid-19 confirmed cases have led to the low performance in temporal feature learning process through recent RNN-based models. To meet this challenge, in this paper, we proposed a novel dual attention-based sequential auto-encoding architecture, called as: DAttAE. Our proposed model supports to effectively learn and predict the new Covid-19 cases in forms of chaotic and non-smooth time series dataset. Specifically, the integration between dual self-attention mechanism in a given Bi-LSTM based auto-encoder in our proposed model supports to directly focus the model on a specific time-range sequence in order to achieve better prediction. We evaluated the performance of our proposed DAttAE model by comparing with multiple traditional and state-of-the-art deep learning-based techniques for time-series prediction task upon different real-world datasets. Experimental outputs demonstrated the effectiveness of our proposed attention-based deep neural approach in comparing with state-of-the-art RNN-based architectures for time series based Covid-19 outbreak prediction task.

Credit Card Fraud Detection project

Abstract: For some time, there has been a strong interest in the ethics of banking (Molyneaux, 2007; George, 1992), as well as the moral complexity of fraudulent behavior (Clarke, 1994). Fraud means obtaining services/goods and/or money by unethical means, and is a growing problem all over the world nowadays. Fraud deals with cases involving criminal purposes that, mostly, are difficult to identify. Credit cards are one of the most famous targets of fraud but not the only one; fraud can occur with any type of credit products, such as personal loans, home loans, and retail. Furthermore, the face of fraud has changed dramatically during the last few decades as technologies have changed and developed. A critical task to help businesses and financial institutions including banks is to take steps to prevent fraud and to deal with it efficiently and effectively, when it does happen (Anderson, 2007). Anderson (2007) has identified and explained the different types of fraud, which are as many and varied as the financial institution’s products and technologies, such as Transaction products: credit and debit cards and checks, Relationship to accounts first, second and third parties, Business processes: application and transaction, Manner and timing short versus long term, Identify misrepresentation: embellishment, theft and fabrication, Handling of transaction: lost or stolen, not received, skimming and at hand, Utilization counterfeit, not present, altered or unaltered, Technologies ATM and Internet. Solutions for integrating sequential information in the feature set exist in the literature. The predominant one consists in creating a set of features which are descriptive statistics obtained by aggregating the sequences of transactions of the cardholders (sum of amount, count of transactions, location from where the payment is being made etc..). We used this method as a benchmark feature engineering method for credit card fraud detection. However, this feature engineering strategy raised several research questions. First of all, we assumed that these descriptive statistics cannot fully describe the sequential properties of fraud and genuine patterns and that modelling the sequences of transactions could be beneficial for fraud detection. Moreover, the creation of these aggregated features is guided by expert knowledge whereas sequence modelling could be automated thanks to the class labels available for past transactions. Finally, the aggregated features are point estimates that may be complemented by a multi-perspective univariate description of the transaction context. We proposed a multi-perspective HMM-based automated feature engineering strategy in order to incorporate a broad spectrum of sequential information in the transactions feature sets. In fact, we model the genuine and fraudulent behaviors of the merchants and the card-holders according to two univariate features: the country from where the payment is being made and the amount of each of the transactions being made. Moreover, the HMMbased features are created in a supervised way and therefore lower the need of expert knowledge for the creation of the fraud detection system. In the end, our multiple perspectives HMM-based approach offers automated feature engineering to model temporal correlations so as to complement and possibly supplement the use of transaction aggregation strategies in order to improve the effectiveness of the classification task. Experiments conducted on a large real world credit card transaction dataset (46 million transactions from belgium card-holders between March and May 2015) have shown that the proposed HMMbased feature engineering allows for an increase in the detection of fraudulent transactions when combined with the state-ofthe-art expert-based feature engineering strategy for credit card fraud detection. To conclude, this work leads to a better understanding of what can be considered contextual knowledge for a credit card fraud detection task and how to include it in the classification task in order to get an increase in fraud detection. The method proposed can be extended to any supervised task with sequential datasets. The main aims are, firstly, to identify the different types of credit card fraud, and, secondly, to review alternative techniques that have been used in fraud detection. Indeed, transaction products, including credit cards, are the most vulnerable to fraud. On the other hand, other products such as personal loans and retail are also at risk, and have serious ethical conflicts. Keywords: Behavior and Location Analysis (BLA); Fraud Detection System (FDS); Automated Teller Machine (ATM); Credit Card Fraud Detection; DB: Database.

Root-zone soil moisture estimation based on remote sensing data and deep learning

Soil moisture in the root zone is the most important factor in eco-hydrological processes. Even though soil moisture can be obtained by remote sensing, limited to the top few centimeters (<5 cm). Researchers have attempted to estimate root-zone soil moisture using multiple regression, data assimilation, and data-driven methods. However, correlations between root-zone soil moisture and its related variables, including surface soil moisture, always appear nonlinear, which is difficult to extract and express using typical statistical methods. The artificial intelligence (AI) method, which is advantageous for nonlinear relationship analysis and extraction is applied for root-zone soil moisture estimation, but by only considering its separate temporal or spatial correlations. The convolutional long short-term memory (ConvLSTM) model, known to capture spatiotemporal patterns of large-scale sequential datasets with the advantage of dealing with spatiotemporal sequence-forecasting problem, was used in this study to estimate root-zone soil moisture based on remote sensing-based variables. Owing to limitation of regional soil moisture observation data, the physical model Hydrus-1D was used to generate large and spatiotemporal vertical soil moisture dataset for the ConvLSTM model training and verification. Then, normalized difference vegetation index (NDVI) etc. remote sensing-based factors were selected as predictive variables. Results of the ConvLSTM model showed that the fitting coefficients (R2) of the root-zone soil moisture estimation significantly increased compared to those achieved by Global Land Data Assimilation System products, especially for deep layers. For example, R2 increased from 0.02 to 0.60 at depth of 40 cm. This study suggests that a combination of the physical model and AI is a flexible tool capable of predicting spatiotemporally continuous root-zone soil moisture with good accuracy on a large scale.

EchoVPR: Echo State Networks for Visual Place Recognition

Recognising previously visited locations is an important, but unsolved, task in autonomous navigation. Current visual place recognition (VPR) benchmarks typically challenge models to recover the position of a query image (or images) from sequential datasets that include both spatial and temporal components. Recently, Echo State Network (ESN) varieties have proven particularly powerful at solving machine learning tasks that require spatio-temporal modelling. These networks are simple, yet powerful neural architectures that&#x2014;exhibiting memory over multiple time-scales and non-linear high-dimensional representations&#x2014;can discover temporal relations in the data while still maintaining linearity in the learning time. In this letter, we present a series of ESNs and analyse their applicability to the VPR problem. We report that the addition of ESNs to pre-processed convolutional neural networks led to a dramatic boost in performance in comparison to non-recurrent networks in five out of six standard benchmarks (GardensPoint, SPEDTest, ESSEX3IN1, Oxford RobotCar, and Nordland), demonstrating that ESNs are able to capture the temporal structure inherent in VPR problems. Moreover, we show that models that include ESNs can outperform class-leading VPR models which also exploit the sequential dynamics of the data. Finally, our results demonstrate that ESNs improve generalisation abilities, robustness, and accuracy further supporting their suitability to VPR applications.

On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools

Recurrent neural networks (RNNs) have been successfully applied to a variety of problems involving sequential data, but their optimization is sensitive to parameter initialization, architecture, and optimizer hyperparameters. Considering RNNs as dynamical systems, a natural way to capture stability, i.e., the growth and decay over long iterates, are the Lyapunov Exponents (LEs), which form the Lyapunov spectrum. The LEs have a bearing on stability of RNN training dynamics since forward propagation of information is related to the backward propagation of error gradients. LEs measure the asymptotic rates of expansion and contraction of non-linear system trajectories, and generalize stability analysis to the time-varying attractors structuring the non-autonomous dynamics of data-driven RNNs. As a tool to understand and exploit stability of training dynamics, the Lyapunov spectrum fills an existing gap between prescriptive mathematical approaches of limited scope and computationally-expensive empirical approaches. To leverage this tool, we implement an efficient way to compute LEs for RNNs during training, discuss the aspects specific to standard RNN architectures driven by typical sequential datasets, and show that the Lyapunov spectrum can serve as a robust readout of training stability across hyperparameters. With this exposition-oriented contribution, we hope to draw attention to this under-studied, but theoretically grounded tool for understanding training stability in RNNs.

Dimensionality Reduction and Louvain Agglomerative Hierarchical Clustering for Cluster-Specified Frequent Biomarker Discovery in Single-Cell Sequencing Data.

The major interest domains of single-cell RNA sequential analysis are identification of existing and novel types of cells, depiction of cells, cell fate prediction, classification of several types of tumor, and investigation of heterogeneity in different cells. Single-cell clustering plays an important role to solve the aforementioned questions of interest. Cluster identification in high dimensional single-cell sequencing data faces some challenges due to its nature. Dimensionality reduction models can solve the problem. Here, we introduce a potential cluster specified frequent biomarkers discovery framework using dimensionality reduction and hierarchical agglomerative clustering Louvain for single-cell RNA sequencing data analysis. First, we pre-filtered the features with fewer number of cells and the cells with fewer number of features. Then we created a Seurat object to store data and analysis together and used quality control metrics to discard low quality or dying cells. Afterwards we applied global-scaling normalization method “LogNormalize” for data normalization. Next, we computed cell-to-cell highly variable features from our dataset. Then, we applied a linear transformation and linear dimensionality reduction technique, Principal Component Analysis (PCA) to project high dimensional data to an optimal low-dimensional space. After identifying fifty “significant”principal components (PCs) based on strong enrichment of low p-value features, we implemented a graph-based clustering algorithm Louvain for the cell clustering of 10 top significant PCs. We applied our model to a single-cell RNA sequential dataset for a rare intestinal cell type in mice (NCBI accession ID:GSE62270, 23,630 features and 1872 samples (cells)). We obtained 10 cell clusters with a maximum modularity of 0.885 1. After detecting the cell clusters, we found 3871 cluster-specific biomarkers using an expression feature extraction statistical tool for single-cell sequencing data, Model-based Analysis of Single-cell Transcriptomics (MAST) with a log 2 FC threshold of 0.25 and a minimum feature detection of 25%. From these cluster-specific biomarkers, we found 1892 most frequent markers, i.e., overlapping biomarkers. We performed degree hub gene network analysis using Cytoscape and reported the five highest degree genes (Rps4x, Rps18, Rpl13a, Rps12 and Rpl18a). Subsequently, we performed KEGG pathway and Gene Ontology enrichment analysis of cluster markers using David 6.8 software tool. In summary, our proposed framework that integrated dimensionality reduction and agglomerative hierarchical clustering provides a robust approach to efficiently discover cluster-specific frequent biomarkers, i.e., overlapping biomarkers from single-cell RNA sequencing data.

Quantitatively Assessing the Future Land-Use/Land-Cover Changes and Their Driving Factors in the Upper Stream of the Awash River Based on the CA–Markov Model and Their Implications for Water Resources Management

Despite the rapid economic and population growth, the risks related to the current dynamics of land use and land cover (LULC) have attracted a lot of attention in Ethiopia. Therefore, a complete investigation of past and future LULC changes is essential for sustainable water resources and land-use planning and management. Since the 1980s, LULC change has been detected in the upper stream of the Awash River basin. The main purpose of this research was to investigate the current dynamics of LULC and use the combined application of the cellular automata and the Markov chain (CA–Markov) model to simulate the year 2038 LULC in the future; key informant interviews, household surveys, focus group discussions, and field observations were used to assess the consequences and drivers of LULC changes in the upstream Awash basin (USAB). This research highlighted the importance of remote sensing (RS) and geographic information system (GIS) techniques for analyzing the LULC changes in the USAB. Multi-temporal cloud-free Landsat images of three sequential data sets for the periods (1984, 2000, and 2019) were employed to classify based on supervised classification and map LULC changes. Satellite imagery enhancement techniques were performed to improve and visualize the image for interpretation. ArcGIS10.4 and IDRISI software was used for LULC classification, data processing, and analyses. Based on Landsat 5 TM-GLS 1984, Landsat 7 ETM-GLS 2000, and Landsat 8 2019 OLI-TIRS, the supervised maximum likelihood image classification method was used to map the LULC dynamics. Landsat images from 1984, 2000, and 2019 were classified to simulate possible LULC in 2019 and 2038. The result reveals that the maximum area is covered by agricultural land and shrubland. It showed, to the areal extent, a substantial increase in agricultural land and urbanization and a decrease in shrubland, forest, grassland, and water. The LULC dynamics showed that those larger change rates were observed from forest and shrubland to agricultural areas. The results of the study show the radical changes in LULC during 1984–2019; the main reasons for this were agricultural expansion and urbanization. From 1984 to 2019, agriculture increased by 62%, urban area increased by 570.5%, and forest decreased by 88.7%. In the same year, the area of shrubland decreased by 68.6%, the area of water decreased by 65.5%, and the area of grassland decreased by 57.7%. In view of the greater increase in agricultural land and urbanization, as well as the decrease in shrubland, it means that the LULC of the region has changed. This research provides valuable information for water resources managers and land-use planners to make changes in the improvement of future LULC policies and development of sub-basin management strategies in the context of sustainable water resources and land-use planning and management.

EMG-Based Grasping Force Estimation for Robot Skill Transfer Learning

In this study, we discuss a new machine learning architecture, the multilayer preceptron-random forest regressors pipeline (MLP-RF model), which stacks two ML regressors of different kinds to estimate the generated gripping forces from recorded surface electromyographic activity signals (EMG) during a gripping task. We evaluate our proposed approach on a publicly available dataset, putEMG-Force, which represents a sEMG-Force data profile. The sEMG signals were then filtered and preprocessed to get the features-target data frame that will be used to train the proposed ML model. The proposed ML model is a pipeline of stacking 2 different natural ML models; a random forest regressor model (RF regressor) and a multiple layer perceptron artificial neural network (MLP regressor). The models were stacked together, and the outputs were penalized by a Ridge regressor to get the best estimation of both models. The model was evaluated by different metrics; mean squared error and coefficient of determination, or $r^{2}$ score, to improve the model prediction performance. We tuned the most significant hyperparameters of each of the MLP-RF model components using a random search algorithm followed by a grid search algorithm. Finally, we evaluated our MLP-RF model performance on the data by training a recurrent neural network consisting of 2 LSTM layers, 2 dropouts, and one dense layer on the same data (as it is the common approach for problems with sequential datasets) and comparing the prediction results with our proposed model. The results show that the MLP-RF outperforms the RNN model.

Performance Assessment of Global Horizontal Irradiance Models in All-Sky Conditions

Solar irradiance models contribute to mitigating the lack of measurement data at a ground station. Conventionally, the models relied on physical calculations or empirical correlations. Recently, machine learning as a sophisticated statistical method has gained popularity due to its accuracy and potential. While some studies compared machine learning models with other models, a study has not yet been performed that compares them side-by-side to assess their performance using the same datasets in different locations. Therefore, this study aims to evaluate the accuracy of three representative models for estimating solar irradiance using atmospheric variables measurement and cloud amount derived from satellite images as the input parameters. Based on its applicability and performance, this study selected the fast all-sky radiation model for solar applications (FARMS) derived from the radiative transfer approach, the Hammer model that simplified atmospheric correlation, and the long short-term memory (LSTM) model specialized in sequential datasets. Global horizontal irradiance (GHI) data were modeled for five distinct locations in South Korea and compared with hourly measurement data of two years to yield the error metrics. When identical input parameters were used, LSTM outperformed the FARMS and the Hammer model in terms of relative root mean square difference (rRMSD) and relative mean bias difference (rMBD). Training an LSTM model using the input parameters of FARMS, such as ozone, nitrogen, and precipitable water, yielded more accurate results than using the Hammer model. The result shows unbiased and accurate estimation with an rRMSD and rMBD of 23.72% and 0.14%, respectively. Conversely, the FARMS has a faster processing speed and does not require significant data to make a fair estimation.

Aging detection of plant control system components using recurrent neural network

A control system detects the aging faults of instruments on the basis of whether the signal is outside of a pre-defined range. Potential adverse effects to plant safety caused by instrument aging increase until triggering the set-point. Detecting an aging fault early can contribute to the safe operation of a nuclear plant. Recently, the recurrent neural network (RNN) has performed well in analyzing time-series data. This study proposes a mechanism for aging detection using long short-term memory (LSTM), which is an improved model of a RNN. The mechanism consists of three phases, input preprocessing, the LSTM network, and output evaluation. The LSTM is trained with a sequential dataset assembled through preprocessing of raw data logged in a compact nuclear simulator. Output processing includes classification to distinguish a normal or aging state from the output of the LSTM and evaluation of the diagnosis results. The experiment results show that the proposed approach provides stable detection capability.

Deep Ensemble Learning Approaches in Healthcare to Enhance the Prediction and Diagnosing Performance: The Workflows, Deployments, and Surveys on the Statistical, Image-Based, and Sequential Datasets.

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

Consensus Patterns of a Set of Time Series via a Wavelet-Based Temporal Localization: Emphasizing the Utility over Point-Wise Averaging and Averaging under Dynamic Time Warping

Summarizing or averaging a sequential data set (i.e., a set of time series) can be comprehensively approached as a result of sophisticated computational tools. Averaging under Dynamic Time Warping (DTW) is one such tool that captures consensus patterns. DTW acts as a similarity measure between time series, and subsequently, an averaging method must be executed upon the behaviour of DTW. However, averaging under DTW somewhat neglects temporal aspect since it is on the search of similar appearances rather than stagnating on corresponding time-points. On the contrary, the mean series carrying point-wise averages provides only a weak consensus pattern as it may over-smooth important temporal variations. As a compromise, a pool of consensus series termed Ultimate Tamed Series (UTS) is studied here that adheres to temporal decomposition supported by the discrete Haar wavelet. We claim that UTS summarizes localized patterns, which would not be reachable via the series under DTW or the mean series. Neighbourhood of localization can be altered as a user can customize different levels of decomposition. In validation, comparisons are carried out with the series under DTW and the mean series via Euclidean distance and the distance resulted by DTW itself. Two sequential data sets are selected for this purpose from a standard repository.

PDE-guided reservoir computing for image denoising with small data.

While network-based techniques have shown outstanding performance in image denoising in the big data regime requiring massive datasets and expensive computation, mathematical understanding of their working principles is very limited. Not to mention, their relevance to traditional mathematical approaches has not attracted much attention. Therefore, we suggest how reservoir computing networks can be strengthened in combination with conventional partial differential equation (PDE) methods for image denoising, especially in the small data regime. Given image data, PDEs generate sequential datasets enhancing desired image features, which provide the network with a better guideline for training in reservoir computing. The proposed procedure, reservoir computing in collaboration with PDEs (RCPDE), offers a synergetic combination of data-driven network-based methods and mathematically well-established PDE methods. It turns out that RCPDE outperforms both the usual reservoir computing and existing PDE approaches in image denoising. Furthermore, RCPDE also excels deep neural networks such as a convolutional neural network both in quality and in time in the small data regime. We believe that RCPDE reveals the great potential of reservoir computing in collaboration with various mathematically justifiable dynamics for better performance as well as for better mathematical understanding.

A Hybrid Validity Index to Determine K Parameter Value of k-Means Algorithm for Time Series Clustering

Time series is a set of sequential data point in time order. The sizes and dimensions of the time series datasets are increasing day by day. Clustering is an unsupervised data mining technique that groups objects based on their similarities. It is used to analyze various datasets, such as finance, climate, and bioinformatics datasets. [Formula: see text]-means is one of the most used clustering algorithms. However, it is challenging to determine the value of [Formula: see text] parameter, which is the number of clusters. One of the most used methods to determine the number of clusters (such as [Formula: see text]) is cluster validity indexes. Several internal and external validity indexes are used to find suitable cluster numbers based on characteristics of datasets. In this study, we propose a hybrid validity index to determine the value of [Formula: see text] parameter of [Formula: see text]-means algorithm. The proposed hybrid validity index comprises four internal validity indexes, such as Dunn, Silhouette, C index, and Davies–Bouldin indexes. The proposed method was applied to nine real-life finance and benchmarks time series datasets. The financial dataset was obtained from Yahoo Finance, consisting of daily closing data of stocks. The other eight benchmark datasets were obtained from UCR time series classification archive. Experimental results showed that the proposed hybrid validity index is promising for finding the suitable number of clusters with respect to the other indexes for clustering time-series datasets.

Sum of variance for quantifying the variation of multiple sequential data for the crispness evaluation of chicken nugget.

Crispness is one of the words most frequently used to describe the texture of fried or dried food in addition to being a key to the determination of freshness for many non-fried foods. In this study, a new feature value called the sum of variance was assessed for its contribution to the estimation of crispness. Dynamic time warping and its averaging algorithms were employed to determine the sum of variance from a set of sequential force data measured using an instrument. The sum of variance is a feature value that expresses the variance of multiple sequential data. In an experiment, seven chicken nugget samples were prepared, and five panels evaluated their texture according to six Japanese word descriptors. An instrument experiment determined the six feature values, including the sum of variance from the measurement data, whereas multiple linear regression was applied to determine the relationship between the sensory values and feature values. For three of the six textures, the sum of variance reduced the error between the sensory values and their estimated values by up to 50%, confirming that this feature contributes to the textural estimation of food crispness.

An Enhanced Gated Recurrent Unit with Auto-Encoder for Solving Text Classification Problems

Classification has become an important task for automatically categorizing documents based on their respective group. The purpose of classification is to assign the pre-specified group or class to an instance based on the observed features related to that instance. For accurate text classification, feature selection techniques are normally used to identify important features and to remove irrelevant, undesired and noisy features for minimizing the dimensionality of feature space. Therefore, in this research, a new model namely Encoder Simplified GRU (ES-GRU) is proposed to reduce dimension of data using an auto-encoder (AE). Gated Recurrent Unit (GRU) is a deep learning algorithm that contains update gate and reset gate, which is considered as one of the most efficient text classification technique, specifically on sequential datasets. Accordingly, the reset gate is replaced with an update gate in order to reduce the redundancy and complexity in the standard GRU. The proposed model has been evaluated on five benchmark text datasets and compared with six baseline well-known text classification approaches, which includes standard GRU, AE, Long Short-Term Memory, Convolutional Neural Network, Support Vector Machine, and Naïve Bayes. Based on various types of performance evaluation parameters, a considerable amount of improvement has been observed in the performance of the proposed model as compared to state-of-the-art approaches.

Multi-Modal Learning from Video, Eye Tracking, and Pupillometry for Operator Skill Characterization in Clinical Fetal Ultrasound.

This paper presents a novel multi-modal learning approach for automated skill characterization of obstetric ultrasound operators using heterogeneous spatio-temporal sensory cues, namely, scan video, eye-tracking data, and pupillometric data, acquired in the clinical environment. We address pertinent challenges such as combining heterogeneous, small-scale and variable-length sequential datasets, to learn deep convolutional neural networks in real-world scenarios. We propose spatial encoding for multi-modal analysis using sonography standard plane images, spatial gaze maps, gaze trajectory images, and pupillary response images. We present and compare five multi-modal learning network architectures using late, intermediate, hybrid, and tensor fusion. We build models for the Heart and the Brain scanning tasks, and performance evaluation suggests that multi-modal learning networks outperform uni-modal networks, with the best-performing model achieving accuracies of 82.4% (Brain task) and 76.4% (Heart task) for the operator skill classification problem.

Farey Trees Explain Sequential Effects in Choice Response Time.

The latencies of successive two-alternative, forced-choice response times display intricately patterned sequential effects, or dependencies. They vary as a function of particular trial-histories, and in terms of the order and identity of previously presented stimuli and registered responses. This article tests a novel hypothesis that sequential effects are governed by dynamic principles, such as those entailed by a discrete sine-circle map adaptation of the Haken Kelso Bunz (HKB) bimanual coordination model. The model explained the sequential effects expressed in two classic sequential dependency data sets. It explained the rise of a repetition advantage, the acceleration of repeated affirmative responses, in tasks with faster paces. Likewise, the model successfully predicted an alternation advantage, the acceleration of interleaved affirmative and negative responses, when a task’s pace slows and becomes more variable. Detailed analyses of five studies established oscillatory influences on sequential effects in the context of balanced and biased trial presentation rates, variable pacing, progressive and differential cognitive loads, and dyadic performance. Overall, the empirical patterns revealed lawful oscillatory constraints governing sequential effects in the time-course and accuracy of performance across a broad continuum of recognition and decision activities.