Proposed Method Research Articles

Cognitive Hybrid Deep Learning-based Multi-modal Sentiment Analysis for Online Product Reviews

Recently the field of sentiment analysis has gained a lot of attraction in literature. The idea that a machine can dynamically spot the text’s sentiments is fascinating. In this paper, we propose a method to classify the textual sentiments in Twitter feeds. In particular, we focus on analyzing the tweets of products as either positive or negative. The proposed technique utilizes a deep learning schema to learn and predict the sentiment by extracting features directly from the text. Specifically, we use Convolutional Neural Networks with different convolutional layers. Further, we experiment with LSTMs and try an ensemble of multiple models to get the best results. We employ an n-gram-based word embeddings approach to get the machine-level word representations. Testing of the method is conducted on real-world datasets. We have discovered that the ensemble technique yields the best results after conducting experiments on a huge corpus of more than one million tweets. To be specific, we get an accuracy of 84.95%. The proposed method is also compared with several existing methods. An extensive numerical investigation has revealed the superiority of the proposed work in actual deployment scenarios.

Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views.

View missing and label missing are two challenging problems in the applications of multi-view multi-label classification scenery. In the past years, many efforts have been made to address the incomplete multi-view learning or incomplete multi-label learning problem. However, few works can simultaneously handle the challenging case with both the incomplete issues. In this article, we propose a new incomplete multi-view multi-label learning network to address this challenging issue. The proposed method is composed of four major parts: view-specific deep feature extraction network, weighted representation fusion module, classification module, and view-specific deep decoder network. By, respectively, integrating the view missing information and label missing information into the weighted fusion module and classification module, the proposed method can effectively reduce the negative influence caused by two such incomplete issues and sufficiently explore the available data and label information to obtain the most discriminative feature extractor and classifier. Furthermore, our method can be trained in both supervised and semi-supervised manners, which has important implications for flexible deployment. Experimental results on five benchmarks in supervised and semi-supervised cases demonstrate that the proposed method can greatly enhance the classification performance on the difficult incomplete multi-view multi-label classification tasks with missing labels and missing views.

Taming Self-Supervised Learning for Presentation Attack Detection: De-Folding and De-Mixing.

Biometric systems are vulnerable to presentation attacks (PAs) performed using various PA instruments (PAIs). Even though there are numerous PA detection (PAD) techniques based on both deep learning and hand-crafted features, the generalization of PAD for unknown PAI is still a challenging problem. In this work, we empirically prove that the initialization of the PAD model is a crucial factor for generalization, which is rarely discussed in the community. Based on such observation, we proposed a self-supervised learning-based method, denoted as DF-DM. Specifically, DF-DM is based on a global-local view coupled with de-folding and de-mixing to derive the task-specific representation for PAD. During de-folding, the proposed technique will learn region-specific features to represent samples in a local pattern by explicitly minimizing the generative loss. While de-mixing drives detectors to obtain the instance-specific features with global information for more comprehensive representation by minimizing the interpolation-based consistency. Extensive experimental results show that the proposed method can achieve significant improvements in terms of both face and fingerprint PAD in more complicated and hybrid datasets when compared with the state-of-the-art methods. When training in CASIA-FASD and Idiap Replay-Attack, the proposed method can achieve an 18.60% equal error rate (EER) in OULU-NPU and MSU-MFSD, exceeding the baseline performance by 9.54%. The source code of the proposed technique is available at https://github.com/kongzhecn/dfdm.

Explaining Genetic Programming-Evolved Routing Policies for Uncertain Capacitated Arc Routing Problems

Genetic programming has been successfully used to evolve routing policies that can make real-time routing decisions for uncertain arc routing problems. Although the evolved routing policies are highly effective, they are typically very large and complex, and hard to be understood and trusted by real users. Existing studies have attempted to improve the interpretability by developing new genetic programming approaches to evolve both effective and interpretable (e.g., with smaller program size) routing policies. However, they still have limitations due to the trade-off between effectiveness and interpretability. To address this issue, we propose a new post-hoc explanation approach to explaining the effective but complex routing policies evolved by genetic programming. The new approach includes a local ranking explanation and a global explanation module. The local ranking explanation uses particle swarm optimisation to learn an interpretable linear model that accurately explains the local behaviour of the routing policy for each decision situation. Then, the global explanation module uses a clustering technique to summarise the local explanations into a global explanation. The experimental results and case studies on the benchmark datasets show that the proposed method can obtain accurate and understandable explanations of the routing policies evolved for uncertain arc routing problems. Our explanation approach is not restricted to uncertain arc routing, but has a great potential to be generalised to other optimisation and machine learning problems such as learning classifier systems and reinforcement learning.

Federated Many-Task Bayesian Optimization

Bayesian optimization is a powerful surrogate-assisted algorithm for solving expensive black-box optimization problems. While Bayesian optimization was developed for centralized optimization, the availability of massive distributed data has attracted increased interests in exploring federated Bayesian optimization that can use data on multiple clients without leaking the raw data. However, existing federated Bayesian optimization (FBO) approaches assume that either all clients jointly solve the same optimization task, or only one client solves one target optimization task by transferring knowledge from others in a federated way, making them unsuited for many real-world applications. In this paper, we consider FBO for the scenario where multiple related local black-box tasks associated with different clients are jointly optimized by sharing knowledge between tasks without leaking the data privacy. An efficient federated many-task Bayesian optimization framework is proposed to address not independent and identically distributed (non-IID) data while protecting the data privacy in the federated setting. A novel federated knowledge transfer paradigm is developed for dynamic many-task model aggregation according to a dissimilarity matrix. The dissimilarity is measured based on the rank of the predictions and only the hyperparameters in the local Gaussian process models are shared. In addition, a federated ensemble acquisition function is constructed by integrating the predictions of two surrogates using the global and local hyperparameters, respectively, to effectively search for the optimal solution. Experimental results show that our proposed method has reliable performance on both benchmark problems and a real machine learning problem also in the presence of non-IID data.

Synchronization of Coupled Neural Networks With Constant Time-Delay Using Sampled-Data Information.

In this article, a synchronization control method is studied for coupled neural networks (CNNs) with constant time delay using sampled-data information. A distributed control protocol relying on the sampled-data information of neighboring nodes is proposed. Lyapunov functional is constructed to analyze the synchronization of CNNs with constant time delay. Using Park's integral inequality and improved free-weight matrix integral inequality, sufficient conditions are provided for CNNs to achieve synchronization with less conservatism. In addition, the maximum sampling interval is determined by transforming the sufficient conditions into an optimization problem, and an aperiodic sampling control technique is implemented to reduce the communication energy load. Finally, numerical simulations are provided to demonstrate that the proposed method is capable of achieving synchronization.

A Spatial-Temporal Video Quality Assessment Method via Comprehensive HVS Simulation.

The quality of videos is the primary concern of video service providers. Built upon deep neural networks, video quality assessment (VQA) has rapidly progressed. Although existing works have introduced the knowledge of the human visual system (HVS) into VQA, there are still some limitations that hinder the full exploitation of HVS, including incomplete modeling with few HVS characteristics and insufficient connection among these characteristics. In this article, we present a novel spatial-temporal VQA method termed HVS-5M, wherein we design five modules to simulate five characteristics of HVS and create a bioinspired connection among these modules in a cooperative manner. Specifically, on the side of the spatial domain, the visual saliency module first extracts a saliency map. Then, the content-dependency and the edge masking modules extract the content and edge features, respectively, which are both weighted by the saliency map to highlight those regions that human beings may be interested in. On the other side of the temporal domain, the motion perception module extracts the dynamic temporal features. Besides, the temporal hysteresis module simulates the memory mechanism of human beings and comprehensively evaluates the video quality according to the fusion features from the spatial and temporal domains. Extensive experiments show that our HVS-5M outperforms the state-of-the-art VQA methods. Ablation studies are further conducted to verify the effectiveness of each module toward the proposed method. The source code is available at https://github.com/GZHU-DVL/HVS-5M.

Stabilization of Periodic Piecewise Time-Varying Systems With Time-Varying Delay Under Multiple Cyber Attacks: An Augmented Lyapunov Functional Approach.

This article investigates the asymptotic stabilization of periodic piecewise time-varying systems with time-varying delay under various cyber attacks, particularly deception and acrlong DoS attacks. The addressed system is reformed into a number of time-varying subsystems based on the time interval for each period. Following that, a state-feedback controller with periodic time-varying gain parameters is developed to solve the stabilization problem. The control design depicts the possibility of the aforementioned cyber attacks with two mutually exclusive stochastic Bernoulli distributed parameters. Then, an augmented Lyapunov-Krasovskii functional with periodically varying matrices is used to determine the conditions for designing the proposed controller that ensures the mean-square asymptotic stability of the addressed system. The results of numerical examples support the conclusion that the proposed method is effective and superior, regardless of the cyber attacks involved.

One Size Fits All: A Unified Traffic Predictor for Capturing the Essential Spatial-Temporal Dependency.

Traffic prediction is a keystone for building smart cities in the new era and has found wide applications in traffic scheduling and management, environment policy making, public safety, and so on. Instead of creating a traffic predictor for each city, this article focuses on designing a unified network model that could be directly applied for traffic prediction in any city, by learning the essential spatial-temporal dependencies, i.e., the mutual relationship between traffic and the corresponding fine-grained road network. To achieve this goal, this article proposes a joint knowledge- and data-driven mechanism that novelly divides dependencies into three kinds of correlations, i.e., road segment, intra-intersection, and inter-intersection correlation, which capture the microcosmic, middle, and macroscopic dependencies between traffic and the road network, respectively. Specifically, we first construct traffic datasets that could cover all road segments from real-world trajectory datasets, which makes it possible to model the whole road network as a graph, with the help of fine-grained road topology. Then, we propose meta road segment learner, connection-aware spatial-temporal graph convolutional network (GCN), and multiscale residual networks for capturing the microcosmic, middle, and macroscopic dependencies, respectively. Our experiments on three real-world datasets demonstrate that our proposed method could: 1) achieve better prediction accuracy compared with several approaches and 2) capture the mutual relationship between traffic and the fine-grained road network since our model trained only using data from the source city achieves good performance when it is directly applied for traffic prediction in the target city, without any fine-tuning. The codes will be made publicly available.

Value-Based Subgoal Discovery and Path Planning for Reaching Long-Horizon Goals.

Learning to reach long-horizon goals in spatial traversal tasks is a significant challenge for autonomous agents. Recent subgoal graph-based planning methods address this challenge by decomposing a goal into a sequence of shorter-horizon subgoals. These methods, however, use arbitrary heuristics for sampling or discovering subgoals, which may not conform to the cumulative reward distribution. Moreover, they are prone to learning erroneous connections (edges) between subgoals, especially those lying across obstacles. To address these issues, this article proposes a novel subgoal graph-based planning method called learning subgoal graph using value-based subgoal discovery and automatic pruning (LSGVP). The proposed method uses a subgoal discovery heuristic that is based on a cumulative reward (value) measure and yields sparse subgoals, including those lying on the higher cumulative reward paths. Moreover, LSGVP guides the agent to automatically prune the learned subgoal graph to remove the erroneous edges. The combination of these novel features helps the LSGVP agent to achieve higher cumulative positive rewards than other subgoal sampling or discovery heuristics, as well as higher goal-reaching success rates than other state-of-the-art subgoal graph-based planning methods.

Automatic Learning Rate Adaption for Memristive Deep Learning Systems.

As a possible device to further enhance the performance of the hybrid complementary metal oxide semiconductor (CMOS) technology in the hardware, the memristor has attracted widespread attention in implementing efficient and compact deep learning (DL) systems. In this study, an automatic learning rate tuning method for memristive DL systems is presented. Memristive devices are utilized to adjust the adaptive learning rate in deep neural networks (DNNs). The speed of the learning rate adaptation process is fast at first and then becomes slow, which consist of the memristance or conductance adjustment process of the memristors. As a result, no manual tuning of learning rates is required in the adaptive back propagation (BP) algorithm. While cycle-to-cycle and device-to-device variations could be a significant issue in memristive DL systems, the proposed method appears robust to noisy gradients, various architectures, and different datasets. Moreover, fuzzy control methods for adaptive learning are presented for pattern recognition, such that the over-fitting issue can be well addressed. To our best knowledge, this is the first memristive DL system using an adaptive learning rate for image recognition. Another highlight of the presented memristive adaptive DL system is that quantized neural network architecture is utilized, and there is therefore a significant increase in the training efficiency, without the loss of testing accuracy.

Nonfactoid Question Answering as Query-Focused Summarization With Graph-Enhanced Multihop Inference.

Nonfactoid question answering (QA) is one of the most extensive yet challenging applications and research areas in natural language processing (NLP). Existing methods fall short of handling the long-distance and complex semantic relations between the question and the document sentences. In this work, we propose a novel query-focused summarization method, namely a graph-enhanced multihop query-focused summarizer (GMQS), to tackle the nonfactoid QA problem. Specifically, we leverage graph-enhanced reasoning techniques to elaborate the multihop inference process in nonfactoid QA. Three types of graphs with different semantic relations, namely semantic relevance, topical coherence, and coreference linking, are constructed for explicitly capturing the question-document and sentence-sentence interrelationships. Relational graph attention network (RGAT) is then developed to aggregate the multirelational information accordingly. In addition, the proposed method can be adapted to both extractive and abstractive applications as well as be mutually enhanced by joint learning. Experimental results show that the proposed method consistently outperforms both existing extractive and abstractive methods on two nonfactoid QA datasets, WikiHow and PubMedQA, and possesses the capability of performing explainable multihop reasoning.

Multidomain Object Detection Framework Using Feature Domain Knowledge Distillation.

Object detection techniques have been widely studied, utilized in various works, and have exhibited robust performance on images with sufficient luminance. However, these approaches typically struggle to extract valuable features from low-luminance images, which often exhibit blurriness and dim appearence, leading to detection failures. To overcome this issue, we introduce an innovative unsupervised feature domain knowledge distillation (KD) framework. The proposed framework enhances the generalization capability of neural networks across both low- and high-luminance domains without incurring additional computational costs during testing. This improvement is made possible through the integration of generative adversarial networks and our proposed unsupervised KD process. Furthermore, we introduce a region-based multiscale discriminator designed to discern feature domain discrepancies at the object level rather than from the global context. This bolsters the joint learning process of object detection and feature domain distillation tasks. Both qualitative and quantitative assessments shown that the proposed method, empowered by the region-based multiscale discriminator and the unsupervised feature domain distillation process, can effectively extract beneficial features from low-luminance images, outperforming other state-of-the-art approaches in both low- and sufficient-luminance domains.

Mechanism and Data-Driven Dual Estimation of Coupling Hydraulic-Thermal States for Steam Heating Networks Considering Multi-Time-Scale Characteristics.

For a steam heating network (SHN), which is one of the most important parts of the integrated energy systems in an industrial park, it is essential to provide accurate and reliable estimation of hydraulic and thermal states, to ensure its operational safety and economy. Considering two different time-scale characteristics of the hydraulic and thermal processes in the SHN, this article proposes a mechanism and data-driven dual estimation method for the coupling hydraulic-thermal dynamic states. Due to the fact that the existing coupled estimation methods with one time scale may suffer from heavy computational burden, a data-driven dual sequential acrlong DSE method of the SHN is proposed based on the two-time scales, in which the dynamics of slow thermal states can also be captured when performing the acrlong SE of the fast hydraulic process. Furthermore, to improve the computational and communicational efficiency, a distributed interaction strategy based on the nodal transformation matrix is designed for large-scale steam systems. To verify the effectiveness of the proposed method, a single pipeline system and two real-world industrial superheated steam networks are employed. Compared to other state-of-the-art methods, the proposed method achieves the best tradeoff between the estimation accuracy and computational efficiency.

EEG-Based Motor Imagery Recognition Framework via Multisubject Dynamic Transfer and Iterative Self-Training.

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

DecouplingNet: A Stable Knowledge Distillation Decoupling Net for Fault Detection of Rotating Machines Under Varying Speeds.

Fault detection, also known as anomaly detection (AD), is at the heart of prediction and health management (PHM), which plays a vital role in ensuring the safe operation of mechanical equipment. Nonetheless, the lack of anomaly data creates a significant obstacle to the AD of the mechanical system. In particular, the complex modulation effects induced by time-varying speeds make AD much more challenging. For rapid and accurate AD, a stable knowledge distillation decoupling net (DecouplingNet) is provided to overcome these difficulties. First, an adversarial network consisting of an encoder, a decoder, and an encoder-discriminator is developed to model normal samples well by imposing constraints on the latent space. Then, a causal decoupling framework is suggested to disentangle equipment state-related information from operating conditions-related features, enabling stable condition monitoring at varying speeds. Finally, feature-based knowledge distillation is employed to boost the efficiency of AD while maintaining the detection accuracy. The proposed method is tested on two experimental scenarios and compared with some typical AD methods. The finding demonstrates that the net outperforms others in terms of accuracy and efficiency when it comes to detecting anomalies in the mechanical equipment that runs under varying speeds.

Enhanced Results on Sampled-Data Synchronization for Chaotic Neural Networks With Actuator Saturation Using Parameterized Control.

This article investigates a novel sampled-data synchronization controller design method for chaotic neural networks (CNNs) with actuator saturation. The proposed method is based on a parameterization approach which reformulates the activation function as the weighted sum of matrices with the weighting functions. Also, controller gain matrices are combined by affinely transformed weighting functions. The enhanced stabilization criterion is formulated in terms of linear matrix inequalities (LMIs) based on the Lyapunov stability theory and weighting function's information. As shown in the comparison results of the bench marking example, the presented method much outperforms previous methods, and thus the enhancement of the proposed parameterized control is verified.

A Hybrid Neuromorphic Object Tracking and Classification Framework for Real-Time Systems.

Deep learning inference that needs to largely take place on the "edge" is a highly computational and memory intensive workload, making it intractable for low-power, embedded platforms such as mobile nodes and remote security applications. To address this challenge, this article proposes a real-time, hybrid neuromorphic framework for object tracking and classification using event-based cameras that possess desirable properties such as low-power consumption (5-14 mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches of using event-by-event processing, this work uses a mixed frame and event approach to get energy savings with high performance. Using a frame-based region proposal method based on the density of foreground events, a hardware-friendly object tracking scheme is implemented using the apparent object velocity while tackling occlusion scenarios. The frame-based object track input is converted back to spikes for TrueNorth (TN) classification via the energy-efficient deep network (EEDN) pipeline. Using originally collected datasets, we train the TN model on the hardware track outputs, instead of using ground truth object locations as commonly done, and demonstrate the ability of our system to handle practical surveillance scenarios. As an alternative tracker paradigm, we also propose a continuous-time tracker with C++ implementation where each event is processed individually, which better exploits the low latency and asynchronous nature of neuromorphic vision sensors. Subsequently, we extensively compare the proposed methodologies to state-of-the-art event-based and frame-based methods for object tracking and classification, and demonstrate the use case of our neuromorphic approach for real-time and embedded applications without sacrificing performance. Finally, we also showcase the efficacy of the proposed neuromorphic system to a standard RGB camera setup when simultaneously evaluated over several hours of traffic recordings.

Cross-Scene Joint Classification of Multisource Data With Multilevel Domain Adaption Network.

Domain adaption (DA) is a challenging task that integrates knowledge from source domain (SD) to perform data analysis for target domain. Most of the existing DA approaches only focus on single-source-single-target setting. In contrast, multisource (MS) data collaborative utilization has been extensively used in various applications, while how to integrate DA with MS collaboration still faces great challenges. In this article, we propose a multilevel DA network (MDA-NET) for promoting information collaboration and cross-scene (CS) classification based on hyperspectral image (HSI) and light detection and ranging (LiDAR) data. In this framework, modality-related adapters are built, and then a mutual-aid classifier is used to aggregate all the discriminative information captured from different modalities for boosting CS classification performance. Experimental results on two cross-domain datasets show that the proposed method consistently provides better performance than other state-of-the-art DA approaches.

Graph-Based Restricted and Arbitrary Switching for Switched Positive Systems via a Weak CLCLF.

This article studies the stability problem of discrete-time switched positive linear systems (SPLSs) with marginally stable subsystems. Based on the weak common linear copositive Lyapunov function (weak CLCLF) approach, the switching property and the state component property are combined to ensure the asymptotic stability of SPLSs under three types of switching signals. First, considering the transfer-restricted switching signal described by the switching digraph, novel cycle-dependent joint path conditions are proposed in combination with state component digraphs. Second, under the time interval sequence, two types of path conditions are constructed for designing switching schemes. Third, necessary and sufficient conditions for the asymptotic stability of SPLSs under arbitrary switching are established. Finally, three examples are provided to illustrate the effectiveness of the proposed method.