Discriminative Correlation Research Articles

Transferable driver facial expression recognition based on joint discriminative correlation alignment network with enhanced feature attention

AbstractFacial expression is closely related to the emotions of drivers, thus facilitating safe driving detection in advanced driving assistance system (ADAS). Recently, deep learning techniques have become prevalent for facial expression recognition. However, for driving scenarios, the facial expression recognition is mainly challenged by the problems of small sample size as well as effective feature representation. To address the above issues, this paper puts forward a transfer learning‐based method for driver facial expression recognition by fully exploiting the other sources of facial expression data that may obey different distributions. Specifically, an enhanced feature attention module is firstly devised such that affluent features with multi‐scales can be extracted and refined based on the spatial and channel attention mechanisms. Then, a joint correlation alignment loss is presented ensuring that the samples in the source and target domains are transformed into the shared common subspace to reduce difference of both the marginal and conditional distributions. Multiple transfer learning tasks on real‐world data are carried out to evaluate the proposed method. The experimental results show that our model achieves better recognition accuracy for driver facial expression compared with several traditional and deep learning‐based transfer learning algorithms.

Learning Context Restrained Correlation Tracking Filters via Adversarial Negative Instance Generation.

The tracking performance of discriminative correlation filters (DCFs) is often subject to unwanted boundary effects. Many attempts have already been made to address the above issue by enlarging searching regions over the last years. However, introducing excessive background information makes the discriminative filter prone to learn from the surrounding context rather than the target. In this article, we propose a novel context restrained correlation tracking filter (CRCTF) that can effectively suppress background interference via incorporating high-quality adversarial generative negative instances. Concretely, we first construct an adversarial context generation network to simulate the central target area with surrounding background information at the initial frame. Then, we suggest a coarse background estimation network to accelerate the background generation in subsequent frames. By introducing a suppression convolution term, we utilize generative background patches to reformulate the original ridge regression objective through circulant property of correlation and a cropping operator. Finally, our tracking filter is efficiently solved by the alternating direction method of multipliers (ADMM). CRCTF demonstrates the accuracy performance on par with several well-established and highly optimized baselines on multiple challenging tracking datasets, verifying the effectiveness of our proposed approach.

Scale-Aware Tracking Method with Appearance Feature Filtering and Inter-Frame Continuity.

Visual object tracking is a fundamental task in computer vision that requires estimating the position and scale of a target object in a video sequence. However, scale variation is a difficult challenge that affects the performance and robustness of many trackers, especially those based on the discriminative correlation filter (DCF). Existing scale estimation methods based on multi-scale features are computationally expensive and degrade the real-time performance of the DCF-based tracker, especially in scenarios with restricted computing power. In this paper, we propose a practical and efficient solution that can handle scale changes without using multi-scale features and can be combined with any DCF-based tracker as a plug-in module. We use color name (CN) features and a salient feature to reduce the target appearance model's dimensionality. We then estimate the target scale based on a Gaussian distribution model and introduce global and local scale consistency assumptions to restore the target's scale. We fuse the tracking results with the DCF-based tracker to obtain the new position and scale of the target. We evaluate our method on the benchmark dataset Temple Color 128 and compare it with some popular trackers. Our method achieves competitive accuracy and robustness while significantly reducing the computational cost.

Multimodal construction of human movement dataset under the development of physical dance teaching in colleges and universities

Abstract First, this paper proposes a multimodal fusion-based human motion recognition technique applied to college physical dance teaching, where feature-level or decision-level fusion maps human kinematic semantic information from datasets to classifiers under a single-level stage. Secondly, a multi-level multimodal fusion approach for human motion recognition is proposed to make it more adaptable to practical application scenarios. The depth data is converted into a depth motion projection map at the input side, and the inertial data is converted into a signal image. Finally, all the modalities are trained by a convolutional neural network for extracting features, and the extracted features are fused at the feature level by discriminant correlation analysis. The results show that the multi-level multimodal fusion framework achieves 99.8% and 99.9% recognition accuracy on the two datasets, respectively, with 100% accuracy in the Throw and Catch action recognition and the lowest recognition rate of 95.36% in the Clap action test, with an average recognition rate of 97.89%, which has a high recognition accuracy. The multi-level multimodal fusion model can obtain movement data close to the actual movement data after optimization in low-precision movement data, which provides data support for physical dance teaching and learning.

Tracking by segmentation with future motion estimation applied to person-following robots.

Person-following is a crucial capability for service robots, and the employment of vision technology is a leading trend in building environmental understanding. While most existing methodologies rely on a tracking-by-detection strategy, which necessitates extensive datasets for training and yet remains susceptible to environmental noise, we propose a novel approach: real-time tracking-by-segmentation with a future motion estimation framework. This framework facilitates pixel-level tracking of a target individual and predicts their future motion. Our strategy leverages a single-shot segmentation tracking neural network for precise foreground segmentation to track the target, overcoming the limitations of using a rectangular region of interest (ROI). Here we clarify that, while the ROI provides a broad context, the segmentation within this bounding box offers a detailed and more accurate position of the human subject. To further improve our approach, a classification-lock pre-trained layer is utilized to form a constraint that curbs feature outliers originating from the person being tracked. A discriminative correlation filter estimates the potential target region in the scene to prevent foreground misrecognition, while a motion estimation neural network anticipates the target's future motion for use in the control module. We validated our proposed methodology using the VOT, LaSot, YouTube-VOS, and Davis tracking datasets, demonstrating its effectiveness. Notably, our framework supports long-term person-following tasks in indoor environments, showing promise for practical implementation in service robots.

Self-supervised learning representation for abnormal acoustic event detection based on attentional contrastive learning

Most abnormal acoustic event detection (AAED) is completed by supervised training of deep learning methods, but manually labeled samples are costly and scarce. This work proposes a self-supervised learning representation for AAED based on contrastive learning to overcome the abovementioned problem. Auditory and visual data augmentations are applied simultaneously to create positive sample pairs. An attention mechanism is introduced into the encoder during self-supervised pre-training. A comparison between fused features by discriminant correlation analysis and a single feature is made to verify the ability of feature grasping for the self-supervised pre-trained model. The pre-training is completed on an abnormal acoustic dataset with noise. Research results show that the self-supervised pre-trained model can achieve an accuracy of 87.72% in linear evaluation and 88.70% in the downstream task with a pure small AAED dataset, which directly exceeds the results of supervised learning. This work releases the stress of the demand for abnormal acoustic event labels.

Learning dynamic spatial-temporal regularized correlation filter tracking with response deviation suppression via multi-feature fusion

Visual object tracking (VOT) for intelligent video surveillance has attracted great attention in the current research community, thanks to advances in computer vision and camera technology. Meanwhile, discriminative correlation filter (DCF) trackers garnered significant interest owing to their high accuracy and low computing cost. Many researchers have introduced spatial and temporal regularization into the DCF framework to achieve a more robust appearance model and further improve tracking performance. However, these algorithms typically set fixed spatial and temporal regularization parameters, which limit flexibility and adaptability under cluttered and challenging scenarios. To overcome these problems, in this work, we propose a new dynamic spatial–temporal regularization for the DCF tracking model that emphasizes the filter to concentrate on more reliable regions during the training stage. Furthermore, we present a response deviation-suppressed regularization term for responses to encourage temporal consistency and avoid model degradation by suppressing relative response changes between two consecutive frames. Moreover, we introduce a multi-memory tracking framework to exploit various features and each memory contributes to tracking the target across all frames. Significant experiments on the OTB-2013, OTB-2015, TC-128, UAV-123, UAVDT, and DTB-70 datasets have revealed that the performance thereof outperformed many state-of-the-art trackers based on DCF and deep-based frameworks in terms of tracking accuracy and tracking success rate.

GRM: Gaussian response module for visual tracking

Online Discriminative Correlation Filters have excellent performance in visual object tracking. It always divides the tracking network as a classification network and a regression network, which makes the regression network lack classification information and makes it hard to leverage its advantages. To address this problem, we propose the ATOM_GRM tracker to append the classification information to the network. Specifically, we design static and dynamic Gaussian response modules to encode the raw target position from classifier to regression network by multistage. In Particular, the parameters and fusion method of two Gaussian response modules is designed in different ways according to their location in the network. Moreover, we propose the ratio of height and width of the bounding box instead of intersection of union as the predictor, and two auxiliary training heads are used in the regression network. It makes the regression network better distinguish bounding boxes. Extensive experiments conducted on four public benchmarks, i.e., OTB100, GOT10k, LaSOT, Trackingnet, demonstrate the effectiveness of the proposed method. The proposed ATOM_GRM achieves 0.556 → 0.596 AO compared with the baseline ATOM on GOT10k.

All-Day Object Tracking for Unmanned Aerial Vehicle

Unmanned aerial vehicle (UAV) has facilitated a wide range of real-world applications and attracted extensive research in the mobile computing field. Specially, developing real-time robust visual onboard trackers for all-day aerial maneuver can remarkably broaden the scope of intelligent deployment of UAV. However, prior tracking methods have merely focused on robust tracking in the well-illuminated scenes, while ignoring trackers capabilities to be deployed in the dark. In darkness, the conditions can be more complex and harsh, easily posing inferior robust tracking or even tracking failure. To this end, this work proposes a novel discriminative correlation filter-based tracker with illumination adaptive and anti-dark capability, namely ADTrack. ADTrack firstly exploits image illuminance information to enable adaptability of the model to the given light condition. Then, by virtue of an efficient enhancer, ADTrack carries out image pretreatment where a target aware mask is generated. Benefiting from the mask, ADTrack aims to solve a novel dual regression problem where dual filters are online trained with mutual constraint. Besides, this work also constructs a UAV nighttime tracking benchmark UAVDark135. Exhaustive experiments on authoritative benchmarks and onboard tests are implemented to validate the superiority and robustness of ADTrack in all-day conditions.

A Re-detection Target Tracking Framework after Loss for Unmanned Aerial Vehicle

Currently, the target tracking algorithm based on discriminative correlation filter (DCF) and deep learning (DL) plays an increasingly important role in UAV (unmanned aerial vehicle). However, existing algorithms have limitations such as limited search region, difficulty in re-capturing targets when tracking is lost, and high computational complexity, which makes them difficult to apply to the UAV platform. In this paper, a target re-detection tracking algorithm (RDT) after tracking loss is designed for mobile platforms. RDT uses DCF in simple scenes to track targets and establish a target model between different frames. To ensure the algorithm when tracking is lost due to occlusion or moving out of view, an efficient switching criterion is designed to indirectly invoke the detection algorithm to re-capture the target and guide online learning of DCF. And the experiment results on the OTB benchmark show that RDT can re-capture the target after tracking is lost, and the speed of operation on the CPU is 85fps.

Robust Visual Object Tracking Based on Feature Channel Weighting and Game Theory

Although the discriminative correlation filter- (DCF)-based tracker improves tracking performance, some object representation issues can still be further optimized. On the one hand, the DCF tracker’s deep convolutional features contain many noisy channels, and assigning the same weights to multiple channels cannot distinguish the importance of different channels. On the other hand, a simple weighted fusion approach cannot fully utilize the benefits of different feature types. We propose a visual object tracking algorithm based on adaptive channel weighting and feature game fusion to solve these problems. In this study, an adaptive channel weighting strategy is designed to assign suitable weights to each channel based on the average energy ratio of the target and background regions in the feature channels and prune the channels with low weights to improve feature robustness and reduce computational complexity. Simultaneously, the game theory concept is introduced in the multifeature fusion. The handcrafted features are combined with shallow and deep convolutional features according to feature complementarity. Then, the two combined features are seen as two sides of the game, continuously gamed during the tracking process to generate a feature model with a higher representation capacity. Extensive experiments are conducted on four mainstream visual tracking benchmark datasets, including OTB2015, VOT2018, LaSOT, and UAV123. The experimental results show that the proposed algorithm performs outstandingly compared to the state-of-the-art trackers.

Pedestrian Tracking in UAV Images With Kalman Filter Motion Estimator and Correlation Filter

Visual object tracking is one of the significant parts of systems varying from autonomous driving to drone-based surveillance/tracking. Discriminative correlation filter (DCF) based trackers have proved their prominence in the past few years. These utilize visual information present in images for monitoring. This work uses the Kalman Filter (KF) to derive a motion estimation model. It combines it with two DCF-based trackers, namely Kernelized Correlation Filter (KCF) and Discriminative Correlation Filter with Channel and Spatial Reliability (CSRDCF) for pedestrian tracking. Camera motion-compensated versions of the trackers are also presented. The performance of the proposed methodology is presented in terms of Success Rate (SR) and Precision (P). Real-time power consumption, memory occupancy, and speed of trackers on the Jetson Nano (ARM Cortex-A57, 4 GB RAM) board have been presented.

Learning Background-Suppressed Dual-Regression Correlation Filters for Visual Tracking

The discriminative correlation filter (DCF)-based tracking method has shown good accuracy and efficiency in visual tracking. However, the periodic assumption of sample space causes unwanted boundary effects, restricting the tracker’s ability to distinguish between the target and background. Additionally, in the real tracking environment, interference factors such as occlusion, background clutter, and illumination changes cause response aberration and, thus, tracking failure. To address these issues, this work proposed a novel tracking method named the background-suppressed dual-regression correlation filter (BSDCF) for visual tracking. First, we utilize the background-suppressed function to crop out the target features from the global features. In the training step, while introducing the spatial regularity constraint and background response suppression regularization, we construct a dual regression structure to train the target and global filters separately. The aim is to exploit the difference between the output response maps for mutual constraint to highlight the target and suppress the background interference. Furthermore, in the detection step, the global response can be enhanced by a weighted fusion of the target response to further improve the tracking performance in complex scenes. Finally, extensive experiments are conducted on three public benchmarks (including OTB100, TC128, and UAVDT), and the experimental results indicate that the proposed BSDCF tracker achieves tracking performance comparable to many state-of-the-art (SOTA) trackers in a variety of complex situations.

SOH estimation and RUL prediction of lithium batteries based on multidomain feature fusion and CatBoost model

AbstractIn this paper, a lithium‐ion battery State of Health (SOH) estimation algorithm is proposed based on the fusion of multidomain features and the application of a CatBoost model. The aim is to address the issue of low prediction accuracy in SOH caused by the utilization of single‐feature extraction techniques. The algorithm encompasses the extraction of various features from the original charge–discharge data, including time‐domain, frequency‐domain, entropy, and time‐series features. Following the evaluation of feature importance, a feature selection process is conducted to eliminate redundant features that provide a limited contribution to the predictive results. Subsequently, a multiple‐set discriminative correlation analysis is employed to integrate high‐dimensional features. To attain accurate predictions, the CatBoost model is further optimized through the utilization of a sparrow search algorithm. Experimental results demonstrate that the proposed algorithm achieves accurate SOH estimations within individual batteries, as evidenced by mean square error values consistently below 4e−4 and goodness‐of‐fit values exceeding or equal to 0.98. Additionally, the algorithm exhibits reliable prediction capabilities across different batteries operating under the same charge/discharge strategy. Comparative analysis indicates that the adoption of the multidomain feature fusion approach yields improved prediction accuracy in contrast to the utilization of a single feature extraction method.

RGBT tracking based on prior least absolute shrinkage and selection operator and quality aware fusion of deep and handcrafted features

Object tracking has always been one of the most challenging topics in machine vision. In recent years, trackers have used RGB-T datasets to overcome the limitations of single-modality data. This paper introduces a Prior Least Absolute Shrinkage and Selection Operator (PLASSO)-based cost function optimized by the Alternating Direction Method of Multipliers (ADMM) for a Discriminative Correlation Filter (DCF)-based tracker. Detailed closed-form solutions for these optimization problems are also given in this paper. The proposed PLASSO-based approach adaptively determines the scale, parameters, and features of the PLASSO-ADSPF tracker. PLASSO- ADSPF first finds the response map in the Fourier domain for the input images and then uses an efficient adaptive function to fuse the results. Based on the response maps, the proposed tracker adaptively and consciously extracts features (handcraft and Deep Features (DF)) to improve the speed of the tracking algorithm. PLASSO-ADSPF extracts DF from the convolution layer output of a modified pre-trained deep network via SoftMax Tsallis Entropy (TsEn) as a proposed cost function in the last layer. Also, a graph is proposed to determine longitudinal and transverse scales independently and accurately. Moreover, a new bounding box regression is proposed based on PLASSO and Cholesky factorization to refine the target box. Extensive experiments have been performed for the PLASSO-ADSPF tracker on five popular datasets. The results show the superiority of the proposed approach compared to the state-of-the-art trackers. Our code is available at PLASSO-ADSPF-tracker.

Fighting against COVID‐19: Innovations and applications

Fighting against <scp>COVID</scp>‐19: Innovations and applications

Neurophysiological correlates of tactile width discrimination in humans.

Tactile information processing requires the integration of sensory, motor, and cognitive information. Width discrimination has been extensively studied in rodents, but not in humans. Here, we describe Electroencephalography (EEG) signals in humans performing a tactile width discrimination task. The first goal of this study was to describe changes in neural activity occurring during the discrimination and the response periods. The second goal was to relate specific changes in neural activity to the performance in the task. Comparison of changes in power between two different periods of the task, corresponding to the discrimination of the tactile stimulus and the motor response, revealed the engagement of an asymmetrical network associated with fronto-temporo-parieto-occipital electrodes and across multiple frequency bands. Analysis of ratios of higher [Ratio 1: (0.5-20 Hz)/(0.5-45 Hz)] or lower frequencies [Ratio 2: (0.5-4.5 Hz)/(0.5-9 Hz)], during the discrimination period revealed that activity recorded from frontal-parietal electrodes was correlated to tactile width discrimination performance between-subjects, independently of task difficulty. Meanwhile, the dynamics in parieto-occipital electrodes were correlated to the changes in performance within-subjects (i.e., between the first and the second blocks) independently of task difficulty. In addition, analysis of information transfer, using Granger causality, further demonstrated that improvements in performance between blocks were characterized by an overall reduction in information transfer to the ipsilateral parietal electrode (P4) and an increase in information transfer to the contralateral parietal electrode (P3). The main finding of this study is that fronto-parietal electrodes encoded between-subjects' performances while parieto-occipital electrodes encoded within-subjects' performances, supporting the notion that tactile width discrimination processing is associated with a complex asymmetrical network involving fronto-parieto-occipital electrodes.

Psychological and physiological correlates of stimulus discrimination in adults.

The discrimination of cues in the environment that signal danger ("fear cue") is important for survival but depends critically on the discernment of such cues from ones that pose no threat ("safety cues"). In rodents, we previously demonstrated the underlying neurobiological mechanisms that support fear versus safety discrimination and documented that these mechanisms extend to the discrimination of reward as well. While learning about reward is equally important for survival, it remains an under-studied area of research, particularly in human studies of conditional discrimination. In the present study, we translated our rodent task of fear reward and neutral discrimination (fear, reward, and neutral discrimination [FRND]) for use in humans. Undergraduate students (N = 53) completed the FRND while electrodermal activity was recorded. Skin conductance response (SCR) amplitude, a marker of arousal response, was derived for fear, reward, and neutral cues that signaled no outcome; critical trials assessed conditional discrimination using combined fear + neutral and reward + neutral cues. Participants provided likeability ratings for each cue type. Results demonstrated that participants rated reward cues the best, fear cues the worst, and neutral cues in between, while SCR amplitude was largest for fear and reward cues and lowest for neutral cues. SCR amplitudes were reduced for fear + neutral (compared to fear) and reward + neutral cues (compared to reward). Results demonstrate that the FRND is a useful paradigm for the assessment of psychological and physiological discrimination of fear and reward. Implications and directions for future work are discussed.

Learning spatio-temporal discriminative model for affine subspace based visual object tracking

Discriminative correlation filters (DCF) with powerful feature descriptors have proven to be very effective for advanced visual object tracking approaches. However, due to the fixed capacity in achieving discriminative learning, existing DCF trackers perform the filter training on a single template extracted by convolutional neural networks (CNN) or hand-crafted descriptors. Such single template learning cannot provide powerful discriminative filters with guaranteed validity under appearance variation. To pinpoint the structural relevance of spatio-temporal appearance to the filtering system, we propose a new tracking algorithm that incorporates the construction of the Grassmannian manifold learning in the DCF formulation. Our method constructs the model appearance within an online updated affine subspace. It enables joint discriminative learning in the origin and basis of the subspace, achieving enhanced discrimination and interpretability of the learned filters. In addition, to improve tracking efficiency, we adaptively integrate online incremental learning to update the obtained manifold. To this end, specific spatio-temporal appearance patterns are dynamically learned during tracking, highlighting relevant variations and alleviating the performance degrading impact of less discriminative representations from a single template. The experimental results obtained on several well-known datasets, i.e., OTB2013, OTB2015, UAV123, and VOT2018, demonstrate the merits of the proposed method and its superiority over the state-of-the-art trackers.

Similarity-sequenced multi-view discriminant feature extraction for image recognition

Traditional multi-view feature extraction methods based on manifold learning frequently overlook the similarity sequence between samples, failing to capture the intrinsic manifold structure of raw nonlinear samples and restricting the recognition performance of multi-view learning. In this paper, we propose a novel similarity-sequenced multi-view discriminant feature extraction method, called Similarity -sequenced Multi-view Discriminant Correlation Analysis (SMDCA), which explicitly considers the sample sequences based on similarity. The method constructs similarity-sequenced discriminant scatters for preserving the sequence structure of within-class samples and develops between-class correlations with the similarity-sequence structure information for further constraining intrinsic manifold structure of cross-view samples. SMDCA can also simultaneously extract low-dimensional sequence features with well-discriminative power from multiple views. Extensive experiments exhibit that SMDCA can provide higher recognition accuracy and stronger robustness in image recognition tasks.