Spatio-temporal Features Research Articles

Gear Classification in Skating Cross-Country Skiing Using Inertial Sensors and Deep Learning.

The aim of this current work is to identify three different gears of cross-country skiing utilizing embedded inertial measurement units and a suitable deep learning model. The cross-country style studied was the skating style during the uphill, which involved three different gears: symmetric gear pushing with poles on both sides (G3) and two asymmetric gears pushing with poles on the right side (G2R) or to the left side (G2L). To monitor the technique, inertial measurement units (IMUs) were affixed to the skis, recording acceleration and Euler angle data during the uphill tests performed by two experienced skiers using the gears under study. The initiation and termination points of the tests were controlled via Bluetooth by a smartphone using a custom application developed with Android Studio. Data were collected on the smartphone and stored on the SD memory cards included in each IMU. Convolutional neural networks combined with long short-term memory were utilized to classify and extract spatio-temporal features. The performance of the model in cross-user evaluations demonstrated an overall accuracy of 90%, and it achieved an accuracy of 98% in the cross-scene evaluations for individual users. These results indicate a promising performance of the developed system in distinguishing between different ski gears within skating styles, providing a valuable tool to enhance ski training and analysis.

Emotion Recognition Using EEG Signals and Audiovisual Features with Contrastive Learning.

Multimodal emotion recognition has emerged as a promising approach to capture the complex nature of human emotions by integrating information from various sources such as physiological signals, visual behavioral cues, and audio-visual content. However, current methods often struggle with effectively processing redundant or conflicting information across modalities and may overlook implicit inter-modal correlations. To address these challenges, this paper presents a novel multimodal emotion recognition framework which integrates audio-visual features with viewers' EEG data to enhance emotion classification accuracy. The proposed approach employs modality-specific encoders to extract spatiotemporal features, which are then aligned through contrastive learning to capture inter-modal relationships. Additionally, cross-modal attention mechanisms are incorporated for effective feature fusion across modalities. The framework, comprising pre-training, fine-tuning, and testing phases, is evaluated on multiple datasets of emotional responses. The experimental results demonstrate that the proposed multimodal approach, which combines audio-visual features with EEG data, is highly effective in recognizing emotions, highlighting its potential for advancing emotion recognition systems.

Fast Spatiotemporal Sequence Graph Convolutional Network-based transient flow prediction around different airfoils

A novel Spatiotemporal Sequence Graph Convolutional Network (ST-SGCN) data-driven model is proposed to predict transient fluid dynamics around airfoils using complex and unstructured flow field data, with the aim of reducing dimensions and expediting predictions. Graph Neural Networks directly interact with the flow field grid, capturing spatiotemporal physical features of grid nodes and their interconnections, while eliminating the need for complex preprocessing steps. The ST-SGCN model integrates a Graph Convolutional Network and a Graph Attention Network with a Deep Recurrent Neural Network that uses a Gate Recurrent Unit as the kernel, adeptly extracting spatial and temporal physical features of the flow field to accurately predict transient flow states. Preliminary airfoil flow experiments demonstrated the model's ability to continuously predict transient flow fields, achieving an average accuracy of 97% for both velocity and pressure field predictions, with a maximum error of approximately 10% in the testing dataset. Further experiments, varying angles of attack, airfoils, and Reynolds numbers, demonstrated the model's generalizability, extensibility, and adaptability, with prediction errors below 5% and a speedup of over 20 times.

Remote Gait Analysis Using Ultra-Wideband Radar Technology Based on Joint Range-Doppler-Time Representation.

In recent years, radar technology has been extensively utilized in contactless human behavior monitoring systems. The unique capabilities of ultra-wideband (UWB) radars compared to conventional radar technologies, due to time-of-flight measurements, present new untapped opportunities for in-depth monitoring of human movement during overground locomotion. This study aims to investigate the deployability of UWB radars in accurately capturing the gait patterns of healthy individuals with no known walking impairments. A novel algorithm was developed that can extract ten clinical spatiotemporal gait features using the Doppler information captured from three monostatic UWB radar sensors during a 6-meter walking task. Key gait events are detected from lower-extremity movements based on the joint range-Doppler-time representation of recorded radar data. The estimated gait parameters were validated against a gold-standard optical motion tracking system using 12 healthy volunteers. On average, nine gait parameters can be consistently estimated with 90-98% accuracy, while capturing 94.5% of participants' gait variability and 90.8% of inter-limb symmetry. Correlation and Bland-Altman analysis revealed a strong correlation between radar-based parameters and the ground-truth values, with average discrepancies consistently close to 0. Results prove that radar sensing can provide accurate biomarkers to supplement clinical human gait analysis, with quality similar to gold standard assessment. Radars can potentially allow a transition from expensive and cumbersome lab-based gait analysis tools toward a completely unobtrusive and affordable solution for in-home deployment, enabling continuous long-term monitoring of individuals for research and healthcare applications.

Revealing various change characteristics and drivers of ecological vulnerability in the mountains of southwest China

Ecological vulnerability (EV) assessment is an important means of reflecting the evolutionary process of the local ecological environment, which is crucial for ecological security. The current EV assessment still faces challenges in comprehensively revealing its spatio-temporal change characteristics due to diversity, multi-scale complexity and non-linearity of ecosystems. In this study, we comprehensively revealed the spatio-temporal change features and driver mechanisms of EV in Yunnan Province (YP) using methods such as breaks for additive seasonal and trend (BFAST) and Geodetector. According to the findings, (1) the ecological vulnerability index (EVI) in YP decreased by 0.0265 on the raster scale between 2000 and 2018, and the ecological vulnerability level (EVL) was mainly dominated by level III. At the city scale, EVL in YP was dominated by level IV. At the basin scale, the average EV of the Jinsha River basin was much greater other basins. (2) The linear trend of EV in the YP mainly showed an insignificant decrease, mostly concentrated in the YP’s eastern region. The non-linear trend mainly showed a monotonous decrease, with the mutation time concentrated in 2009. The future persistence trend of EV under different coupling modes was dominated by anti-persistence decrease (Sen-MK + Hurst) and anti-persistence monotonous decrease (BFAST + Hurst), with an area percentage of 40.19 % and 25.13 %, respectively. (3) Gross primary productivity was the priority factor influencing YP’s EV (q = 0.4137). This study not only enriches the cases of EV assessment studies in the high mountain valley area but also bridges the gap of analysing the multi-spatial scale characteristics and change trends of regional EV.

Moving ships detection via the trajectory feature extraction from spatiotemporal slices of infrared maritime videos

In practical application scenarios, maritime infrared videosoften contain different types of sea state scenes, weather conditions, shooting time, shooting distance, etc. In these different types of maritime videos, ship targets have great differences in size, grayscaledistribution and contrast, which brings difficulties to ship target detection.At the same time, the diversity of fluctuation, grayscale distribution and reflected light of the sea surface will bring unpredictable interference noise to ship target detection.How to accurately detect ship targets in complex and changeable maritime infrared videos is a challenging task and research focus.The key to achieving accurate ship detection is to extract robust target features that can effectively distinguish targets from all thebackground noises.In this paper, a novel infrared video ship target detection algorithm based on spatiotemporal slice target trajectory features extraction is proposed. The algorithm is very sensitive to real targets and has excellentanti-noise ability.The main innovation of the algorithm is to extract the target trajectory feature from the spatiotemporal slice of the sequence image and generate the trajectory feature map.We use the trajectory texture formed by the ship target in the spatiotemporal slice to extract the target feature, which can greatly suppress the background noise.The adaptive dilation linear model algorithm can effectively detect the target trajectory line in the spatiotemporal slice. In addition, we also make full use of the gradient of the target trajectory line to distinguish different target trajectory pixels, and propose an adaptive iterative dilation target region localization algorithm combined with gradient consistency.For object segmentation, we calculate the segmentation double-threshold using the adjacent surrounding background pixels of the target, so as to achievetarget segmentationof multiple grayscaledistribution types. Finally, in the comparison experiment, our algorithm shows superior target detection performance, especially when detecting ships from a large number of highlighted sea clutter background, the robust anti-noise ability of the algorithm can be highlighted.

AONet: Attention network with optional activation for unsupervised video anomaly detection

AbstractAnomaly detection in video surveillance is crucial but challenging due to the rarity of irregular events and ambiguity of defining anomalies. We propose a method called AONet that utilizes a spatiotemporal module to extract spatiotemporal features efficiently, as well as a residual autoencoder equipped with an attention network for effective future frame prediction in video anomaly detection. AONet utilizes a novel activation function called OptAF that combines the strengths of the ReLU, leaky ReLU, and sigmoid functions. Furthermore, the proposed method employs a combination of robust loss functions to address various aspects of prediction errors and enhance training effectiveness. The performance of the proposed method is evaluated on three widely used benchmark datasets. The results indicate that the proposed method outperforms existing state‐of‐the‐art methods and demonstrates comparable performance, achieving area under the curve values of 97.0%, 86.9%, and 73.8% on the UCSD Ped2, CUHK Avenue, and ShanghaiTech Campus datasets, respectively. Additionally, the high speed of the proposed method enables its application to real‐time tasks.

Mechanistic investigations into the removal of Hg0 over Pt-based sorbents by warm syngas cleanup

Commercial sorbents for Hg0 removal from syngas operate at low temperatures (<120 ºC), which impacts the thermal efficiency of integrated gasification and chemical production processes. For the first time, supported Pt-based sorbents with different supports, viz., Pt/SBA-15, Pt/S-40, Pt/ZrO2, Pt/CeO2, and Pt/TiO2 are synthesized for high-temperature (≥200 ºC) Hg0 removal. Pt/SBA-15 shows the highest cumulative Hg0 removal efficiency (>99%) at 270 ºC in N2, which is retained for much higher durations (>20h) compared to the best-reported Pd-based and metal oxide sorbents. Besides amalgam formation, Hg0 removal on Pt/ZrO2 and Pt/CeO2 occurs via oxidation at high temperatures (> 270 ºC), which explains the reduced negative effect of high temperature over these sorbents. In contrast to other supports, the removal efficiency over Pt/SBA-15 is above 99% even in the presence of H2, H2O, and CO2, and is attributed to the hydrophobic mesoporous silica support. DRIFTS studies indicate CO adsorption on the active Pt sites over all sorbents. Based on the inferences obtained, a kinetic model is developed and integrated into a reactor-scale model which effectively simulates the spatio-temporal features of Hg0 sorption over Pt/SBA-15.

Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction.

Human motion prediction is challenging due to the complex spatiotemporal feature modeling. Among all methods, graph convolution networks (GCNs) are extensively utilized because of their superiority in explicit connection modeling. Within a GCN, the graph correlation adjacency matrix drives feature aggregation, and thus, is the key to extracting predictive motion features. State-of-the-art methods decompose the spatiotemporal correlation into spatial correlations for each frame and temporal correlations for each joint. Directly parameterizing these correlations introduces redundant parameters to represent common relations shared by all frames and all joints. Besides, the spatiotemporal graph adjacency matrix is the same for different motion samples, and thus, cannot reflect samplewise correspondence variances. To overcome these two bottlenecks, we propose dynamic spatiotemporal decompose GC (DSTD-GC), which only takes 28.6% parameters of the state-of-the-art GC. The key of DSTD-GC is constrained dynamic correlation modeling, which explicitly parameterizes the common static constraints as a spatial/temporal vanilla adjacency matrix shared by all frames/joints and dynamically extracts correspondence variances for each frame/joint with an adjustment modeling function. For each sample, the common constrained adjacency matrices are fixed to represent generic motion patterns, while the extracted variances complete the matrices with specific pattern adjustments. Meanwhile, we mathematically reformulate GCs on spatiotemporal graphs into a unified form and find that DSTD-GC relaxes certain constraints of other GC, which contributes to a better representation capability. Moreover, by combining DSTD-GC with prior knowledge like body connection and temporal context, we propose a powerful spatiotemporal GCN called DSTD-GCN. On the Human3.6M, Carnegie Mellon University (CMU) Mocap, and 3D Poses in the Wild (3DPW) datasets, DSTD-GCN outperforms state-of-the-art methods by 3.9%-8.7% in prediction accuracy with 55.0%-96.9% fewer parameters. Codes are available at https://github.com/Jaakk0F/DSTD-GCN.

A coupled multi-factor synergistic graph network model for traffic prediction

Abstract Since only the spatio-temporal structure itself is considered in the current traffic prediction process, and there is a lack of multi-factor information about the scene, which leads to problems such as lack of universality and authenticity in the spatio-temporal scene, we proposed a coupled multi-factor synergistic graph network (CM-SGN). For this network, this paper firstly adopts a fusion mechanism to fuse traffic features with coupled scene features, constructs a complex feature correlation matrix, and enhances the multi-feature fusion effect of spatio-temporal scenes; secondly, it introduces a spatio-temporal feature network constrained by an attention mechanism to ensure the spatio-temporal stability of the fused features; and lastly, it reduces the sensitivity of the scene-traffic interdependent targets by using the regression layer combinations of the loss function to improve the prediction generalization capability. Experiments were conducted on two real datasets. Experiments were conducted on two real datasets, and the results show that the model proposed in this paper meets the complex spatio-temporal scenarios in traffic prediction, effectively captures the intrinsic spatio-temporal dependencies, and has good stability and generalization ability.

Spatio-temporal features of ionospheric disturbances resulting from March 2023 geomagnetic storm: Comparisons with March 2015 St. Patrick’s Day storm

This study explores the ionospheric disturbances induced by the March 2023 geomagnetic storm, offering insights into the complex interplay between space weather events and the Earth’s upper atmosphere. In this regard, data from ionospheric maps (global and regional electron contents) and topside plasma density (provided by the Swarm satellites) have been used. Furthermore, the findings are compared with those of the March 2015 St. Patrick’s Day storm of solar cycle 24, which exhibited notably similar onset conditions. The Global Electron Content (GEC) displays substantial positive surges in the African, Pacific, and American sectors, with a notable enhancement in the American sector on March 24, 2023. During the recovery phase (March-23 storm), negative storm effects are observed across all longitudinal sectors, with greater intensity at low-latitudes compared to mid-latitudes. Moreover, the study highlights discrepancies in positive storm effects when compared to the St. Patrick’s Day storm. During the March-2023, there was no positive storm effect observed in the pacific mid-latitude regions. This longitudinal difference in occurrence of positive storm may be attributed to potential influences from variations in the z-component of the interplanetary magnetic field and energy inputs into the magnetosphere. A super fountain effect is observed exclusively in the American sectors during both storms, exhibiting a noticeable hemispheric asymmetry. The non-uniform planetary distribution of disturbed thermospheric winds likely played a major role in the ionospheric asymmetry in the American region during the 2023 event.

AMTrack:Transformer tracking via action information and mix-frequency features

Nowadays, Transformer-based visual tracking algorithms have been developing quickly because of the self-attention mechanism of Transformer, which has the capability to model global information. Although the self-attention mechanism in Transformer can effectively capture long-range dependencies in feature space, they only use flattened two-dimensional features and are unable to capture long-range temporal dependencies. Furthermore, since the self-attention in Transformer functions as a low-pass filter, it picks up on low-frequency features of the target while ignoring high-frequency features. This research suggests a Transformer tracker based on action information and mix-frequency features (AMTrack) to address these problems. Specifically, to address the lack of temporal remote dependencies, we introduce the target action aware module and the target action offset module. The target action aware module sets up several pathways to extract spatio-temporal, channel, and motion feature independently. In contrast, the target action offset module computes the target’s offset information by computing relative feature maps. Furthermore, in order to address the imbalance between high and low frequency features, we propose a mix-frequency attention and multi-frequency self-attention convolutional block. The mix-frequency attention uses high-frequency features within partitioned local windows as input for the high-frequency branch and average-pooled low-frequency features as the input for the low-frequency branch, calculating attention scores in both branches respectively. The multi-frequency self-attention convolutional block uses self-attention to capture low-frequency features and convolution to capture high-frequency features. Extensive experiments are carried out on eight challenging tracking datasets (e.g., OTB100 (Object Tracking Benchmark 100), NFS (Need For Speed), UAV123 (Unmanned Aerial Vehicles 123), TC128 (Temple Color 128), VOT2018 (Visual Object Tracking 2018), LaSOT (Large-scale Single Object Tracking), TrackingNet (Tracking Network), GOT-10k (Generic Object Tracking-10k)), and the experimental results show that our tracker achieves excellent tracking performance when compared with several state-of-the-art tracking algorithms. The experimental results show that on LaSOT, the success rates AUC (Area Under Curve), PNorm, and P reach 65.8%, 69.2%, and 68.0%, respectively, where the AUC value is 2.1% higher than the baseline algorithm TrDiMP (Transformer Discriminative Model Prediction). On other datasets, our tracker also achieves excellent tracking performance.

STFM: Accurate Spatio-Temporal Fusion Model for Weather Forecasting

Meteorological prediction is crucial for various sectors, including agriculture, navigation, daily life, disaster prevention, and scientific research. However, traditional numerical weather prediction (NWP) models are constrained by their high computational resource requirements, while the accuracy of deep learning models remains suboptimal. In response to these challenges, we propose a novel deep learning-based model, the Spatiotemporal Fusion Model (STFM), designed to enhance the accuracy of meteorological predictions. Our model leverages Fifth-Generation ECMWF Reanalysis (ERA5) data and introduces two key components: a spatiotemporal encoder module and a spatiotemporal fusion module. The spatiotemporal encoder integrates the strengths of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), effectively capturing both spatial and temporal dependencies. Meanwhile, the spatiotemporal fusion module employs a dual attention mechanism, decomposing spatial attention into global static attention and channel dynamic attention. This approach ensures comprehensive extraction of spatial features from meteorological data. The combination of these modules significantly improves prediction performance. Experimental results demonstrate that STFM excels in extracting spatiotemporal features from reanalysis data, yielding predictions that closely align with observed values. In comparative studies, STFM outperformed other models, achieving a 7% improvement in ground and high-altitude temperature predictions, a 5% enhancement in the prediction of the u/v components of 10 m wind speed, and an increase in the accuracy of potential height and relative humidity predictions by 3% and 1%, respectively. This enhanced performance highlights STFM’s potential to advance the accuracy and reliability of meteorological forecasting.

A deep learning approach for deriving wheat phenology from near-surface RGB image series using spatiotemporal fusion

Accurate monitoring of wheat phenological stages is essential for effective crop management and informed agricultural decision-making. Traditional methods often rely on labour-intensive field surveys, which are prone to subjective bias and limited temporal resolution. To address these challenges, this study explores the potential of near-surface cameras combined with an advanced deep-learning approach to derive wheat phenological stages from high-quality, real-time RGB image series. Three deep learning models based on three different spatiotemporal feature fusion methods, namely sequential fusion, synchronous fusion, and parallel fusion, were constructed and evaluated for deriving wheat phenological stages with these near-surface RGB image series. Moreover, the impact of different image resolutions, capture perspectives, and model training strategies on the performance of deep learning models was also investigated. The results indicate that the model using the sequential fusion method is optimal, with an overall accuracy (OA) of 0.935, a mean absolute error (MAE) of 0.069, F1-score (F1) of 0.936, and kappa coefficients (Kappa) of 0.924 in wheat phenological stages. Besides, the enhanced image resolution of 512 × 512 pixels and a suitable image capture perspective, specifically a sensor viewing angle of 40° to 60° vertically, introduce more effective features for phenological stage detection, thereby enhancing the model’s accuracy. Furthermore, concerning the model training, applying a two-step fine-tuning strategy will also enhance the model’s robustness to random variations in perspective. This research introduces an innovative approach for real-time phenological stage detection and provides a solid foundation for precision agriculture. By accurately deriving critical phenological stages, the methodology developed in this study supports the optimization of crop management practices, which may result in improved resource efficiency and sustainability across diverse agricultural settings. The implications of this work extend beyond wheat, offering a scalable solution that can be adapted to monitor other crops, thereby contributing to more efficient and sustainable agricultural systems.

Maintained volitional activation of the muscle alters the cortical processing of proprioceptive afference from the ankle joint

Cortical proprioceptive processing of intermittent, passive movements can be assessed by extracting evoked and induced electroencephalographic (EEG) responses to somatosensory stimuli. Although the existent prior research on somatosensory stimulations, it remains unknown to what extent ongoing volitional muscle activation modulates the proprioceptive cortical processing of passive ankle-joint rotations.Twenty-five healthy volunteers (28.8 ± 7 yr, 14 males) underwent a total of 100 right ankle-joint passive rotations (4° dorsiflexions, 4 ± 0.25 s inter-stimulus interval, 30°/s peak angular velocity) evoked by a movement actuator during passive condition with relaxed ankle and active condition with a constant plantarflexion torque of 5 ± 2.5 Nm. Simultaneously, EEG, electromyographic (EMG) and kinematic signals were collected. Spatiotemporal features of evoked and induced EEG responses to the stimuli were extracted to estimate the modulation of the cortical proprioceptive processing between the active and passive conditions.Proprioceptive stimuli during the active condition elicited robustly ∼26 % larger evoked response and ∼38 % larger beta suppression amplitudes, but ∼42 % weaker beta rebound amplitude over the primary sensorimotor cortex than the passive condition, with no differences in terms of response latencies.These findings indicate that the active volitional motor task during naturalistic proprioceptive stimulation of the ankle joint enhances related cortical activation and reduces related cortical inhibition with respect to the passive condition. Possible factors explaining these results include mechanisms occurring at several levels of the proprioceptive processing from the peripheral muscle (i.e. mechanical, muscle spindle status, etc.) to the different central (i.e. spinal, sub-cortical and cortical) levels.

UAV Anomaly Detection Method Based on Convolutional Autoencoder and Support Vector Data Description with 0/1 Soft-Margin Loss

Unmanned aerial vehicles (UAVs) are becoming more widely used in various industries, raising growing concerns about their safety and reliability. The flight data of UAVs can directly reflect their flight health status; however, the rarity of abnormal flight data and the spatiotemporal characteristics of these data represent a significant challenge for constructing accurate and reliable anomaly detectors. To address this, this study proposes an anomaly detection framework that fully considers the temporal correlations and distribution characteristics of flight data. This framework first combines a one-dimensional convolutional neural network (1DCNN) with an autoencoder (AE) to establish a feature extraction model. This model leverages the feature extraction capabilities of the 1DCNN and the reconstruction capabilities of the AE to thoroughly extract the spatiotemporal features from UAV flight data. Then, to address the challenge of adaptive anomaly detection thresholds, this research proposes a nonlinear model of support vector data description (SVDD) utilizing a 0/1 soft-margin loss, referred to as L0/1-SVDD. This model replaces the traditional hinge loss function in SVDD with a 0/1 loss function, with the goal of enhancing the accuracy and robustness of anomaly detection. Since the 0/1 loss function is a bounded, non-convex, and non-continuous function, this paper proposes the Bregman ADMM algorithm to solve the L0/1-SVDD. Finally, the difference between the reconstructed and the actual value is employed to train the L0/1-SVDD, resulting in a hypersphere classifier that is capable of detecting UAV anomaly data. The experimental results using real flight data show that, compared with methods such as AE, LSTM, and LSTM-AE, the proposed method exhibits superior performance across five evaluation metrics.

Bridging spatiotemporal feature gap for video salient object detection

The mutual transfer of spatiotemporal features is the main challenge for the two-stream video salient object detection. Current methods adopt the spatiotemporal feature interaction to achieve it. However, these methods still have two issues: modal feature gap and layer feature gap. To address these, we propose a Bridging Spatiotemporal feature Gap Network (BSGNet) with a global correspondence interaction and gate filtering (GCGF) module, a global-local distribution consistency (GLDC) module, and a modality-layer feature fusion framework (MLFF). Compared with previous works, BSGNet not only explores more effective interaction by GCGF, but also bridges modality and layer feature gaps by GLDC and MLFF. Firstly, GCGF achieves the spatiotemporal feature interaction by modeling intra-modal and inter-modal global correspondences. Besides, GCGF employs a gate mechanism to control the proportion of message transfer between appearance and motion information, which characterizes the contribution provided by spatiotemporal features. Secondly, at both global and local levels, GLDC pushes the spatiotemporal feature distribution between same scenes, and pulls the spatiotemporal feature distribution between different scenes. This can enhance the distribution consistency to align spatiotemporal features and bridge modal feature gap. Finally, MLFF designs an inter-modal and inter-layer feature fusion framework to bridge the layer feature gap brought by the different modalities and different receptive fields. Extensive experiments on five benchmarks reveal that our BSGNet outperforms state-of-the-arts.

Spatio-temporal fusion with motion masks for the moving small target detection from remote-sensing videos

Moving small target detection is an eye-catching research topic, full of great challenges in object detection field. Its primary difficulties exist in (i) the very small size proportion of the target region in background images, (ii) the quite weak contrast to background and (iii) the perception of slight target motion. Currently-existing detection methods mainly utilize the image features from spatial domain. This kind of features is quite weak to small targets. More new feature kinds need to be concerned. In view of these, besides traditional image clues, motion features are currently attracting more and more attention in small target detection. In order to promote detection performance, this paper proposes a Spatio-Temporal Fusion Network (STFNet) with two parallel feature extraction branches for detecting moving small targets. In this network, one branch is designed to capture the traditional semantic features from spatial domain, and the other is devised to perceive the slight target motion features hidden in temporal domain. Meanwhile, to optimize the extraction of spatio-temporal features, a group of motion masks is specially designed to guide feature extractors to pay more precise attention to the positions where small targets appear. Moreover, we design a new bridging module to enhance the cross-domain and cross-scale fusion of spatio-temporal features. Through extensive comparison and ablation experiments on seven sub-datasets, it is proved that our new STFNet is effective and it is obviously superior to the compared ones in detecting moving small targets from satellite sequence images. It achieves a Precision of 0.93, an mAP50 of 71.10% and an F1 of 0.84, evidently higher than existing state-of-the-art methods. Our codes are available at https://github.com/UESTC-nnLab/STF.

Depression detection based on the temporal-spatial-frequency feature fusion of EEG

Depression is a prevalent affective psychiatric disorder projected to be the leading contributor to the world’s disease burden by 2030. Due to its high prevalence and low recognition rate, an objective and effective detection method is urgently needed. Deep learning methods based on electroencephalography (EEG) have shown significant potential in depression detection. However, excessive channels can increase redundancy and computational complexity in EEG, while irrelevant channels may reduce accuracy. Additionally, existing models often overlook the complementarity between the temporal-, spatial-, and frequency-domain features of EEG, limiting their detection capabilities. To address these issues, we propose a method that fuses the temporal, spatial, and frequency domain features of EEG to enhance the detection accuracy while eliminating redundant channels. We introduce an EEG channel selection method based on frequency domain weighting that automatically adjusts the channel weights to select the EEG channels that best capture spatial information across the delta, theta, alpha, beta, and gamma bands, thereby optimizing the extraction of spatial-frequency features. In addition, we designed a multiscale spatiotemporal convolutional attention network to extract the spatiotemporal features of EEG. In this network, the multiscale convolutional attention module enhanced the model’s ability to capture spatial features, whereas the temporal trend-aware self-attention module extracted long-term temporal features by analyzing global correlations across different time points. Experimental results on the MODMA dataset show that our method achieved a 97.24% detection accuracy, surpassing current state-of-the-art models. This study offers a novel approach for constructing depression detection models, providing a foundation for future research and application.

Unveiling the power of Haar frequency domain: Advancing small target motion detection in dim light

Visual small target motion detection finds successful applications in varied scenarios. However, dim-light conditions, such as the tunnel scenes and nighttime environments, present significant challenges to existing detection methods which mainly operate within the spatiotemporal domain. This is because the transmission of small target motion information suffers from the inevitable interference of image noise caused by dim light in the spatiotemporal domain, resulting in the detriment of extracting essential spatiotemporal features of the target motion. Given the significant obstacles posed by dim-light imaging to small target motion detection within the spatiotemporal domain, the exploration of an alternative observation domain for small target motion, alongside the development of a corresponding detection method, emerges as a viable solution. To address this, in this paper, we discovered the remarkable potential of the Haar frequency domain in characterizing the small target motion in dim light. To investigate the advantages of integrating Haar frequency processing in small target motion detection, we introduce a Haar-windowed summation mechanism into an existing bio-inspired small target motion detection model. The proposed mechanism integrates visual information in spatiotemporal windows regulated by frequency parameters of Haar wavelets and effectively discriminates the small target motion from the disturbance of random noise caused by dim light. Theoretical analysis and numerical experiments confirm the superior performance of integrating the Haar frequency processing. This study provides a new vision of small target motion detection through the lens of the frequency domain and extends the limits of existing bio-inspired models for practical applications in dim light.