Fine-grained Manner Research Articles

LMD: Cluster-Independent Multiscale Marker Identification in Single-cell RNA-seq Data.

Identifying accurate cell markers in single-cell RNA-seq data is crucial for understanding cellular diversity and function. Localized Marker Detector (LMD) is a novel tool to identify "localized genes" - genes exclusively expressed in groups of highly similar cells - thereby characterizing cellular diversity in a multi-resolution and fine-grained manner. LMD constructs a cell-cell affinity graph, diffuses the gene expression value across the cell graph, and assigns a score to each gene based on its diffusion dynamics. LMD's candidate markers can be grouped into functional gene modules, which accurately reflect cell types, subtypes, and other sources of variation such as cell cycle status. We apply LMD to mouse bone marrow and hair follicle dermal condensate datasets, where LMD facilitates cross-sample comparisons, identifying shared and sample-specific gene signatures and novel cell populations without requiring batch effect correction or integration methods. Furthermore, we assessed the performance of LMD across nine single-cell RNA sequencing datasets, compared it with six other methods aimed at achieving similar objectives, and found that LMD outperforms the other methods evaluated.

GAP: A group-based automatic pruning algorithm via convolution kernel fusion

In recent years, the deployment and operation of convolution neural networks on edge devices with limited computing capabilities have become increasingly challenging due to the large network structure and computational cost. Currently, the mainstream structured pruning algorithms mainly compress the network at the filter or layer level. However, these methods introduce too much human intervention with large granularities, which may lead to unpredictable performance after compression. In this paper, we propose a group-based automatic pruning algorithm(GAP) via kernel fusion to automatically search for the optimal pruning structure in a more fine-grained manner. Specifically, we first adopt a novel nonlinear dimensionality reduction clustering algorithm to divide the filters of each convolution layer into groups of equal size. Afterwards, we encode the mutual distribution similarity of the kernels within each group, and its KL divergence is employed as an importance indicator to determine the retained kernel groups through weighted fusion. Subsequently, we introduce an intelligent searching module that automatically explore and optimize the pruned structure of each layer. Finally, the pruned filters are permutated to form a dense group convolution and fine-tuned. Sufficient experiments show that, on two image classification datasets, for five advanced CNN models, our GAP algorithm outperforms most extant SOTA schemes, reduces artificial intervention, and enables efficient end-to-end training of compact models.

Revealing the exploitability of heap overflow through PoC analysis

The exploitable heap layouts are used to determine the exploitability of heap vulnerabilities in general-purpose applications. Prior studies have focused on using fuzzing-based methods to generate more exploitable heap layouts. However, the exploitable heap layout cannot fully demonstrate the exploitability of a vulnerability, as it is uncertain whether the attacker can control the data covered by the overflow. In this paper, we propose the Heap Overflow Exploitability Evaluator (Hoee), a new approach to automatically reveal the exploitability of heap buffer overflow vulnerabilities by evaluating proof-of-concepts (PoCs) generated by fuzzers. Hoee leverages several techniques to collect dynamic information at runtime and recover heap object layouts in a fine-grained manner. The overflow context is carefully analyzed to determine whether the sensitive pointer is corrupted, tainted, or critically used. We evaluate Hoee on 34 real-world CVE vulnerabilities from 16 general-purpose programs. The results demonstrate that Hoee accurately identifies the key factors for developing exploits in vulnerable contexts and correctly recognizes the behavior of overflow.

A micro Reinforcement Learning architecture for Intrusion Detection Systems

This paper proposes an Intrusion Detection System (IDS) that utilizes Deep Reinforcement Learning (DRL) in a fine-grained manner to enhance the performance of binary and multiclass intrusion classification tasks. The proposed system, named Micro Reinforcement Learning Classifier (MRLC), is evaluated using three standard datasets. MRLC architecture utilizes a fine-grained learning approach to enhance IDS accuracy. Simulation studies demonstrate that MRLC has a high efficiency in discriminating different intrusion classes, outperforming state-of-the-art RL-based methods. The average accuracy of MRLC is 99.56%, 99.99%, 99.01% for NSL-KDD, CIC-IDS2018, and UNSW-NB15 datasets respectively. The implementation codes are available at https://github.com/boshradarabi/MICRO-RL-IDS.

Risk identification of public opinion on social media: a new approach based on cross-spatial network analysis

PurposeThis study aims to use geographical information on social media for public opinion risk identification during a crisis.Design/methodology/approachThis study constructs a double-layer network that associates the online public opinion with geographical information. In the double-layer network, Gaussian process regression is used to train the prediction model for geographical locations. Second, cross-space information flow is described using local government data availability and regional internet development indicators. Finally, the structural characteristics and information flow of the double-layer network are explored to capture public opinion risks in a fine-grained manner. This study used the early stages of the COVID-19 outbreak for validation analyses, and it collected more than 90,000 pieces of public opinion data from microblogs.FindingsIn the early stages of the COVID-19 outbreak, the double-layer network exhibited a radiating state, and the information dissemination was more dependent on the nodes with higher in-degree. Moreover, the double-layer network structure showed geographical differences. The risk contagion was more significant in areas where information flow was prominent, but the influence of nodes was reduced.Originality/valuePublic opinion risk identification that incorporates geographical scenarios contributes to enhanced situational awareness. This study not only effectively extends geographical information on social media, but also provides valuable insights for accurately responding to public opinion.

Spatiotemporal gated traffic trajectory simulation with semantic-aware graph learning

Traffic trajectories of various vehicles, bicycles and pedestrians can help understand the traffic dynamics in a fine-grained manner like traffic flow, traffic congestion and ride-hailing demand. The comprehensive usage of traffic trajectory data has not been fully investigated due to the prevalent privacy concerns and commercial limitations. The traffic trajectory simulation task has emerged to generate high-fidelity trajectories in demand for downstream tasks to fill the gap between the scarce trajectory data and the widespread applications. Previous state-of-the-art methods build the spatiotemporal dependencies of trajectories with Graph Neural Networks (GNNs) under generative adversarial training, yielding better yet unstable trajectory quality. We observe that the unsatisfied synthetic trajectories are caused by the insufficient spatiotemporal modeling of road networks and trajectory semantics. In this paper, we propose a novel SpatioTEmporal GAted (STEGA) framework equipped with semantic-aware graph learning for traffic trajectory simulation to enable the explicit modeling of spatiotemporal dependencies throughout the learning pipeline. On the one hand, STEGA employs a graph encoder with the semantics of road networks for the spatial points of a trajectory, together with a time encoder for the time points. On the other hand, STEGA devises two spatiotemporal gates with the semantic graphs for the predictions of the future trajectory. Boosted by the semantic-aware graph learning, the proposed STEGA outperforms the state-of-the-art counterparts consistently at both macro- and micro-level metrics on two datasets. Elaborate ablation studies and downstream tasks of the synthetic trajectories further demonstrate the superiority of STEGA. Our code is available at https://github.com/Star607/STEGA.

SA²VP: Spatially Aligned-and-Adapted Visual Prompt

As a prominent parameter-efficient fine-tuning technique in NLP, prompt tuning is being explored its potential in computer vision. Typical methods for visual prompt tuning follow the sequential modeling paradigm stemming from NLP, which represents an input image as a flattened sequence of token embeddings and then learns a set of unordered parameterized tokens prefixed to the sequence representation as the visual prompts for task adaptation of large vision models. While such sequential modeling paradigm of visual prompt has shown great promise, there are two potential limitations. First, the learned visual prompts cannot model the underlying spatial relations in the input image, which is crucial for image encoding. Second, since all prompt tokens play the same role of prompting for all image tokens without distinction, it lacks the fine-grained prompting capability, i.e., individual prompting for different image tokens. In this work, we propose the Spatially Aligned-and-Adapted Visual Prompt model (SA^2VP), which learns a two-dimensional prompt token map with equal (or scaled) size to the image token map, thereby being able to spatially align with the image map. Each prompt token is designated to prompt knowledge only for the spatially corresponding image tokens. As a result, our model can conduct individual prompting for different image tokens in a fine-grained manner. Moreover, benefiting from the capability of preserving the spatial structure by the learned prompt token map, our SA^2VP is able to model the spatial relations in the input image, leading to more effective prompting. Extensive experiments on three challenging benchmarks for image classification demonstrate the superiority of our model over other state-of-the-art methods for visual prompt tuning. Code is available at https://github.com/tommy-xq/SA2VP.

Reusing Convolutional Neural Network Models through Modularization and Composition

With the widespread success of deep learning technologies, many trained deep neural network (DNN) models are now publicly available. However, directly reusing the public DNN models for new tasks often fails due to mismatching functionality or performance. Inspired by the notion of modularization and composition in software reuse, we investigate the possibility of improving the reusability of DNN models in a more fine-grained manner. Specifically, we propose two modularization approaches named CNNSplitter and GradSplitter, which can decompose a trained convolutional neural network (CNN) model for N -class classification into N small reusable modules. Each module recognizes one of the N classes and contains a part of the convolution kernels of the trained CNN model. Then, the resulting modules can be reused to patch existing CNN models or build new CNN models through composition. The main difference between CNNSplitter and GradSplitter lies in their search methods: the former relies on a genetic algorithm to explore search space, while the latter utilizes a gradient-based search method. Our experiments with three representative CNNs on three widely used public datasets demonstrate the effectiveness of the proposed approaches. Compared with CNNSplitter, GradSplitter incurs less accuracy loss, produces much smaller modules (19.88% fewer kernels), and achieves better results on patching weak models. In particular, experiments on GradSplitter show that (1) by patching weak models, the average improvement in terms of precision, recall, and F1-score is 17.13%, 4.95%, and 11.47%, respectively, and (2) for a new task, compared with the models trained from scratch, reusing modules achieves similar accuracy (the average loss of accuracy is only 2.46%) without a costly training process. Our approaches provide a viable solution to the rapid development and improvement of CNN models.

Channel semantic mutual learning for visible-thermal person re-identification.

Visible-infrared person re-identification (VI-ReID) is a cross-modality retrieval issue aiming to match the same pedestrian between visible and infrared cameras. Thus, the modality discrepancy presents a significant challenge for this task. Most methods employ different networks to extract features that are invariant between modalities. While we propose a novel channel semantic mutual learning network (CSMN), which attributes the difference in semantics between modalities to the difference at the channel level, it optimises the semantic consistency between channels from two perspectives: the local inter-channel semantics and the global inter-modal semantics. Meanwhile, we design a channel-level auto-guided double metric loss (CADM) to learn modality-invariant features and the sample distribution in a fine-grained manner. We conducted experiments on RegDB and SYSU-MM01, and the experimental results validate the superiority of CSMN. Especially on RegDB datasets, CSMN improves the current best performance by 3.43% and 0.5% on the Rank-1 score and mINP value, respectively. The code is available at https://github.com/013zyj/CSMN.

MTSC-Net: A Semi-Supervised Counting Network for Estimating the Number of Slash pine New Shoots.

The new shoot density of slash pine serves as a vital indicator for assessing its growth and photosynthetic capacity, while the number of new shoots offers an intuitive reflection of this density. With deep learning methods becoming increasingly popular, automated counting of new shoots has greatly improved in recent years but is still limited by tedious and expensive data collection and labeling. To resolve these issues, this paper proposes a semi-supervised counting network (MTSC-Net) for estimating the number of slash pine new shoots. First, based on the mean-teacher framework, we introduce the improved VGG19 to extract multiscale new shoot features. Second, to connect local new shoot feature information with global channel features, attention feature fusion module is introduced to achieve effective feature fusion. Finally, the new shoot density map and density probability distribution are processed in a fine-grained manner through multiscale dilated convolution of the regression head and classification head. In addition, a masked image modeling strategy is introduced to encourage the contextual understanding of global new shoot features and improve the counting performance. The experimental results show that MTSC-Net outperforms other semi-supervised counting models with labeled percentages ranging from 5% to 50%. When the labeled percentage is 5%, the mean absolute error and root mean square error are 17.71 and 25.49, respectively. These findings demonstrate that our work can be used as an efficient semi-supervised counting method to provide automated support for tree breeding and genetic utilization.

Eraser: Eliminating Performance Regression on Learned Query Optimizer

Efficient query optimization is crucial for database management systems. Recently, machine learning models have been applied in query optimizers to generate better plans, but the unpredictable performance regressions prevent them from being truly applicable. To be more specific, while a learned query optimizer commonly outperforms the traditional query optimizer on average for a workload of queries, its performance regression seems inevitable for some queries due to model under-fitting and difficulty in generalization. In this paper, we propose a system called Eraser to resolve this problem. Eraser aims at eliminating performance regressions while still attaining considerable overall performance improvement. To this end, Eraser applies a two-stage strategy to estimate the model accuracy for each candidate plan, and helps the learned query optimizer select more reliable plans. The first stage serves as a coarse-grained filter that removes all highly risky plans with feature values that are seen for the first time. The second stage clusters plans in a more fine-grained manner and evaluates each cluster according to the prediction quality of learned query optimizers for selecting the final execution plan. Eraser can be deployed as a plugin on top of any learned query optimizer. We implement Eraser and demonstrate its superiority on PostgreSQL and Spark. In our experiments, Eraser eliminates most of the regressions while bringing very little negative impact on the overall performance of learned query optimizers, no matter whether they perform better or worse than the traditional query optimizer. Meanwhile, it is adaptive to dynamic settings and generally applicable to different database systems.

Label-aware debiased causal reasoning for Natural Language Inference

Recently, researchers have argued that the impressive performance of Natural Language Inference (NLI) models is highly due to the spurious correlations existing in training data, which makes models vulnerable and poorly generalized. Some work has made preliminary debiased attempts by developing data-driven interventions or model-level debiased learning. Despite the progress, existing debiased methods either suffered from the high cost of data annotation processing, or required elaborate design to identify biased factors. By conducting detailed investigations and data analysis, we argue that label information can provide meaningful guidance to identify these spurious correlations in training data, which has not been paid enough attention. Thus, we design a novel Label-aware Debiased Causal Reasoning Network (LDCRN). Specifically, according to the data analysis, we first build a causal graph to describe causal relations and spurious correlations in NLI. Then, we employ an NLI model (e.g., RoBERTa) to calculate total causal effect of input sentences to labels. Meanwhile, we design a novel label-aware biased module to model spurious correlations and calculate their causal effect in a fine-grained manner. The debiasing process is realized by subtracting this causal effect from total causal effect. Finally, extensive experiments over two well-known NLI datasets and multiple human-annotated challenging test sets are conducted to prove the superiority of LDCRN. Moreover, we have developed novel challenging test sets based on MultiNLI to facilitate the community.

CONTROL CLOUD DATA ACESS PRIVELAGE AND ANONYMITY WITH FULLY ANONYMOUS ATTRIBUTE BASED ENCRYPTION

Cloud computing is a revolutionary computing paradigm, which enables flexible, ondemand, and low-cost usage of computing resources, but the data is outsourced to some cloud servers, and various privacy concerns emerge from it. Various schemes based on the attribute-based encryption have been proposed to secure the cloud storage. However, most work focuses on the data contents privacy and the access control, while less attention is paid to the privilege control and the identity privacy. In this paper, we present a semi-anonymous privilege control scheme AnonyControl to address not only the data privacy, but also the user identity privacy in existing access control schemes. AnonyControl decentralizes the central authority to limit the identity leakage and thus achieves semianonymity. Besides, it also generalizes the file access control to the privilege control, by which privileges of all operations on the cloud data can be managed in a fine-grained manner. Subsequently, we present the AnonyControl-F, which fully prevents the identity leakage and achieve the full anonymity. Our security analysis shows that both AnonyControl and AnonyControl-F are secure under the decisional bilinear Diffie–Hellman assumption, and our performance evaluation exhibits the feasibility of our schemes.

Adaptive Zone Learning for Weakly Supervised Object Localization.

Weakly supervised object localization (WSOL) stands as a pivotal endeavor within the realm of computer vision, entailing the location of objects utilizing merely image-level labels. Contemporary approaches in WSOL have leveraged FPMs, yielding commendable outcomes. However, these existing FPM-based techniques are predominantly confined to rudimentary strategies of either augmenting the foreground or diminishing the background presence. We argue for the exploration and exploitation of the intricate interplay between the object's foreground and its background to achieve efficient object localization. In this manuscript, we introduce an innovative framework, termed adaptive zone learning (AZL), which operates on a coarse-to-fine basis to refine FPMs through a triad of adaptive zone mechanisms. First, an adversarial learning mechanism (ALM) is employed, orchestrating an interplay between the foreground and background regions. This mechanism accentuates coarse-grained object regions in a mutually adversarial manner. Subsequently, an oriented learning mechanism (OLM) is unveiled, which harnesses local insights from both foreground and background in a fine-grained manner. This mechanism is instrumental in delineating object regions with greater granularity, thereby generating better FPMs. Furthermore, we propose a reinforced learning mechanism (RLM) as the compensatory mechanism for adversarial design, by which the undesirable foreground maps are refined again. Extensive experiments on CUB-200-2011 and ILSVRC datasets demonstrate that AZL achieves significant and consistent performance improvements over other state-of-the-art WSOL methods.

Audio-Video Analysis Method of Public Speaking Videos to Detect Deepfake Threat

Aim: The purpose of the article is to present the hypothesis that the use of discrepancies in audiovisual materials can significantly increase the effectiveness of detecting various types of deepfake and related threats. In order to verify this hypothesis, the authors proposed a new method that reveals inconsistencies in both multiple modalities simultaneously and within individual modalities separately, enabling them to effectively distinguish between authentic and altered public speaking videos. Project and methods: The proposed approach is to integrate audio and visual signals in a so-called fine-grained manner, and then carry out binary classification processes based on calculated adjustments to the classification results of each modality. The method has been tested using various network architectures, in particular Capsule networks – for deep anomaly detection and Swin Transformer – for image classification. Pre-processing included frame extraction and face detection using the MTCNN algorithm, as well as conversion of audio to mel spectrograms to better reflect human auditory perception. The proposed technique was tested on multimodal deepfake datasets, namely FakeAVCeleb and TMC, along with a custom dataset containing 4,700 recordings. The method has shown high performance in identifying deepfake threats in various test scenarios. Results: The method proposed by the authors achieved better AUC and accuracy compared to other reference methods, confirming its effectiveness in the analysis of multimodal artefacts. The test results confirm that it is effective in detecting modified videos in a variety of test scenarios which can be considered an advance over existing deepfake detection techniques. The results highlight the adaptability of the method in various architectures of feature extraction networks. Conclusions: The presented method of audiovisual deepfake detection uses fine inconsistencies of multimodal features to distinguish whether the material is authentic or synthetic. It is distinguished by its ability to point out inconsistencies in different types of deepfakes and, within each individual modality, can effectively distinguish authentic content from manipulated counterparts. The adaptability has been confirmed by the successful application of the method in various feature extraction network architectures. Moreover, its effectiveness has been proven in rigorous tests on two different audiovisual deepfake datasets. Keywords: analysis of audio-video stream, detection of deepfake threats, analysis of public speeches

Robust Posegraph Optimization Using Proximity Points

This paper proposes a robust posegraph optimization (PGO) method for posegraphs with keypoints. In the conventional PGO formulation, a loop constraint is defined between a pair of nodes, whereas in the proposed method, it is defined between a pair of keypoints. In this manner, robust PGO based on switch variables can be realized in a more fine-grained manner. Loop constraint is defined based on the unique geometric property of proximity point, and implemented as a new edge type of the g2o solver. The proposed method is compared with other robust PGO methods using real world data recorded in Nakanoshima Robot Challenge 2021.

The contribution of individual parameters to perceived iconicity and transparency in gesture-sign pairs

It is often assumed that gestures are more iconic than signs, as they do not have to conformto a linguistic system. This study introduces an expanded methodology to explore(a) the relative transparency and iconicity of silent gestures and signs, and (b) the iconicityof three individual parameters (handshape, location and movement). We elicitedmeaning guesses and iconicity ratings (both whole-item and for each parameter) fromsign-naive participants for both gestures and signs. Pilot data provide no evidence fordifferences in transparency and iconicity of gestures and signs, butwe do find interestingexamples of signs rated as more iconic than gestures. The iconicity of all three parametersis correlated with the iconicity of the whole item in both gestures and signs, butthere may be a role for iconic strategies and the saliency of individual parameters. Withthis method, we provide a novel, more fine-grained manner of investigating iconicity inthe manual modality.

GCRec: Graph-Augmented Capsule Network for Next-Item Recommendation.

Next-item recommendation has been a hot research, which aims at predicting the next action by modeling users' behavior sequences. While previous efforts toward this task have been made in capturing complex item transition patterns, we argue that they still suffer from three limitations: 1) they have difficulty in explicitly capturing the impact of inherent order of item transition patterns; 2) only a simple and crude embedding is insufficient to yield satisfactory long-term users' representations from limited training sequences; and 3) they are incapable of dynamically integrating long-term and short-term user interest modeling. In this work, we propose a novel solution named graph-augmented capsule network (GCRec), which exploits sequential user behaviors in a more fine-grained manner. Specifically, we employ a linear graph convolution module to learn informative long-term representations of users. Furthermore, we devise a user-specific capsule module and a position-aware gating module, which are sensitive to the relative sequential order of the recently interacted items, to capture sequential patterns at union-level and point-level. To aggregate the long-term and short-term user interests as a representative vector, we design a dual-gating mechanism, which could decide the contribution ratio of each module given different contextual information. Through extensive experiments on four benchmarks, we validate the rationality and effectiveness of GCRec on the next-item recommendation task.

GMAT-DU: Traffic Anomaly Prediction With Fine Spatiotemporal Granularity in Sparse Data

The fine-grained prediction of traffic anomalies is crucial for Traffic Management Bureau to alleviate congestion and avoid public safety incidents. While in practice, the fine-grained prediction is very challenging due to two issues. 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Data sparsity</i> . At the fine-grained setting, missing data is inevitable and widespread on spatial and temporal dimension. Existing methods have weak performance as they do not handle missing data properly. 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Data distribution mutation</i> . At the fine-grained setting, the traffic conditions of adjacent road segments are sometimes completely different, invalidating existing spatiotemporal smoothing-based methods. This paper proposes GMAT-DU, a novel model that aims to predict traffic anomaly from sparse data in fine-grained manner. To solve the first issue, we propose a Decay Unrolling (DU) mechanism to make the model applicable to sparse datasets. The performance will be progressively enhanced by the spatiotemporal unrolling of high-impact neighbors. For the second issue, we combine the meta-features of roads with correlations between roads, which are learnt from road semantic information and historical spatiotemporal data, and make the model focusing on the high-impact neighbors by a Graph Meta-features based ATtention (GMAT) mechanism. Extensive experiments on two real-world datasets validate the effectiveness of our method. The experiment results show the significant advantages against the state-of-the-art models.

Improving Computation and Memory Efficiency for Real-world Transformer Inference on GPUs

Transformer models have emerged as a leading approach in the field of natural language processing (NLP) and are increasingly being deployed in production environments. Graphic processing units (GPUs) have become a popular choice for the transformer deployment and often rely on the batch processing technique to ensure high hardware performance. Nonetheless, the current practice for transformer inference encounters computational and memory redundancy due to the heavy-tailed distribution of sequence lengths in NLP scenarios, resulting in low practical performance. In this article, we propose a unified solution for improving both computation and memory efficiency of the real-world transformer inference on GPUs. The solution eliminates the redundant computation and memory footprint across a transformer model. At first, a GPU-oriented computation approach is proposed to process the self-attention module in a fine-grained manner, eliminating its redundant computation. Next, the multi-layer perceptron module continues to use the word-accumulation approach to eliminate its redundant computation. Then, to better unify the fine-grained approach and the word-accumulation approach, it organizes the data layout of the self-attention module in block granularity. Since aforementioned approaches make the required memory size largely reduce and constantly fluctuate, we propose the chunk-based approach to enable a better balance between memory footprint and allocation/free efficiency. Our experimental results show that our unified solution achieves a decrease of average latency by 28% on the entire transformer model, 63.8% on the self-attention module, and reduces memory footprint of intermediate results by 7.8×, compared with prevailing frameworks.