Query Items Research Articles

Local–global normality learning and discrepancy normalizing flow for unsupervised image anomaly detection

The unsupervised detection and localization of image anomalies hold significant importance across various domains, particularly in industrial quality inspection. Despite its widespread utilization, this task remains inherently challenging due to its reliance solely on defect-free normal knowledge. This paper presents the local–global normality learning and discrepancy normalizing flow, a new state-of-the-art model for unsupervised image anomaly detection and localization. In contrast to existing methods, It adopts a two-stream approach that considers both local and global semantics, ensuring stable detection of abnormalities. The framework comprises two key components: the dual-branch Transformer and the discrepancy normalizing flow, facilitating reconstruction and discrimination. The proposed framework leverages pre-trained convolutional neural networks to extract multi-scale feature embeddings, followed by a novel dual-branch transformer that achieves feature reconstruction from local and global perspectives. The local reconstruction employs self-attention, while the global reconstruction incorporates global prototype tokens and semantic query tokens by the aggregation-cross attention mechanism. Moreover, discrepancy normalizing flow is developed to estimate the likelihood of anomalies based on the discrepancy between pre-trained features and local/global reconstruction results. Extensive validation on established public benchmarks confirms that our method achieves state-of-the-art performance with the proposed local–global reconstruction and discrimination dual-stream framework.

VerFormer: Vertebrae-Aware Transformer for Automatic Spine Segmentation from CT Images.

The accurate and efficient segmentation of the spine is important in the diagnosis and treatment of spine malfunctions and fractures. However, it is still challenging because of large inter-vertebra variations in shape and cross-image localization of the spine. In previous methods, convolutional neural networks (CNNs) have been widely applied as a vision backbone to tackle this task. However, these methods are challenged in utilizing the global contextual information across the whole image for accurate spine segmentation because of the inherent locality of the convolution operation. Compared with CNNs, the Vision Transformer (ViT) has been proposed as another vision backbone with a high capacity to capture global contextual information. However, when the ViT is employed for spine segmentation, it treats all input tokens equally, including vertebrae-related tokens and non-vertebrae-related tokens. Additionally, it lacks the capability to locate regions of interest, thus lowering the accuracy of spine segmentation. To address this limitation, we propose a novel Vertebrae-aware Vision Transformer (VerFormer) for automatic spine segmentation from CT images. Our VerFormer is designed by incorporating a novel Vertebrae-aware Global (VG) block into the ViT backbone. In the VG block, the vertebrae-related global contextual information is extracted by a Vertebrae-aware Global Query (VGQ) module. Then, this information is incorporated into query tokens to highlight vertebrae-related tokens in the multi-head self-attention module. Thus, this VG block can leverage global contextual information to effectively and efficiently locate spines across the whole input, thus improving the segmentation accuracy of VerFormer. Driven by this design, the VerFormer demonstrates a solid capacity to capture more discriminative dependencies and vertebrae-related context in automatic spine segmentation. The experimental results on two spine CT segmentation tasks demonstrate the effectiveness of our VG block and the superiority of our VerFormer in spine segmentation. Compared with other popular CNN- or ViT-based segmentation models, our VerFormer shows superior segmentation accuracy and generalization.

Towards forward secure verifiable data streaming with support for keyword query

Verifiable data streaming (VDS) allows users to continuously upload their encrypted data items to untrusted cloud servers to reduce the burden of local storage. Meanwhile, it enables users to update outsourced data and initiate queries to verify data integrity. However, most previous works focus on queries for a certain index and are not compatible with general keyword queries. To this end, we propose a VDS protocol that supports keyword queries. Specifically, we first present an efficient dynamic symmetric searchable encryption (DSSE) with range query, which can represent multiple queried keywords within a range as the concise query token, thereby achieving communication-optimized keyword queries while achieving unbounded data appending. Furthermore, we construct a new VDS protocol supporting keyword queries by integrating the proposed range DSSE. Specifically, suppose the user searches for multiple keywords within a certain range. In that case, cloud servers can match the corresponding data items based on the user's query token, and users can verify the integrity of these data items. Security analysis demonstrates that our VDS protocol achieves forward security. Experimental results show that it sacrifices acceptable costs in exchange for keyword retrieval capability.

MGCoT: Multi-Grained Contextual Transformer for table-based text generation

Recent advances in Transformer have led to the revolution of table-based text generation. However, most existing Transformer-based architectures ignore the rich contexts among input tokens distributed in multi-level units (e.g., cell, row, or column), leading to sometimes unfaithful text generation that fails to establish accurate association relationships and misses vital information. In this paper, we propose Multi-Grained Contextual Transformer (MGCoT), a novel architecture that fully capitalizes on the multi-grained contexts among input tokens and thus strengthens the capacity of table-based text generation. The key primitive, Multi-Grained Contexts (MGCo) module, involves two components: a local context sub-module that adaptively gathers neighboring tokens to form the token-wise local context features, and a global context sub-module that consistently aggregates tokens from a broader range to form the shared global context feature. The former aims at modeling the short-range dependencies that reflect the salience of tokens within similar fine-grained units (e.g., cell and row) attending to the query token, while the latter aims at capturing the long-range dependencies that reflect the significance of each token within similar coarse-grained units (e.g., multiple rows or columns). Based on the fused multi-grained contexts, MGCoT can flexibly and holistically model the content of a table across multi-level structures. On three benchmark datasets, ToTTo, FeTaQA, and Tablesum, MGCoT outperforms strong baselines by a large margin on the quality of the generated texts, demonstrating the effectiveness of multi-grained context modeling. Our source codes are available at https://github.com/Cedric-Mo/MGCoT.

MGQFormer: Mask-Guided Query-Based Transformer for Image Manipulation Localization

Deep learning-based models have made great progress in image tampering localization, which aims to distinguish between manipulated and authentic regions. However, these models suffer from inefficient training. This is because they use ground-truth mask labels mainly through the cross-entropy loss, which prioritizes per-pixel precision but disregards the spatial location and shape details of manipulated regions. To address this problem, we propose a Mask-Guided Query-based Transformer Framework (MGQFormer), which uses ground-truth masks to guide the learnable query token (LQT) in identifying the forged regions. Specifically, we extract feature embeddings of ground-truth masks as the guiding query token (GQT) and feed GQT and LQT into MGQFormer to estimate fake regions, respectively. Then we make MGQFormer learn the position and shape information in ground-truth mask labels by proposing a mask-guided loss to reduce the feature distance between GQT and LQT. We also observe that such mask-guided training strategy has a significant impact on the convergence speed of MGQFormer training. Extensive experiments on multiple benchmarks show that our method significantly improves over state-of-the-art methods.

Let All Be Whitened: Multi-Teacher Distillation for Efficient Visual Retrieval

Visual retrieval aims to search for the most relevant visual items, e.g., images and videos, from a candidate gallery with a given query item. Accuracy and efficiency are two competing objectives in retrieval tasks. Instead of crafting a new method pursuing further improvement on accuracy, in this paper we propose a multi-teacher distillation framework Whiten-MTD, which is able to transfer knowledge from off-the-shelf pre-trained retrieval models to a lightweight student model for efficient visual retrieval. Furthermore, we discover that the similarities obtained by different retrieval models are diversified and incommensurable, which makes it challenging to jointly distill knowledge from multiple models. Therefore, we propose to whiten the output of teacher models before fusion, which enables effective multi-teacher distillation for retrieval models. Whiten-MTD is conceptually simple and practically effective. Extensive experiments on two landmark image retrieval datasets and one video retrieval dataset demonstrate the effectiveness of our proposed method, and its good balance of retrieval performance and efficiency. Our source code is released at https://github.com/Maryeon/whiten_mtd.

TCNet: Continuous Sign Language Recognition from Trajectories and Correlated Regions

A key challenge in continuous sign language recognition (CSLR) is to efficiently capture long-range spatial interactions over time from the video input. To address this challenge, we propose TCNet, a hybrid network that effectively models spatio-temporal information from Trajectories and Correlated regions. TCNet's trajectory module transforms frames into aligned trajectories composed of continuous visual tokens. This facilitates extracting region trajectory patterns. In addition, for a query token, self-attention is learned along the trajectory. As such, our network can also focus on fine-grained spatio-temporal patterns, such as finger movement, of a region in motion. TCNet's correlation module utilizes a novel dynamic attention mechanism that filters out irrelevant frame regions. Additionally, it assigns dynamic key-value tokens from correlated regions to each query. Both innovations significantly reduce the computation cost and memory. We perform experiments on four large-scale datasets: PHOENIX14, PHOENIX14-T, CSL, and CSL-Daily. Our results demonstrate that TCNet consistently achieves state-of-the-art performance. For example, we improve over the previous state-of-the-art by 1.5\% and 1.0\% word error rate on PHOENIX14 and PHOENIX14-T, respectively. Code is available at https://github.com/hotfinda/TCNet

Persepsi Simetode Mengajar Guru Pendidikan Agama Islam pada Siswa SMPN 3 Tanjung Medan

This test expects to figure out understudies' perspectives on PAI teachers' presentation methods for understudies at SMPN 3 Tanjung Medan. This investigation method utilizes a realistic quantitative procedure. The populace in this study was 81 Muslim understudies at SMPN 3 Tanjung Medan. The assessment technique utilizes a full scale assessment, so the populace and test takers have similar number, to be specific 81 understudies. Data was gathered utilizing an open survey and a proportion of understudies' impressions of the teacher's exhibition procedure assessed utilizing a Likert scale. The data checking system utilized is realistic examination. Based on the consequences of data testing, the query items show that understudy appraisals are named 39.5% satisfied, 23.5% extremely fulfilled, 21% frustrated, and 16% exceptionally disheartened. Hence, one might say that the most predominant gathering of understudies' comprehension was satisfied at 39.5% and the gathering with the least understanding was exceptionally disheartened at 16%.

Dilated-Windows-based Vision Transformer with Efficient-Suppressive-self-attention for insect pests classification

Transformers with long-range dependency and data specificity act as an effective means of classifying insect pests in agricultural engineering. Although many methods have been proposed to confine the range of self-attention within a local region to reduce the computation complexity, none of them can reduce the number of model parameters. Moreover, the self-attention mechanism usually causes query tokens to focus excessively on image patches, which limits the effective receptive field and the long-range dependence. To address these issues, this paper establishes a novel Dilated-Windows-based Vision Transformer with Efficient-Suppressive-self-attention (DWViT-ES) architecture, which includes efficient-self-attention (ESA), dilated window (DW), and suppressive-self-attention (SSA) as its core components. The ESA simplifies the successive linear Transformations to reduce the number of model parameters and computational costs. Meanwhile, the DW and SSA expand the effective receptive field of self-attention mechanism to prevent query tokens from focusing on similar and close regions, thereby preventing the loss of useful information. Finally, experiments show that the DWViT-ES only has 19.6 M parameters and 3.5G FLOPs (over 20% reductions vs. 19.6 M and 4.5G of Swin-T). Meanwhile, the DWViT-ES training from scratch has 71.6% top-1 accuracy on the IP102 dataset (2.4% absolute improvement of Swin-T); after fine-tuning on Imagenet-1K, the DWViT-ES achieves 76.0% and 78.7% top-1 accuracy on IP102 and CPB (0.1% and 0.9% absolute improvement of Swin-T), respectively. Meanwhile, practical deployment on a mobile-embedded device is presented, which validates the feasibility of the DWViT-ES.

Secure Cloud Storage and Retrieval of Personal Health Data From Smart Wearable Devices With Privacy-Preserving Techniques

With the increasing awareness of personal health, personal health data management has become an important part of people's lives. Smart wearable devices (SWDs) collect people's personal health data, and then store the data on cloud. Authorized entities access the data to provide personalized health services. However, these personal health data contain a large amount of sensitive information, which may pose a significant threat to people's lives and property. To address this, this paper proposes a privacy-preserving solution. SWD data is encrypted, and secure indexes are created using Bloom filter and 0-1 encoding. Encrypted data and indexes are stored in a semi-trusted cloud. Only authorized entities can access the ciphertexts, ensuring secure personalized health management. Extensive experiments validate the scheme's efficiency in index construction, query token generation, and ciphertext search. Security analysis confirms no external entity, including the cloud, gains additional information during retrieval.

VerFHS: Verifiable Image Retrieval on Forward Privacy in Blockchain-Enabled IoT

In the scenario of the Internet of Things (IoT) co-located with the cloud, many applications such as face recognition, traffic monitoring and medical diagnosis usually outsource a large amount of generated image data to cloud servers to reduce the burden of local storage. Many secure encrypted image retrieval schemes have been proposed to protect data privacy. However, existing work incurs storage and communication burdens, lacks verifiability of query results and has potential forward security threats. To solve these issues, we propose the VerFHS framework in this paper, which can satisfy Verifiability, Feedback, High-Security. Specifically, we first present an extended secure k-NN algorithm to protect indexes, cleverly design ciphertext inner product for similarity comparison, and use the reward mechanism of blockchain to build a monitoring and feedback mechanism for cloud servers. Then we demonstrate an enhanced VerFHS scheme in the dynamic setting (VerFHSD) that uses a permutation matrix to process image encryption against adaptive attacks during dynamic updates. VerFHSD prevents cloud servers from making search queries over newly added images via previous tokens, thereby achieving forward security. The formal security analysis shows that our schemes protect the privacy of images, indexes & query tokens and forward security. And extensive experiments using the real-world dataset demonstrate that our scheme not only has the highest search accuracy all the time, but also achieves efficient queries at the millisecond level, when compared with other advanced image retrieval schemes.

Active Token Mixer

The three existing dominant network families, i.e., CNNs, Transformers and MLPs, differ from each other mainly in the ways of fusing spatial contextual information, leaving designing more effective token-mixing mechanisms at the core of backbone architecture development. In this work, we propose an innovative token-mixer, dubbed Active Token Mixer (ATM), to actively incorporate contextual information from other tokens in the global scope into the given query token. This fundamental operator actively predicts where to capture useful contexts and learns how to fuse the captured contexts with the query token at channel level. In this way, the spatial range of token-mixing can be expanded to a global scope with limited computational complexity, where the way of token-mixing is reformed. We take ATMs as the primary operators and assemble them into a cascade architecture, dubbed ATMNet. Extensive experiments demonstrate that ATMNet is generally applicable and comprehensively surpasses different families of SOTA vision backbones by a clear margin on a broad range of vision tasks, including visual recognition and dense prediction tasks. Code is available at https://github.com/microsoft/ActiveMLP.

Enhanced Multi-Relationships Integration Graph Convolutional Network for Inferring Substitutable and Complementary Items

Understanding the relationships between items can improve the accuracy and interpretability of recommender systems. Among these relationships, the substitute and complement relationships attract the most attention in e-commerce platforms. The substitutable items are interchangeable and might be compared with each other before purchasing, while the complementary items are used in conjunction and are usually bought together with the query item. In this paper, we focus on two issues of inferring the substitutable and complementary items: 1) how to model their mutual influence to improve the performance of downstream tasks, 2) how to further discriminate them by considering the strength of relationship for different item pairs. We propose a novel multi-task learning framework named Enhanced Multi-Relationships Integration Graph Convolutional Network (EMRIGCN). We regard the relationship inference task as a link prediction task in heterogeneous graph with different types of edges between nodes (items). To model the mutual influence between substitute and complement, EMRIGCN adopts a two-level integration module, i.e., feature and structure integration, based on experts sharing mechanism during message passing. To obtain the strength of relationship for item pairs, we build an auxiliary loss function to further increase or decrease the distances between embeddings of items with weak or strong relation in latent space. Extensive experiments on both public and industrial datasets prove that EMRIGCN significantly outperforms the state-of-the-art solutions. We also conducted A/B tests on real world recommender systems of Meituan Maicai, an online supermarket platform in China, and obtained 15.3% improvement on VBR and 15.34% improvement on RPM.

Better Localness for Non-Autoregressive Transformer

The Non-Autoregressive Transformer, due to its low inference latency, has attracted much attention from researchers. Although, the performance of the non-autoregressive transformer has been significantly improved in recent years, there is still a gap between the non-autoregressive transformer and the autoregressive transformer. Considering the success of localness on the autoregressive transformer, in this work, we consider incorporating localness into the non-autoregressive transformer. Specifically, we design a dynamic mask matrix according to the query tokens, key tokens, and relative distance, and unify the localness module for self-attention and cross-attention module. We conduct experiments on several benchmark tasks, and the results show that our model can significantly improve the performance of the non-autoregressive transformer.

TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation.

Video frame interpolation (VFI) aims to synthesize an intermediate frame between two consecutive frames. State-of-the-art approaches usually adopt a two-step solution, which includes 1) generating locally-warped pixels by calculating the optical flow based on pre-defined motion patterns (e.g., uniform motion, symmetric motion), 2) blending the warped pixels to form a full frame through deep neural synthesis networks. However, for various complicated motions (e.g., non-uniform motion, turn around), such improper assumptions about pre-defined motion patterns introduce the inconsistent warping from the two consecutive frames. This leads to the warped features for new frames are usually not aligned, yielding distortion and blur, especially when large and complex motions occur. To solve this issue, in this paper we propose a novel Trajectory-aware Transformer for Video Frame Interpolation (TTVFI). In particular, we formulate the warped features with inconsistent motions as query tokens, and formulate relevant regions in a motion trajectory from two original consecutive frames into keys and values. Self-attention is learned on relevant tokens along the trajectory to blend the pristine features into intermediate frames through end-to-end training. Experimental results demonstrate that our method outperforms other state-of-the-art methods in four widely-used VFI benchmarks. Both code and pre-trained models will be released at https://github.com/ChengxuLiu/TTVFI.git.

Subjective Functionality and Comfort Prediction for Apartment Floor Plans and Its Application to Intuitive Online Property Search

This paper presents a new user experience for online apartment search using functionality and comfort as query items. Specifically, it has three technical contributions. First, we present a new dataset on the perceived functionality and comfort scores of residential floor plans using nine question statements about the level of comfort, openness, privacy, etc. Second, we propose an algorithm to predict the scores from the floor plan images. Lastly, we implement a new apartment search system and conduct a large-scale usability study using crowdsourcing. The experimental results show that our apartment search system can provide a better user experience. To the best of our knowledge, this is the first work to propose a highly accurate machine learning model for predicting the subjective functionality and comfort of apartments.

A response generator with response-aware encoder for generating specific and relevant responses

The dialogue data usually consist of the pairs of a query and its response, but no previous response generators have exploited the responses explicitly in their training while a response provides significant information about the meaning of a query. Therefore, this paper proposes a sequence-to-sequence response generator with a response-aware encoder. The proposed generator exploits golden responses by reflecting them into query representation. For this purpose, the response-aware encoder adds a relevancy scorer layer to the transformer encoder that calculates the relevancy of query tokens to a response. However, golden responses are available only during training of the response generator and unavailable at inference time. As a solution to this problem, the joint learning of a teacher and a student relevancy scorer is adopted. That is, at the training time, both the teacher and the student relevancy scorers are optimized but the decoder generates a response using only the relevancy of the teacher scorer. However, at the inference time, the decoder uses that of the student scorer. Since the student scorer is trained to minimize the difference from the teacher scorer, it can be used to compute the relevancy of a prospective response. The proposed model is the first attempt to use a golden response directly for generating a query representation, whereas previous studies used the responses for its implicit and indirect reflection. As a result, it achieved higher dialogue evaluation score than the current state-of-the-art model for Reddit, Persona-Chat, and DailyDialog data sets.

Towards a unified search: Improving PubMed retrieval with full text

ObjectiveA significant number of recent articles in PubMed have full text available in PubMed Central®, and the availability of full texts has been consistently growing. However, it is not currently possible for a user to simultaneously query the contents of both databases and receive a single integrated search result. In this study, we investigate how to score full text articles given a multitoken query and how to combine those full text article scores with scores originating from abstracts and achieve an overall improved retrieval performance. Materials and methodsFor scoring full text articles, we propose a method to combine information coming from different sections by converting the traditionally used BM25 scores into log odds ratio scores which can be treated uniformly. We further propose a method that successfully combines scores from two heterogenous retrieval sources – full text articles and abstract only articles – by balancing the contributions of their respective scores through a probabilistic transformation. We use PubMed click data that consists of queries sampled from PubMed user logs along with a subset of retrieved and clicked documents to train the probabilistic functions and to evaluate retrieval effectiveness. Results and conclusionsRandom ranking achieves 0.579 MAP score on our PubMed click data. BM25 ranking on PubMed abstracts improves the MAP by 10.6%. For full text documents, experiments confirm that BM25 section scores are of different value depending on the section type and are not directly comparable. Naïvely using the body text of articles along with abstract text degrades the overall quality of the search. The proposed log odds ratio scores normalize and combine the contributions of occurrences of query tokens in different sections. By including full text where available, we gain another 0.67%, or 7% relative improvement over abstract alone. We find an advantage in the more accurate estimate of the value of BM25 scores depending on the section from which they were produced. Taking the sum of top three section scores performs the best.

Extricating web pages from deep web using deaima architecture

Due to remarkable development of advanced data, it has gotten drawn-out to recover applicable query items from the web crawler. The customary web search tool furnishes pertinent site pages however with the blend of unessential pages with less coverage rate and high slithering time. To overcome the issues, this paper proposes an Intelligent Multi-Agent design that utilizes Differential Evolution Algorithm (DEA) to extricate more pertinent site pages with high coverage rate and less slithering time. Three intelligent agents, specifically, Trajectory pattern mining agent, Query clustering agent and Web page classification agent are employed to track the relevancy of the pages, hidden web links and classify the searchable and non-searchable forms respectively. The extrication of relevant pages is optimized by the DEA algorithm. DEA calculation like selection, crossover and mutation are utilized for optimizing the extraction process. In any case, in Genetic Algorithm (GA) determination of chromosome depends on the wellness esteem, where the low quality chromosomes don't get by in the future. The Differential Evolutionary activity help the helpless chromosome to get by in the opposition and it will create the upgraded results. The learnt by experience of the intelligent agents help to work on the presentation of the framework. The exploration result shows that the proposed architecture gives the high precision and recall ratesover the current web crawlers. The creeping time for recovering the website pages is likewise diminished utilizing the proposed DEAIMA architecture.

Shrinking Temporal Attention in Transformers for Video Action Recognition

Spatiotemporal modeling in an unified architecture is key for video action recognition. This paper proposes a Shrinking Temporal Attention Transformer (STAT), which efficiently builts spatiotemporal attention maps considering the attenuation of spatial attention in short and long temporal sequences. Specifically, for short-term temporal tokens, query token interacts with them in a fine-grained manner in dealing with short-range motion. It then shrinks to a coarse attention in neighborhood for long-term tokens, to provide larger receptive field for long-range spatial aggregation. Both of them are composed in a short-long temporal integrated block to build visual appearances and temporal structure concurrently with lower costly in computation. We conduct thorough ablation studies, and achieve state-of-the-art results on multiple action recognition benchmarks including Kinetics400 and Something-Something v2, outperforming prior methods with 50% less FLOPs and without any pretrained model.