Embedding Scheme Research Articles

Multi-temporal Sequential Recommendation Model Based on the Fused Learning Preferences

Since sequential recommendation can effectively capture the user’s dynamic interaction pattern according to the user’s historical interaction, it plays one of the important roles in the field of recommendation systems. Nowadays, the sequential recommendation models based on deep learning approaches have achieved good results in reality due to their excellent performance in information retrieval and filtering. However, there are still some challenges that still should be overcome of them. A major disadvantage of them is that they are only considering users’ short-time interests while ignoring their long-term preferences. Moreover, they are incapable of considering the influence of time information in the original interaction sequence, which could be helpful to fully extract various patterns via the different temporal embedding forms. Therefore, this paper proposes a novel model named multi-temporal sequential recommendation model based on the fused learning preference named MTSR-FLP in short. In the proposed framework, this paper adopts an MLP-based state generation module to consider the user’s long-term preferences and the short-time interests simultaneously. In particular, the proposed MTSR-FLP model designs a global representation learning approach to obtain the user’s global preference, and a local representation learning approach to capture the users’ local preference via its historical information. Moreover, the proposed model develops a multiple temporal embedding scheme to encode the positions of user–item interactions of a sequence, in which multiple kernels are utilized for the absolute or relative timestamps to establish unique embedding matrices. Finally, compared with other advanced sequence recommendation models on five public real-world datasets, the experimental results show that the proposed MTSR-FLP model has improved the performance of HR@10 from the 6.68% through 31.10% and NDCG@10 from the 8.60% through 42.54%.

High-Quality Temporal Link Prediction for Weighted Dynamic Graphs via Inductive Embedding Aggregation

Temporal link prediction (TLP) is an inference task on dynamic graphs that predicts future topology using historical graph snapshots. Existing TLP methods are usually designed for unweighted graphs with fixed node sets. Some of them cannot be generalized to the prediction of weighted graphs with non-fixed node sets. Although several methods can still be used to predict weighted graphs, they can only derive <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">low-quality</i> prediction snapshots sensitive to large edge weights but fail to distinguish small and zero weights in adjacency matrices. In this study, we consider the challenging <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">high-quality</i> TLP on weighted dynamic graphs and propose a novel inductive dynamic embedding aggregation (IDEA) method, inspired by the high-resolution video prediction. IDEA combines conventional error minimization objectives with a scale difference minimization objective, which can generate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">high-quality</i> weighted prediction snapshots, distinguishing differences among large, small, and zero weights in adjacency matrices. Since IDEA adopts an inductive dynamic embedding scheme with an attentive node aligning unit and adaptive embedding aggregation module, it can also tackle the TLP on weighted graphs even with non-fixed node sets. Experiments on datasets of various scenarios validate that IDEA can derive <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">high-quality</i> prediction results for weighted dynamic graphs and tackle the variation of node sets.

Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis.

Transformer has been widely used in histopathology whole slide image analysis. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits its effectiveness and efficiency when applied to gigapixel histopathology images. In this paper, we propose a novel kernel attention Transformer (KAT) for histopathology WSI analysis and assistant cancer diagnosis. The information transmission in KAT is achieved by cross-attention between the patch features and a set of kernels related to the spatial relationship of the patches on the whole slide images. Compared to the common Transformer structure, KAT can extract the hierarchical context information of the local regions of the WSI and provide diversified diagnosis information. Meanwhile, the kernel-based cross-attention paradigm significantly reduces the computational amount. The proposed method was evaluated on three large-scale datasets and was compared with 8 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI analysis and is superior to the state-of-the-art methods.

IRMSwin-T: A lightweight shifted windows transformer based on inverted residual structure and residual multi-layer perceptron for rolling bearing fault diagnosis.

The data-driven fault diagnosis method has achieved many good results. However, classical convolutional and recurrent neural networks have problems with large parameters and poor anti-noise performance. To solve these problems, we propose a lightweight shifted windows transformer based on inverted residual structure and residual multi-layer perceptron (IRMSwin-T) for fault diagnosis of rolling bearings. First, the original data are expanded by using overlapping sampling technology. Then, the collected one-dimensional vibration signals are vector serialized by using the patch embedding strategy. Finally, the IRMSwin-T network is developed to extract features of vector sequences and classify faults. The experimental results showed that compared with mainstream lightweight models, the IRMSwin-T model in this paper has fewer parameters and higher diagnostic accuracy.

Research on Oil–Gas Two-Phase Flow Characteristics and Improvement of Aero-Engine Bearing Chamber

In order to study the oil–gas two-phase flow characteristics of an aero-engine bearing chamber and improve the scavenge effect of lubricating oil, the two-phase flow solution model of a bearing chamber based on the Euler–Euler method was established. Three improvement schemes were proposed for the ventilation structure and scavenge structure of the bearing chamber. The flow characteristics and scavenge characteristics of a conventional bearing chamber and three improvement schemes under different working conditions were analyzed in depth. The results show that after the conventional bearing chamber ventilation structure is embedded (Embed) and improved, with the increase in the embedding depth, the oil in the cavity is further blocked in the cavity, the amount of oil flowing out from the vent is further reduced, and the scavenge efficiency is further improved. After the slope improvement of the scavenge structure of the conventional bearing chamber, due to the increase in the depth of the oil return groove, the drag effect of the air shear force in the cavity on the oil in the oil return groove is further weakened, and the oil accumulation area on the right side of the scavenge port is further suppressed. The volume fraction of the oil in the cavity is further reduced, and the scavenge efficiency is further improved. The combined improvement scheme (ES) can take into account the advantages of embedding and slope improvement schemes, and further improve the scavenge efficiency. Compared to the conventional bearing chamber, when the oil flow rate is 200 L/h and the speed is 15,000 r/min, the oil return efficiency of the embedded (h = 12 mm), slope (l = 56 mm) and combined improvement schemes are increased by 20.19%, 13.43%, and 37.94%, respectively.

Exploiting Emotions via Composite Pretrained Embedding and Ensemble Language Model

Decisions in the modern era are based on more than just the available data; they also incorporate feedback from online sources. Processing reviews known as Sentiment analysis (SA) or Emotion analysis. Understanding the user's perspective and routines is crucial now-a-days for multiple reasons. It is used by both businesses and governments to make strategic decisions. Various architectural and vector embedding strategies have been developed for SA processing. Accurate representation of text is crucial for automatic SA. Due to the large number of languages spoken and written, polysemy and syntactic or semantic issues were common. To get around these problems, we developed effective composite embedding (ECE), a method that combines the advantages of vector embedding techniques that are either context-independent (like glove &amp; fasttext) or context-aware (like XLNet) to effectively represent the features needed for processing. To improve the performace towards emotion or sentiment we proposed stacked ensemble model of deep lanugae models.ECE with Ensembled model is evaluated on balanced dataset to prove that it is a reliable embedding technique and a generalised model for SA.In order to evaluate ECE, cutting-edge ML and Deep net language models are deployed and comapared. The model is evaluated using benchmark datset such as MR, Kindle along with realtime tweet dataset of user complaints . LIME is used to verify the model's predictions and to provide statistical results for sentence.The model with ECE embedding provides state-of-art results with real time dataset as well.

Investigation of Coulomb blockade diamonds of 7-helicenes interacting with gold nanoparticles

Helicene molecules show promising properties for future applications in optoelectronics due to their impressive optical properties and long effective conjugation. In this work, we thus study the properties of [7]-Helicenes in junctions with gold nanoparticles with emphasis on charge transport. The two helicenes are positioned between gold nanoparticles of different sizes creating to emulate molecular junctions and we utilize a combined quantum mechanical/molecular mechanical methodology using the polarizable embedding scheme to compute Coulomb blockade diamonds for the helicenes. Both molecules create Coulomb blockade diamonds following the 1-3-2 pattern.

High-speed and low-power embedded TEC BCH scheme for ReRAM array

High-speed and low-power embedded TEC BCH scheme for ReRAM array

A blind color image watermarking algorithm based on Hadamard transform and TLBO algorithm

For the sake of better protect the copyright of color images, an efficient color image watermarking algorithm based on Hadamard transform and teaching-learning-based optimization (TLBO) algorithm is proposed in this paper. In the stage of embedding watermark, different watermark embedding strategies are employed in 4 × 4 blocks based on the impact level of pixel caused by the high and low bit of watermark and taking into account the energy concentration characteristics of the Hadamard matrix. To enhance the robustness of the watermarking algorithm, the voting system is designed for the high-four bits of watermark information. Simultaneously, the permutation ordered binary (POB) number system is used to compress the low-four bits of watermark information for improving the invisibility of the watermarked image and the embedding capacity. When the watermark needs to be extracted, the inverse process of POB number system transform is optimized, and the time of watermark extraction is shortened. Finally, the TLBO algorithm is used to optimize the watermarking algorithm parameters for saving the experimental time while considering both invisibility and robustness of the algorithm. In comparison to the other algorithms examined in this research, the experimental results demonstrate that the proposed watermarking algorithm exhibits several performance enhancements, these include aspects such as robustness, running time, watermark embedding capacity, security and invisibility.

Visual image encryption algorithm based on compressed sensing and 2D cosine -type logistic map

A new two-dimensional cosine-type Logistic map (2D-CTLM) is proposed in this paper. Performance analysis shows that 2D-CTLM has good ergodicity, complex behavior, and a wide range of chaotic regions. To study its application, an adaptive embedded high visual security image encryption scheme is proposed by combining the system with two-dimensional compression sensing to address the shortcomings of fixed embedding positions and low transmission efficiency in existing schemes. The embedding position is fixed when the ciphertext image is embedded into the carrier image. When embedded in an inappropriate location, it will affect the decryption quality and the imperceptibility of the steganographic image. Firstly, the initial value of 2D-CTLM system is generated by using the information entropy of plaintext image and counter to generate random measurement matrix, which is used to compress and encrypt plaintext image from two directions at the same time. Secondly, the compressed cipher image is scrambled by the initial matrix generated by the GOL (game of life) rules in two-dimensional cellular automata to obtain the ciphertext image. Finally, the most suitable embedding position in the carrier image is found by combining information entropy and edge entropy. Then the unquantized ciphertext image is embedded into it by ST (slant transform) embedding method to obtain a high visual security image. To improve the transmission efficiency, the color carrier image can be selected to transmit three gray plaintext images at the same time, maintain high-quality reconstruction effect. SHA-512 and the information entropy of plaintext image are used to generate the initial value of encryption and measurement matrix, enhance the correlation between the algorithm and plaintext image, and effectively resist known plaintext and selective plaintext attacks. Simulation results and performance analysis show that compared with the existing encryption schemes, the encryption scheme has excellent visual effect, good decryption quality and good robustness.

Which programmes and policies across health and community settings will generate the most significant impacts for youth suicide prevention in Australia and the UK? Protocol for a systems modelling and simulation study

IntroductionSuicide is a leading cause of mortality among young people aged 15–24 globally. Despite the deployment of comprehensive suicide prevention strategies, we still do not know which interventions, for which...

A dilating and embedding scheme for constructing sparse MIMO radar array

AbstractIn this letter, a dilating and embedding scheme for constructing sparse multiple‐input and multiple‐output radar arrays is addressed, which seeks to increase the uniform degrees of freedom (uDOFs) and reduce the mutual coupling effects for a given number of transmitters and receivers (Tx/Rx). According to this scheme, a dilated maximum inter‐element spacing constraint (MISC) array with an embedded thinned coprime array in the collocated MIMO radar for DOA estimation is proposed. Theoretical analysis shows that the proposed array can provide higher uDOFs and is robust in a scenario with heavy levels of mutual coupling. Numerical simulations are conducted to validate its aforementioned advantages in line with the design scheme.

Multi-proximity based embedding scheme for learning vector quantization-based classification of biochemical structured data

In this paper, we propose a data embedding technique for structured data that allows for the direct application of standard vector-based machine learning models without the need for explicit feature extraction. Our approach relies on multiple notions of data proximity, making it suitable for handling mixed data types or incorporating domain knowledge. Our method also reduces the computational costs of pairwise proximity calculations, resulting in improved efficiency and scalability. We demonstrate the effectiveness of our technique on graph and sequence datasets from the biochemical domain. In particular, we show that the method can efficiently be combined with interpretable machine learning approaches like relevance-based learning vector quantization for sophisticated classification learning.

Research on a Knowledge Graph Embedding Method Based on Improved Convolutional Neural Networks for Hydraulic Engineering

In response to the shortcomings of existing knowledge graph embedding strategies, such as weak feature interaction and latent knowledge representation, a unique hydraulic knowledge graph embedding method is suggested. The proposed method incorporates spatial position features into the entity-relation embedding process, thereby enhancing the representation capability of latent knowledge. Furthermore, it utilizes a multi-layer convolutional neural network to fuse features at different levels, effectively capturing more abundant semantic information. Additionally, the method employs multi-scale dilated convolution kernels to capture rich explicit interaction features across different scales of space. In this study, the effectiveness of the proposed model was validated on the link prediction task. Experimental results demonstrated that, compared to the ConvE model, the proposed model achieved a significant improvement of 14.8% in terms of mean reciprocal rank (MRR) on public datasets. Additionally, the suggested model outperformed the ConvR model on the hydraulic dataset, leading to a 10.1% increase in MRR. The results indicate that the proposed approach exhibits good applicability and performance in the task of hydraulic knowledge graph complementation. This suggests that the method has the potential to offer significant assistance for knowledge discovery and application research in the field of hydraulics.

Modulus difference-induced embedding strategy to construct iontronic pressure sensor with high sensitivity and wide linear response range

Sensitivity and linearity are two crucial indices to assess the sensing capability of pressure sensors; unfortunately, the two mutually exclusive parameters usually result in limited applications. Although a series of microengineering strategies including micropatterned, multilayered, and porous approach have been provided in detail, the conflict between the two parameters still continues. Here, we present an efficient strategy to resolve this contradiction via modulus difference-induced embedding deformation. Both the microscopic observation and finite element simulation results confirm the embedding deformation behavior ascribed to the elastic modulus difference between soft electrode and rigid microstructures. The iontronic pressure sensor with high sensitivity (35kPa-1) and wide linear response range (0-250kPa) is further fabricated and demonstrates the potential applications in monitoring of high-fidelity pulse waveforms and human motion. This work provides an alternative strategy to guide targeted design of all-around and comprehensive pressure sensor.

Social world knowledge: Modeling and applications.

Social world knowledge is a key ingredient in effective communication and information processing by humans and machines alike. As of today, there exist many knowledge bases that represent factual world knowledge. Yet, there is no resource that is designed to capture social aspects of world knowledge. We believe that this work makes an important step towards the formulation and construction of such a resource. We introduce SocialVec, a general framework for eliciting low-dimensional entity embeddings from the social contexts in which they occur in social networks. In this framework, entities correspond to highly popular accounts which invoke general interest. We assume that entities that individual users tend to co-follow are socially related, and use this definition of social context to learn the entity embeddings. Similar to word embeddings which facilitate tasks that involve text semantics, we expect the learned social entity embeddings to benefit multiple tasks of social flavor. In this work, we elicited the social embeddings of roughly 200K entities from a sample of 1.3M Twitter users and the accounts that they follow. We employ and gauge the resulting embeddings on two tasks of social importance. First, we assess the political bias of news sources in terms of entity similarity in the social embedding space. Second, we predict the personal traits of individual Twitter users based on the social embeddings of entities that they follow. In both cases, we show advantageous or competitive performance using our approach compared with task-specific baselines. We further show that existing entity embedding schemes, which are fact-based, fail to capture social aspects of knowledge. We make the learned social entity embeddings available to the research community to support further exploration of social world knowledge and its applications.

MetaTransformer: deep metagenomic sequencing read classification using self-attention models.

Deep learning has emerged as a paradigm that revolutionizes numerous domains of scientific research. Transformers have been utilized in language modeling outperforming previous approaches. Therefore, the utilization of deep learning as a tool for analyzing the genomic sequences is promising, yielding convincing results in fields such as motif identification and variant calling. DeepMicrobes, a machine learning-based classifier, has recently been introduced for taxonomic prediction at species and genus level. However, it relies on complex models based on bidirectional long short-term memory cells resulting in slow runtimes and excessive memory requirements, hampering its effective usability. We present MetaTransformer, a self-attention-based deep learning metagenomic analysis tool. Our transformer-encoder-based models enable efficient parallelization while outperforming DeepMicrobes in terms of species and genus classification abilities. Furthermore, we investigate approaches to reduce memory consumption and boost performance using different embedding schemes. As a result, we are able to achieve 2× to 5× speedup for inference compared to DeepMicrobes while keeping a significantly smaller memory footprint. MetaTransformer can be trained in 9hours for genus and 16hours for species prediction. Our results demonstrate performance improvements due to self-attention models and the impact of embedding schemes in deep learning on metagenomic sequencing data.

Provably Secure and Anonymous V2I and V2V Authentication Protocol for VANETs

Vehicular Ad-hoc Networks (VANETs) enable the connection and information exchange between vehicles and transportation infrastructure (V2I), vehicle and vehicle (V2V) to improve the safety and efficiency of the transportation system. In previous protocols, V2I and V2V authentication protocols based on bilinear pairings or identity-based cryptography are computationally heavy or difficult to protect the user’s identity and privacy. In addition, lightweight authentication protocols have not achieve both V2I and V2V authentication, and may suffer from Onboard Unit (OBU) intrusion attacks, Roadside Unit (RSU) captured attacks, and difficult to track malicious vehicles. Therefore, a lightweight and anonymous V2I and V2V authentication protocol with batch verification based on Elliptic Curves Cryptography (ECC) is proposed. We use Physical Unclonable Function (PUF) and biological key to avoid RSU captured attacks and OBU intrusion attacks, design a feature embedding strategy with dynamic pseudo-identity to recover the malicious vehicle’s identity by the Trusted Authority (TA). In the application test simulating the actual scenario, our protocol is more efficient because of lower overhead and batch authentication strategy. The semantic security of the protocol is formally proved under the random oracle model. The comparative analysis and security proof results show that our protocol is more superior to others.

Towards improving the performance of blind image steganalyzer using third-order SPAM features and ensemble classifier

The success rate for blind or universal steganalysis lies in the ability to extract the statistical footprints of image features. Further, the choice of machine learning (ML) algorithm is crucial to distinguish the stego image more precisely from the untouched clean images. Literature suggests that most steganalysis approaches report less favorable detection accuracy despite considering many features. This study presents a three-step process to accurately identify the clean and stego images to solve this issue. We used the curvelet denoising as an initial phase during the first step to suppress the natural noise residuals (NRs) by producing the stego NRs. Secondly, it extracts the Third-order Markov-chain sample transition probability matrices as features. Finally, the oblique decision tree ensemble using a multisurface proximal support vector machine (SVM) classifier has been utilized to achieve greater detection accuracy than the state-of-the-art classifiers. The experiments are performed on an extensive database comprising clean and stego images generated from nine embedding schemes with varying payloads. The experimental results suggest that an accuracy of 93.12 has been achieved using the proposed Third order subtractive pixel adjacency matrix (SPAM) features with an ensemble classifier.

A Learnable Radial Basis Positional Embedding for Coordinate-MLPs

We propose a novel method to enhance the performance of coordinate-MLPs (also referred to as neural fields) by learning instance-specific positional embeddings. End-to-end optimization of positional embedding parameters along with network weights leads to poor generalization performance. Instead, we develop a generic framework to learn the positional embedding based on the classic graph-Laplacian regularization, which can implicitly balance the trade-off between memorization and generalization. This framework is then used to propose a novel positional embedding scheme, where the hyperparameters are learned per coordinate (i.e instance) to deliver optimal performance. We show that the proposed embedding achieves better performance with higher stability compared to the well-established random Fourier features (RFF). Further, we demonstrate that the proposed embedding scheme yields stable gradients, enabling seamless integration into deep architectures as intermediate layers.