Position Coding Research Articles

Short-term heating load forecasting model based on SVMD and improved informer

Accurate heat load forecasting can not only effectively reduce energy consumption, but also improve the efficiency of the heating system and user comfort. In order to improve the accuracy of heat load forecasting, we proposed a forecasting method combining SVMD algorithm and improved Informer model and applied it to the heat load forecasting of district heating system. First, SVMD algorithm is used to decompose the heat load data, and then relative position coding is introduced into Informer model, and causal convolution and jump connection mechanisms are adopted to better capture the dependence of sequence data and enhance the ability of local information extraction. In order to verify the model performance, experiments were conducted on multiple data sets. The results show that the R2 evaluation index of the model reaches 98.7 %, which is 10.8 %, 17.1 %, 6.3 %, 4.9 %, 2.3 %, 1.5 % and 16.6 % higher than SVR, RNN, LSTM, Transformer, SH-Informer, iTransformer and DLinear respectively. The validity of the model is further verified by DM test, which provides a reference for further improving the timeliness of intelligent heating.

Interest HD: An Interest Frame Model for Recommendation Based on HD Image Generation.

This work is inspired by high-definition (HD) image generation techniques. When the user's interests are viewed as different frames of varying clarity, the unclear parts of one interest frame can be clarified by other interest frames. The user's overall HD interest portrait can be viewed as a fusion of multiple interest frames through detail compensation. Based on this inspiration, we propose a model for generating HD interest portrait called interest frame for recommendation (IF4Rec). First, we present a fine-grained pixel-level user interest mining method, Pixel embedding (PE) uses positional coding techniques to mine atomic-level interest pixel matrices in multiple dimensions, such as time, space, and frequency. Then, using an atomic-level interest pixel matrix, we propose Item2Frame to generate several interest frames for a user. The similarity score of each item is calculated to fill the multi-interest pixel clusters, through an improved self-attention mechanism. Finally, stimulated by HD image generation techniques, we initially present an interest frame noise compensation method. By utilizing the multihead attention mechanism, pixel-level optimization and noise complementation are performed between multi-interest frames, and an HD interest portrait is achieved. Experiments show that our model mines users' interests well. On five publicly available datasets, our model outperforms the baselines.

Readers encode absolute letter positions

Reading research has long been concerned with the question of whether the reading brain accesses lexical representations via absolute or relative letter position information. In recent years, important results have been obtained with the flanker lexical decision task. Studies have shown faster decisions about target words (e.g., ‘rock’) when flanked by related letters (‘ro rock ck’) than unrelated letters (‘st rock ep')—and crucially, equal facilitation upon switching flanker positions (‘ck rock ro'), pointing to relative rather than absolute letter position coding. Yet, a later study employing longer targets and flankers yielded detrimental effects of switching flanker positions. In order to get a better grasp on the equivocal evidence thus far, here we carried out an extensive test of flanker relatedness and position effects, using various target and flanker lengths, all within a single experiment. We observed a clear reduction of flanker relatedness effects upon switching flanker positions, and this held true across target and flanker lengths. The present results unambiguously suggest that lexical access is driven by absolute letter position information, and furthermore, are accurately predicted by the recent PONG model (Snell, 2024b).

An optimized Transformer algorithm: Implementation for intelligent English translation

There is an increasing demand for high-quality translations in the realm of intelligent English translation. This paper optimized the traditional Transformer algorithm by enhancing position coding and the softmax layer. Bidirectional long short-term memory (BiLST) was employed to realize position encoding, capturing both contextual and positional information simultaneously. Additionally, the softmax function was replaced with the sparsemax function to obtain sparser results. The translation performance of some algorithms on Chinese and English datasets was compared and analyzed. It was found that that the optimized Transformer algorithm performed better than the RNNSearch, ConvS2S, and Transformer-base algorithms in terms of bilingual evaluation understudy (BLEU) score on the test set. It achieved an average BLEU score of 23.72, representing an improvement of 1.56 over the RNNSearch algorithm, 1.17 over the ConvS2S algorithm, and 0.73 over the Transformer-base algorithm. The parameter quantity of the optimized algorithm was 6.83 M, which was 0.05 M higher than the Transformer-base algorithm. Furthermore, its running time was 4,862.76 s, showing a marginal increase of 0.21% compared to the Transformer-base algorithm. These findings validate the reliability of the optimized Transformer algorithm for intelligent English translation and its potential practical application.

Few-Shot Learning Sensitive Recognition Method Based on Prototypical Network

Traditional machine learning-based entity extraction methods rely heavily on feature engineering by experts, and the generalization ability of the model is poor. Prototype networks, on the other hand, can effectively use a small amount of labeled data to train models while using category prototypes to enhance the generalization ability of the models. Therefore, this paper proposes a prototype network-based named entity recognition (NER) method, namely the FSPN-NER model, to solve the problem of difficult recognition of sensitive data in data-sparse text. The model utilizes the positional coding model (PCM) to pre-train the data and perform feature extraction, then computes the prototype vectors to achieve entity matching, and finally introduces a boundary detection module to enhance the performance of the prototype network in the named entity recognition task. The model in this paper is compared with LSTM, BiLSTM, CRF, Transformer and their combination models, and the experimental results on the test dataset show that the model outperforms the comparative models with an accuracy of 84.8%, a recall of 85.8% and an F1 value of 0.853.

Flexible word position coding in reading: Roles for attention and memory.

Readers may fail to notice the error in 'Do love you me?': this is the recently established transposed-word (TW) phenomenon. Word position coding is a novel cognitive construct, and researchers are presently debating the underlying mechanisms. Here I investigated roles for attention and memory. Participants (N = 54) made grammaticality judgements to four-word sequences that formed correct sentences ('The man can run', 'The dog was here'), TW sentences ('The can man run', 'The was dog here'), or ungrammatical control sentences ('The man dog run', 'The was can here'). Sequences were replaced by post-masks after 200ms, and that post-mask was accompanied by a 50-ms retro-cue in the form of an 'X' presented at a critical location (where one could have locally inferred grammaticality; e.g., between the first and second word of 'The was dog here') or a non-critical location (e.g., between the third and fourth word of 'The was dog here'). TW sentences were harder to reject than control sentences - the classic TW effect - and crucially, this effect was modulated by cue validity, with valid cues attenuating TW effects compared to invalid cues. The present results suggest that focused attention aids the process of binding words to locations. Furthermore, as cues appeared after sentence offset, these results suggest that word position coding may take place in memory.

Vaccine patterns among older adults with Guillain-Barré syndrome and matched comparators, 2006-2019.

Some vaccines have a small risk of triggering Guillain-Barré syndrome (GBS), an autoimmune disorder where nerve damage leads to paralysis. There is a CDC precaution for patients whose GBS was associated with an influenza or tetanus toxoid-containing vaccine (GBS occurring within 42 days following vaccination). We described vaccine patterns before and after a GBS diagnosis with a matched cohort design in a 20% random sample of fee-for-service Medicare enrollees. We defined the index date as an ICD-9-CM or ICD-10-CM GBS diagnosis code in the primary position of an inpatient claim. We matched each GBS patient to five non-GBS comparators on sex, exact age, racial and ethnic category, state of residence and the month of preventive health visits during baseline; used weighting to balance covariates; and measured frequency of vaccines received per 100 people during year before and after the index date using the weighted mean cumulative count (wMCC). We identified 1567 patients with a GBS diagnosis with at least 1 year of prior continuous enrollment in Medicare A and B that matched to five comparators each. The wMCCs in the 1 year before the index date were similar for both groups, with a wMCC of 74 vaccines/100 people in the GBS group (95% CI 71, 77). Within 1 year after the index date, patients with GBS had received 26 vaccines/100 people (95% CI 23, 28), which was 41 fewer vaccines than matched non-GBS comparators (95% CI -44, -38). Among GBS patients, 11% were diagnosed with GBS within 42 days after a vaccine. GBS diagnosis has a strong impact on reducing subsequent vaccination even though there is no warning or precaution about future vaccines for most patients diagnosed with GBS. These data suggest discordance between clinical practice and current vaccine recommendations.

Reconstruction of OFDM Signals Using a Dual Discriminator CGAN with BiLSTM and Transformer.

Communication signal reconstruction technology represents a critical area of research within communication countermeasures and signal processing. Considering traditional OFDM signal reconstruction methods' intricacy and suboptimal reconstruction performance, a dual discriminator CGAN model incorporating LSTM and Transformer is proposed. When reconstructing OFDM signals using the traditional CNN network, it becomes challenging to extract intricate temporal information. Therefore, the BiLSTM network is incorporated into the first discriminator to capture timing details of the IQ (In-phase and Quadrature-phase) sequence and constellation map information of the AP (Amplitude and Phase) sequence. Subsequently, following the addition of fixed position coding, these data are fed into the core network constructed based on the Transformer Encoder for further learning. Simultaneously, to capture the correlation between the two IQ signals, the VIT (Vision in Transformer) concept is incorporated into the second discriminator. The IQ sequence is treated as a single-channel two-dimensional image and segmented into pixel blocks containing IQ sequence through Conv2d. Fixed position coding is added and sent to the Transformer core network for learning. The generator network transforms input noise data into a dimensional space aligned with the IQ signal and embedding vector dimensions. It appends identical position encoding information to the IQ sequence before sending it to the Transformer network. The experimental results demonstrate that, under commonly utilized OFDM modulation formats such as BPSK, QPSK, and 16QAM, the time series waveform, constellation diagram, and spectral diagram exhibit high-quality reconstruction. Our algorithm achieves improved signal quality while managing complexity compared to other reconstruction methods.

Transformer-inspired stacked-GAN for hyperspectral target detection

ABSTRACT Hyperspectral images (HSI) can provide abundant spectral feature information and have been successfully used in the field of object detection. The use of Generative Adversarial Network (GAN) to reconstruct the background spectral to achieve differential detection has shown excellent performance in hyperspectral object detection (HTD) tasks. However, due to factors such as noise interference and the complexity of the background, such algorithms also face many challenges such as lack of background sample mixing, poor reconstruction effect, and generation of out-of-class samples. A Transformer-inspired stacked GAN network (STGAN) is proposed to solve these problems. First, in the data preparation phase, we adopted a method of extracting pure background samples at a deep level. Then, in the encoder module, the background spectral features are mined using a Transformer-Encoder with Position Coding and Multihead Self-Attention Mechanism (MHSA). Finally, the same GAN is stacked based on the above network structure, and the latent feature mapping of the first GAN is used as input to reduce the out-of-class samples and achieve a more detailed reconstruction of the background. The comparison experiments of our model with other algorithms on datasets in different scenarios prove its advanced performance.

Validating new coding algorithms to improve identification of alcohol-associated and nonalcohol-associated cirrhosis hospitalizations in administrative databases.

Alcohol (AC) and nonalcohol-associated cirrhosis (NAC) epidemiology studies are limited by available case definitions. We compared the diagnostic accuracy of previous and newly developed case definitions to identify AC and NAC hospitalizations. We randomly selected 700 hospitalizations from the 2008 to 2022 Canadian Discharge Abstract Database with alcohol-associated and cirrhosis-related International Classification of Diseases 10th revision codes. We compared standard approaches for AC (ie, AC code alone and alcohol use disorder and nonspecific cirrhosis codes together) and NAC (ie, NAC codes alone) case identification to newly developed approaches that combine standard approaches with new code combinations. Using electronic medical record review as the reference standard, we calculated case definition positive and negative predictive values, sensitivity, specificity, and AUROC. Electronic medical records were available for 671 admissions; 252 had confirmed AC and 195 NAC. Compared to previous AC definitions, the newly developed algorithm selecting for the AC code, alcohol-associated hepatic failure code, or alcohol use disorder code with a decompensated cirrhosis-related condition or NAC code provided the best overall positive predictive value (91%, 95% CI: 87-95), negative predictive value (89%, CI: 86-92), sensitivity (81%, CI: 76-86), specificity (96%, CI: 93-97), and AUROC (0.88, CI: 0.85-0.91). Comparing all evaluated NAC definitions, high sensitivity (92%, CI: 87-95), specificity (82%, CI: 79-86), negative predictive value (96%, CI: 94-98), AUROC (0.87, CI: 0.84-0.90), but relatively low positive predictive value (68%, CI: 62-74) were obtained by excluding alcohol use disorder codes and using either a NAC code in any diagnostic position or a primary diagnostic code for HCC, unspecified/chronic hepatic failure, esophageal varices without bleeding, or hepatorenal syndrome. New case definitions show enhanced accuracy for identifying hospitalizations for AC and NAC compared to previously used approaches.

Rapid Serial Visual Presentation of transposed-word sequences in the grammatical decision task: an examination of the roles of temporal and spatial cues to word order

ABSTRACT Transposing two words in a sentence (e.g. “cat” and “was” in “the white cat was big”) creates a sequence that is harder to classify as ungrammatical than control sequences (e.g. “the white was cat slowly”), suggesting that word position coding is noisy and can be affected by syntactic expectations. In the present research, this transposed-word effect was examined more closely using Rapid Serial Visual Presentation (RSVP) formats which provided either clear temporal cues to word order but no spatial cues, or both types of cues. Compared to when all words were presented simultaneously, the two RSVP formats reduced the transposed-word effect to the same degree while having no parallel impact on another ungrammatical comparison condition involving no transposition. These results are discussed in the context of serial and parallel models of reading as well as models that propose a later processing stage for the locus of the transposed-word effect.

Leakage detection of an acoustic emission pipeline based on an improved transformer network

Pipeline leakage detection is an integral part of pipeline integrity management. Combining AE (Acoustic Emission) with deep learning is currently the most commonly used method for pipeline leakage detection. However, this approach is usually applicable only to specific situations and requires powerful signal analysis and computational capabilities. To address these issues, this paper proposes an improved Transformer network model for diagnosing faults associated with abnormal working conditions in acoustic emission pipelines. First, the method utilizes the temporal properties of the GRU and the positional coding of the Transformer to capture and feature extract the data point sequence position information to suppress redundant information, and introduces the largest pooling layer into the Transformer model to alleviate the overfitting phenomenon. Second, while retaining the original attention learning mechanism and identity path in the original DRSN, a new soft threshold function is introduced to replace the ReLU activation function with a new threshold function, and a new soft threshold module and adaptive slope module are designed to construct the improved residual shrinkage unit (ASB-STRSBU), which is used to adaptively set the optimal threshold. Finally, pipeline leakage is classified. The experimental results show that the NDRSN model is able to make full use of global and local information when considering leakage signals and can automatically learn and acquire the important parameters of the input features in the spatial and channel domains. By optimizing the GRU improved Transformer network recognition model, the method significantly reduces the model training time and computational resource consumption while maintaining high leakage recognition accuracy. The average accuracy reached 93.97%. This indicates that the method has good robustness in acoustic emission pipeline leakage detection.

Beyond fixed sets: boundary conditions for obtaining SNARC-like effects with continuous semantic magnitudes.

Previous research has demonstrated the presence of an effect (i.e., the spatial-numerical association of response codes or SNARC) in both numerical parity and magnitude judgment tasks in which smaller numerical magnitudes are manually responded to faster on the left side and larger numerical magnitudes on the right side. Such a result has typically been attributed to a spatially based representation of numerical magnitude in long-term memory, the format of which has recently been postulated to be positional in line with learning of a canonically ordered number sequence. As a test of this view, in the current research, participants made classification judgments involving either the size (N = 88) or the living-nonliving status (N = 114) corresponding to the names of animals/objects etc. to which no learned canonical ordering of size exists. Names were taken from a very large set of 400 animals/objects etc. and each name was presented only once in an experimental session. Responses were made using left and right manual keypresses. In this work, the relation between response time and the relative sizes of the animals/objects did not differ across the left-right side of response indicating that SNARC-like effects did not occur. As such, the results suggest that space is not an inherent aspect of the long-term representation of magnitude in the brain and that some form of positional coding of magnitude is necessary for SNARC-like effects to occur.

Hybrid diffractive-refractive lens for chromatic confocal measurement system

A novel chromatic confocal measurement (CCM) method using a hybrid diffractive- refractive lens is presented. This hybrid diffractive- refractive lens is designed to optimize the linearity of chromatic dispersion and minimize the size of the optical system. The hybrid diffractive- refractive lens is fabricated by etching a diffractive surface onto a quartz aspheric lens through lithography, which combines the high numerical aperture (NA) of a refractive lens with the unique dispersion properties of the diffractive optical elements (DOE). The lens is incorporated as a dispersive objective lens in a CCM experimental system. The system has a measurement range of 514.8 µm, calibrated using a laser displacement interferometer. The experimental results show that the wavelength-to-axial position coding of the CCM system achieves high linearity (R2= 0.9999) in the working wavelength range (500-700 nm). The system has an axial resolution of 0.08 µm and a displacement measurement nonlinear error of less than 2.05 µm.

Segmentation of liver CT images based on weighted medical transformer model.

Deep convolutional neural networks have made significant strides in the field of medical image segmentation. Although existing convolutional structures enhance performance by leveraging local image information, they often lose the interdependence information between contexts. Therefore, the article utilizes the multi-attention mechanism of the Transformer structure to more comprehensively express relationships between contexts and introduced the Transformer network architecture into the field of medical image segmentation. Most models based on this Transformer structure typically require large datasets for training. However, in the medical field, the limited size of datasets makes training models with the Transformer structure challenging. To address this, the article propose a Weighted Medical Transformer (WMT) model that imposes low requirements on dataset quantity. The weighting mechanism in the WMT model aims to improve the issue of inaccurate relative positional coding when dealing with small medical datasets. Additionally, a coarse-grained and fine-grained segmentation mechanism is introduced, focusing on both the detailed aspects within image blocks and the boundary information connecting blocks. Experimental results on a liver dataset demonstrate that the model achieves F1 and IoU scores of 88.48% and 79.41%, respectively. Results on the MoNuSeg dataset show comparable high F1 and IoU scores of 79.58% and 66.19%, respectively. The model's accuracy surpasses that of U-Net++ and U-Net models. Compared to other models, this approach is applicable to scenarios with limited datasets, exhibiting high execution efficiency and accuracy.

Electrophysiological Properties of the Medial Mammillary Bodies across the Sleep-Wake Cycle.

The medial mammillary bodies (MBs) play an important role in the formation of spatial memories; their dense inputs from hippocampal and brainstem regions makes them well placed to integrate movement-related and spatial information, which is then extended to the anterior thalamic nuclei and beyond to the cortex. While the anatomical connectivity of the medial MBs has been well studied, much less is known about their physiological properties, particularly in freely moving animals. We therefore carried out a comprehensive characterization of medial MB electrophysiology across arousal states by concurrently recording from the medial MB and the CA1 field of the hippocampus in male rats. In agreement with previous studies, we found medial MB neurons to have firing rates modulated by running speed and angular head velocity, as well as theta-entrained firing. We extended the characterization of MB neuron electrophysiology in three key ways: (1) we identified a subset of neurons (25%) that exhibit dominant bursting activity; (2) we showed that ∼30% of theta-entrained neurons exhibit robust theta cycle skipping, a firing characteristic that implicates them in a network for prospective coding of position; and (3) a considerable proportion of medial MB units showed sharp-wave ripple (SWR) responsive firing (∼37%). The functional heterogeneity of MB electrophysiology reinforces their role as an integrative node for mnemonic processing and identifies potential roles for the MBs in memory consolidation through propagation of SWR-responsive activity to the anterior thalamus and prospective coding in the form of theta cycle skipping.

Repetition Suppression Reveals Cue-Specific Spatial Representations for Landmarks and Self-Motion Cues in the Human Retrosplenial Cortex.

The efficient use of various spatial cues within a setting is crucial for successful navigation. Two fundamental forms of spatial navigation, landmark-based and self-motion-based, engage distinct cognitive mechanisms. The question of whether these modes invoke shared or separate spatial representations in the brain remains unresolved. While nonhuman animal studies have yielded inconsistent results, human investigation is limited. In our previous work (Chen et al., 2019), we introduced a novel spatial navigation paradigm utilizing ultra-high field fMRI to explore neural coding of positional information. We found that different entorhinal subregions in the right hemisphere encode positional information for landmarks and self-motion cues. The present study tested the generalizability of our previous finding with a modified navigation paradigm. Although we did not replicate our previous finding in the entorhinal cortex, we identified adaptation-based allocentric positional codes for both cue types in the retrosplenial cortex (RSC), which were not confounded by the path to the spatial location. Crucially, the multi-voxel patterns of these spatial codes differed between the cue types, suggesting cue-specific positional coding. The parahippocampal cortex exhibited positional coding for self-motion cues, which was not dissociable from path length. Finally, the brain regions involved in successful navigation differed from our previous study, indicating overall distinct neural mechanisms recruited in our two studies. Taken together, the current findings demonstrate cue-specific allocentric positional coding in the human RSC in the same navigation task for the first time and that spatial representations in the brain are contingent on specific experimental conditions.

Chinese text dual attention network for aspect-level sentiment classification.

English text has a clear and compact subject structure, which makes it easy to find dependency relationships between words. However, Chinese text often conveys information using situational settings, which results in loose sentence structures, and even most Chinese comments and experimental summary texts lack subjects. This makes it challenging to determine the dependency relationship between words in Chinese text, especially in aspect-level sentiment recognition. To solve this problem faced by Chinese text in the field of sentiment recognition, a Chinese text dual attention network for aspect-level sentiment recognition is proposed. First, Chinese syntactic dependency is proposed, and sentiment dictionary is introduced to quickly and accurately extract aspect-level sentiment words, opinion extraction and classification of sentimental trends in text. Additionally, in order to extract context-level features, the CNN-BILSTM model and position coding are also introduced. Finally, to better extract fine-grained aspect-level sentiment, a two-level attention mechanism is used. Compared with ten advanced baseline models, the model's capabilities are being further optimized for better performance, with Accuracy of 0.9180, 0.9080 and 0.8380 respectively. This method is being demonstrated by a vast array of experiments to achieve higher performance in aspect-level sentiment recognition in less time, and ablation experiments demonstrate the importance of each module of the model.

On the distinction between position and order information when processing strings of characters.

To probe the processing of gaze-dependent positional information and gaze-independent order information when matching strings of characters, we compared effects of visual similarity (hypothesized to affect gaze-centered position coding) with the effects of character transpositions (hypothesized to affect the processing of gaze-independent order information). In Experiment 1, we obtained empirical measures of visual similarity for pairs of characters, separately for uppercase consonants and keyboard symbols. These similarity values were then used in Experiment 2 to create pairs of four-character stimuli (four letters or four symbols) that could differ by substituting one character with a different character from the same category that was visually similar (e.g., FJDK-FJBK) or dissimilar (e.g., FJVK-FJBK). We also compared the effects of transposing two characters (e.g., FBJK-FJBK) with substituting two characters (e.g., FHSK-FJBK). "Different" responses were harder to make in the single substitution condition when the substituted character was visually similar, and this effect was not conditioned by character type. On the other hand, transposition costs (i.e., greater difficulty in detecting a difference with transpositions compared with double substitutions) were greater for letters compared with symbols. We conclude that visual similarity mainly affects the generic gaze-dependent processing of complex visual features, and that the encoding of letter order involves a mechanism that is specific to reading.

No position-specific interference from prior lists in cued recognition: A challenge for position coding (and other) theories of serial memory

Position-specific intrusions of items from prior lists are rare but important phenomena that distinguish broad classes of theory in serial memory. They are uniquely predicted by position coding theories, which assume items on all lists are associated with the same set of codes representing their positions. Activating a position code activates items associated with it in current and prior lists in proportion to their distance from the activated position. Thus, prior list intrusions are most likely to come from the coded position. Alternative “item dependent” theories based on associations between items and contexts built from items have difficulty accounting for the position specificity of prior list intrusions. We tested the position coding account with a position-cued recognition task designed to produce prior list interference. Cuing a position should activate a position code, which should activate items in nearby positions in the current and prior lists. We presented lures from the prior list to test for position-specific activation in response time and error rate; lures from nearby positions should interfere more. We found no evidence for such interference in 10 experiments, falsifying the position coding prediction. We ran two serial recall experiments with the same materials and found position-specific prior list intrusions. These results challenge all theories of serial memory: Position coding theories can explain the prior list intrusions in serial recall and but not the absence of prior list interference in cued recognition. Item dependent theories can explain the absence of prior list interference in cued recognition but cannot explain the occurrence of prior list intrusions in serial recall.