Training Sequence Research Articles

Modeling Continuous Motion for 3D Point Cloud Object Tracking

The task of 3D single object tracking (SOT) with LiDAR point clouds is crucial for various applications, such as autonomous driving and robotics. However, existing approaches have primarily relied on appearance matching or motion modeling within only two successive frames, thereby overlooking the long-range continuous motion property of objects in 3D space. To address this issue, this paper presents a novel approach that views each tracklet as a continuous stream: at each timestamp, only the current frame is fed into the network to interact with multi-frame historical features stored in a memory bank, enabling efficient exploitation of sequential information. To achieve effective cross-frame message passing, a hybrid attention mechanism is designed to account for both long-range relation modeling and local geometric feature extraction. Furthermore, to enhance the utilization of multi-frame features for robust tracking, a contrastive sequence enhancement strategy is proposed, which uses ground truth tracklets to augment training sequences and promote discrimination against false positives in a contrastive manner. Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art method by significant margins on multiple benchmarks.

Re-evaluation of the methodology for estimating the US specialty physician workforce.

Increasing pursuit of subspecialized training has quietly revolutionized physician training, but the potential impact on physician workforce estimates has not previously been recognized. The Physicians Specialty Data Reports of the Association of American Medical Colleges, derived from specialty designations in the American Medical Association (AMA) Physician Professional Data (PPD), are the reference source for US physician workforce estimates; by 2020, the report for pathologists was an undercount of 39% when compared with the PPD. Most of the difference was due to the omission of pathology subspecialty designations. The rest resulted from reliance on only the first of the AMA PPD's 2 specialty data fields. Placement of specialty designation in these 2 fields is sensitive to sequence of training and is thus affected by multiple or intercalated (between years of residency training) fellowships. Both these phenomena have become progressively more common and are not unique to pathology. Our findings demonstrate the need to update definitions and methodology underlying estimates of the US physician workforce for pathology and suggest a like need in other specialties affected by similar trends.

Flattening the curve-How to get better results with small deep-mutational-scanning datasets.

Proteins are used in various biotechnological applications, often requiring the optimization of protein properties by introducing specific amino-acid exchanges. Deep mutational scanning (DMS) is an effective high-throughput method for evaluating the effects of these exchanges on protein function. DMS data can then inform the training of a neural network to predict the impact of mutations. Most approaches use some representation of the protein sequence for training and prediction. As proteins are characterized by complex structures and intricate residue interaction networks, directly providing structural information as input reduces the need to learn these features from the data. We introduce a method for encoding protein structures as stacked 2D contact maps, which capture residue interactions, their evolutionary conservation, and mutation-induced interaction changes. Furthermore, we explored techniques to augment neural network training performance on smaller DMS datasets. To validate our approach, we trained three neural network architectures originally used for image analysis on three DMS datasets, and we compared their performances with networks trained solely on protein sequences. The results confirm the effectiveness of the protein structure encoding in machine learning efforts on DMS data. Using structural representations as direct input to the networks, along with data augmentation and pretraining, significantly reduced demands on training data size and improved prediction performance, especially on smaller datasets, while performance on large datasets was on par with state-of-the-art sequence convolutional neural networks. The methods presented here have the potential to provide the same workflow as DMS without the experimental and financial burden of testing thousands of mutants. Additionally, we present an open-source, user-friendly software tool to make these data analysis techniques accessible, particularly to biotechnology and protein engineering researchers who wish to apply them to their mutagenesis data.

Original article Formalization of the task of developing wartime curricula and programs

Managing higher education involves solving a number of problems that are difficult to formalize. Such tasks include ensuring consistency of the logical-time sequence of training in real conditions of continuous improvement of documents regulating the educational process (federal state educational standards, professional standards, approximate basic educational programs, curricula, basic professional educational programs, work programs of disciplines, indicators of achievement of competencies and so on.).Increasing the efficiency of higher education management is associated with the introduction of specialized information systems into the educational process of a university, designed to automate basic processes and tasks.The range of tasks solved by the higher education management system is also influenced by the actual sociopolitical situation. In modern conditions, it seems relevant to resolve the issue of developing curricula for shortened training or wartime plans in certain areas of professional activity.Works have been published in which formalized methods have been developed to ensure consistency of the logical-time sequence of learning in the higher education system. These methods create favorable preconditions for their further improvement and application in solving related problems.The applicability of graph theory methods for a formalized solution to the problem of developing wartime curricula and programs is shown.

Symbol cyclic shift-based DFT precoding for OTFS-VLC systems

In this paper, a symbol cyclic shift-based discrete Fourier transform (SCS-DFT) precoding for orthogonal time frequency space (OTFS) is proposed and experimentally demonstrated in the visible light communication (VLC) system. The proposed scheme effectively mitigates the channel fading effect of edge subcarriers caused by inter-symbol interference (ISI) when there is no cyclic prefix (CP) in OTFS. In addition, only a training sequence (TS) symbol can realize channel estimation and equalization, and it can reduce the implementation complexity. The experimental results demonstrated that, compared with the conventional OTFS modulation, the proposed SCS-DFT OTFS can enhance the transmission performance while maintaining the high spectral efficiency of the VLC system.

Deepstacked-AVPs: predicting antiviral peptides using tri-segment evolutionary profile and word embedding based multi-perspective features with deep stacking model

BackgroundViral infections have been the main health issue in the last decade. Antiviral peptides (AVPs) are a subclass of antimicrobial peptides (AMPs) with substantial potential to protect the human body against various viral diseases. However, there has been significant production of antiviral vaccines and medications. Recently, the development of AVPs as an antiviral agent suggests an effective way to treat virus-affected cells. Recently, the involvement of intelligent machine learning techniques for developing peptide-based therapeutic agents is becoming an increasing interest due to its significant outcomes. The existing wet-laboratory-based drugs are expensive, time-consuming, and cannot effectively perform in screening and predicting the targeted motif of antiviral peptides.MethodsIn this paper, we proposed a novel computational model called Deepstacked-AVPs to discriminate AVPs accurately. The training sequences are numerically encoded using a novel Tri-segmentation-based position-specific scoring matrix (PSSM-TS) and word2vec-based semantic features. Composition/Transition/Distribution-Transition (CTDT) is also employed to represent the physiochemical properties based on structural features. Apart from these, the fused vector is formed using PSSM-TS features, semantic information, and CTDT descriptors to compensate for the limitations of single encoding methods. Information gain (IG) is applied to choose the optimal feature set. The selected features are trained using a stacked-ensemble classifier.ResultsThe proposed Deepstacked-AVPs model achieved a predictive accuracy of 96.60%%, an area under the curve (AUC) of 0.98, and a precision-recall (PR) value of 0.97 using training samples. In the case of the independent samples, our model obtained an accuracy of 95.15%, an AUC of 0.97, and a PR value of 0.97.ConclusionOur Deepstacked-AVPs model outperformed existing models with a ~ 4% and ~ 2% higher accuracy using training and independent samples, respectively. The reliability and efficacy of the proposed Deepstacked-AVPs model make it a valuable tool for scientists and may perform a beneficial role in pharmaceutical design and research academia.

Self-paced semi-supervised feature selection with application to multi-modal Alzheimer’s disease classification

Semi-supervised multi-modal learning has attracted much attention due to the expense and scarcity of data labels, especially in disease diagnosis field. Most existing methods follow the paradigm by iteratively inferring the pseudo-labels of unlabeled data and add them into training sequence, but they ignore the reliability of those pseudo-labels, where inaccurate and wrong supervision will lead to negative influence on model learning. In this paper, we propose a Self-paced Semi-supervised Multi-modal Feature Selection (SSMFS) method, and apply it to Alzheimer’s disease classification. Specifically, SSMFS projects multi-modal biomedical data into the common label space with discriminative feature selection. Under the guidance of prior multi-modal similarity graphs, a unified graph is adaptively learned and embedded to preserve the neighborhood structures. More importantly, SSMFS dynamically investigates the discriminability and credibility of pseudo-labels, and adaptively assigns a weight to each unlabeled sample via self-paced learning such that the negative influence of wrong supervision can be reduced. Finally, a multi-kernel support vector machine is used to fuse the selected multi-modal features for final disease prediction. Experimental results on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate the effectiveness of our method.

Submillimeter balanced SSFP BOLD-functional MRI accelerated with 3D stack-of-spirals at 9.4 T.

This work aims to improve the speed of balanced SSFP (bSSFP) acquisition with segmented 3D stack-of-spirals for functional brain studies at ultrahigh field. Functional experiments were performed with an accelerated 3D stack-of-spirals sequence with water excitation for fat suppression. The resulting data were reconstructed using an iterative algorithm with corrections for system imperfections such as trajectory deviations and B0 inhomogeneity. In the first set of experiments, we evaluated the signal change and stability with respect to echo and TR for a full-field checkerboard stimulus. To demonstrate the high spatio-temporal resolution of the developed method, the results of three optimized protocols at submillimeter resolution (0.6-mm isotropic and 0.8-mm isotropic) and at 1.2 mm isotropic resolution for whole-brain coverage were shown. Water excitation and the model-based iterative reconstruction improved image quality. The BOLD-related signal changes increased with longer TE and longer TR. We observed an increase in thermal noise performance at lower TE and higher TR. However, signal stability deteriorates at higher TE and TR. Therefore, optimized protocols used shorter TE and moderately long TR to maximize the sensitivity and speed. Reproducible activations were detected along the gray-matter gyri in the submillimeter protocols with a median signal change of approximately 4% across subjects. Three-dimensional stack-of-spirals enables passband balanced SSFP functional imaging at a much higher spatial and temporal scale, compared with conventional spoiled gradient-echo train sequences.

Maximizing the Downlink Data Rates in Massive Multiple Input Multiple Output with Frequency Division Duplex Transmission Mode Using Power Allocation Optimization Method with Limited Coherence Time

The expected development of the future generation of wireless communications systems such as 6G aims to achieve an ultrareliable and low-latency communications (URLLCs) while maximizing the data rates. These requirements push research into developing new advanced technologies. To this end, massive multiple input multiple output (MMIMO) is introduced as a promising transmission approach to fulfill these requirements. However, maximizing the downlink-achievable sum rate (DASR) in MMIMO with a frequency division duplex (FDD) transmission mode and limited coherence time (LCT) is very challenging. To address this challenge, this paper proposes a DASR maximization approach using a feasible power allocation optimization method. The proposed approach is based on smartly allocating the total transmit power between the data transmission and training sequence transmission for channel estimation. This can be achieved by allocating more energy to the training signal than the data transmission during the channel estimation process to improve the quality of channel estimation without compromising more training sequence length, thus maximizing the DASR. Additionally, the theory of random matrix approach is exploited to derive an asymptotic closed-form expression for the DASR with a regularized zero-forcing precoder (RZFP), which allows the power optimization process to be achieved without the need for computationally complex Monte Carlo simulations. The results provided in this paper indicate that a considerable enhancement in the DASR performance is achieved using the proposed power allocation method in comparison with the conventional uniform power allocation method.

Comparative study to determine the proper sequence of simulation training, pelvic trainer versus virtual reality simulator: a pilot study

BackgroundIncreased surgical efficacy has led to a remarkable increase in the usage of minimally invasive surgical procedures since their inception. The use of simulation in surgical teaching has grown significantly during the past 10 years. Several laparoscopic simulators have been built. Virtual reality (VR) simulators and box trainers (BTs), often known as pelvic trainers, are the two primary training modalities used in hospitals and clinical training institutes for the development and acquisition of laparoscopic skills. Our study aimed to evaluate the proper sequence of pelvic trainers and VR simulator training to improve laparoscopic gynecological skills.MethodologyWe carried out this pilot study at the Virtual Endoscopic Simulation and Skills Acquisition Laboratory at the Obstetrics and Gynecology Department in the Kasr Al Ainy Hospital, Faculty of Medicine, Cairo University, Egypt, from February to August 2022. All residents with minimal or without laparoscopic experience (twenty residents) were divided into two groups and classified as (group A versus group B). Group A’s training began with a pelvic trainer, which was tested using a checklist. Later, the group trained on a virtual reality simulator, which tested them using an electronic autoassessment. After training on a virtual reality simulator and passing an electronic autoassessment test, group B moved on to pelvic trainers and had a checklist-based assessment.ResultsWe compared pelvic trainer tasks between the training groups, and detected no significant differences in camera navigation, cutting pattern, peg transfer, or running stitches (P values 0.646, 0.341, 0.179, and 0.939 respectively); when we compared VR simulator tasks between the training groups, there were no significant differences in camera navigation, cutting pattern, peg transfer, or running stitches (P values 0.79, 0.3, 0.33, and 0.06, respectively).ConclusionThere was no difference in training, between residents who started on a pelvic trainer or the VR simulator; therefore, both could be used in laparoscopic training with no preferred order.Trial registrationThe trial was registered at clinicaltrials.gov with the name “Pelvic trainer vs VRS” and the identifier “NCT05255614.” The registration date was January 19, 2022, and the trial was prospectively registered. URL: https://register.clinicaltrials.gov/prs/app/action/ViewOrUnrelease?uid=U0004GED&ts=22&sid=S000BR5D&cx=t6mc14

A novel IQ mismatch compensation method in wideband direct-conversion receivers.

Direct-conversion receivers (DCRs) have been widely used in recent years due to their small size and low power consumption. However, the mismatch between the in-phase (I) and the quadrature (Q) branches will seriously affect the performance of the DCRs. This paper proposes a novel blind compensation method to suppress the interference introduced by IQ mismatch. Based on the Hilbert transform, our proposed method can obtain the orthogonal signal of the I-channel signal by utilizing the Weaver architecture. Compared with traditional compensation methods, the main difference of the proposed method is that it ignores prior information, training sequences, and additional hardware circuits. Furthermore, the complexity of the proposed blind compensation method is low because no iterative operations are involved in the compensation process. The simulation results show that the proposed method has an excellent compensation performance, especially in wideband applications.

Neural activity of retinal ganglion cells under continuous, dynamically-modulated high frequency electrical stimulation

Objective. Current retinal prosthetics are limited in their ability to precisely control firing patterns of functionally distinct retinal ganglion cell (RGC) types. The aim of this study was to characterise RGC responses to continuous, kilohertz-frequency-varying stimulation to assess its utility in controlling RGC activity. Approach. We used in vitro patch-clamp experiments to assess electrically-evoked ON and OFF RGC responses to frequency-varying pulse train sequences. In each sequence, the stimulation amplitude was kept constant while the stimulation frequency (0.5–10 kHz) was changed every 40 ms, in either a linearly increasing, linearly decreasing or randomised manner. The stimulation amplitude across sequences was increased from 10 to 300 µA. Main results. We found that continuous stimulation without rest periods caused complex and irreproducible stimulus-response relationships, primarily due to strong stimulus-induced response adaptation and influence of the preceding stimulus frequency on the response to a subsequent stimulus. In addition, ON and OFF populations showed different sensitivities to continuous, frequency-varying pulse trains, with OFF cells generally exhibiting more dependency on frequency changes within a sequence. Finally, the ability to maintain spiking behaviour to continuous stimulation in RGCs significantly reduced over longer stimulation durations irrespective of the frequency order. Significance. This study represents an important step in advancing and understanding the utility of continuous frequency modulation in controlling functionally distinct RGCs. Our results indicate that continuous, kHz-frequency-varying stimulation sequences provide very limited control of RGC firing patterns due to inter-dependency between adjacent frequencies and generally, different RGC types do not display different frequency preferences under such stimulation conditions. For future stimulation strategies using kHz frequencies, careful consideration must be given to design appropriate pauses in stimulation, stimulation frequency order and the length of continuous stimulation duration.

An Optimized Sequence for Sparse Channel Estimation in a 5G MIMO System

ABSTRACT Recently, the massive Multiple-Input Multiple-Output (MIMO) system has been integrated with machine learning approaches to realise automatic channel state information detection. These methods need high computational complexity due to the lack of optimal training sequences and the failure to control the sparsity assembly based on massive MIMO. Moreover, it does not consider optimised training sequences based on a compressive sensing approach in the presence of contamination. Thus, this paper proposed an enhanced deep learning model with an optimised pilot training sequence for sparse channel estimation in a 5 G massive MIMO system. Initially, the system and sparse block channel impulse response model will be constructed by considering the time-domain synchronous Generalized Frequency Division Multiplexing (GFDM) system and additive white Gaussian noise (AWGN). In the proposed work, a seagull optimisation algorithm is developed to design the training pilot sequence depending on the coherence properties of the sensing matrix, which is used to recover the channel impulse response. Then, the sparse massive MIMO channel is evaluated by proposing a new deep channel estimator with an enhanced stacked auto encoder (D-ESAE). The proposed channel MSE is 10−6, the bit error rate is 0.04, and NMSE is 10−9 at 40 dB SNR through Matlab simulation.

Integration of physics-based data in deep learning model training for predicting the effect of sulfur content in the directed energy deposition process

The training of a machine learning model solely on experimental data, encompassing both pre- and post-process information, can reveal the general relationship of the directed energy deposition process. However, models trained in this manner encounter limitations in capturing critical in-process information occurring during deposition. This paper details the training of a deep learning model through the integration of in-process physics-based simulation information and a pre-process experiment dataset. The sulfur content of stainless steel 316L was selected as critical in-process information affecting the final track geometry and was captured using computational fluid dynamics simulation of a single-track deposition process, which cannot be captured accurately through experimentation. The physics-based simulation dataset was generated by obtaining the contour of deposition and dilution of the solidified track cross-section. The experiment was conducted using central composite design, and data augmentation was achieved through curve fitting using a response surface methodology regression model. Statistical analysis assessing the quality of simulation and experiment data was conducted. Among six baseline models, a deep learning model with a specified training sequence of experiment and simulation data, denoted as DL-AugExp-Sim-Exp, exhibited the best-performing R2 and root mean square error prediction accuracy for cross-section track shape. Notably, deep learning models trained with both experiment and simulation information demonstrated a lower R2 value compared to models trained solely with experiment data, revealing a tradeoff between R2 value and additional prediction capability. In summary, in this study, the integration of a physics-based simulation dataset demonstrated the additional prediction capability concerning the effect of sulfur content on track geometry.

Highly secure non-orthogonal multiple access based on key accompanying transmission in training sequence.

This paper proposes a high-security chaotic encrypted power sparse coding division (CE-PSCD) scheme for 7-core fiber based on non-orthogonal multiple access (NOMA) technology. The method utilizes power multiplexing to realize parallel transmission of two signals. Joint encryption of the four-dimensional region is realized using constellation mapping encryption, carrier frequency encryption, symbol scrambling, and sparse code scrambling. What we believe to be a new dimension for encryption of autonomously designed sparse codes is proposed. Meanwhile, we hide the chaotic key in training sequence (TS) to realize the co-transmission of the key and the message. A 70 Gb/s CE-PSCD signal transmission over 2 km of 7-core fiber is demonstrated experimentally. At the limit of forward error correction (FEC) ∼3.8 × 10-3, the difference in the encrypted sensitivity among different users at the equal power level is 0.36 dB, which means that the fairness of users will not be destroyed. The key space can reach 10134, with a bit error rate (BER) of about 0.5 for brute-force cracking at illegal receivers. As long as the key bits in the hidden TS are wrong by one bit, the BER stays around 0.5. The results show no significant attenuation of the signal before and after encryption at either high or low power, verifying the high-security performance of our proposed scheme.

Extensive antibody search with whole spectrum black-box optimization

In designing functional biological sequences with machine learning, the activity predictor tends to be inaccurate due to shortage of data. Top ranked sequences are thus unlikely to contain effective ones. This paper proposes to take prediction stability into account to provide domain experts with a reasonable list of sequences to choose from. In our approach, multiple prediction models are trained by subsampling the training set and the multi-objective optimization problem, where one objective is the average activity and the other is the standard deviation, is solved. The Pareto front represents a list of sequences with the whole spectrum of activity and stability. Using this method, we designed VHH (Variable domain of Heavy chain of Heavy chain) antibodies based on the dataset obtained from deep mutational screening. To solve multi-objective optimization, we employed our sequence design software MOQA that uses quantum annealing. By applying several selection criteria to 19,778 designed sequences, five sequences were selected for wet-lab validation. One sequence, 16 mutations away from the closest training sequence, was successfully expressed and found to possess desired binding specificity. Our whole spectrum approach provides a balanced way of dealing with the prediction uncertainty, and can possibly be applied to extensive search of functional sequences.

AI Can Identify Solar System Instability Billions of Years in Advance

Rare event schemes require an approximation of the probability of the rare event as a function of system state. Finding an appropriate reaction coordinate is typically the most challenging aspect of applying a rare event scheme. Here we develop an artificial intelligence (AI) based reaction coordinate that effectively predicts which of a limited number of simulations of the solar system will go unstable using a convolutional neural network classifier. The performance of the algorithm does not degrade significantly even 3.5 billion years before the instability. We overcome the class imbalance intrinsic to rare event problems using a combination of minority class oversampling, increased minority class weighting, and pulling multiple non-overlapping training sequences from simulations. Our success suggests that AI may provide a promising avenue for developing reaction coordinates without detailed theoretical knowledge of the system.

Supplemental Material for General Knowledge and Detailed Memory Benefit From Different Training Sequences

Supplemental Material for General Knowledge and Detailed Memory Benefit From Different Training Sequences

Training Sequences Design for Simultaneously Transceiver IQ Skew Estimation in Coherent Systems

Training Sequences Design for Simultaneously Transceiver IQ Skew Estimation in Coherent Systems

Federated Cross-Incremental Self-Supervised Learning for Medical Image Segmentation.

Federated cross learning has shown impressive performance in medical image segmentation. However, it encounters the catastrophic forgetting issue caused by data heterogeneity across different clients and is particularly pronounced when simultaneously facing pixelwise label deficiency problem. In this article, we propose a novel federated cross-incremental self-supervised learning method, coined FedCSL, which not only can enable any client in the federation incrementally yet effectively learn from others without inducing knowledge forgetting or requiring massive labeled samples, but also preserve maximum data privacy. Specifically, to overcome the catastrophic forgetting issue, a novel cross-incremental collaborative distillation (CCD) mechanism is proposed, which distills explicit knowledge learned from previous clients to subsequent clients based on secure multiparty computation (MPC). Besides, an effective retrospect mechanism is designed to rearrange the training sequence of clients per round, further releasing the power of CCD by enforcing interclient knowledge propagation. In addition, to alleviate the need of large-scale densely annotated pretraining medical datasets, we also propose a two-stage training framework, in which federated cross-incremental self-supervised pretraining paradigm first extracts robust yet general image-level patterns across multi-institutional data silos via a novel round-robin distributed masked image modeling (MIM) pipeline; then, the resulting visual concepts, e.g., semantics, are transferred to the federated cross-incremental supervised fine-tuning paradigm, favoring various cross-silo medical image segmentation tasks. The experimental results on public datasets demonstrate the effectiveness of the proposed method as well as the consistently superior performance of our method over most state-of-the-art methods quantitatively and qualitatively.