In this work, we consider the H<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub>-optimal model reduction of dynamical systems that are linear in the state equation with up to a quadratic nonlinearity in the output equation. As our primary theoretical contributions, we derive gradients of the squared H<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> system error with respect to the reduced model quantities and, from the stationary points of these gradients, introduce Gramian based first-order necessary conditions for the H<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub>-optimal approximation of a linear quadratic output (LQO) system. Theresulting H<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub>-optimality framework generalizes the anal ogous Gramian-based optimality framework for purely linear systems. Computationally, we demonstrate how to en force the necessary optimality conditions using the Petrov Galerkin projection; the corresponding projection matrices are derived from a pair of Sylvester equations. Based on this result, we propose an iteratively corrected algorithm for the H<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub>-optimal model reduction of LQO systems, which werefer to asthelinear quadratic output two-sided iteration algorithm (LQO-TSIA). Numerical examples are included to illustrate the effectiveness of the proposed computational method against other existing approaches.

The high memory demands of the Key-Value (KV) Cache during the inference of Large Language Models (LLMs) severely restrict their deployment in resource-constrained platforms. Quantization can effectively alleviate the memory pressure caused by KV Cache. However, existing methods either rely on static one-size-fits-all precision allocation or fail to dynamically prioritize critical KV in long-context tasks, forcing memory-accuracy-throughput tradeoffs. In this work, we propose a novel mixed-precision quantization method for KV Cache named KVmix. KVmix leverages gradient-based importance analysis to evaluate how individual Key and Value projection matrices affect the model loss, enabling layer-specific bit-width allocation for mix-precision quantization. It dynamically prioritizes higher precision for important layers while aggressively quantizing less influential ones, achieving a tunable balance between accuracy and efficiency. KVmix introduces a dynamic long-context optimization strategy that adaptively keeps full-precision KV pairs for recent pivotal tokens and compresses older ones, achieving high-quality sequence generation with low memory usage. Additionally, KVmix provides efficient low-bit quantization and CUDA kernels to optimize computational overhead. On LLMs such as Llama and Mistral, KVmix achieves near-lossless inference performance with extremely low quantization configuration (Key 2.19bit Value 2.38bit), while delivering a remarkable 4.9× memory compression and a 5.3× speedup in inference throughput.

With the increasing demand for both accuracy and efficiency in transient electromagnetic (TEM) simulations, conventional 3-D forward modeling methods face growing challenges. This study presents a high-accuracy and high-efficiency 3-D forward modeling approach that combines the spectral-element method (SEM) with a model order reduction (MOR) scheme. High-order orthogonal basis functions are employed, and the computational domain is discretized in a finite-element manner to improve modeling accuracy. During element-level analysis, a reduced-integration strategy is introduced to enhance the sparsity of the double-curl and conductivity matrices, thereby reducing the computational time and memory consumption required for matrix assembly. For temporal treatment, a shift-and-invert Krylov (SAI-Krylov) subspace algorithm is adopted: the basis and projection matrices are constructed using only one matrix factorization and tens of back-substitutions, after which low-dimensional matrix exponential functions are evaluated to efficiently obtain electromagnetic responses at arbitrary times. Comparisons with other numerical methods demonstrate the superior efficiency and accuracy of the proposed approach. Finally, simulations on a 3-D sulfide ore-body model are performed to investigate TEM field propagation for both galvanic and loop sources, confirming the capability of the method to model electromagnetic responses in complex geological settings.

Mathematical models that describe sequence-function relationships are widely used in computational biology. A key challenge when interpreting these models is that their parameters are not uniquely determined: many different parameter choices can encode the same sequence-function landscape. These ambiguities, known as “gauge freedoms,” must be resolved before parameter values can be meaningfully interpreted. Resolving gauge freedoms requires imposing mathematical constraints on parameters that remove these degrees of freedom, a procedure called “fixing the gauge.” We recently developed mathematical methods for fixing the gauge of a large class of commonly used models, but the direct computational implementation of these methods is often impractical due to the need for projection matrices whose memory requirements scale quadratically with the number of parameters. Here we introduce GaugeFixer, a Python package that exploits the specific mathematical structure of gauge-fixing projections to achieve linear scaling, thus enabling application to models with millions of parameters. To demonstrate GaugeFixer, we analyze the local structure of peaks in an empirical fitness landscape for translation initiation. GaugeFixer reveals striking similarities, but also fine-scaled variation, in ribosome binding preferences at different positions relative to the start codon, thereby facilitating the interpretation of an otherwise unwieldy fitness landscape. GaugeFixer thus fills an unmet need in the computational tools available for biologically interpreting sequence-function relationships.

Abstract This paper investigates model order reduction (MOR) methods for parametric bilinear systems using Walsh functions. First, the coefficient matrices are expanded via Taylor series to transform the system considered into a polynomial parametric system. The resulting system is then expanded using Walsh functions, where the bilinear terms are effectively handled, leading to a generalized Sylvester equation. Then, LU factorization combines with a hybrid iterative strategy to obtain the solution of the generalized Sylvester equation, which can significantly enhance the convergence rate of the generalized minimal residual method and reduce computational costs. Finally, orthogonal projection matrices are constructed from the obtained expansion coefficients to generate the parametric reduced‐order systems. Theoretical analysis shows that the reduced‐order systems can match the first several expansion coefficients of the output of the original system. Numerical experiments demonstrate the feasibility and effectiveness of the proposed methods.

The local population structures of two short-lived perennial species, Androsace albana and Eritrichium caucasicum, classified by ontogenetic stages were observed annually for 15 years (2009–2023) at permanent sites in the alpine belt of the Northwest Caucasus. The uniquely long series of these data made it possible to discover the effects of vegetation dormancy in the life cycle of a short-lived species, which was fundamentally impossible with short series of about three to five years. Data of the “identified individuals” (A. albana) and “identified individuals from unknown parents” (E. caucasicum) types enable us to calibrate the corresponding matrix models of discrete-structured population dynamics and obtain the so-called annual population projection matrices (PPMs). The analysis of PPMs by mathematical means yields various quantitative characteristics of the monitored object, in particular, the viability measure of the local population. However, the revealed effects of vegetation dormancy make changes to the data series and raise the issue to revise the previous models and ensued characteristics. We show that including an additional state of death or vegetation dormancy into the life cycle, which is quite a logical move from the viewpoint of the model, does not make any sense in the task of assessing the population viability. When adjusted to fit the revised data, the calibration procedure does naturally increase the previous estimates of the viability measure, thereby confirming an important role that the vegetation dormancy plays as a mechanism to adapt the plant to a stressful environment.

Text-to-image (T2I) generation, a core component of generative artificial intelligence(AI), is increasingly important for creative industries and human–computer interaction. Despite impressive progress in realism and diversity, diffusion models still exhibit critical security blind spots particularly in the Transformer key-value mapping mechanism that underpins cross-modal alignment. Existing backdoor attacks often rely on large-scale data poisoning or extensive fine-tuning, leading to low efficiency and limited stealth. To address these challenges, we propose two efficient backdoor attack methods AttnBackdoor and SemBackdoor grounded in the Transformer’s key-value storage principle. AttnBackdoor injects precise mappings between trigger prompts and target instances by fine-tuning the key-value projection matrices in U-Net cross-attention layers (≈5% of parameters). SemBackdoor establishes semantic-level mappings by editing the text encoder’s MLP projection matrix (≈0.3% of parameters). Both approaches achieve high attack success rates (&gt;90%), with SemBackdoor reaching 98.6% and AttnBackdoor 97.2%. They also reduce parameter updates and training time by 1–2 orders of magnitude compared to prior work while preserving benign generation quality. Our findings reveal dual vulnerabilities at visual and semantic levels and provide a foundation for developing next generation defenses for secure generative AI.

Legal judgment prediction (LJP) increasingly relies on large language models whose full fine-tuning is memory-intensive and susceptible to catastrophic forgetting. We present LawLLM-DS, a two-stage Low-Rank Adaptation (LoRA) framework that first performs legal knowledge pre-tuning with an aggressive learning rate and subsequently refines judgment relations with conservative updates, using dedicated LoRA adapters, 4-bit quantization, and targeted modification of seven Transformer projection matrices to keep only 0.21% of parameters trainable. From a structural perspective, the twenty annotated legal elements form a symmetric label co-occurrence graph that exhibits both cluster-level regularities and asymmetric sparsity patterns, and LawLLM-DS implicitly captures these graph-informed dependencies while remaining compatible with downstream GNN-based representations. Experiments on 5096 manually annotated divorce cases show that LawLLM-DS lifts macro F1 to 0.8893 and achieves an accuracy of 0.8786, outperforming single-stage LoRA and BERT baselines under the same data regime. Ablation studies further verify the contributions of stage-wise learning rates, adapter placement, and low-rank settings. These findings demonstrate that curriculum-style, parameter-efficient adaptation provides a practical path toward lightweight yet structure-aware LJP systems for judicial decision support.

Hyperspectral images present significant challenges for conventional dimensionality reduction methods due to their high dimensionality, spectral redundancy, and complex spatial–spatial dependencies. While kernel-based sparse representation methods have shown promise in handling spectral non-linearities, they often fail to preserve spatial consistency and semantic discriminability during feature transformation. To address these limitations, we propose a novel semantic-guided kernel low-rank sparse preserving projection (SKLSPP) framework. Unlike previous approaches that primarily focus on spectral information, our method introduces three key innovations: a semantic-aware kernel representation that maintains discriminability through label constraints, a spatially adaptive manifold regularization term that preserves local pixel affinities in the reduced subspace, and an efficient optimization framework that jointly learns sparse codes and projection matrices. Extensive experiments on benchmark datasets demonstrate that SKLSPP achieves superior performance compared to state-of-the-art methods, showing enhanced feature discrimination, reduced redundancy, and improved robustness to noise while maintaining spatial coherence in the dimensionality-reduced features.

Cross-domain feature fusion and classification for weak target in sea clutter based on metric learning

Federated learning (FL) enables collaborative model training across distributed devices without sharing raw data, but applying FL to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">multi-modal</i> settings introduces significant challenges. Clients typically possess heterogeneous modalities and model architectures, making it difficult to align feature spaces efficiently while preserving privacy and minimizing communication costs. To address this, we introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoMFed</i>, a Communication-Efficient Multi-Modal Federated Learning framework that uses learnable projection matrices to generate compressed latent representations. A latent-space regularizer aligns these representations across clients, improving cross-modal consistency and robustness to outliers. Experiments on human activity recognition benchmarks show that CoMFed achieves competitive accuracy with minimal overhead.

Massive multiple-input multiple-output (MIMO) systems offer high spectral efficiency but generate high-dimensional downlink channel state information (CSI), posing challenges for real-time channel acquisition and precoding, particularly in centimeter-wave (cmWave) vehicle-to-infrastructure (V2I) communications where rapid channel variations due to high mobility further complicate CSI acquisition. To address this, we propose an uplink sounding reference signal (SRS)-aided joint design of downlink CSI reference signal (CSI-RS), CSI feedback, and base-station (BS) precoding with end-to-end (E2E) deep learning. Firstly, we design a multi-axis multi-layer perceptron (MAXIM)-based multi-domain CSI-RS network, which takes the uplink sounding reference signals (SRS) as input and outputs a frequency-, beam-, and port-domain projection matrices. Secondly, user equipment (UE) then compresses/quantizes the received CSI-RS and feeds a compact representation to the BS. Thirdly, at the BS, two complementary branches produce candidate precoders: one is named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">feedback-only</i> precoding network driven by quantized CSI feedback, and the other is named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SRS-only</i> precoding network driven by uplink SRS. These candidate precoders are subsequently combined by a precoding fusion network to yield the final transmit precoder. Finally, all these modules are trained with a spectral-efficiency-oriented loss in an E2E deep learning manner. Simulation results in high-mobility vehicular scenarios demonstrate that the proposed approach effectively harnesses both SRS-derived and CSI-feedback information, achieving markedly better performance than conventional baselines, especially under severe channel aging conditions.

Domain adaptation methods have been extensively studied for rolling bearing fault diagnosis under various conditions. However, some existing methods only consider the one-way embedding of original space into a low-dimensional subspace without backward validation, which leads to inaccurate embeddings of data and poor diagnostic performance. In this paper, a rolling bearing fault diagnosis method based on multi-source domain joint structure preservation transfer with autoencoder (MJSPTA) is proposed. Firstly, similar source domains are screened by inter-domain metrics; then, the high-dimensional data of both the source and target domains are projected into a shared subspace with different projection matrices, respectively, during the encoding stage. Finally, the decoding stage reconstructs the low-dimensional data back to the original high-dimensional space to minimize the reconstruction accuracy. In the shared subspace, the difference between source and target domains is reduced through distribution matching and sample weighting. Meanwhile, graph embedding theory is introduced to maximally preserve the local manifold structure of the samples during domain adaptation. Next, label propagation is used to obtain the predicted labels, and a voting mechanism ultimately determines the fault type. The effectiveness and robustness of the method are verified through a series of diagnostic tests.

The challenge of traffic sign detection and recognition for driving vehicles has become more critical with recent advances in autonomous and assisted driving technologies. Although object recognition problems, particularly traffic sign recognition, have been extensively studied, most Vision Transformer (ViT) models still rely on static attention mechanisms with fixed projection matrices (Q, K, and V). Using this mechanism limits the ViTs to handle real-world problems such as object detection and traffic sign recognition, etc. Problems, such as partially or fully obscured signs, changes in illumination, and weather conditions, result in subpar feature extraction, which compounds the misclassification problem. To overcome this challenge, a Conditional Visual Transformer (CViT) is proposed in this research, which dynamically adapts feature aggregation, Q, K, and V projections, as well as attention-based mechanisms, based on the input sign type. Its main component consists of a controlled failure deep learning model using a CViT that targets specific types of traffic signs through varying feature extraction and attention adjustments, resulting in high classification performance and minimizing misclassifications. Furthermore, an adaptive gating technique is employed that optimally adjusts the projection matrix across different traffic signs. The proposed CViT achieved an overall accuracy of 99.87%, with a Micro Precision of 99.07%, a Macro Recall of 94.3%, and a Macro F1 Score of 99.07%, respectively. These results demonstrate the potential of CViT to improve both the efficiency and reliability of traffic sign recognition in autonomous driving applications.

The challenge of traffic sign detection and recognition for driving vehicles has become more critical with recent advances in autonomous and assisted driving technologies. Although object recognition problems, particularly traffic sign recognition, have been extensively studied, most Vision Transformer (ViT) models still rely on static attention mechanisms with fixed projection matrices (Q, K, and V). Using this mechanism limits the ViTs to handle real-world problems such as object detection and traffic sign recognition, etc. Problems, such as partially or fully obscured signs, changes in illumination, and weather conditions, result in subpar feature extraction, which compounds the misclassification problem. To overcome this challenge, a Conditional Visual Transformer (CViT) is proposed in this research, which dynamically adapts feature aggregation, Q, K, and V projections, as well as attention-based mechanisms, based on the input sign type. Its main component consists of a controlled failure deep learning model using a CViT that targets specific types of traffic signs through varying feature extraction and attention adjustments, resulting in high classification performance and minimizing misclassifications. Furthermore, an adaptive gating technique is employed that optimally adjusts the projection matrix across different traffic signs. The proposed CViT achieved an overall accuracy of 99.87%, with a Micro Precision of 99.07%, a Macro Recall of 94.3%, and a Macro F1 Score of 99.07%, respectively. These results demonstrate the potential of CViT to improve both the efficiency and reliability of traffic sign recognition in autonomous driving applications.

With the increasing demands for process safety and manufacturing efficiency, process monitoring has garnered significant attention from both academia and industry over the past few decades. Process monitoring aims to detect deviations from normal operating conditions by analyzing data features extracted under predefined normal states. However, the inherent non-stationarity of real industrial processes can compromise the accurate definition of these normal conditions, thereby limiting the effectiveness of traditional multivariate statistical process monitoring (MSPM) methods. A common strategy to address non-stationarity is to employ projection matrices that transform non-stationary time series into stationary ones, upon which monitoring statistics are constructed. Nevertheless, this approach often overlooks the valuable information contained in the non-stationary subspace, leading to insufficient extraction of fault-relevant features. Fault signatures may manifest in both stationary and non-stationary components of the process data. To overcome these limitations, an integrated monitoring framework that combines Stationary Subspace Analysis (SSA), a Stacked Autoencoder (SAE), and Support Vector Data Description (SVDD) is proposed in this research. Specifically, SSA was first applied to decompose the process data into stationary and non-stationary subspaces. Monitoring statistics were then constructed directly in the stationary subspace, while reconstruction errors from the SAE were used to capture features in the non-stationary subspace. Finally, SVDD was used to fuse the dual-space statistical indicators, enabling comprehensive fault detection. The proposed method was validated by the Tennessee Eastman and real industrial processes. Comparative results demonstrate that it outperformed existing non-stationary monitoring techniques in terms of monitoring performance.

Learning to estimate and classify brain functional networks (BFNs) has become an increasingly important way of predicting neurological or mental disorders at their early stages. The traditional methods conduct BFN estimation and classification in two separate steps, thus preventing the interaction and joint optimization. In contrast, Transformer provides a natural architecture to learn BFNs with downstream tasks in an end-to-end manner. Despite their great potential, Transformer-based methods involve a large number of parameters that need to be learnt from Big Data and often lead to poor model interpretability. Considering the challenge in acquiring data and the high demand for model interpretability in medical scenarios, in this paper, we propose a minimalist Transformer architecture, referred to as Miniformer, by simplifying the projection matrices in the self-attention module into a single diagonal matrix, which greatly reduces the number of parameters, alleviates the risk of overfitting, and improves the interpretability. Additionally, the clear physical meaning of parameters in Miniformer makes the integration of domain knowledge or prior easier and more natural. Therefore, we further develop two variants of Miniformer by incorporating sparsity for removing potentially noisy time points from fMRI signals, and smoothness for capturing the temporal correlations in fMRI signals, respectively. To evaluate the effectiveness of the proposed methods, we perform brain disease diagnosis experiments on three public datasets. The results show that Miniformer and its variants tend to achieve higher classification performance than comparison methods with good interpretability.

Abstract Pose estimation algorithms are an extensively studied research topic in the field of computer vision and machine learning. Even though many algorithms attempt to solve the problem, most algorithms are still not accurate enough to recover poses in real-world applications. Therefore, we have developed a new approach that utilizes depth cues and optical flow measurements that presents improved pose recovery in real-world pose estimation applications. We also present a camera calibration method that creates projection matrices for pose estimation from cameras, which enables angular comparison for relative pose estimates from two sensor systems positioned at different locations. We applied and tested the proposed algorithm in the laboratory settings and compared our findings with a commercial and a gold standard pose estimation system. Angular pose errors were reported.

Objective.Anatomical changes in target volumes and surrounding organs-at-risk (OARs) commonly occur during radiation therapy (RT). Relying solely on the initial treatment plan can lead to suboptimal dose delivery and increased risk to healthy tissues. This study investigates a fluence map (FM) prediction-based method (FM_PD) for rapid plan adaptation. It enables online adaptive RT (OART) to better account for structural changes throughout treatment and assess its potential for improved normal tissue sparing.Approach.The planning target volumes (PTVs) and corresponding dose distribution were converted into 2D projection matrices during training. A 2D Dense-U-Net model incorporating a PTV-specific loss function (PTV_loss) was trained on a dataset of 93 intensity-modulated RT plans for hypopharyngeal carcinoma. Nine re-planning scenarios (time intervals: 32-47 days) were used for testing to simulate an OART setting. Predicted FMs were applied to the daily CTs to calculate updated dose distributions. These doses were compared to the original (non-adapted) plans to evaluate the dosimetric impact on OARs.Main results.FM_PD significantly reduced the dose to normal tissues while maintaining tumor coverage. The D2of the PTV decreased by 1.13 ± 5.85%, moreover, substantial dose decreases were observed in critical structures: Dmaxto the lens, optic nerves, and brainstem decreased by 18.67 ± 19.04%, 19.17 ± 19.57%, and 14.54%, respectively. The total body Dmeandecreased by 25.65 ± 15.44%. In cases where the PTV was adjacent to lung tissue, the Dmeandropped significantly by 46.40 ± 36.89%.Significance.FM_PD offers a rapid and effective approach for adapting RT plans in response to anatomical changes, significantly reducing doses to healthy tissues. Compared to maintaining the initial plan, FM_PD is a recommended strategy for cross-fraction adaptation scenarios in clinical OART practice.

Principal component analysis (PCA) is a widely used technique for dimension reduction. As datasets continue to grow in size, distributed-PCA (DPCA) has become an active research area. A key challenge in DPCA lies in efficiently aggregating results across multiple machines or computing nodes due to computational overhead. Fan et al. introduced a pioneering DPCA method to estimate the leading rank-r eigenspace, aggregating local rank-r projection matrices by averaging. However, their method does not use eigenvalue information. In this article, we propose a novel DPCA method that incorporates eigenvalue information to aggregate local results via the matrix β -mean, which we call β -DPCA. The matrix β -mean offers a flexible and robust aggregation method through the adjustable choice of β values. Notably, for β = 1 , it corresponds to the arithmetic mean; for β = − 1 , the harmonic mean; and as β → 0 , the geometric mean. Moreover, the matrix β -mean is shown to associate with the matrix β -divergence, a subclass of the Bregman matrix divergence, to support the robustness of β -DPCA. We also study the stability of eigenvector ordering under perturbations for β -DPCA. The performance of our proposal is evaluated through numerical studies. Supplementary materials for this article are available online.