Linear Computational Complexity Research Articles

A recurrence for the surface area of (n, k)-arrangement graphs

An interesting property of an interconnected network is the number of nodes at distance i from an arbitrary processor, namely the node-centred surface area. This is an important property due to its applications in various fields of study. The ( n , k ) -arrangement graphs, denoted as A ( n , k ) are important class of graphs as they address the scalability issue in simpler topologies such as the hypercube and the star graph. Much work has been done to obtain a closed-form formula for the surface area of this class of graphs, but generally, it is not trivial to find an algorithm to compute the surface area of such graphs in polynomial time or to find an explicit formula with polynomially many terms in regard to the graph's parameters. In this paper, we present a simple recurrence that has linear computational complexity for A ( n , k ) for any arbitrary n and k in their defined range.

Deep cryogenic silicon etching for 3D integrated capacitors: A numerical perspective

One promising approach to increase the capacity density of integral microcapacitors, microsupercapacitors, and microbatteries is three-dimensional structure design, where electrodes are exposed in three dimensions instead of conventional in-plane electrodes. Such structures include nanowires, nanotubes, nanopillars, nanoholes, nanosheets, and nanowalls. In this work, a cryogenic silicon etching process suitable for fabrication of structures with high electrode area is proposed. A numeric model of this process is experimentally calibrated and used for pillar array structure sidewall area optimization. The use of adaptive Runge–Kutta–Fehlberg time integrator allows to achieve almost linear overall computation complexity as a function of simulated etching time, despite the linear increase in conductance computation complexity with depth. A rule for choosing optimal geometric structure parameters under technological constraints is formulated. An optimized trefoil-like structure is proposed, resulting in a total 5.5% increase in sidewall area with respect to the hexagonal array of circular pillars, resulting in 20.33 sidewall area per unit chip area for 30 min long etch or 31.80 for 60 min long etch.

Multi-resolution visual Mamba with multi-directional selective mechanism for retinal disease detection.

Retinal diseases significantly impact patients' quality of life and increase social medical costs. Optical coherence tomography (OCT) offers high-resolution imaging for precise detection and monitoring of these conditions. While deep learning techniques have been employed to extract features from OCT images for classification, convolutional neural networks (CNNs) often fail to capture global context due to their focus on local receptive fields. Transformer-based methods, on the other hand, suffer from quadratic complexity when handling long-range dependencies. To overcome these limitations, we introduce the Multi-Resolution Visual Mamba (MRVM) model, which addresses long-range dependencies with linear computational complexity for OCT image classification. The MRVM model initially employs convolution to extract local features and subsequently utilizes the retinal Mamba to capture global dependencies. By integrating multi-scale global features, the MRVM enhances classification accuracy and overall performance. Additionally, the multi-directional selection mechanism (MSM) within the retinal Mamba improves feature extraction by concentrating on various directions, thereby better capturing complex, orientation-specific retinal patterns. Experimental results demonstrate that the MRVM model excels in differentiating retinal images with various lesions, achieving superior detection accuracy compared to traditional methods, with overall accuracies of 98.98\% and 96.21\% on two public datasets, respectively. This approach offers a novel perspective for accurately identifying retinal diseases and could contribute to the development of more robust artificial intelligence algorithms and recognition systems for medical image-assisted diagnosis.

Amortizing Circuit-PSI in the Multiple Sender/Receiver Setting

Private set intersection (PSI) is a cryptographic functionality for two parties to learn the intersection of their input sets, without leaking any other information. Circuit-PSI is a stronger PSI functionality where the parties learn only a secret-shared form of the desired intersection, thus without revealing the intersection directly. These secret shares can subsequently serve as input to a secure multiparty computation of any function on this intersection. In this paper we consider several settings in which parties take part in multiple Circuit-PSI executions with the same input set, and aim to amortize communications and computations. To that end, we build up a new framework for Circuit-PSI around generalizations of oblivious (programmable) PRFs that are extended with offline setup phases. We present several efficient instantiations of this framework with new security proofs for this setting. As a side result, we obtain a slight improvement in communication and computation complexity over the state-of-the-art semi-honest Circuit-PSI protocol by Bienstock et al. (USENIX '23). Additionally, we present a novel Circuit-PSI protocol from a PRF with secret-shared outputs, which has linear communication and computation complexity in the parties' input set sizes, and is able to realize a stronger security notion. Lastly, we derive the potential amortizations over multiple protocol executions, and observe that each of the presented instantiations is favorable in at least one of the multiple-execution settings.

TS‐Mixer: A lightweight text representation model based on context awareness

AbstractLarge pre‐trained models (PTMs) have shown their powerful ability in multiple natural language processing tasks. However, using them in practical application remains a challenge due to the significant computational cost and memory requirements. In order to achieve the balance of computational cost and accuracy, MLP architecture can be used as an alternative to the self‐attention module, such as pNLP‐Mixer and Hyper‐Mixer. Experiments indicate that, MLP‐based models can attain competitive performance with low cost. They maintain the balance of computation cost and accuracy successfully, yet, this is at the expense of not being able to capture short‐range dependencies. In this paper, a novel MLP‐based model, termed TS‐Mixer, is proposed which can capture local dependencies by shifting operation. Compared with other MLP‐based models, the parameters of TS‐Mixer are decoupled from the sequence length, hence it has a smaller model size in long sequence tasks. In addition, TS‐Mixer has linear computational complexity, therefore it can be used as a lightweight alternative to the self‐attention model. Experiments show that the TS‐Mixer outperforms other MLP‐based models, which achieves higher accuracy with fewer parameters in multiple downstream tasks. Notably, compared with pre‐trained models, TS‐Mixer can reach more than 90% of their accuracy with 1% or even one thousandth of the parameters (0.174 ~ 1.2 M). These results demonstrate that TS‐Mixer can achieve a better balance between the computing resources and accuracy. Code is available at: https://github.com/wyl-privacy-project/TS-Mixer.

Neural Memory State Space Models for Medical Image Segmentation.

With the rapid advancement of deep learning, computer-aided diagnosis and treatment have become crucial in medicine. UNet is a widely used architecture for medical image segmentation, and various methods for improving UNet have been extensively explored. One popular approach is incorporating transformers, though their quadratic computational complexity poses challenges. Recently, State-Space Models (SSMs), exemplified by Mamba, have gained significant attention as a promising alternative due to their linear computational complexity. Another approach, neural memory Ordinary Differential Equations (nmODEs), exhibits similar principles and achieves good results. In this paper, we explore the respective strengths and weaknesses of nmODEs and SSMs and propose a novel architecture, the nmSSM decoder, which combines the advantages of both approaches. This architecture possesses powerful nonlinear representation capabilities while retaining the ability to preserve input and process global information. We construct nmSSM-UNet using the nmSSM decoder and conduct comprehensive experiments on the PH2, ISIC2018, and BU-COCO datasets to validate its effectiveness in medical image segmentation. The results demonstrate the promising application value of nmSSM-UNet. Additionally, we conducted ablation experiments to verify the effectiveness of our proposed improvements on SSMs and nmODEs.

DCSS-UNet: UNet based on State Space Model for Polyp Segmentation

Early and accurate segmentation of medical images can provide valuable information for medical treatment. In recent years, the automatic and accurate segmentation of polyps in colonoscopy images has received extensive attention from the research community of artificial intelligence and computer vision. Many researchers have conducted in-depth research on models based on CNN and Transformer. However, CNN have limited ability to model remote dependencies, which makes it challenging to fully utilize semantic information in images. On the other hand, the complexity of the secondary computation poses a challenge to the transformer. Recently, state-space models (SSMS), such as Mamba, have been recognized as a promising approach. They not only show superior performance in remote interaction, but also maintain linear computational complexity. Inspired by Mamba, we propose DCSS-UNet, where we utilize visual state space (VSS) blocks in VMamba to capture a wide range of contextual information. In the Skip connection phase, we propose Skip Connects Feature Attention modules(SFA) to better communicate information from the encoder. In the decoder stage, we innovatively combined the Temporal Fusion Attention Module(TFAM) to effectively fuse the feature information. In addition, we introduced a custom Loss calculation method, Tversky Loss, for the model to achieve faster convergence and improve segmentation along polyp boundaries. Our model was trained on the Kvasir-SEG and CVC-ClinicDB datasets, and validated on datasets Kvasir-SEG, CVC-ColonDB, CVC-300, and ETIS. The results show that the model achieves good segmentation accuracy and generalization performance with a low number of parameters. We are 6.1% ahead in the Kavirs-SEG dataset and 3.1% ahead in the CVC-ClinicDB dataset compared to VM-UNet.

Periodformer: An efficient long-term time series forecasting method based on periodic attention

As Transformer-based models have achieved impressive performance across various time series tasks, Long-Term Series Forecasting (LTSF) has garnered extensive attention in recent years. The intricate complexity of the Attention mechanism leads to pronounced overfitting of time series data, thereby impeding the efficient and accurate forecasting of long-term sequences. To better cope with this, we design a lightweight Period-Attention mechanism (Periodformer), which renovates the aggregation of long-term subseries via explicit periodicity and short-term subseries via built-in proximity. Meanwhile, a gating mechanism is embedded into Periodformer to regulate the influence of the attention module on the prediction results. This gives Periodformer a much more powerful and flexible sequence modeling capability with linear computational complexity, guaranteeing higher prediction performance and shorter runtime on real devices. Experimental results show that Periodformer consistently performs best on six widely used benchmark datasets. Compared with the state-of-the-art methods, the prediction error of Periodformer was reduced by 13% and 26% for multivariate and univariate forecasting, respectively.

A novel multilevel finite element method for a generalized nonlinear Schrödinger equation

In this article, we focus on an efficient multilevel finite element method to solve the time-dependent nonlinear Schrödinger equation which is one of the most important equations of mathematical physics. For the time derivative, we adopt implicit schemes including the backward Euler method and the Crank–Nicolson method. Based on these stable implicit schemes, the proposed method requires solving a nonlinear elliptic problem at each time step. For these nonlinear elliptic equations, a multilevel mesh sequence is constructed. At each mesh level, we first derive a rough approximation by correcting the approximation of the previous mesh level in a special correction subspace. The correction subspace is composed of a coarse finite element space and an additional approximate solution derived from the previous mesh level. Next, we only need to solve a linearized elliptic equation by inserting the rough approximation into the nonlinear term. Then, we derive an accurate approximate solution by performing the aforementioned solving process on the multilevel mesh sequence until we reach the final mesh level. Owing to the special construct of the correction subspace, we derive a multilevel finite element method to solve the nonlinear Schrödinger equation for the first time, and meanwhile we also derive an optimal error estimate with linear computational complexity. Additionally, unlike the existing multilevel methods for nonlinear problems, that typically require bounded second-order derivatives of the nonlinear terms, the nonlinear term in our study requires only one-order derivatives. Numerical results are provided to support our theoretical analysis and demonstrate the efficiency of the presented method.

When wavelet decomposition meets external attention: a lightweight cloud server load prediction model

Load prediction tasks aim to predict the dynamic trend of future load based on historical performance sequences, which are crucial for cloud platforms to make timely and reasonable task scheduling. However, existing prediction models are limited while capturing complicated temporal patterns from the load sequences. Besides, the frequently adopted global weighting strategy (e.g., the self-attention mechanism) in temporal modeling schemes has quadratic computational complexity, hindering the immediate response of cloud servers in complex real-time scenarios. To address the above limitations, we propose a Wavelet decomposition-enhanced External Transformer (WETformer) to provide accurate yet efficient load prediction for cloud servers. Specifically, we first incorporate discrete wavelet transform to progressively extract long-term trends, highlighting the intrinsic attributes of temporal sequences. Then, we propose a lightweight multi-head External Attention (EA) mechanism to simultaneously consider the inter-element relationships within load sequences and the correlations across different sequences. Such an external component has linear computational complexity, mitigating the encoding redundancy prevalent and enhancing prediction efficiency. Extensive experiments conducted on Alibaba Cloud’s cluster tracking dataset demonstrate that WETformer achieves superior prediction accuracy and the shortest inference time compared to several state-of-the-art baseline methods.

COLAFormer: Communicating local–global features with linear computational complexity

Local and sparse attention effectively reduce the high computational cost of global self-attention. However, they suffer from non-global dependency and coarse feature capturing, respectively. While some subsequent models employ effective interaction techniques for better classification performance, we observe that the computation and memory of these models overgrow as the resolution increases. Consequently, applying them to downstream tasks with large resolutions takes time and effort. In response to this concern, we propose an effective backbone network based on a novel attention mechanism called Concatenating glObal tokens in Local Attention (COLA) with a linear computational complexity. The implementation of COLA is straightforward, as it incorporates global information into the local attention in a concatenating manner. We introduce a learnable condensing feature (LCF) module to capture high-quality global information. LCF possesses the following properties: (1) performing a function similar to clustering, aggregating image patches into a smaller number of tokens based on similarity. (2) a constant number of aggregated tokens regardless of the image size, ensuring that it is a linear complexity operator. Based on COLA, we build COLAFormer, which achieves global dependency and fine-grained feature capturing with linear computational complexity and demonstrates impressive performance across various vision tasks. Specifically, our COLAFormer-S achieves 84.5% classification accuracy, surpassing other advanced models by 0.4% with similar or less resource consumption. Furthermore, our COLAFormer-S can achieve a better object detection performance while consuming only 1/4 of the resources compared to other state-of-the-art models. The code and models will be made publicly available.

Towards Resource-Efficient DDoS Detection in IoT: Leveraging Feature Engineering of System and Network Usage Metrics

The Internet of Things (IoT) is omnipresent, exposing a large number of devices that often lack security controls to the public Internet. In the modern world, many everyday processes depend on these devices, and their service outage could lead to catastrophic consequences. There are many Deep Packet Inspection (DPI) based intrusion detection systems (IDS). However, their linear computational complexity induced by the event-driven nature poses a power-demanding obstacle in resource-constrained IoT environments. In this paper, we shift away from the traditional IDS as we introduce a novel and lightweight framework, relying on a time-driven algorithm to detect Distributed Denial of Service (DDoS) attacks by employing Machine Learning (ML) algorithms leveraging the newly engineered features containing system and network utilization information. These features are periodically generated, and there are only ten of them, resulting in a low and constant algorithmic complexity. Moreover, we leverage IoT-specific patterns to detect malicious traffic as we argue that each Denial of Service (DoS) attack leaves a unique fingerprint in the proposed set of features. We construct a dataset by launching some of the most prevalent DoS attacks against an IoT device, and we demonstrate the effectiveness of our approach with high accuracy. The results show that standalone IoT devices can detect and classify DoS and, therefore, arguably, DDoS attacks against them at a low computational cost with a deterministic delay.

LPAM: A lightweight medical segmentation network based on Mamba improved by prompt attention

AbstractPresently, State Space Models (SSMs), including frameworks like Mamba, have been incorporated into the realm of computer vision. These models not only sustain remote interactions and encapsulate global semantic information effectively, but also preserve linear computational complexity, offering a balance between performance and computational efficiency. Given that Mamba inherently adheres to the principle of selectivity when constructing sequence models, the goal is to further unleash the potential of Mamba through this innovative combination of convolution and self‐attention, improve accuracy and minimize the number of parameters while achieving linear complexity. Mamba is employed as an encoder to distill semantic information from the image, and it is supplemented with convolutional blocks, thereby conserving the details of the image. Concurrently, embedding prompts at a deeper level enhances its adaptability to cater to diverse requirements. Lastly, a bidirectional attention mechanism is incorporated for inference, striving to retain both global connections and local details to the maximum extent. This culminates in a novel, lightweight medical segmentation model. Exhaustive experiments were executed on six public datasets. The empirical results show that the proposed model exhibits competitive performance in medical image segmentation tasks.

A Novel Mamba Architecture with a Semantic Transformer for Efficient Real-Time Remote Sensing Semantic Segmentation

Real-time remote sensing segmentation technology is crucial for unmanned aerial vehicles (UAVs) in battlefield surveillance, land characterization observation, earthquake disaster assessment, etc., and can significantly enhance the application value of UAVs in military and civilian fields. To realize this potential, it is essential to develop real-time semantic segmentation methods that can be applied to resource-limited platforms, such as edge devices. The majority of mainstream real-time semantic segmentation methods rely on convolutional neural networks (CNNs) and transformers. However, CNNs cannot effectively capture long-range dependencies, while transformers have high computational complexity. This paper proposes a novel remote sensing Mamba architecture for real-time segmentation tasks in remote sensing, named RTMamba. Specifically, the backbone utilizes a Visual State-Space (VSS) block to extract deep features and maintains linear computational complexity, thereby capturing long-range contextual information. Additionally, a novel Inverted Triangle Pyramid Pooling (ITP) module is incorporated into the decoder. The ITP module can effectively filter redundant feature information and enhance the perception of objects and their boundaries in remote sensing images. Extensive experiments were conducted on three challenging aerial remote sensing segmentation benchmarks, including Vaihingen, Potsdam, and LoveDA. The results show that RTMamba achieves competitive performance advantages in terms of segmentation accuracy and inference speed compared to state-of-the-art CNN and transformer methods. To further validate the deployment potential of the model on embedded devices with limited resources, such as UAVs, we conducted tests on the Jetson AGX Orin edge device. The experimental results demonstrate that RTMamba achieves impressive real-time segmentation performance.

New approach for learning structured graph with Laplacian rank constraint

Many existing graph-clustering methods get results in a two-stage manner, including constructing a graph from data and partitioning it. It always leads to sub-optimal performances. To address the drawbacks, imposing rank constraint on the graph’s Laplacian matrix is an effective way. It can learn a structured graph that connected components are exactly equal to the cluster numbers. According to separate connected components, it directly obtains clustering assignments. However, previous works heuristically solve it during the optimization without any theory guarantee. To handle this issue, we propose a novel solver to optimize this problem with theoretical advantages, enabling the structured bipartite graphs to directly indicate clustering results. Initially, we discover that pairwise and bipartite graphs exhibit the same connected components after undergoing a specific operation. Then, we derive an efficient algorithm to optimize two different clustering models with rank constraints based on this property. Moreover, our proposed method exhibits linear computational complexity to the data scale. Extensive empirical results on both synthetic and real benchmark data demonstrate that the proposed method outperforms existing methods.

Fast Dynamic Time Warping for Temperature Compensation in Guided Waves

The paper discusses Structural Health Monitoring (SHM) based on ultrasonic guided waves for damage detection in structures. Guided waves allow inspection over long distances and inaccessible features, but are also sensitive to changes in environmental and operating conditions (EOC). The focus of this paper is on temperature compensation methods for guided waves. The compensation techniques include Baseline Signal Stretch (BSS), Optimal Baseline Selection (OBS), OBS+BSS, and Dynamic Time-Warping (DTW). In particular, a new, fast approximation of DTW is evaluated and compared with conventional but computationally expensive DTW. The FDTW algorithm utilizes a multilevel approach inspired by graph bisection principles to achieve precise warping path determination with linear computational complexity. The study evaluates the compensation performance of FDTW using a single baseline at a single temperature, thus addressing the complexity and inaccessibility issues of obtaining an extensive database of baseline signals in practical applications. The Open Guided Wave (OGW) dataset is employed for a fair comparison with other compensation methods. Results indicate that FDTW performs well, demonstrating comparable warping performance to DTW but with significantly reduced computational complexity. The analysis also includes comparisons with BSS, OBS+BSS, and DTW across a range of temperatures, highlighting the effectiveness of FDTW in mitigating errors introduced by larger temperature variations.

Scalable and parameter-free fusion graph learning for multi-view clustering

Multi-view clustering aims to capture the consistency and complementary information present in view-specific data to achieve clustering alignment. However, existing multi-view clustering methods often rely on different regularization terms to quantify the importance of various views, which inevitably introduces additional hyperparameters. It is challenging to fine-tune these additional parameters in real-world applications. Additionally, these methods suffer from high time complexity and impose substantial constraints when applied in large-scale scenarios. To address these limitations, we propose a parameter-free and time-efficient graph fusion method for multi-view clustering that can integrate view-specific graphs and directly generate clustering labels. Specifically, we introduce an anchor strategy and generate bipartite graphs on different views to enhance efficiency. Subsequently, we employ a self-weighted graph fusion strategy to merge the view-specific bipartite graphs. Finally, we propose a new solver to handle these problems, enabling the structured bipartite graphs to directly indicate clustering results. In contrast to previous clustering methods, our approach does not introduce any additional parameters and entirely relies on self-weighting for the fusion of view-specific graphs. As a result, our proposed method exhibits linear computational complexity to the data scale. Extensive experimental results on various benchmark datasets demonstrate the effectiveness and efficiency of our approach. Our code is available at https://github.com/DuannYu/MvSST.

Ensemble Approach Using k-Partitioned Isolation Forests for the Detection of Stock Market Manipulation

Stock market manipulation, defined as any attempt to artificially influence stock prices, poses significant challenges by causing financial losses and eroding investor trust. The prevalent reliance on supervised learning models for detecting such manipulations, while showing promise, faces notable hurdles due to the dearth of labeled data and the inability to recognize novel manipulation tactics beyond those explicitly labeled. This study ventures into addressing these gaps by proposing a novel detection framework aimed at identifying suspicious hourly manipulation blocks through an unsupervised learning approach, thereby circumventing the limitations of data labeling and enhancing the adaptability to emerging manipulation strategies. Our methodology involves the innovative creation of features reflecting the behavior of stocks across various time windows followed by the segmentation of the dataset into k subsets. This setup facilitates the identification of potential manipulation instances via a voting ensemble composed of k isolation forest models, which have been chosen for their efficiency in pinpointing anomalies and their linear computational complexity—attributes that are critical for analyzing vast datasets. Evaluated against eight real stocks known to have undergone manipulation, our approach demonstrated a remarkable capability to identify up to 89% of manipulated blocks, thus significantly outperforming previous methods that do not utilize a voting ensemble. This finding not only surpasses the detection rates reported in prior studies but also underscores the enhanced robustness and adaptability of our unsupervised model in uncovering varied manipulation schemes. Through this research, we contribute to the field by offering a scalable and efficient unsupervised learning strategy for stock manipulation detection, thereby marking a substantial advancement over traditional supervised methods and paving the way for more resilient financial markets.

Flipping-based iterative surface reconstruction for unoriented points

In this paper, we propose a novel surface reconstruction method for unoriented points by establishing and solving a nonlinear equation system. By treating normals as unknown parameters and imposing the conditions that the implicit field is constant and its gradients parallel to the normals on the input point cloud, we establish a nonlinear equation system involving the oriented normals. To simplify the system, we transform it into a 0-1 integer programming problem solely focusing on orientation by incorporating inconsistent oriented normal information through PCA. We solve the simplified problem using flipping-based iterative algorithms and propose two novel criteria for flipping based on theoretical analysis.Extensive experiments on renowned datasets demonstrate that our flipping-based method with wavelet surface reconstruction achieves state-of-the-art results in orientation and reconstruction. Furthermore, it exhibits linear computational and storage complexity by leveraging the orthogonality and compact support properties of wavelet bases. The source code is available at https://github.com/mayueji/FISR_code.

A Joint Time-Frequency Domain Transformer for multivariate time series forecasting

In order to enhance the performance of Transformer models for long-term multivariate forecasting while minimizing computational demands, this paper introduces the Joint Time-Frequency Domain Transformer (JTFT). JTFT combines time and frequency domain representations to make predictions. The frequency domain representation efficiently extracts multi-scale dependencies while maintaining sparsity by utilizing a small number of learnable frequencies. Simultaneously, the time domain (TD) representation is derived from a fixed number of the most recent data points, strengthening the modeling of local relationships and mitigating the effects of non-stationarity. Importantly, the length of the representation remains independent of the input sequence length, enabling JTFT to achieve linear computational complexity. Furthermore, a low-rank attention layer is proposed to efficiently capture cross-dimensional dependencies, thus preventing performance degradation resulting from the entanglement of temporal and channel-wise modeling. Experimental results on eight real-world datasets demonstrate that JTFT outperforms state-of-the-art baselines in predictive performance.