Kernel Design Research Articles

HawkEye Conv-Driven YOLOv10 with Advanced Feature Pyramid Networks for Small Object Detection in UAV Imagery

Current mainstream computer vision algorithms focus on designing suitable network architectures and loss functions to fit training data. However, the accuracy of small object detection remains lower than for other scales, and the design of convolution operators limits the model’s performance. For UAV small object detection, standard convolutions, due to their fixed kernel size, cannot adaptively capture small object spatial information. Many convolutional variants have scattered sampling points, leading to blurred boundaries and reduced accuracy. In response, we propose HawkEye Conv (HEConv), which utilizes stable sampling and dynamic offsets with random selection. By varying the convolution kernel design, HEConv reduces the accuracy gap between small and larger objects while offering multiple versions and plug-and-play capabilities. We also develop HawkEye Spatial Pyramid Pooling and Gradual Dynamic Feature Pyramid Network modules to validate HEConv. Experiments on the RFRB agricultural and VisDrone2019 urban datasets demonstrate that, compared to YOLOv10, our model improves AP50 by 11.9% and 6.2%, APS by 11.5% and 5%, and F1-score by 5% and 7%. Importantly, it enhances small object detection without sacrificing large object accuracy, thereby reducing the multi-scale performance gap.

Compatible and Applicable Platform design of Secure Real-Time Operating System Kernel

Compatible and Applicable Platform design of Secure Real-Time Operating System Kernel

NPFormer: Interpretable rotating machinery fault diagnosis architecture design under heavy noise operating scenarios

While existing models have achieved significant progress on fault diagnosis tasks, what interpretable temporal dependencies they capture to resist noise remains unclear. Previous studies have ignored the inherent non-stationary property of fault data, which is the basis for capturing reliable temporal dependencies under heavy noise operating scenarios. To tackle the aforementioned problems, we propose a non-stationary perception network (NPFormer). Firstly, we revisit the large kernel design in the early stage of the network, and we demonstrate that this can make the attention map more stable for modeling non-stationary series, which can be a more powerful paradigm with heavy noise interference. Secondly, an adaptive non-stationarity embedding block is developed, which breaks the raw data into multiple components and adaptively attenuates the non-stationarity of mechanical signals for better predictability. Finally, we design multi-scale fusion attention to capture fine-grained features with the global scope, while achieving localized spatial–temporal correlations. Extensive experiments have been conducted on three public datasets and the practical aero-engine engineering application, showing that NPFormer significantly outperforms existing state-of-the-art methods. The weight information is visualized to reveal how NPFormer works, which can demonstrate that the stable attention map is a clear clue of anti-noise. We quantitatively reveal the effect of noise on the model, and the model’s anti-noise clues, during model inference. Code and models will be available for further study.

Asymmetric convolutional modulation network for efficient image super-resolution

Thanks to the effectiveness of large kernel convolutions in acquiring large receptive fields, lightweight single image super-resolution (SISR) approaches based on large kernel designs have achieved significant performance. However, these state-of-the-art methods still require high computational costs, and their model efficiency remains to be improved. To address these challenges, we propose an asymmetric convolutional modulation network (ACMN) for efficient image super-resolution. In detail, we design an asymmetric convolutional modulation unit (ACMU) that combines large kernel design and modulation mechanism for adaptively selecting representative features from a large receptive field. Furthermore, we adopt depth-wise asymmetric convolutions to further reduce computational burden. To supplement the local context information and promote inter-channel interaction, we further develop a channel shuffle feedforward network (CSFN) to extract local feature information and facilitate inter-channel information interaction. Experimental results demonstrate that our ACMN outperforms other state-of-the-art efficient SR methods with fewer parameters and Multi-Adds.

Adaptive Joint Carrier and DOA Estimations of FHSS Signals Based on Knowledge-Enhanced Compressed Measurements and Deep Learning.

As one of the most widely used spread spectrum techniques, the frequency-hopping spread spectrum (FHSS) has been widely adopted in both civilian and military secure communications. In this technique, the carrier frequency of the signal hops pseudo-randomly over a large range, compared to the baseband. To capture an FHSS signal, conventional non-cooperative receivers without knowledge of the carrier have to operate at a high sampling rate covering the entire FHSS hopping range, according to the Nyquist sampling theorem. In this paper, we propose an adaptive compressed method for joint carrier and direction of arrival (DOA) estimations of FHSS signals, enabling subsequent non-cooperative processing. The compressed measurement kernels (i.e., non-zero entries in the sensing matrix) have been adaptively designed based on the posterior knowledge of the signal and task-specific information optimization. Moreover, a deep neural network has been designed to ensure the efficiency of the measurement kernel design process. Finally, the signal carrier and DOA are estimated based on the measurement data. Through simulations, the performance of the adaptively designed measurement kernels is proved to be improved over the random measurement kernels. In addition, the proposed method is shown to outperform the compressed methods in the literature.

Fault diagnosis of rolling bearings under varying speeds based on gray level co-occurrence matrix and DCCNN

Rolling bearings are widely used in various industries, including rail transit, aerospace, and wind power generation, playing a critical role. However, bearing failures can lead to serious consequences, impacting equipment operation and even causing safety accidents. Hence, the diagnosis of bearing faults is of utmost importance. However, the variable speed conditions experienced during bearing operation pose significant challenges to fault diagnosis. To overcome the limitations of traditional methods in diagnosing bearing faults under variable speed conditions, this paper proposes a fault diagnosis method based on the gray-level co-occurrence matrix (GLCM) and Dual Channel Convolutional Neural Network (DCCNN). The method introduces a two-dimensional grayscale matrix construction (2D-GMC) technique to extract grayscale texture features for fault diagnosis. Additionally, an unconventional kernel design method, based on grayscale image contrast, is proposed to reduce the complexity associated with traditional square kernels. A new DCCNN architecture is developed accordingly. Furthermore, the transfer learning concept is utilized to train the proposed DCCNN model using fault signals at specific rotational speeds. The method intercepts the variable speed into multi-speed short-time series, then constructs gray image under different speed to realize the rapid fault diagnosis of bearings under variable speed conditions.

LSKANet: Long Strip Kernel Attention Network for Robotic Surgical Scene Segmentation.

Surgical scene segmentation is a critical task in Robotic-assisted surgery. However, the complexity of the surgical scene, which mainly includes local feature similarity (e.g., between different anatomical tissues), intraoperative complex artifacts, and indistinguishable boundaries, poses significant challenges to accurate segmentation. To tackle these problems, we propose the Long Strip Kernel Attention network (LSKANet), including two well-designed modules named Dual-block Large Kernel Attention module (DLKA) and Multiscale Affinity Feature Fusion module (MAFF), which can implement precise segmentation of surgical images. Specifically, by introducing strip convolutions with different topologies (cascaded and parallel) in two blocks and a large kernel design, DLKA can make full use of region- and strip-like surgical features and extract both visual and structural information to reduce the false segmentation caused by local feature similarity. In MAFF, affinity matrices calculated from multiscale feature maps are applied as feature fusion weights, which helps to address the interference of artifacts by suppressing the activations of irrelevant regions. Besides, the hybrid loss with Boundary Guided Head (BGH) is proposed to help the network segment indistinguishable boundaries effectively. We evaluate the proposed LSKANet on three datasets with different surgical scenes. The experimental results show that our method achieves new state-of-the-art results on all three datasets with improvements of 2.6%, 1.4%, and 3.4% mIoU, respectively. Furthermore, our method is compatible with different backbones and can significantly increase their segmentation accuracy. Code is available at https://github.com/YubinHan73/LSKANet.

A distance-based kernel for classification via Support Vector Machines.

Support Vector Machines (SVMs) are a type of supervised machine learning algorithm widely used for classification tasks. In contrast to traditional methods that split the data into separate training and testing sets, here we propose an innovative approach where subsets of the original data are randomly selected to train the model multiple times. This iterative training process aims to identify a representative data subset, leading to improved inferences about the population. Additionally, we introduce a novel distance-based kernel specifically designed for binary-type features based on a similarity matrix that efficiently handles both binary and multi-class classification problems. Computational experiments on publicly available datasets of varying sizes demonstrate that our proposed method significantly outperforms existing approaches in terms of classification accuracy. Furthermore, the distance-based kernel achieves superior performance compared to other well-known kernels from the literature and those used in previous studies on the same datasets. These findings validate the effectiveness of our proposed classification method and distance-based kernel for SVMs. By leveraging random subset selection and a unique kernel design, we achieve notable improvements in classification accuracy. These results have significant implications for diverse classification problems in Machine Learning and data analysis.

Incremental learning-based optimal design of BFN kernel for online spacecraft disturbance rejection control

Low earth orbit (LEO) space environment is glutted with unknown and uncertain disturbance, which impede the regular operation and dynamic stability of spacecraft. In recent decades, Basis Function Network (BFN) has attracted much attention benefiting from its potential adaptability for unknown dynamic disturbance. However, it also suffers from the shortcomings of parameter drift, under fitting and long time consuming, which restricts its applications in practical space missions. This paper proposes a new online network reconstruction algorithm, which enables BFN a better excitation effect and more rapid response with less time consumption. The algorithm selects less but more relevant basis functions than routine neural network to approximate the disturbance, which consequently helps to achieve a better characterization effect. Two BFN selection strategies based on iterative learning and batch learning are proposed to further increase the efficiency of online network reconstruction. Detailed theoretical derivation proves the feasibility of the new algorithm. Numerical simulation of a real space station shows that the algorithm has strong adaptability and time efficiency under the interference of structural uncertainty.

Spherical harmonic coefficients of isotropic polynomial functions with applications to gravity field modeling

Various aspects of gravity field modeling rely upon analytical mathematical functions for calculating spherical harmonic coefficients. Such functions allow quick and efficient evaluation of cumbersome convolution integrals defined on the sphere. In this work, we present a new analytical method for determining spherical harmonic coefficients of isotropic polynomial functions. This method in computationally flexible and efficient, since it makes use of recurrence relations. Also, its use is universal and could be extended to piecewise polynomials and polynomials with compact support. Our numerical investigation of the proposed method shows that certain recurrence relations lose accuracy as the order of implemented polynomials increases because of accumulation of numerical errors. Propagation of these errors could be mitigated by hybrid methods or using extended precision arithmetic. We demonstrate the relevance of our method in gravity field modeling and discuss two areas of application. The first one is the design of B-spline windows and filter kernels for the low-pass filtering of gravity field functionals (e.g., GRACE Follow-On monthly geopotential solutions). The second one is the calculation of spherical harmonic coefficients of isotropic polynomial covariance functions.

LGP-YOLO: an efficient convolutional neural network for surface defect detection of light guide plate

Light guide plate (LGP) is a key component of liquid crystal display (LCD) display systems, so its quality directly affects the display effect of LCD. However, LGPs have complex background texture, low contrast, varying defect size and numerous defect types, which makes realizing efficient and accuracy-satisfactory surface defect automatic detection of LGPS still a big challenge. Therefore, combining its optical properties, dot distribution, defect imaging characteristics and detection requirements, a surface defect detection algorithm based on LGP-YOLO for practical industrial applications is proposed in this paper. To enhance the feature extraction ability of the network without dimensionality reduction, expand the effective receptive field and reduce the interference of invalid targets, we built the receptive field module (RFM) by combining the effective channel attention network (ECA-Net) and reviewing large kernel design in CNNs (RepLKNet). For the purpose of optimizing the performance of the network in downstream tasks, enhance the network's expression ability and improve the network’s ability of detecting multi-scale targets, we construct the small detection module (SDM) by combining space-to-depth non-strided convolution (SPDConv) and omini-dimensional dynamic convolution (ODConv). Finally, an LGP defect dataset is constructed using a set of images collected from industrial sites, and a multi-round experiment is carried out to test the proposed method on the LGP detect dataset. The experimental results show that the proposed LGP-YOLO network can achieve high performance, with mAP and F1-score reaching 99.08% and 97.45% respectively, and inference speed reaching 81.15 FPS. This demonstrates that LGP-YOLO can strike a good balance between detection accuracy and inference speed, capable of meeting the requirements of high-precision and high-efficiency LGP defect detection in LGP manufacturing factories.

On kernel design for regularized non-causal system identification

Through one decade’s development, the kernel-based regularization method (KRM) has become a complement to the classical maximum likelihood/prediction error method and an emerging new system identification paradigm. One recent example is its application in the non-causal system identification, and the key issue lies in the design and analysis of kernels for non-causal systems. In this paper, we develop systematic ways to deal with this issue. In particular, we first introduce the guidelines for kernel design and then extend the system theoretic framework to design the so-called non-causal simulation-induced (NCSI) kernel, and we also study its structural properties, including stability and semiseparability. Finally, we consider some special cases of the NCSI kernel and show their advantage over the existing kernels through numerical simulations.

High-Performance Acceleration of 2-D and 3-D CNNs on FPGAs Using Static Block Floating Point.

Over the past few years, 2-D convolutional neural networks (CNNs) have demonstrated their great success in a wide range of 2-D computer vision applications, such as image classification and object detection. At the same time, 3-D CNNs, as a variant of 2-D CNNs, have shown their excellent ability to analyze 3-D data, such as video and geometric data. However, the heavy algorithmic complexity of 2-D and 3-D CNNs imposes a substantial overhead over the speed of these networks, which limits their deployment in real-life applications. Although various domain-specific accelerators have been proposed to address this challenge, most of them only focus on accelerating 2-D CNNs, without considering their computational efficiency on 3-D CNNs. In this article, we propose a unified hardware architecture to accelerate both 2-D and 3-D CNNs with high hardware efficiency. Our experiments demonstrate that the proposed accelerator can achieve up to 92.4% and 85.2% multiply-accumulate efficiency on 2-D and 3-D CNNs, respectively. To improve the hardware performance, we propose a hardware-friendly quantization approach called static block floating point (BFP), which eliminates the frequent representation conversions required in traditional dynamic BFP arithmetic. Comparing with the integer linear quantization using zero-point, the static BFP quantization can decrease the logic resource consumption of the convolutional kernel design by nearly 50% on a field-programmable gate array (FPGA). Without time-consuming retraining, the proposed static BFP quantization is able to quantize the precision to 8-bit mantissa with negligible accuracy loss. As different CNNs on our reconfigurable system require different hardware and software parameters to achieve optimal hardware performance and accuracy, we also propose an automatic tool for parameter optimization. Based on our hardware design and optimization, we demonstrate that the proposed accelerator can achieve 3.8-5.6 times higher energy efficiency than graphics processing unit (GPU) implementation. Comparing with the state-of-the-art FPGA-based accelerators, our design achieves higher generality and up to 1.4-2.2 times higher resource efficiency on both 2-D and 3-D CNNs.

Physics-constrained Gaussian process model for prediction of hydrodynamic interactions between wave energy converters in an array

To improve the efficiency of wave farms and achieve maximum power generation, the layout of wave energy converters (WECs) in an array needs to be carefully designed so that the hydrodynamic interactions can be positively exploited. For this, the hydrodynamic characteristics of the WEC array in different layouts need to be calculated. However, such calculations using numerical models usually entail significant computational cost, especially for large arrays of WECs. To address the computational challenge, a physics-constrained Gaussian process (GP) model is proposed to replace the original expensive numerical model and predict the hydrodynamic characteristics of the WECs for any array layout. By exploring the relationship between the WEC array (i.e., the input) and different hydrodynamic characteristics (i.e., the output), we summarize a set of physical constraints/features, including invariance, symmetry, and additivity. This prior knowledge about the input-output relationship is then directly embedded in the constructed GP model through the design of physics-constrained kernels. In particular, a double-sum invariant kernel is first developed to incorporate the invariance and symmetry features, and then an additive kernel is developed to incorporate the additive feature of the problem. The invariant kernel and the additive kernel are then integrated to construct the physics-constrained GP model. Compared to the standard GP model, the proposed physics-constrained GP models require less training data to achieve the desired accuracy in predicting the hydrodynamic characteristics and are also less vulnerable to the curse of dimensionality (i.e., good scalability for large arrays) due to the use of an additive kernel. The efficiency, accuracy, and scalability of the proposed approach are demonstrated through an application to predict the hydrodynamic characteristics for WEC arrays of different sizes and layouts.

A note on microlocal kernel design for some slow–fast stochastic differential equations with critical transitions and application to EEG signals

This technical note presents an extension of kernel model decomposition (KMD) for detecting critical transitions in some fast–slow random dynamical systems. The approach rests upon modifying KMD for reconstructing an observable by using a novel data-based time-frequency-phase kernel that allows to approximate signals with critical transitions. In particular, we apply the developed method for approximating the solution and detecting critical transitions in some prototypical slow–fast SDEs with critical transitions. We also apply it to detecting seizures in a multi-scale mesoscale nine-dimensional SDE model of brain activity.

A Novel Variable Convolution Kernel Design According to Time-frequency Resolution Altering in Bearing Fault Diagnosis

A Novel Variable Convolution Kernel Design According to Time-frequency Resolution Altering in Bearing Fault Diagnosis

Analysis of Kernel Matrices via the von Neumann Entropy and Its Relation to RVM Performances.

Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high computational costs of kernel-based methods make it extremely inefficient to use standard model selection methods, such as cross-validation, creating a need for careful kernel design and parameter choice. These reasons justify the prior analyses of kernel matrices, i.e., mathematical objects generated by the kernel functions. This paper explores these topics from an entropic standpoint for the case of kernelized relevance vector machines (RVMs), pinpointing desirable properties of kernel matrices that increase the likelihood of obtaining good model performances in terms of generalization power, as well as relate these properties to the model's fitting ability. We also derive a heuristic for achieving close-to-optimal modeling results while keeping the computational costs low, thus providing a recipe for efficient analysis when processing resources are limited.

Error Bounds for Kernel-Based Linear System Identification With Unknown Hyperparameters

Applying regularization in reproducing kernel Hilbert spaces has been successful in linear system identification using stable kernel designs. From a Gaussian process perspective, it automatically provides probabilistic error bounds for the identified models from the posterior covariance, which are useful in robust and stochastic control. However, the error bounds require knowledge of the true hyperparameters in the kernel design. They can be inaccurate with estimated hyperparameters for lightly damped systems or in the presence of high noise. In this work, we provide reliable quantification of the estimation error when the hyperparameters are unknown. The bounds are obtained by first constructing a high-probability set for the true hyperparameters from the marginal likelihood function. Then the worst-case posterior covariance is found within the set. The proposed bound is proven to contain the true model with a high probability and its validity is demonstrated in numerical simulation.

Is Formal Verification of seL4 Adequate to Address the Key Security Challenges of Kernel Design?

Is Formal Verification of seL4 Adequate to Address the Key Security Challenges of Kernel Design?

Knowledge-Enhanced Compressed Measurements for Detection of Frequency-Hopping Spread Spectrum Signals Based on Task-Specific Information and Deep Neural Networks

The frequency-hopping spread spectrum (FHSS) technique is widely used in secure communications. In this technique, the signal carrier frequency hops over a large band. The conventional non-compressed receiver must sample the signal at high rates to catch the entire frequency-hopping range, which is unfeasible for wide frequency-hopping ranges. In this paper, we propose an efficient adaptive compressed method to measure and detect the FHSS signals non-cooperatively. In contrast to the literature, the FHSS signal-detection method proposed in this paper is achieved directly with compressed sampling rates. The measurement kernels (the non-zero coefficients in the measurement matrix) are designed adaptively, using continuously updated knowledge from the compressed measurement. More importantly, in contrast to the iterative optimizations of the measurement matrices in the literature, the deep neural networks are trained once using task-specific information optimization and repeatedly implemented for measurement kernel design, enabling efficient adaptive detection of the FHSS signals. Simulations show that the proposed method provides stably low missing detection rates, compared to the compressed detection with random measurement kernels and the recently proposed method. Meanwhile, the measurement design in the proposed method is shown to provide improved efficiency, compared to the commonly used recursive method.