A novel non-convex penalty function-based STAP algorithm for airborne passive radar systems

Abstract

In conventional sparse Space-Time Adaptive Processing (STAP) algorithms, the l0-norm is often used instead of the l1-norm to relax convexity constraints. The approach presented in this paper can be mathematically reformulated as a linear problem, which can be efficiently solved using various convex optimization techniques. While this method effectively circumvents the NP-Hard complexity associated with the l0-norm, it does come with certain potential challenges. For example, the observation matrix must meet the criteria of the Restricted Isometry Property (RIP) and other stringent conditions. In this research, a novel STAP algorithm is introduced, which is based on a non-convex penalty function. This innovative algorithm substitutes the traditional l0-norm with a custom-designed non-convex penalty function and is applied to the Direct Filter Processor (DFP), solved using the recursive least squares (RLS) algorithm. The resulting algorithm, known as gp-RLS, is demonstrated through Monte Carlo simulations to outperform other l1-based and reduced-rank STAP algorithms in terms of faster convergence, improved signal-to-clutter-plus-noise ratio (SCNR), and enhanced target detection performance.