Radar Clutter Covariance Estimation: A Nonlinear Spectral Shrinkage Approach

In this paper, we exploit the spiked covariance structure of the clutter plus noise covariance matrix for adaptive radar signal processing. Using state of the art techniques from mathematical finance and high dimensional statistics we propose a nonlinear shrinkage-based rotation invariant spiked covariance matrix estimator. We compare the proposed estimator with Rank Constrained Maximum Likelihood (RCML)-Expected Likelihood (EL) covariance estimator using the Challenge dataset generated from RFView. We demonstrate that the computation-time for the proposed estimator is less than the RCML-EL estimator with identical Signal to Clutter plus Noise (SCNR) performance for a variety of radar scenarios simulated by RFView. We derive the lower bound and upper bound for the normalized SCNR and empirically show that RCML-EL and the proposed estimator perform within these derived bounds for the Challenge dataset. We state the convergence for the spiked eigenvalues of the estimator.