Colored Noise and Regularization Parameter Selection for Waveform Metrology
Andrew M. Dienstfrey, Paul D. Hale
We study six regularization parameter selection algorithms applied to deconvolution problems relevant for characterization of high-speed communication measurement systems. In particular we investigate the performance of these selectors in the presence of unspecified noise correlation. Using Monte Carlo, we interpret selectors as components of a multivariate random variable and study their joint distribution. We find that several selectors, despite their widespread use, are not robust to unspecified noise correlations. Specifically, the discrepancy principle fails to return adequate regularizations for rough noise, while the generalized cross validation, unbiased predictive risk, and information complexity selectors can fail for smooth noise. For some experimental configurations generalized cross validation failed completely, returning zero successful inversions out of 500 attempts. These selectors share in the characteristic that they do not contain mechanisms to monitor quantities derived from the parameter-dependent solution vector. By contrast, the L-curve and quasi-optimality criteria exhibited significantly fewer failures and correlated highly with the optimal inversion across all noise levels and correlations.
IEEE Transactions on Instrumentation and Measurement