Towards An Intrinsic Definition Of Robustness For A Classifier

Finding good measures of robustness -- i.e. the ability to correctly classify corrupted input signals -- of a trained classifier is an important question for sensitive practical applications. In this paper, we point out that averaging the radius of robustness of samples in a validation set is a statistically weak measure. We propose instead to weight the importance of samples depending on their difficulty. We motivate the proposed score by a theoretical case study using logistic regression. We also empirically demonstrate the ability of the proposed score to measure robustness of classifiers with little dependence on the choice of samples in more complex settings, including deep convolutional neural networks and real datasets.