Selective Brain Damage: Measuring the Disparate Impact of Model Pruning
Neural network pruning techniques have demonstrated it is possible to remove the majority of weights in a network with surprisingly little degradation to test set accuracy. However, this measure of performance conceals significant differences in how different classes and images are impacted by pruning. We find that certain examples, which we term pruning identified exemplars (PIEs), and classes are systematically more impacted by the introduction of sparsity. Removing PIE images from the test-set greatly improves top-1 accuracy for both pruned and non-pruned models. These hard-to-generalize-to images tend to be mislabelled, of lower image quality, depict multiple objects or require fine-grained classification. These findings shed light on previously unknown trade-offs, and suggest that a high degree of caution should be exercised before pruning is used in sensitive domains. (website,paper,code)
Neural network pruning techniques have demonstrated it is possible to remove the majority of weights in a network with surprisingly little degradation to test set accuracy. However, this measure of performance conceals significant differences in how different classes and images are impacted by pruning. We find that certain examples, which we term pruning identified exemplars (PIEs), and classes are systematically more impacted by the introduction of sparsity. Removing PIE images from the test-set greatly improves top-1 accuracy for both pruned and non-pruned models. These hard-to-generalize-to images tend to be mislabelled, of lower image quality, depict multiple objects or require fine-grained classification. These findings shed light on previously unknown trade-offs, and suggest that a high degree of caution should be exercised before pruning is used in sensitive domains. (website,paper,code)
The State of Sparsity in Deep Neural Networks
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet. Across thousands of experiments, we demonstrate that complex bayesian techniques shown to yield high compression rates on smaller datasets perform inconsistently, and that simple magnitude pruning approaches achieve comparable or better results. Based on insights from our experiments, we achieve a new state-of-the-art sparsity-accuracy trade-off for ResNet-50 using only magnitude pruning. Additionally, we consider the contributions of research work that suggests pruning is a type of architecture search. We find that on our large scale setup, that unstructured sparse architectures learned through pruning cannot be trained from scratch to the same test set performance as a model trained with joint sparsification and optimization.
paper, code
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet. Across thousands of experiments, we demonstrate that complex bayesian techniques shown to yield high compression rates on smaller datasets perform inconsistently, and that simple magnitude pruning approaches achieve comparable or better results. Based on insights from our experiments, we achieve a new state-of-the-art sparsity-accuracy trade-off for ResNet-50 using only magnitude pruning. Additionally, we consider the contributions of research work that suggests pruning is a type of architecture search. We find that on our large scale setup, that unstructured sparse architectures learned through pruning cannot be trained from scratch to the same test set performance as a model trained with joint sparsification and optimization.
paper, code
Evaluating Feature Importance Estimates
Interpretability methods should be both meaningful to a human and correctly explain model behavior. In this work, we propose ROAR, RemOve And Retrain, a formal measure to evaluate the latter. ROAR measures the relative accuracy of commonly used interpretability methods and a set of recently proposed ensemble-based derivative approaches. Our results across several large-scale image classification datasets are consistent and thought-provoking -- we find that the interpretability methods we consider produce estimates that are less accurate or on par with a random designation of feature importance. However, certain derivative approaches that ensemble these estimates far outperform such a random guess. The manner of ensembling remains critical, we show that some approaches do no better than the underlying method but carry a far higher computational burden.
Slides, code, paper.
Interpretability methods should be both meaningful to a human and correctly explain model behavior. In this work, we propose ROAR, RemOve And Retrain, a formal measure to evaluate the latter. ROAR measures the relative accuracy of commonly used interpretability methods and a set of recently proposed ensemble-based derivative approaches. Our results across several large-scale image classification datasets are consistent and thought-provoking -- we find that the interpretability methods we consider produce estimates that are less accurate or on par with a random designation of feature importance. However, certain derivative approaches that ensemble these estimates far outperform such a random guess. The manner of ensembling remains critical, we show that some approaches do no better than the underlying method but carry a far higher computational burden.
Slides, code, paper.
The (Un)reliability of Saliency Methods
Saliency methods aim to explain the predictions of deep neural networks. These methods lack reliability when the explanation is sensitive to factors that do not contribute to the model prediction. We use a simple and common pre-processing step —adding a constant shift to the input data— to show that a transformation with no effect on the model can cause numerous methods to incorrectly attribute.
In order to guarantee reliability, we posit that methods should fulfill input invariance, the requirement that a saliency method mirror the sensitivity of the model with respect to transformations of the input. We show, through several examples, that saliency methods that do not satisfy input invariance result in misleading attribution.
Paper, Slides.
Saliency methods aim to explain the predictions of deep neural networks. These methods lack reliability when the explanation is sensitive to factors that do not contribute to the model prediction. We use a simple and common pre-processing step —adding a constant shift to the input data— to show that a transformation with no effect on the model can cause numerous methods to incorrectly attribute.
In order to guarantee reliability, we posit that methods should fulfill input invariance, the requirement that a saliency method mirror the sensitivity of the model with respect to transformations of the input. We show, through several examples, that saliency methods that do not satisfy input invariance result in misleading attribution.
Paper, Slides.