Learning representational invariance instead of categorization

The current most accurate models of image object cat-
egorization are deep neural networks trained on large la-
beled data sets. Minimizing a classification loss between
the predictions of the network and the true labels has proven
an effective way to learn discriminative functions of the ob-
ject classes. However, recent studies have suggested that
such models learn highly discriminative features that are
not aligned with visual perception and might be at the root
of adversarial vulnerability. Here, we propose to replace
the classification loss with the joint optimization of invari-
ance to identity-preserving transformations of images (data
augmentation invariance), and the invariance to objects of
the same category (class-wise invariance). We hypothesize
that optimizing these invariance objectives might yield fea-
tures more aligned with visual perception, more robust to
adversarial perturbations, while still suitable for accurate
object categorization.