[1811.12231] ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Abstract:Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes. Some recent studies suggest a more important role of image textures. We here put these conflicting hypotheses to a quantitative test by evaluating CNNs and human observers on images with a texture-shape cue conflict. We show that ImageNet-trained CNNs are strongly biased towards recognising textures rather than shapes, which is in stark contrast to human behavioural evidence and reveals fundamentally different classification strategies. We then demonstrate that the same standard architecture (ResNet-50) that learns a texture-based representation on ImageNet is able to learn a shape-based representation instead when trained on "Stylized-ImageNet", a stylized version of ImageNet. This provides a much better fit for human behavioural performance in our well-controlled psychophysical lab setting (nine experiments totalling 48,560 psychophysical trials across 97 observers) and comes with a number of unexpected emergent benefits such as improved object detection performance and previously unseen robustness towards a wide range of image distortions, highlighting advantages of a shape-based representation.

[1811.12231] ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness --> Computer Science > Computer Vision and Pattern Recognition arXiv:1811.12231 (cs) [Submitted on 29 Nov 2018 ( v1 ), last revised 9 Nov 2022 (this version, v3)] Title: ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Authors: Robert Geirhos , Patricia Rubisch , Claudio Michaelis , Matthias Bethge , Felix A. Wichmann , Wieland Brendel View a PDF of the paper titled ImageNet-trained CNNs are biased towards texture; increas

saved by

• Kolade Alabi

related reading

• Convolutional Neural Networks, Explained | Towards Data Sciencetowardsdatascience.com
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• A Recipe for Training Neural Networkskarpathy.github.io
• [1411.5908] Understanding image representations by measuring their equivariance and equivalencearxiv.org
• Feature Visualizationdistill.pub
• The “Fraction Sense” Emerges from a Deep Convolutional Neural Networkcognitivesciencesociety.org
• CS231n Deep Learning for Computer Visioncs231n.github.io
• The Platonic Representation Hypothesisphillipi.github.io
• ImageNet Classification with Deep Convolutional Neural Networksproceedings.neurips.cc
• A Beginner's Guide To Understanding Convolutional Neural Networks – Adit Deshpande – Engineering at Forward | UCLA CS '19adeshpande3.github.io
• A Recipe for Training Neural Networkskarpathy.github.io
• [2201.03545] A ConvNet for the 2020sarxiv.org