✳flâneur — a map of the web's best reading

LaCoOT: Layer Collapse through Optimal Transport

arxiv.org · 16,022 words · saved by 1 readers

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. Although deep neural networks are well-known for their outstanding performance in tackling complex tasks, their hunger for computational resources remains a significant hurdle, posing energy-consumption issues and restricting their deployment on resource-constrained devices, preventing their widespread adoption. In this paper, we present an optimal transport-based method to reduce the depth of over-parametrized deep neural networks, alleviating their computational burden. More specifically, we propose a new regularization strategy based on the Max-Sliced Wasserstein distance to minimize the distance between the intermediate feature distributions in the neural network. We show that minimizing this distance enables the complete removal of intermediate laye

LaCoOT: Layer Collapse through Optimal Transport Victor Quétu 1 Zhu Liao 1 Nour Hezbri 2 Fabio Pizzati 3 Enzo Tartaglione 1 1 LTCI, Télécom Paris, Institut Polytechnique de Paris, France 2 ENSAE, Institut Polytechnique de Paris, France 3 MBZUAI, UAE victor.quetu@telecom-paris.fr Abstract Although deep neural networks are well-known for their outstanding performance in tackling complex tasks, their hunger for computational resources remains a significant hurdle, posing energy-consumption issues and restricting their deployment on resource-constrained devices, preventing their widespread adoptio

Explore this link on the map →

related reading