Neural Manifold Ordinary Differential Equations

To better conform to data geometry, recent deep generative modelling techniques adapt Euclidean constructions to non-Euclidean spaces. In this paper, we study normalizing flows on manifolds. Previous work has developed flow models for specific cases; however, these advancements hand craft layers on a manifold-by-manifold basis, restricting generality and inducing cumbersome design constraints. We overcome these issues by introducing Neural Manifold Ordinary Differential Equations, a manifold generalization of Neural ODEs, which enables the construction of Manifold Continuous Normalizing Flows (MCNFs). MCNFs require only local geometry (therefore generalizing to arbitrary manifolds) and compute probabilities with continuous change of variables (allowing for a simple and expressive flow construction). We find that leveraging continuous manifold dynamics produces a marked improvement for both density estimation and downstream tasks.

Neural Manifold Ordinary Differential Equations Aaron Lou*, Derek Lim*, Isay Katsman*, Leo Huang*, Qingxuan Jiang Cornell University {al968, dl772, isk22, ah839, qj46}@cornell.edu arXiv:2006.10254v1 [stat.ML] 18 Jun 2020 Ser-Nam Lim Christopher De Sa…

saved by

• Roland Riachi

related reading

• Scalable Gradients for Stochastic Differential Equationsarxiv.org
• [1707.01415] Machine Learning, Deepest Learning: Statistical Data Assimilation Problemsarxiv.org
• Modular Manifolds - Thinking Machines Labthinkingmachines.ai
• Neural Ordinary Differential Equationsarxiv.org
• 1806.07366arxiv.org
• Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
• Flow Matching for Generative Modelingalphaxiv.org
• Riemannian Metric Matching for Scalable Geometric Modeling of Distributionsarxiv.org
• Normalizing Flows for Probabilistic Modeling and Inferencealphaxiv.org
• [2604.04891] Muon Dynamics as a Spectral Wasserstein Flowarxiv.org
• Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitarxiv.org
• Denoising Density Estimatorsarxiv.org