[2406.14453] The Effective Number of Parameters in Kernel Density Estimation
Abstract:The quest for a formula that satisfactorily measures the effective degrees of freedom in kernel density estimation (KDE) is a long standing problem with few solutions. Starting from the orthogonal polynomial sequence (OPS) expansion for the ratio of the empirical to the oracle density, we show how convolution with the kernel leads to a new OPS with respect to which one may express the resulting KDE. The expansion coefficients of the two OPS systems can then be related via a kernel sensitivity matrix, and this then naturally leads to a definition of effective parameters by taking the trace of a symmetrized positive semi-definite normalized version. The resulting effective degrees of freedom (EDoF) formula is an oracle-based quantity; the first ever proposed in the literature. Asymptotic properties of the empirical EDoF are worked out through influence functions. Numerical investigations confirm the theoretical insights.
The Effective Number of Parameters in Kernel Density Estimation Sofia Guglielmini∗ Igor Volobouev† A. Alexandre Trindade‡ arXiv:2406.14453v2 [stat.ME] 23 Mar 2025 March 25, 2025 Abstract We devise a formula for measuring the…
saved by
related reading
- High-Dim Stat Rigolletarxiv.org
- [2408.05807] Kernel Density Estimators in Large Dimensionsarxiv.org
- Nonparametric statistics - Wikipediaen.wikipedia.org
- Approximating KL Divergencejoschu.net
- Gregory Gundersengregorygundersen.com
- [2110.03922] The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networksarxiv.org
- Some Math behind Neural Tangent Kernel | Lil'Loglilianweng.github.io
- Kullback–Leibler divergence - Wikipediaen.wikipedia.org
- Denoising Density Estimatorsarxiv.org
- German tank problem - Wikipediaen.wikipedia.org
- Deep Energy Estimator.pdfarxiv.org
- Kalman filter - Wikipediaen.wikipedia.org