On the Link Between Polynomials and Optimization, Part 1

There's a fascinating link between minimization of quadratic functions and polynomials. A link that goes deep and allows to phrase optimization problems in the language of polynomials and vice versa. Using this connection, we can tap into centuries of research in the theory of polynomials and shed new light on …

There's a fascinating link between minimization of quadratic functions and polynomials. A link that goes deep and allows to phrase optimization problems in the language of polynomials and vice versa. Using this connection, we can tap into centuries of research in the theory of polynomials and shed new light on old problems. $$ \def\HH{\boldsymbol a} \def\rr{\boldsymbol r} \def\HH{\boldsymbol A} \def\HH{\boldsymbol H} \def\EE{\mathbb E} \def\II{\boldsymbol I} \def\CC{\boldsymbol C} \def\DD{\boldsymbol D} \def\KK{\boldsymbol K} \def\eeps{\boldsymbol \varepsilon} \def\tr{\text{tr}} \def\LLambda{\

related reading

• Why Momentum Really Worksdistill.pub
• Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
• Universal Complexity Bounds for Universal Gradient Methods in Nonlinear Optimizationarxiv.org
• Polynomial Regression - An Alternative For Neural Networks? | Towards Data Sciencetowardsdatascience.com
• Closed-form dynamics beyond quadratics – Machine Learning Research Blogfrancisbach.com
• prox_algs.pdfweb.stanford.edu
• Mathematical optimization - Wikipediaen.wikipedia.org
• bv_cvxbook.pdfweb.stanford.edu
• [2605.15522] Stochastic Non-Smooth Convex Optimization with Unbounded Gradientsarxiv.org
• Keep the gradient flowingfa.bianp.net
• [1605.08101] Global rates of convergence for nonconvex optimization on manifoldsarxiv.org
• Proofs involving ordinary least squares - Wikipediaen.wikipedia.org