Emmeran Johnson, David Martínez-Rubio, Ciara Pike-Burke, Patrick Rebeschini
Adaptive regularization is necessary for optimal online learning on high-dimensional $oldsymbol{ ext{ell}_p}$-balls, as fixed regularization cannot achieve optimality across all dimension regimes.
In the field of online learning, researchers are exploring how to make predictions or decisions over time in a way that minimizes mistakes or 'regret'. This study focuses on a specific mathematical setting involving $oldsymbol{ ext{ell}_p}$-balls, which are geometric shapes in high-dimensional spaces. The key finding is that to achieve the best performance in these settings, the method of regularization (a technique to prevent overfitting) needs to adapt based on the dimensions of the problem. Fixed, non-adaptive regularization methods fall short in certain scenarios, highlighting the importance of adaptability.
