Kevin Kurian Thomas Vaidyan, Michael P. Friedlander, Ahmet Alacaoglu
This paper establishes optimal convergence rates for the last iterate of stochastic proximal algorithms without assuming bounded variance, applicable to problems in multi-task and federated learning.
The study investigates two algorithms used for optimizing problems where the goal is to minimize a function that is a combination of smooth and nonsmooth components. These algorithms are particularly useful in situations like multi-task learning and federated learning, where tasks are interconnected. The authors show that these algorithms can achieve optimal convergence rates without needing a common assumption about variance, making them more broadly applicable. This advancement helps improve the performance of these algorithms in practical applications involving complex data structures.
