Abstract: Stochastic optimization algorithms are widely used to solve large-scale ... To address this issue, we perform a comprehensive convergence rate analysis of stochastic gradient descent (SGD) ...