题    目：梯度稳控修正的随机RMSProp算法在最小二乘问题中的应用

主讲人：邢文训 教授

单    位：清华大学

时    间：2025年5月23日 15:00

地    点：九章学堂南楼C座302

摘   要： Root mean square propagation (abbreviated as RMSProp) is a first-order stochastic algorithm used in machine learning widely. In this talk, a stable gradient-adjusted RMSProp (abbreviated as SGA-RMSProp) with mini-batch stochastic gradient is proposed for the linear least squares problem. R-linear convergence of the algorithm is established on the consistent linear least squares problem. The algorithm is also proved to converge R-linearly to a neighborhood of the minimizer for the inconsistent case, with the region of the neighborhood being controlled by the batch size. Furthermore, numerical experiments are conducted to compare the performances of SGA-RMSProp and stochastic gradient descend (abbreviated as SGD) with different batch sizes. The faster initial convergence rate of SGA-RMSProp is observed through numerical experiments and an adaptive strategy for switching from SGA-RMSProp to SGD is proposed, which combines the benefits of these two algorithms.

简    介：清华大学数学科学系教授、博士生导师，北京大学理学学士，清华大学理学博士。目前研究兴趣为非凸/非光滑全局最优化及组合最优化问题，在国内外学术刊物SIAM Journal on Optimization, European Journal of Operational Research, IIE Transactions, Discrete Applied Mathematics, Annals of Operations Research等发表论文70余篇，出版专著1部，教材7部。2007年获得国防科工委国防科学技术进步奖（一等），2008年获国家科学技术进步奖（二等），2001年获中国运筹学会运筹学应用奖（二等）。目前为中国运筹学会监事，JORSC编委等。