Asynchronous Stochastic Block Coordinate Descent with Variance Reduction
Gu, Huo, Huang · cs.LG · 2016-11-14 · 原文
Asynchronous parallel implementations for stochastic optimization have received huge successes in theory and practice recently. Asynchronous implementations with lock-free are more efficient than the one with writing or reading lock. In this paper, we focus on a composite objective function consisting of a smooth convex function f and a block separable convex function, which widely exists in machine learning and computer vision. We propose an asynchronous stochastic block coordinate descent algorithm with the accelerated technology of variance reduction (AsySBCDVR), which are with lock-free in the implementation and analysis. AsySBCDVR is particularly important because it can scale well with the sample size and dimension simultaneously. We prove that AsySBCDVR achieves a linear convergence rate when the function f is with the optimal strong convexity property, and a sublinear rate when f is with the general convexity. More importantly, a near-linear speedup on a parallel system with shared memory can be obtained.
讲义
讲义·推断 依据「原文」自动生成的结构化摘要（推断），非原文表述；以原文为准。
1. 人话版
Asynchronous parallel implementations for stochastic optimization have received huge successes in theory and practice recently.
Asynchronous implementations with lock-free are more efficient than the one with writing or reading lock.
2. 领域脉络
本文类目：cs.LG，属于其所在研究脉络的最新进展。
3. 机制拆解
In this paper, we focus on a composite objective function consisting of a smooth convex function f and a block separable convex function, which widely exists in machine learning and computer vision.
We propose an asynchronous stochastic block coordinate descent algorithm with the accelerated technology of variance reduction (AsySBCDVR), which are with lock-free in the implementation and analysis.
AsySBCDVR is particularly important because it can scale well with the sample size and dimension simultaneously.
4. 证据与数字
摘要未给出量化结果——留意原文的实验与数据。
5. 反例与边界
摘要未声明局限与反例——这是需要警惕的信号，精读时先问边界。
6. 跨领域连接与意外收获
思考本文机制能否迁移到你正在跟进的问题。
7. 可复用方法
把本文机制与你手头项目对照，找一个两周内能验证的最小实验。
8. 术语表
精读时把不熟的术语记入此处，作为下次回忆的锚点。