New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

signSGD with majority vote is communication efficient and fault tolerant #11

Open

nocotan opened this issue May 31, 2021 · 0 comments

Assignees

Labels

gradient quantization signSGD

Member

nocotan commented May 31, 2021 •

edited

Loading

一言でいうと

勾配の符号のみを通信するsignSGDを提案．

論文リンク

https://arxiv.org/pdf/1810.05291.pdf

著者/所属機関

Caltech

投稿日付(yyyy/MM/dd)

ICLR2019

概要

大規模分散学習において望ましい性質としてはいかが挙げられる：

アルゴリズムの高速な収束性能
良好な汎化性能
通信コストの効率性
通信エラーに対する頑健性

これらを全て満たすアルゴリズムとして，勾配の符号のみを通信するsignSGDを提案．

新規性・差分

勾配の符号のみを通信するsignSGDを提案
signSGDの収束レートを導出

手法

理論解析

結果

コメント

nocotan added gradient quantization signSGD labels

nocotan self-assigned this

nocotan changed the title ~~[WIP] signSGD with majority vote is communication efficient and fault tolerant~~ signSGD with majority vote is communication efficient and fault tolerant

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment