Jiajun Fan

ICML 2022

Generalized Data Distribution Iteration

In brief

What the paper does.

GDI tackles sample efficiency and final performance at once by casting both as optimizing the training data distribution: it models and controls the capacity and diversity of the behavior policy and adapts its sampling distribution. Integrated into Generalized Policy Iteration, it yields operator-based versions of RL methods from DQN to Agent57 and surpasses 22 human world records on Atari with only 200M training frames.

Published in
International Conference on Machine Learning (ICML 2022)
Paper
proceedings.mlr.press/v162/fan22c.html
Project page
jiajunfan.com/projects/gdi/

Cite

Cite the paper.

@InProceedings{pmlr-v162-fan22c,
  title     = {Generalized Data Distribution Iteration},
  author    = {Fan, Jiajun and Xiao, Changnan},
  booktitle = {Proceedings of the 39th International Conference on Machine Learning},
  pages     = {6103--6184},
  year      = {2022},
  editor    = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume    = {162},
  series    = {Proceedings of Machine Learning Research},
  month     = {17--23 Jul},
  publisher = {PMLR},
  pdf       = {https://proceedings.mlr.press/v162/fan22c/fan22c.pdf},
  url       = {https://proceedings.mlr.press/v162/fan22c.html}
}