In brief
What the paper does.
GDI tackles sample efficiency and final performance at once by casting both as optimizing the training data distribution: it models and controls the capacity and diversity of the behavior policy and adapts its sampling distribution. Integrated into Generalized Policy Iteration, it yields operator-based versions of RL methods from DQN to Agent57 and surpasses 22 human world records on Atari with only 200M training frames.
Cite
Cite the paper.
@InProceedings{pmlr-v162-fan22c,
title = {Generalized Data Distribution Iteration},
author = {Fan, Jiajun and Xiao, Changnan},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
pages = {6103--6184},
year = {2022},
editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
volume = {162},
series = {Proceedings of Machine Learning Research},
month = {17--23 Jul},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v162/fan22c/fan22c.pdf},
url = {https://proceedings.mlr.press/v162/fan22c.html}
}