Jiajun Fan

NeurIPS 2022 Workshop

CASA: Bridging the Gap between Policy Improvement and Policy Evaluation with Conflict Averse Policy Iteration

In brief

What the paper does.

CASA regularizes the inconsistency between policy evaluation and policy improvement in model-free RL. Treating the evaluation step as a compensation for the improvement step alleviates the gradient conflict between the two, which is equivalent to entropy-regularized policy improvement and keeps the policy out of suboptimal solutions; it outperforms strong baselines on the Arcade Learning Environment.

Published in
NeurIPS 2022 Workshop on Deep Reinforcement Learning
Paper
arxiv.org/abs/2105.03923

Cite

Cite the paper.

@inproceedings{xiao2022casaneurips,
  title={CASA: Bridging the Gap between Policy Improvement and Policy Evaluation with Conflict Averse Policy Iteration},
  author={Xiao, Changnan and Shi, Haosen and Fan, Jiajun and Deng, Shihong and Yin, Haiyan},
  booktitle={Deep Reinforcement Learning Workshop, NeurIPS 2022},
  year={2022}
}