In brief
What the paper does.
CASA regularizes the inconsistency between policy evaluation and policy improvement in model-free RL. Treating the evaluation step as a compensation for the improvement step alleviates the gradient conflict between the two, which is equivalent to entropy-regularized policy improvement and keeps the policy out of suboptimal solutions; it outperforms strong baselines on the Arcade Learning Environment.
Cite
Cite the paper.
@inproceedings{xiao2022casaneurips,
title={CASA: Bridging the Gap between Policy Improvement and Policy Evaluation with Conflict Averse Policy Iteration},
author={Xiao, Changnan and Shi, Haosen and Fan, Jiajun and Deng, Shihong and Yin, Haiyan},
booktitle={Deep Reinforcement Learning Workshop, NeurIPS 2022},
year={2022}
}