TL;DR — Population-based RL methods improve exploration by running diverse policies, but are fundamentally limited by a fixed, predefined population. LBC breaks this limitation by learning a hybrid behavior mapping over all policies, enabling a dramatically enlarged behavior space — and achieves superhuman performance with 78× less data.
The Core Idea
Population-based methods fix a set of exploratory policies and select between them. LBC instead constructs a continuous, learnable behavior mapping space that blends all policies, then uses a bandit-based meta-controller to learn which behaviors to select at each moment:
Hybrid Behavior Mapping
Instead of selecting from a fixed population, LBC parameterizes a convex combination space over all policies — infinite diversity from a finite set of base agents.
Bandit Meta-Controller
A lightweight bandit algorithm learns which behavior mapping to activate for each episode, balancing exploration across the behavior space with exploitation of known good behaviors.
Off-Policy Integration
LBC is integrated into distributed off-policy actor-critic methods — compatible with existing RL infrastructure without major architectural changes.
Unified Perspective
Provides a unified view of diverse RL algorithms as special cases of behavior control, opening new directions for understanding exploration in deep RL.
24 World Records Broken
LBC broke 24 Atari human world records within just 1 billion training frames:
| Method | Human-Norm. Score | Frames Used | World Records |
|---|---|---|---|
| Agent57 (DeepMind) | ~4,766% | 78 Billion | 0 |
| NGU (DeepMind) | ~3,421% | 35 Billion | 0 |
| R2D2 (DeepMind) | ~4,038% | 10 Billion | ~3 |
| LBC (Ours) | 10,077% Best | 1 Billion 78× less | 24 Records! |
Quantitative Comparison
LBC (Ours)
10,077% mean human-normalized score
24 world records
1B training frames
Agent57 (Prior SOTA)
4,766% mean human-normalized score
0 world records
78B training frames (78× more)
R2D2
4,038% mean human-normalized score
10B training frames
Publication Journey
BibTeX
@inproceedings{fan2023learnable,
title={Learnable Behavior Control: Breaking Atari Human World Records
via Sample-Efficient Behavior Selection},
author={Jiajun Fan and Yuzheng Zhuang and Yuecheng Liu and Jianye HAO
and Bin Wang and Jiangcheng Zhu and Hao Wang and Shu-Tao Xia},
booktitle={The Eleventh International Conference on Learning Representations},
year={2023},
url={https://openreview.net/forum?id=FeWvD0L_a4}
}