Three papers from MIG: Offline Meta Reinforcement Learning with In-Distribution Online Adaptation, Symmetry-Aware Robot Design with Structured Subgroups Inbox, What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL, have been accepted by ICML 2023.

Paper from MIG: The Provable Benefit of Unsupervised Data Sharing for Offline Reinforcement Learning, has been accepted by ICLR 2023.

Paper from MIG: Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery , has been accepted by AAAI 2023.

Six papers from MIG: Non-Linear Coordination Graphs , Safe Opponent-Exploitation Subgame Refinement , RORL: Robust Offline Reinforcement Learning via Conservative Smoothing , Low-Rank Modular Reinforcement Learning via Muscle Synergy , CUP: Critic-Guided Policy Reuse , Latent-Variable Advantage-Weighted Policy Optimization for Offline Reinforcement Learning , have been accepted by NeurIPS 2022.

Three papers from MIG: On the Role of Discount Factor in Offline Reinforcement Learning , Self-Organized Polynomial-Time Coordination Graphs , Individual Reward Assisted Multi-Agent Reinforcement Learning , have been accepted by International Conference on Machine Learning (ICML) 2022.

Four papers from MIG: Context-Aware Sparse Deep Coordination Graphs , Active Hierarchical Exploration with Stable Subgoal Representation Learning , Rethinking Goal-Conditioned Supervised Learning and Its Connection to Offline RL , Offline Reinforcement Learning with Value-based Episodic Memory have been accepted by International Conference on Learning Representations (ICLR) 2022.

Paper from MIG: Multi-Agent Incentive Communication via Decentralized Teammate Modeling , has been accepted for presentation at the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22). This year AAAI received a record 9,251 submissions, of which 9,020 were reviewed. Based on a thorough and rigorous review process, AAAI has accepted 1,349 papers. This yields an overall acceptance rate of 15%.

Jianhao Wang, a PhD candidate from MIG group, was awarded with National Scholarship (top 1%) at Tsinghua University, due to his impressing scientific research achievements. Lulu Zheng, a master candidate from MIG group, was awarded with National Scholarship, too. National Scholarship is the most important award in Tsinghua University and this year only 5 graduate students obtained this scholarship in Institute for Interdisciplinary Information Sciences (IIIS). Till now, there are 5 students in total from MIG group who has won National Scholarship.

Six papers from MIG: Celebrating Diversity in Shared Multi-Agent Reinforcement Learning , Offline Reinforcement Learning with Reverse Model-based Imagination, On the Estimation Bias in Double Q-Learning , Episodic Multi-agent Reinforcement Learning with Curiosity-driven Exploration, Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorization, Model-Based Reinforcement Learning via Imagination with Derived Memory, were accepted at NeurIPS 2021. The topics of these papers are diverse, including multi-agent RL, offline RL, and Model-based RL.

Guangxiang Zhu, Terry Liu, and Tonghan Wang graduated from MIG in June, 2021. Guangxiang, as the first student to obtain a PhD's degree in MIG, has won the award for Excellent Doctoral Dissertation of Tsinghua University, 2021, creating a milestone for MIG. Besides, Terry and Tonghan are the first two students to obtain a Master of Computer Sciences in MIG.

Two papers from MIG, MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration , and Generalizable Episodic Memory for Deep Reinforcement Learning were accepted at ICML 2021, There were 5513 submissions to ICML this year, of which the program committee accepted 1184 for presentation at the conference, including 166 long presentations and 1018 short presentations.

The paper from MIG collaborated with NetEase, Reward-Constrained Behavior Cloning , was accepted for presentation at IJCAI-21 (the 30th International Joint Conference on Artificial Intelligence). Out of the 4204 full-paper submissions, 587 papers are finally accepted, at a 13.9% acceptance rate (19.3% out of the 3033 papers which passed the summary-reject review phase and received full reviews).

Four papers from MIG were accepted by International Conference on Learning Representations (ICLR) 2021. Tonghan Wang, Jianhao Wang and Beining Han focused Multi-Agent Reinforcement Learning (MARL). Our group's paper: RODE: Learning Roles to Decompose Multi-Agent Tasks. and Off-Policy Multi-Agent Decomposed Policy Gradients. (cooperated with Beining) are accepted. Another accepted paper QPLEX: Duplex Dueling Multi-Agent Q-Learning encodes the IGM principle into the neural network architecture and thus enables efficient value function learning. Siyuan Li and Lulu Zheng focused on hierarchical Reinforcement Learning and used slow features to define subgoals in their paper Learning Subgoal Representations with Slow Dynamics , which was also accepted in ICLR 2021.

Tonghan Wang, a master candidate from MIG group, was awarded with National Scholarship (top 1%) at Tsinghua University, due to his impressing scientific research achievements. National Scholarship is the most important award in Tsinghua University and only 4 graduate students can obtain this scholarship in Institute for Interdisciplinary Information Sciences (IIIS) every year. Besides, Tonghan is the third student in MIG group to win National Scholarship.

Sample efficiency has been one of the major challenges for deep reinforcement learning. Recently, model-based reinforcement learning has been proposed to address this challenge by performing planning on imaginary trajectories with a learned world model. However, world model learning may suffer from overfitting to training trajectories, and thus model-based value estimation and policy search will be prone to be sucked in an inferior local policy. In this paper, we propose a novel model-based reinforcement learning algorithm, called BrIdging Reality and Dream (BIRD). It maximizes the mutual information between imaginary and real trajectories so that the policy improvement learned from imaginary trajectories can be easily generalized to real trajectories. We demonstrate that our approach improves sample efficiency of model-based planning, and achieves state-of-the-art performance on challenging visual control benchmarks.

ROMA learns sub-task specialization in cooperative agent teams by proposing a role-oriented learning framework. In this framework, roles are emergent, and agents with similar roles tend to share their learning to be specialized on certain sub-tasks. ROMA can learn specialized, dynamic, versatile, and identifiable roles, and push forward the state of the art on the StarCraft II micromanagement benchmark.

Three papers from MIG were accepted by International Conference on Learning Representations (ICLR) 2020. Tonghan Wang and Jianhao Wang focused on exploration and communication problems in Multi-Agent Reinforcement Learning (MARL). Both of their papers: Influence-Based Multi-Agent Exploration and Learning Nearly Decomposable Value-Functions via Communication Minimization were accepted. Influence-Based Multi-Agent Exploration was accepted as spotlight. Guangxiang Zhu focused on combining episodic control with reinforcement learning and his paper Episodic Reinforcement Learning with Associative Memory was also accepted.

This paper presents a novel self-supervised object-oriented framework to enable efficient object-based dynamics learning and planning, which employs a three-level learning architecture from motion detection, to dynamic instance segmentation, and to dynamics learning. This framework can learn model from few interactions with environments and enable an agent to directly plan in unseen environments without retraining. In addition, it learns semantically and visually interpretable representations.

GuangXiang Zhu and Siyuan Li, two PhD candidates from MIG group, were both awarded with National Scholarship (top 1%) at Tsinghua University, for their impressing achievements in scientific research. National Scholarship is the most important award in Tsinghua University and only 4 graduate students can obtain this scholarship in Institute for Interdisciplinary Information Sciences (IIIS) every year.

This paper aims to adapt low-level skills to downstream tasks while maintaining the generality of reward design. Siyuan Li and Rui Wang proposed an HRL framework which sets auxiliary rewards for low-level skill training based on the advantage function of the high-level policy. This auxiliary reward enables efficient, simultaneous learning of the high-level policy and low-level skills without using task-specific knowledge. In addition, they also theoretically prove that optimizing low-level skills with this auxiliary reward will increase the task return for the joint policy.