RL

We propose Curriculum-RLAIF, a data-centric framework that improves reward model generalizability by training on preference pairs of increasing difficulty. This curriculum-based approach addresses data noise, distribution shift, and model-capacity mismatch. Experiments show that Curriculum-RLAIF significantly boosts policy alignment performance without extra inference cost, outperforming non-curriculum and alternative strategies.

This study explores whether LLMs can mentally model decision-making agents by reasoning over their behavior and state transitions from interaction histories. Evaluated on reinforcement learning tasks, results show that while LLMs offer some insight, they fall short of fully modeling agents without further innovation, highlighting both their potential and current limitations for explainable RL.

We propose an LLM-driven framework that enables **robots to autonomously discover useful skills from scratch**. By generating tasks, rewards, and success criteria, the LLM guides reinforcement learning, while a vision-language model verifies outcomes. This allows the robot to build a meaningful skill library without relying on predefined primitives.

We propose reward decomposition methods for better decision-making explainality.

Lafite-RL is a framework that leverages Large Language Models to provide natural language feedback for guiding reinforcement learning in robotic tasks. Tested on RLBench, it improves learning efficiency and success rates without requiring costly human supervision.

We introduce Internally Rewarded Reinforcement Learning (IRRL), where rewards are generated by a jointly learned internal model rather than the environment. This coupling of policy and reward learning can destabilize training. We formalize IRRL, analyze its challenges, and propose a clipped linear reward function that reduces reward noise. Experiments show improved stability, faster convergence, and better performance across tasks.

Explainable Q-Map improves the transparency of RL agents by combining reward decomposition with abstract action spaces, enabling clear, high-level explanations based on task-relevant object properties. We demonstrate visual and textual explanations in robotic scenarios and show how they can be used with LLMs for reasoning and interactive querying.

We propose the Intrinsic Sound Curiosity Module (ISCM) to use sound as an informative modality for unsupervised reinforcement learning. In realistic manipulation scenarios with simulated audio, ISCM guides exploration and representation learning. Experiments show that sound-driven pre-training leads to better representations and faster adaptation than vision-only baselines.