Selected Publications

See my Google Scholar for a comprehensive listing!

*

preprint
We propose PersRM-R1, a reasoning-based reward model that learns personal preferences from just a few examples. Using synthetic data and a two-stage training pipeline, it achieves high accuracy and generalization, outperforming models of similar size and rivaling much larger ones—paving the way for more personalized LLMs.

preprint
We propose Curriculum-RLAIF, a data-centric framework that improves reward model generalizability by training on preference pairs of increasing difficulty. This curriculum-based approach addresses data noise, distribution shift, and model-capacity mismatch. Experiments show that Curriculum-RLAIF significantly boosts policy alignment performance without extra inference cost, outperforming non-curriculum and alternative strategies.

TMLR 2025 18th EWRL
This study explores whether LLMs can mentally model decision-making agents by reasoning over their behavior and state transitions from interaction histories. Evaluated on reinforcement learning tasks, results show that while LLMs offer some insight, they fall short of fully modeling agents without further innovation, highlighting both their potential and current limitations for explainable RL.

IJCAI 2025 Workshop (oral)
We propose REAL (Response Embedding-based Alignment for LLMs), a method to improve alignment efficiency by selecting less ambiguous, dissimilar response pairs for annotation. By leveraging embedding similarity in an off-policy manner, REAL reduces label noise and improves alignment quality. Experiments show it boosts performance while cutting annotation effort by up to 65%.

preprint 2025
LLM+MAP is a bimanual planning framework that combines GPT-4o with multi-agent task planning to enable efficient and logically consistent long-horizon manipulation. It outperforms baseline LLMs on planning time, success rate, and coordination metrics.

CoRL 2024 Workshop ICRA@40
We propose an LLM-driven framework that enables robots to autonomously discover useful skills from scratch. By generating tasks, rewards, and success criteria, the LLM guides reinforcement learning, while a vision-language model verifies outcomes. This allows the robot to build a meaningful skill library without relying on predefined primitives.

COLING 2024 (oral)
We propose LoT (Logical Thoughts), a framework that improves large language models’ reasoning at inference time by applying symbolic logic to verify and correct their step-by-step thought process. LoT enhances performance on diverse reasoning tasks and reduces hallucinations.

ICANN 2024
We introduce OSSA (Object State-Sensitive Agent), a task-planning agent using pre-trained LLMs and VLMs to generate plans sensitive to object states. We compare two methods: a modular approach combining vision and language models, and a monolithic VLM approach. Evaluated on tabletop tasks involving clearing a table, OSSA’s monolithic model outperforms the modular one. A new multimodal benchmark dataset with object state annotations is provided.

Humanoids 2024
LABOR uses LLMs to orchestrate control policies for long-horizon bimanual manipulation tasks. By leveraging task reasoning and coordination via language, it achieves higher success rates on simulated tasks with the NICOL robot and provides insights into LLM-based control challenges.

CLeaR 2024 (oral)
We propose reward decomposition methods for better decision-making explainality.

IROS 2023 (oral)
We present Matcha agent, an interactive perception framework that uses LLMs to guide robots in gathering multimodal sensory data (vision, sound, haptics, proprioception) before executing tasks. Matcha enables high-level reasoning and planning in partially observable environments, showing that LLMs can effectively control robot behavior when grounded with multimodal context.

CoRL 2023 Workshop (oral)
Lafite-RL is a framework that leverages Large Language Models to provide natural language feedback for guiding reinforcement learning in robotic tasks. Tested on RLBench, it improves learning efficiency and success rates without requiring costly human supervision.

ICML 2023
We introduce Internally Rewarded Reinforcement Learning (IRRL), where rewards are generated by a jointly learned internal model rather than the environment. This coupling of policy and reward learning can destabilize training. We formalize IRRL, analyze its challenges, and propose a clipped linear reward function that reduces reward noise. Experiments show improved stability, faster convergence, and better performance across tasks.

ICDL 2023 (oral)
Explainable Q-Map improves the transparency of RL agents by combining reward decomposition with abstract action spaces, enabling clear, high-level explanations based on task-relevant object properties. We demonstrate visual and textual explanations in robotic scenarios and show how they can be used with LLMs for reasoning and interactive querying.

IROS 2022 (oral)
We propose the Intrinsic Sound Curiosity Module (ISCM) to use sound as an informative modality for unsupervised reinforcement learning. In realistic manipulation scenarios with simulated audio, ISCM guides exploration and representation learning. Experiments show that sound-driven pre-training leads to better representations and faster adaptation than vision-only baselines.

CSCWD 2021
DWDS is a density-weighted diversity strategy for active learning in deep learning. It selects informative and representative samples using geometric insights and a beam search for efficient query selection. DWDS consistently outperforms existing methods under limited labeling budgets.

Recent Posts

Learn & Share

Last updated on 6 min read
A probabilistic explanation of Elo rating approach.

Presentations

Activities & Talks

Introduced trending and advancement of agentic intelligence, including the paradigm from reinforcement learning (RL) to large language …

Introduced trending and advancement of agentic intelligence, including the paradigm from reinforcement learning (RL) to large language …

See all

Experience

Education & Work

 
 
 
 
 

Ph.D. in Computer Science

Univerisity of Hamburg

Mar 2021 - Present Hamburg, Germany
  • Doctoral advisor: Prof. Stefan Wermter
  • Ph.D. Thesis: “Environment Exploration and Autonomous Adaptation in Embodied Agents” (under review)
 
 
 
 
 

AI Engineer

JD AI Core Department

Jul 2018 - Jun 2020 Beijing, China

I developed core machine learning algorithms across several domains, including

  • Sales prediction and personalized recommendation (time series modeling)
  • Image quality assessment, aesthetic ranking; drug identification in medical cabinets; and object detection for unmanned cargo machines (computer vision)
  • Toxic dialogue detection (natural language processing)
 
 
 
 
 

M.Sc. in Signal Processing

University of Chinese Academy of Sciences

Sep 2015 - Jun 2018 Beijing, China
  • Advisor: Prof. Daojing Li
  • Conducted research on machine learning and advanced signal processing methods, applied to radar object detection and multipath clutter suppression.
 
 
 
 
 

Signal Processing Engineer

ExtantFuture

Jan 2015 - Jun 2015 Beijing, China
Developed signal denoising and core processing algorithms for a wearable medical device to monitor fetal health indicators, including fetal heart rate and movement, as well as maternal activity tracking.
 
 
 
 
 

B.E. in Electronic Information Engineering

Xidian University

Sep 2010 - Jun 2014 Xi'an, China
Studied digital circuit design, analog circuit design, and microcontroller-based control systems. Participated in electronic design competitions and mathematical modeling contests.

Awards & Grants

Selected recognitions and funding

ICRA@40 Travel Grants

IEEE Robotics and Automation Society 2024

Researcher Access Program

OpenAI 2024

Academic Scholarship

University of Chinese Academy of Sciences 2015 – 2018

Excellent Student Cadre & Triple-A Student

University of Chinese Academy of Sciences 2015

The Second Prize of China Undergraduate Mathematical Contest in Modeling

China Society for Industrial and Applied Mathematics 2013

The Second Prize of Xinghuo Electronic Competition

Xidian University 2012

The First Prize of Xidian’s Mathematical Contest in Modeling

Xidian University 2012

Contact

Connect with me