Yanjun Chen 陈彦筠

I am a PhD candidate in the Department of Computing at The Hong Kong Polytechnic University, advised by Prof. Wenjie Li (Maggie) and Prof. Wei Zhang, with joint doctoral training at the Eastern Institute of Technology (EIT), Ningbo. I am currently a research intern at Microsoft Research Asia, Singapore.

In reinforcement learning, models are trainable. The environments that train them are not. I want to make the environment trainable, the way models are, and with it to lift the ceiling of what AI can become.

Research

Making the pieces around a model trainable starts with knowing what they actually do in training, and which signal can say exactly what each step is worth.

Reward-model accuracy and training outcomes in RLHF. The reward model is the most typical piece of a training environment and is usually judged by its accuracy. Yet moderately accurate reward models train better language models than the most accurate ones, so a piece’s worth has to be judged inside training.

The Accuracy Paradox in RLHF: When Better Reward Models Don’t Yield Better Language Models (EMNLP 2024).

Princeton’s What Makes a Reward Model a Good Teacher? (NeurIPS 2025) opens with this finding; the ACL 2025 tutorial Synthetic Data in the Era of LLMs gives it a full slide.

Exact credit for LLM agent teams. What each message in a team of LLM agents was worth has mostly been predicted, but when the team communicates through a shared context and everything a downstream agent reads is written into the trace, the trace is the state. One message can then be replaced and the run continued to its end, which makes its credit exact. For an LLM, much of the environment arrives as context (retrieved text, tool outputs, prompts), and that is where the same method points next: assigning credit to those pieces as well.

The Trace Is the State: Exact Credit Assignment for LLM Agent Teams (arXiv:2603.06859, in submission).

When a policy absorbs action shaping. Reward shaping has a theorem guaranteeing that a potential-based term can be removed; the same practice on the action channel, a training-time offset, has none. A trainable policy absorbs an offset its own output layer can reproduce exactly, and once absorbed, the offset can be removed with the return almost unchanged.

Action Shaping: Policies Absorb What They Can Express (arXiv:2609.32752, in submission).

Where I’m Going

The destination: an environment that learns alongside the model it trains, from language to embodied agents. The path runs through credit: credit should not stop at the agent’s boundary. Once trustworthy learning signals reach the pieces around the model, the environment can begin to learn.

With thanks to Xiaoyu Shen and Dawei Zhu, whose ongoing mentorship and guidance have shaped much of how I think about research.