Yanjun Chen

PhD Candidate, PolyU博士候选人 · 理大博士候補者 · PolyU

yan‑jun.chen@connect.polyu.hk Hong Kong香港香港 Google Scholar GitHub X ORCID CV简历履歴書

I am a PhD candidate in the Department of Computing at The Hong Kong Polytechnic University, advised by Prof. Wenjie Li (Maggie) and Prof. Wei Zhang, with joint doctoral training at the Eastern Institute of Technology (EIT), Ningbo.

In reinforcement learning, models are trainable. The environments that train them are not. I want to make the environment trainable, the way models are, and with it to lift the ceiling of what AI can become.

Research

What does the environment actually do to the model it trains?

Reward-model evaluation in RLHF. A more accurate reward model does not always train a better policy.

The Accuracy Paradox in RLHF (EMNLP 2024).

Withdrawable shaping on the action interface. Shaping aids are added to help the agent learn, then kept forever.

Under review (2026).

Exact credit for cooperative LLM agents. When agents cooperate, a shared outcome hides what each decision contributed.

Exact Is Easier (arXiv:2603.06859, in submission).

Where I’m Going

The destination: an environment that learns alongside the model it trains, from language to embodied agents.

With thanks to Xiaoyu Shen and Dawei Zhu, whose ongoing mentorship and guidance have shaped much of how I think about research.

我是香港理工大学计算学系的博士候选人，导师为 Wenjie Li (Maggie) 教授与 Wei Zhang 教授，并在东方理工（EIT，宁波）联合培养。

在强化学习中，模型是能训练的。训练模型的环境，还不是。我想让环境也能训练起来，像模型一样，并以此把 AI 的上限抬上去。

研究方向

环境到底对它训练的模型做了什么？

RLHF 中的 reward model 评估。 更准确的 reward model，并不总是训练出更好的 policy。

The Accuracy Paradox in RLHF (EMNLP 2024).

action interface 上可撤回的 shaping。 Shaping 辅助被加入来帮助 agent 学习，然后永远留了下来。

Under review (2026).

协作式 LLM agent 的精确 credit。 当 agent 协作时，一个共享的结果隐藏了每个决策的实际贡献。

Exact Is Easier (arXiv:2603.06859, in submission).

后续方向

终局：环境与模型一起学习，从语言走向具身智能体。

感谢 Xiaoyu Shen 老师与 Dawei Zhu 师兄一直以来的指导与帮助，他们在很多方面塑造了我做研究的方式。

香港理工大学計算学科の博士候補者で、Wenjie Li (Maggie) 教授と Wei Zhang 教授の指導のもと、東方理工（EIT、寧波）との共同育成プログラムに参加しています。

強化学習において、モデルは訓練できる。モデルを訓練する環境は、まだできない。私はその環境を、モデルと同じように訓練できるものにしたい。そしてそれによって、AI の到達点を引き上げたいのです。

研究内容

環境は、訓練するモデルに実際のところ何をしているのか。

RLHF における reward model の評価。 より精度の高い reward model が、必ずしもより良い policy を訓練するわけではない。

The Accuracy Paradox in RLHF (EMNLP 2024).

action interface 上の撤回可能な shaping。 Shaping 補助は agent の学習を助けるために加えられ、そのまま永久に残される。

Under review (2026).

協調的 LLM agent の厳密な credit。 Agent が協調するとき、共有された結果は各決定の実際の寄与を隠してしまう。

Exact Is Easier (arXiv:2603.06859, in submission).

これから

目指す先は、モデルとともに学ぶ環境です。言語から身体性エージェントへ。

Xiaoyu Shen 先生と Dawei Zhu 先輩から受けた継続的なご指導とお力添えに深く感謝いたします。私の研究との向き合い方の多くは、お二人からの影響によるものです。

News近况お知らせ

Jun 24, 20262026年6月24日2026年6月24日	Passed the confirmation of candidature for my PhD at The Hong Kong Polytechnic University, with the thesis Towards Efficient Reinforcement Learning via Environment Measurement and Shaping.通过了香港理工大学的博士候选人资格确认，论文题目为 Towards Efficient Reinforcement Learning via Environment Measurement and Shaping。香港理工大学の博士候補者資格審査に合格しました。論文題目は Towards Efficient Reinforcement Learning via Environment Measurement and Shaping です。
May 22, 20262026年5月22日2026年5月22日	Shortlisted for PolyU Micro Fund 2025/26 Cohort 2 (HK$20,000 cash prize), with a conditional offer to the HKSTP Ideation Programme.入围 PolyU Micro Fund 2025/26 第二轮（HK$20,000 现金奖励），并获得 HKSTP Ideation Programme 的有条件录取。PolyU Micro Fund 2025/26 Cohort 2（賞金 HK$20,000）にショートリスト入り、HKSTP Ideation Programme に条件付きで内定。
May 08, 20262026年5月8日2026年5月8日	Released v2 of Exact Is Easier: Credit Assignment for Cooperative LLM Agents on arXiv:2603.06859.Exact Is Easier: Credit Assignment for Cooperative LLM Agents v2 已发布至 arXiv:2603.06859。Exact Is Easier: Credit Assignment for Cooperative LLM Agents v2 を arXiv:2603.06859 で公開。
Mar 06, 20262026年3月6日2026年3月6日	First arXiv release of Exact Is Easier: Credit Assignment for Cooperative LLM Agents (in submission).Exact Is Easier: Credit Assignment for Cooperative LLM Agents 在 arXiv 首次发布（投稿中）。Exact Is Easier: Credit Assignment for Cooperative LLM Agents を arXiv に初回公開（投稿中）。
May 22, 20252025年5月22日2025年5月22日	Co-authored a comprehensive survey on latent chain-of-thought reasoning (arXiv:2505.16782).与人合作完成了一篇关于隐式思维链推理（latent chain-of-thought reasoning）的综述论文（arXiv:2505.16782）。latent chain-of-thought reasoning に関する包括的なサーベイ論文を共著として発表（arXiv:2505.16782）。
May 15, 20252025年5月15日2025年5月15日	Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning accepted at ACL 2025 Findings (co-author).Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning 被 ACL 2025 Findings 接收（共同作者）。Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning が ACL 2025 Findings に採択（共著）。
Jan 15, 20252025年1月15日2025年1月15日	Fine-Grained and Multi-Dimensional Metrics for Document-Level MT accepted at NAACL 2025 (co-author).Fine-Grained and Multi-Dimensional Metrics for Document-Level MT 被 NAACL 2025 接收（共同作者）。Fine-Grained and Multi-Dimensional Metrics for Document-Level MT が NAACL 2025 に採択（共著）。

Selected Publications代表性论文主要論文

ACL Findings

Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning

Xinghao Chen, Zhixin Sun, Wenjin Guo, and 6 more authors

In Findings of the Association for Computational Linguistics (ACL Findings), 2025

arXiv Bib Code

@inproceedings{chen2025distilling,
  title = {Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning},
  author = {Chen, Xinghao and Sun, Zhixin and Guo, Wenjin and Zhang, Miao and Chen, Yanjun and Sun, Yirong and Su, Hao and Pan, Yu and others},
  booktitle = {Findings of the Association for Computational Linguistics (ACL Findings)},
  year = {2025},
}

NAACL

Fine-Grained and Multi-Dimensional Metrics for Document-Level Machine Translation

Yirong Sun, Dawei Zhu, Yanjun Chen, and 3 more authors

In Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2025

Bib

@inproceedings{sun2025finegrained,
  title = {Fine-Grained and Multi-Dimensional Metrics for Document-Level Machine Translation},
  author = {Sun, Yirong and Zhu, Dawei and Chen, Yanjun and Xiao, Eric and Chen, Xinghao and Shen, Xiaoyu},
  booktitle = {Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)},
  year = {2025},
}

EMNLP

The Accuracy Paradox in RLHF: When Better Reward Models Don’t Yield Better Language Models

Yanjun Chen, Dawei Zhu, Yirong Sun, and 3 more authors

In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

arXiv Bib Code

@inproceedings{chen2024accuracy,
  title = {The Accuracy Paradox in {RLHF}: When Better Reward Models Don't Yield Better Language Models},
  author = {Chen, Yanjun and Zhu, Dawei and Sun, Yirong and Chen, Xinghao and Zhang, Wei and Shen, Xiaoyu},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2024},
}

arXiv

Exact Is Easier: Credit Assignment for Cooperative LLM Agents

Yanjun Chen, Yirong Sun, Hanlin Wang, and 5 more authors

arXiv preprint arXiv:2603.06859, 2026

In submission.

arXiv Bib Code

@article{chen2026exact,
  title = {Exact Is Easier: Credit Assignment for Cooperative {LLM} Agents},
  author = {Chen, Yanjun and Sun, Yirong and Wang, Hanlin and Wang, Jinghan and Zhang, Xinming and Shen, Xiaoyu and Li, Wenjie and Zhang, Wei},
  journal = {arXiv preprint arXiv:2603.06859},
  year = {2026},
  note = {In submission.}
}

arXiv

Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning

Xinghao Chen, Anhao Zhao, Heming Xia, and 7 more authors

arXiv preprint arXiv:2505.16782, 2025

arXiv Bib Code

@article{chen2025reasoning,
  title = {Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning},
  author = {Chen, Xinghao and Zhao, Anhao and Xia, Heming and Lu, Xuan and Wang, Hanlin and Chen, Yanjun and Zhang, Wei and Wang, Jian and Li, Wenjie and others},
  journal = {arXiv preprint arXiv:2505.16782},
  year = {2025},
}

Teaching & Service教学与服务教育・学会活動

Teaching教学教育 (Teaching Assistant, The Hong Kong Polytechnic University)（助教，香港理工大学）（ティーチング・アシスタント、香港理工大学）

2025/26 S2	`COMP5221` Software Project Management
2025/26 S1	`COMP1002` Computational Thinking and Problem Solving
2024/25 S2	`COMP1411` Introduction to Computer Systems
2024/25 S1	`COMP5567` Distributed Algorithms and Protocols for Blockchains

Reviewing审稿査読

2026	ACL ARR (March and May cycles)（3 月与 5 月周期）（3 月・5 月サイクル）
2026	IEEE Transactions on Neural Networks and Learning Systems (TNNLS), invited受邀招待