Experience-Handoff Reinforcement Learning for Recursive Self-Improvement

doyoung kim, Byeongguk Jeon, Kailin Zhang, Sherry Yang

Experience-Handoff Reinforcement Learning for Recursive Self-Improvement: 10 upvotes on Hugging Face Daily Papers, #17 of 34 papers on 2026-11-13. Day-by-day upvote history.

Upvotes over time

It had 1 upvotes by its first day, 4 upvotes by the next day, 10 upvotes by day 3, 10 upvotes by day 7, and has 10 now. It ranked #76 of 167 papers in its week and #231 of 520 in its month. Its code repository has 3 GitHub stars.

It is common to train Large Language Models (LLMs) for agentic tasks with Reinforcement Learning (RL) from scalar rewards. However, this throws away the fine-grained information about the environment that the agent experienced during the trial (e.g., error logs). To utilize this rich information, recent works wrap the agent in a harness that refines its own inputs through reflection or compaction and hands past experience to future rollouts, but the construction of the handed-off artifacts (e.g., a playbook) is itself not learned. We want the best of both worlds: gradient updates from a scalar reward, while also utilizing environment information to hand off experience. As a concrete testbed, we fix the Agentic Context Engineering () harness and train one model to act, reflect, and curate task-local playbooks. Our method, , evaluates candidate experience-handoff artifacts by their downstream effect on future actor rollouts, and co-optimizes the role-specific behaviors (action generator, insight reflector, and playbook curator) under a fixed scaffold. Across agentic benchmarks, achieves higher held-out performance than baseline methods. The results suggest that agent training should optimize not only the actions in the current rollout, but also the model-written experience that future rollouts inherit.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.