LLamaRL
First page
Paper summary
LlamaRL is a fully-distributed, asynchronous reinforcement learning framework designed for efficient large-scale LLM training (8B to 405B+ models). It achieves up to 10.7× speedup over DeepSpeed-Chat by combining co-located model offloading, asynchronous off-policy training (AIPO), and fast GPU-native weight sync (DDMA), while maintaining model quality across tasks like math reasoning.