The SkyRL Team

🗓️ Posted: July 17th 2026

We are happy to announce the release of SkyRL v0.3.0.

SkyRL’s focus in this release revolves around three major themes:
    1. Support for large scale, async RL
    2. Improving SkyRL’s Tinker API for self-hosting Tinker servers
    3. Providing a unified library for SFT and RL

This release comes packed with a number of features: improved support for fully async RL, 
multi-LoRA support with the Tinker API, native SFT support, speculative decoding support,
INT4 Quantization-Aware Training and more!

Large-Scale, Fully-Async RL

HTTP-based inference

We introduced a new HTTP-based inference architecture in v0.2.0. This simplifies inference engine management, allows us to integrate high performance routers like vllm-router and supports advance serving patterns like prefill-decode disaggregation. The legacy inference codepath has now been removed in v0.3.0.

Improved load balancing

****By default, SkyRL uses a fixed session ID per trajectory and maps a given session ID to a given vLLM engine replica with consistent-hashing. This maximizes KV cache reuse for a given trajectory. However, simple session-based routing can be suboptimal for overall generation throughput in async RL. We’ve implemented a custom routing policy with vllm-router called sticky_least_loaded that load balances trajectories across replicas with sticky routing per trajectory. On our benchmarks for fully async RL on a multi-turn agentic task (averaging 40+ turns) with a large MoE model (GLM 4.7), this load balancing improves throughput by upto 1.4x.

Routing policies: `consistent_hash` vs `sticky_least_loaded` routing, benchmarked at different KV    
cache pressure scenarios for an agentic RL training run with GLM 4.7. Consistent hashing leads to   
imbalanced load across replicas, reducing overall generation throughput

Dynamic microbatching

We added support for dynamic microbatching to improve training performance. Dynamic microbatching packs sequences in each worker upto a fixed token count, instead of a fixed number of sequences per forward pass. This balances real token counts across microbatches, improving throughput and memory utilization when sequence lengths are highly variable. Training throughput improvements are dependent on the task as well as the model and cluster shape, but we see gains of 2x or more common math and agentic tasks.

Dynamic sampling

We’ve added support for dynamic sampling in fully async RL. When enabled, SkyRL drops zero variance groups (groups where all the trajectories score the same reward) and continues sampling until the training mini batch is filled. Zero-variance groups can negatively affect learning and destabilize training. While these zero variance groups receive zero advantages with GRPO, they still affect learning by biasing the aggregate mini-batch loss. Dynamic sampling ensures that such groups are filtered out while keeping a consistent batch size during optimization steps.