SGLang 0.5.1 added cache and scheduling work to its serving release
SGLang released version 0.5.1 on August 23, 2025, including HiCache NIXL connector work, CUDA-graph capture changes and new runtime integrations.
Historical reporting: the SGLang project's dated v0.5.1 GitHub release establishes the event. The public 0.5.1 package page was freshly read as a second public distribution record. Neither source proves serving performance for another deployment.
Key takeaways
- SGLang 0.5.1 joined cache transfer, CUDA-graph capture and scheduler work in a single serving-runtime release.
- A release-note entry identifies code available in a version; it does not establish cache correctness or latency under a target request mix.
- Serving upgrades need separate tests for cache state, transport, scheduler behavior and output consistency.
The August 23 release marks a distinct SGLang 0.5 event[source 1][source 2]
SGLang's project repository records version 0.5.1 released on August 23, 2025. The release is a distinct historical event from the 0.4 release already covered in the archive. Its change list includes a HiCache NIXL connector, batch transfer work for RDMA transport, a TensorRT-LLM MLA decode-kernel integration and an optimization that disables garbage collection during CUDA Graph capture. The public 0.5.1 package page identifies SGLang as serving infrastructure and links the project's own documentation and release communications. These records establish a versioned serving-runtime change; they do not establish that every listed integration is enabled, compatible or beneficial in another installation.
Cache-transfer features require state and transport tests[source 1][source 2]
The v0.5.1 change list names both a HiCache NIXL connector and RDMA batch-transfer work. Cache movement can alter where state lives, when it is reusable and what happens when a worker or transport path fails. A version note cannot answer those operating questions for a particular topology. Teams need to test cache hits and misses, eviction, degraded transport, replica behavior, cancellation and restart while capturing request identifiers and output comparisons. They should also distinguish a fast cache path from a correct one: a repeated prefix can reduce work without proving that cache identity, isolation or invalidation is correct for an application's sessions.
CUDA-graph and kernel changes sit inside a wider scheduling path[source 1][source 2]
The release says disabling garbage collection during CUDA Graph capture could speed that capture by up to three times, and it names a TensorRT-LLM MLA decode-kernel integration. The first is a project performance claim tied to capture; the second is an integration claim. Neither gives a general request-throughput guarantee. Capture warmup, graph shapes, memory headroom, GPU generation, batching and host scheduling can all alter a result. A good evaluation reports first-request and steady-state behavior separately and includes a correctness check after cache, graph or kernel selection changes, because a fast repeated run can hide an invalid capture or dispatch fallback.
Serving release notes are a start point for controlled rollout[source 1][source 2]
The 0.5.1 materials show a project evolving its cache, transport and kernel layers. A production decision needs further evidence: locked dependencies, selected runtime flags, target accelerators, model revisions, representative concurrency and a recovery policy. It also needs a defined downgrade because cache and scheduler changes can interact with request state across an upgrade. A controlled canary can make cache warming and graph capture observable before a general traffic shift, while retaining request-level traces for comparison. The historical source supports reporting that SGLang published these changes on August 23. It does not certify an externally operated cluster, its data boundaries or its advertised performance. Those require measurements and incident-aware tests in the actual service.
Sources
- SGLang GitHub: v0.5.1 release (August 23, 2025)
- SGLang package record: 0.5.1
Continue reading
