Ray-Learn（一）：actor、资源账本与三个静默陷阱
主流 RL 框架里有一大半把控制面建在 Ray 上，但它们几乎都不用 Ray 的上层库。这一篇从 task / actor 的语义讲起，把资源账本的「账面数字」和物理占用拆开，并用三个可复现的实验说明：多机 RL 里最常见的故障，往往不报错。
ML Systems Engineer · High-Performance Computing
Hi there, I'm Shengguang Zhu 👋
I'm a developer deeply passionate about AI infrastructure, with a primary focus on reinforcement learning frameworks and LLM inference — contributing to SGLang and FastDeploy. My goal is to make distributed training and inference as fast and scalable as possible, diving into the intersection of machine learning and systems.
What I do
RL Infrastructure
Building and optimizing reinforcement learning training and inference pipelines.
LLM Inference
Contributing to SGLang and FastDeploy, across the serving runtime and the kernels underneath it. SGLangFastDeploy
Router
Session and KV-cache aware routing, contributing to Dynamo. Dynamo
Kernel Optimization
Designing and tuning high-performance custom kernels.
Writing
Maintaining MLsys-Note, an open space for notes and code on AI infrastructure. MLsys-Note
Recent writing
All posts →聊聊 KV Cache 的显存账：一张卡到底能塞多少并发
把 KV Cache 的显存占用算清楚，再看 PagedAttention 的分块管理省下的是哪一部分。含一份可以直接抄去改的估算脚本，以及几个在生产里反复踩到的坑。
Continuous batching, and where the throughput actually goes
A scheduler-level look at continuous batching: why naive static batching wastes most of the GPU, what the arrival-rate math says about queue depth, and the three places real implementations lose the throughput they just won.
论文精读｜TAO：面向 LLM Agent 服务的工具感知联合调度
Agent 会话有九成以上的时间在等 CPU 上的工具调用。TAO 用一个按字典序比较的三元组给等待中的会话估值，再用多背包动态规划一次性决定谁上 GPU、上哪台。这篇精读按论文原始数据复现了关键图表，也标出了它没赢的地方。
Infinite progress.