SGLang now supports AWS Trainium chips, according to a blog post by LMSYS Org the SGLang project1. The beta release enables inference serving and reinforcement learning (RL) rollouts on Trainium2 and Trainium3 chips.
The integration allows SGLang to run open models like Qwen3, GLM-4, and DeepSeek-V2-Lite on AWS Trainium. The beta is designed to support heterogeneous RL pipelines, where rollout generation and policy updates can run on different hardware, potentially optimizing cost and performance.
SGLang's support for Trainium is built on TorchNeuron, AWS's native PyTorch backend for Trainium, allowing models to be served through the same interface used on GPUs. The beta includes features such as data parallelism, tensor parallelism, and expert parallelism, as well as support for BF16 and FP8 data types. The beta is currently limited to a small group of teams, with broader availability planned for later this year. The announcement does not include independent test results or performance benchmarks.
The beta supports models including Qwen3, Qwen3-MoE-30B-A3B, DeepSeek-V2-Lite, GLM-4, and GLM-5.2/5.3. Configuration includes disaggregated serving, where prefill and decode servers can run on separate NeuronCores. The announcement does not specify sample size or provide detailed performance metrics. The beta is described as an early preview, with updates expected at the SGLang Summit in November.
Access is available to a limited group of teams upon request, initially for teams running or planning LLM inference on Trainium2 or Trainium3 interested in heterogeneous RL. The announcement does not include pricing or a specific release date for broader availability.