Trnrt

About Trnrt

One AWS trn2.48xlarge serves GLM-5.3-Flash at up to 28.9 agent requests a minute on SemiAnalysis's AgentX benchmark, 3.57 million tokens a minute, and takes a million-token prompt to its first token in about three minutes. NKI layer megakernels cut its decode step from 32.2 ms to 24.2 ms.

The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.