sgl-project/sglang

Summary TRTLLMHAAttnBackend(FlashInferAttnBackend) (in python/sglang/srt/layers/attention/trtllm_mha_backend.py) inherits from FlashInferAttnBackend, but it reuses very little from the parent. Almost all attention-specific behavior is overridden, and some inherited setup is allocated-but-never-used. We should reconsider this inheritance: if the reuse is small, we should not inherit from FlashInferAttnBackend — inheriting directly from the base AttentionBackend (and pulling in only the small bits of plumbing that are actually shared) would be cleaner. What is actually reused The only real reuse comes from super().__init__(...): req_to_token_pool, token_to_kv_pool max_context_len, skip_prefill, use_sliding_window_kv_pool the kv_indptr / kv_last_page_len buffers (used by the companion TRTLLMHAAttnMultiStepDraftBackend draft steps) forward() dispatch — which actually comes from the base AttentionBackend, not from FlashInfer. What is overridden (not reused) Essentially all the attenti

Lists of models needed to be supported (updating): offline: ideogram4 JoyEcho online: omni-dreams Welcome to join the SGLang-Diffusion team. your contribution would be much appreciated!

Checklist I searched related issues but found no solution. The bug persists in the latest version. Issues without environment info and a minimal reproducible demo are hard to resolve and may receive no feedback. If this is not a bug report but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed. Please use English. Otherwise, it will be closed. Describe the bug It keeps breaking ci on blackwell. So we have to ignore it in https://github.com/sgl-project/sglang/pull/21081/changes After fixing it, we should reenable this test. Reproduction python test/registered/4-gpu-models/test_qwen35_models.py Environment install from latest main

Motivation SGLang's test suite has 600+ test files, but most are E2E tests that launch a full server. While these cover many code paths, core modules like managers/, mem_cache/, entrypoints/, sampling/, parser/, function_call/, and utils/ still need better unit test coverage (tests that run without launching a server). Why this matters: E2E tests launch a server and load model weights, taking minutes. Unit tests run in seconds with mocked dependencies. When an E2E test fails, the root cause could be anywhere. Unit tests pinpoint the exact broken function. What's Already Done Unit test directory test/registered/unit/ with READMEs (test/registered/README.md, test/registered/unit/README.md) documenting conventions 32 unit tests migrated across 12 subdirectories Coverage tooling configured (.coveragerc, pytest-cov) CI coverage reporting in ci-coverage-overview.yml (daily + manual trigger) — example run Contribution guide updated with unit test section Directory Structure test/registe

Checklist If this is not a feature request but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed. Please use English. Otherwise, it will be closed. Motivation Summary Request: Unify all JIT and precompilation cache paths under a single configurable root so that users and operators can manage cache location, size, and persistence in one place. The current situation is fragmented across multiple env vars and default paths (including /tmp), which makes it hard to reason about where compiled artifacts live and to reuse caches across runs or machines. Current State (Fragmented) 1. Triton JIT cache What Env var / source Default / behavior Direct @triton.jit (e.g. allocator, MoE kernels) TRITON_CACHE_DIR (Triton runtime) Triton default: ~/.triton/cache; often overwritten by Inductor to a path under /tmp/torchinductor_* SGLang override SGLANG_TRITON_CACHE_DIR (if implemented) e.g. ~/.triton/cache or ~/.