vllm-project/aibrix

Summary AIBrix already supports Volcano PodGroup gang scheduling through StormService.spec.template.spec.schedulingStrategy.volcanoSchedulingStrategy, but the current usage is not obvious from examples or documentation. We should add documentation and examples showing how to configure gang scheduling from the StormService entry point. Motivation StormService is the main user-facing entry for AIBrix orchestration. Users who want PD-disaggregated serving need to understand how to configure: group-level gang scheduling with minMember role-level pod minimums with Volcano minTaskMember the difference between RoleSet-level scheduling and per-role/PodSet scheduling Without examples, users may assume AIBrix has no role-aware gang support, or may configure RoleSet directly instead of using StormService. Proposed Scope Add StormService examples for: 1. Group-level gang scheduling apiVersion: orchestration.aibrix.ai/v1alpha1 kind: StormService metadata: name: sglang-pd spec: replicas: 1 s

Summary Improve the usability and correctness of Volcano gang scheduling in StormService by adding validation for invalid gang configurations and automatically injecting pod.spec.schedulerName when Volcano gang scheduling is configured. Motivation Today users configure Volcano gang scheduling through: spec: template: spec: schedulingStrategy: volcanoSchedulingStrategy: minMember: 6 minTaskMember: prefill: 4 decode: 2 However, there are two usability gaps: Invalid configurations can be accepted, such as zero or negative minTaskMember values. Users must manually set schedulerName: volcano in every role pod template, which is easy to forget. This makes StormService gang scheduling harder to use and easier to misconfigure. Proposed Scope 1. Add validation for Volcano gang fields Validate StormService.spec.template.spec.schedulingStrategy.volcanoSchedulingStrategy and the corresponding RoleSet fields. Suggested validation rul

Describe the issue The session-affinity header names are currently defined in two places. In pkg/plugins/gateway/types.go: HeaderSessionID = "x-session-id" HeaderSessionKey = "x-aibrix-session-key" In pkg/plugins/gateway/algorithms/simple_session_affinity.go: sessionIDHeader = "x-session-id" sessionKeyHeader = "x-aibrix-session-key" There is already a comment noting that sessionIDHeader must strictly match types.HeaderSessionID to prevent routing failures. sessionKeyHeader has the same duplication pattern with types.HeaderSessionKey. Why this matters If one definition changes without updating the other, request header parsing and the session-affinity router can disagree on the expected header names. That can break sticky routing or session-key based routing in a way that is hard to diagnose. Expected behavior There should be a single source of truth for these gateway header names, or an explicit structure that prevents drift between request parsing and routing algorithms. Possible

Feature Description and Motivation The gateway plugin documentation page currently documents only a subset of the headers under the Target and General Headers section: request-id x-went-into-req-headers target-pod routing-strategy external-filter The implementation defines and uses additional gateway headers that are useful for routing control, observability, session affinity, tracing, and PD routing. Documenting them would make the gateway behavior easier to understand and debug. Use Case When users verify routing behavior or troubleshoot gateway plugin requests, they need to know which headers can be sent by clients, which headers are injected into backend requests, and which headers are returned by the gateway response. Missing or unclear documentation makes it harder to debug session-affinity routing, model config profile selection, trace propagation, and PD prefill/decode routing. Proposed Solution Update docs/source/features/gateway-plugins.rst, especially the Target and Genera

Bug description While smoke-testing PodAutoscaler KPA / APA strategies on minikube, I found two related validation / error-handling issues: Unknown metric names are accepted and then treated as metric value 0, causing KPA / APA to scale down. subTargetSelector.roleName is accepted for a Deployment target under KPA, even though role-level sub-targeting appears to be intended only for StormService. What worked Using a fake vLLM-style Prometheus endpoint and the current central metric registry key num_requests_running, both KPA and APA worked: KPA + POD metric scaled a Deployment from 2 to 4. APA + POD metric scaled a Deployment up to the configured maxReplicas: 6. KPA / APA did not create HPA resources; they directly patched the target workload's spec.replicas. Issues found 1. Unknown metric name is treated as zero and causes scale-down Example PodAutoscaler: spec: scalingStrategy: APA scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: custom-pa-target minRe