defilantech/LLMKube

Problem The per-request balancing work from #1812 shipped in #1819 and #1820, but nothing under docs/ mentions it. #1820's PR checklist flagged this: "docs/site/concepts/model-router.md not yet updated". Users scaling an InferenceService to several replicas still hit the behavior #1812 describes (keep-alive clients pin one pod through the Service) and have no way to learn that ModelRouter fixes it. Proposed solution Update docs/site/concepts/model-router.md (and the API reference if it is hand-maintained) to cover: spec.backends[].resolution (service, the default, or endpoint): what endpoint does (the proxy balances each request across the ready pod endpoints from the backend Service's EndpointSlices, with per-endpoint health), when to use it (multi-replica InferenceServices, especially embeddings and rerank), and when not to (single-replica backends; metal-agent and other out-of-cluster backends have no EndpointSlices and stay Service-level). How endpoint changes reach the proxy: the

Feature Description Two additions to the ModelRouter Proxy data plane that together give multi-replica InferenceServices per-request load balancing: Endpoint-level backends. Compile inferenceServiceRef backends to the pod endpoints behind the InferenceService's Service (EndpointSlices) instead of the Service DNS name, and balance per request across healthy endpoints in the router-proxy. Mount /v1/embeddings and /v1/rerank. The router-proxy handles only POST /v1/chat/completions and GET /v1/models today (internal/router/proxy.go:96-97), so embedding and rerank services cannot be fronted by a ModelRouter at all. Problem Statement replicas on an InferenceService provides capacity, not balance. The operator creates one selector-based ClusterIP Service in front of the Deployment (internal/controller/service_builder.go:33), so a client with a keep-alive connection pool (every OpenAI SDK holds connections open by default) pins its TCP connections through conntrack to a single pod. One busy

Bug Description ForemanFleetNodeHeartbeatStale in charts/foreman/templates/prometheusrule.yaml interpolates {{ $labels.node }}, but the metric it alerts on carries no node label. Both annotations render with a hole where the FleetNode name should be, so the alert cannot say which FleetNode is stale — on a multi-node fleet it gives you nothing to act on. Found while chasing #1778, where the alert stayed critical for hours and the only way to identify the culprit was to diff kubectl get fleetnodes against the live pods by hand. Steps to Reproduce Install the foreman chart with prometheusRule enabled and the foreman-crs kube-state-metrics CustomResourceState config. Let a FleetNode go stale (see #1778 — replace an agent pod). Read the firing alert's annotations. Expected Behavior The summary names the FleetNode, e.g. FleetNode foreman-llmkube-agent-5c9cffbcf7-2vk64 heartbeat stale. Actual Behavior summary: "FleetNode heartbeat stale" description: "FleetNode has not heartbeated

Feature Description A CodeHost and WorkItems implementation for the Gitea API family. One adapter serves Forgejo, Gitea, and Codeberg, since Forgejo is a hard fork of Gitea and the API surface Foreman needs is shared. This is the adapter that makes @phorrr's Forgejo setup work, and the first real test of whether the #1158 boundaries hold. Problem Statement Foreman needs exactly four operations from a forge, which is why this is a small adapter rather than a client library: ResolveCloneURL(repoSlug) EnsureChangeRequest(repoSlug, head, base, title, body) -> (url, created, err) HeadCommitSubject(repoSlug, branch) WorkItems.Get(repoSlug, id) Proposed Solution Hand-rolled against the Gitea /api/v1 surface, consistent with the existing GitHub clients, which are hand-rolled net/http with no forge SDK in go.mod. See #1297 for why every multi-forge library was rejected. Use the base/head route for operation 2. Gitea and Forgejo both expose GET /repos/{owner}/{repo}/pulls/{base}/{head}, so "do