Lightning-AI/LitServe

🐛 Bug On windows, killing To Reproduce I opened Lightning Studio but it redirected me to https://lightning.ai/templates. Below I include fully python code needed for replication. Code sample Code sample: import os import sys import threading import time from litserve import LitAPI, LitServer class _MinimalAPI(LitAPI): def setup(self, device: str) -> None: self.model = lambda x: x * 2 def decode_request(self, request: dict) -> float: return float(request["input"]) def predict(self, x: float) -> float: return self.model(x) def encode_response(self, output: float) -> dict: return {"output": output} if __name__ == "__main__": print(f"[MAIN] PID={os.getpid()} platform={sys.platform}") print("[MAIN] Starting LitServer...") server = LitServer( _MinimalAPI(), accelerator="cpu", devices=1, workers_per_device=1, ) def _prompt(): time.sleep(4) # wait for uvicorn + inference wor

🐛 Bug wrap_litserve_start() is a pytest utility that starts the LitServe server in a context manager for testing. It launches inference worker processes but yields immediately without waiting for them to finish setup() (model loading, etc.). This causes TestClient.post() to hang indefinitely because the worker hasn't started listening on the request queue yet. The request gets enqueued but never processed. To Reproduce import litserve as ls from fastapi.testclient import TestClient class SlowSetupAPI(ls.LitAPI): def setup(self, device): import time time.sleep(5) # simulate slow model loading self.model = lambda x: x def decode_request(self, request): return request["input"] def predict(self, x): return self.model(x) def encode_response(self, output): return {"output": output} def test_predict(): api = SlowSetupAPI() server = ls.LitServer(api, accelerator="cpu") with ls.utils.wrap_litserve_start(serv

The /predict endpoint in the example servers does not define a request body schema. Consequently, Swagger UI provides no input form and sends an empty request body by default when "Execute" is clicked. This results in a 500 Internal Server Error as the server fails to parse the empty JSON payload. To Reproduce 1. Start the server python tests/simple_server_diff_port.py 2. Access Swagger UI Navigate to: http://localhost:8000/docs 3. Trigger the Error Expand the /predict endpoint Click Execute Note: No input field is visible to the user. Actual Results The server crashes with a JSONDecodeError because Swagger sends an empty body: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ... File "src/litserve/server.py", line X, in predict return await request.json() Expected Behavior Swagger UI should display a request body form (e.g., a JSON blob like below). This allows users to provide input values directly in the interface, ensuring the request.json() call has