==> call the api

Chat completion (OpenAI)

POST /v1/llm/chat/completions

OpenAI's Chat Completions. "stream": true streams server-sent events, the last chunk carrying usage. Structured output (response_format with a JSON schema), tools and vLLM's extras (chat_template_kwargs: { enable_thinking: false }) pass through. Any OpenAI client works with the base URL https://api.<host>/v1/llm and the token as its key. Body up to 32 MiB. A personal access token only, for an account the model is open to (modelApi in /v1/me); a few calls in flight per account at most (429 beyond). The answer streams back as the model writes it.

Needs a personal access token: Authorization: Bearer <token>.

Body

application/json

messagesarray of objectrequired
- role"system" | "user" | "assistant" | "tool"required
- contentone of 2required
  Text, or OpenAI content parts (images inline).
  1. string
  2. array of object
streamboolean
max_tokensinteger
temperaturenumber
response_formatobject
toolsarray of object

Answer

200 Success.

Refusals

Each answers { "error": "...", "code"?: "..." } with its status.

status	when
`400`	The request is not valid.
`401`	No valid token.
`403`	This model is not open to the account, or the call did not come with a token.
`413`	The body is too large for this model.
`429`	Too many calls in flight for this account.
`503`	The model is not running.

Example

curl -X POST 'https://api.clusterstuck.sh/v1/llm/chat/completions' \
  -H 'Authorization: Bearer <your token>' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system","content":"..."}],"stream":false,"max_tokens":0,"temperature":0,"response_format":{},"tools":[{}]}'