Skip to main content
POST
Deploy a fine-tuned model with vLLM

Authorizations

Authorization
string
header
required

The access token received from the authorization server in the OAuth 2.0 flow.

Body

application/json

Deploy a fine-tuned model behind vLLM.

name
string
required

Deployment name (DNS-1123 label). Used for the K8s resources (serve-vllm-) and the router endpoint. Unique per served model.

finetune_job_id
string<uuid>
required

A COMPLETE fine-tune training job whose adapter to serve.

served_model_name
string | null

Model id exposed via the router. Defaults to the job's registered model name.

Maximum string length: 200
gpu_count
integer | null

Override GPU count for the serving pod.

Required range: 1 <= x <= 64
min_replicas
integer | null

KEDA min replicas (0 enables scale-to-zero).

Required range: x >= 0
max_replicas
integer | null
Required range: x >= 1
max_model_len
integer | null

vLLM --max-model-len override.

Required range: x >= 1
dtype
string | null

vLLM --dtype (e.g. 'bfloat16', 'auto').

quality_score
number | null

Router quality score for this model.

Response

Deployment created; auto-registered into the router when ready.

deployment_id
string<uuid>
required
name
string
required
status
enum<string>
required
Available options:
pending,
deploying,
active,
failed,
stopped,
archived
domain
string
required
owner
string
required
finetune_job_id
string<uuid>
required
base_model
string
required
registered_model_name
string
required
served_model_name
string
required
created_at
string<date-time>
required
updated_at
string<date-time>
required
model_version
string | null
service_url
string | null
replica_count
integer | null
ready_replicas
integer | null
router_endpoint_name
string | null
router_model_name
string | null
gpu_resources
Gpu Resources · object | null
error_message
string | null
stopped_at
string<date-time> | null