Skip to main content
You can view an inferencing deployment to see whether a fine-tuned model is running and ready for requests.

Prerequisites

Before starting this task, ensure you have the following:
  • Required roles: nx1_finetune_admin, nx1_finetune_user, or nx1_engineer.
  • A previously created inferencing deployment.
You see only the deployments you created. Users with the nx1_finetune_admin role see all deployments.

View an inferencing deployment

Perform these steps to view an inferencing deployment:
  1. Log in to NexusOne.
  2. On the top navigation bar, hover your mouse over AI and then select AutoML & AutoLLM.
  3. Click Fine-tune LLMs (AutoLLM).
  4. Click Inferencing. You are going to see a table of deployments.
  5. Find the deployment you want to view. The table has the following columns:
    • Name
    • Served model
    • Replicas: Shows the running copies of the model server out of the total
    • On router: Shows yes when the model is on an LLM Router
    • Status

Inferencing deployment statuses

An inferencing deployment has one of the following statuses:
  • pending: NexusOne accepted the deployment request.
  • deploying: NexusOne is starting the model and adding it to an LLM Router.
  • active: The model is running and ready for requests.
  • failed: The deployment failed.
  • stopped: You stopped the deployment, and NexusOne removed the model from the LLM Router.
  • archived: You deleted the deployment.

Additional resources

For a high-level overview of inferencing and how it’s used in NexusOne, refer to Fine-tune LLM.