Skip to main content
A base/general Large Language Model (LLM) learns from public data, so it doesn’t know your business. Unique business information from your DataHub glossary terms, internal documents, or how your datasets relate to each other is unknown to the base model. Fine-tune LLM is the AutoML feature that trains a base LLM on your domain knowledge and then serves the result, so the model answers questions about your business without needing to look anything up first. For example, you can start with an open source base model such as Meta-Llama-3-8B, train it on your DataHub domain knowledge, and serve the fine-tuned result. And if the model you want isn’t listed, then ask your platform administrator to add it, because they control which base models are available.

Key features

The Fine-tune LLM feature provides the following:
  • Automatic routing: After you deploy a fine-tuned model, NexusOne adds it to the LLM Router once it’s ready. You don’t edit any router configuration by hand.
  • Autoscaled serving: NexusOne serves a deployed fine-tuned model on vLLM, which scales with demand.
  • Efficient fine-tuning: NexusOne trains LoRA and QLoRA adapters, which are small add-on files that hold what the model learned. They’re shortcuts for teaching a big model something new without retraining the whole thing. This keeps training cheaper.
  • Experiment tracking: When training finishes, NexusOne registers the fine-tuned model in MLflow, tagged with its base model, domain, and adapter type.
  • Human approval before training: Once NexusOne generates a dataset, you review, edit, and remove examples before it spends any GPU time. Nothing trains without your approval.
  • Training data from your catalog: NexusOne builds question and answer examples from the knowledge your domain already has in DataHub. You don’t write training data by hand.

How it works

Fine-tune LLM has three sub-features, which you use in order.
  1. Dataset: Generate and approve the training data.
  2. Training: Fine-tune a base model on the approved training data.
  3. Inferencing: Deploy the fine-tuned model so you can use it.
Each sub-feature produces the input for the next. The following sections give a brief overview of each one.

Dataset

A dataset is a collection of example conversations that show a model how to answer questions about your business domain. NexusOne generates it for you from the knowledge in your DataHub catalog, for the base model you chose. Each dataset example has a question from a user and the answer the model should give. After NexusOne generates a dataset, you can approve it so NexusOne can use it to fine-tune a model.

How NexusOne generates a dataset

To generate a dataset in NexusOne, you provide a DataHub business domain and the base model the dataset is for. NexusOne then runs two AI agents in the background. One agent gathers the knowledge for your domain, and the other writes the dataset, which is a collection of example conversations. Every dataset example records the source it came from, so you can trace an answer back to the glossary term, dataset, or document behind it. When the agents complete their task, NexusOne pauses and waits for you. You can review a sample of the examples and remove any you don’t want. Once you approve, NexusOne saves the final dataset to object storage and registers it in MLflow.

Dataset lifecycle

A dataset moves through the following statuses top to bottom:

Training

A training job takes a completed dataset and fine-tunes a base model on it. To run the fine-tuning, NexusOne renders an Airflow DAG, which is the workflow that carries out the training, and uploads it so Airflow can run it. As with AutoML training, you can trigger the DAG right away or later. The difference is that this DAG runs on a GPU.

How NexusOne trains a model

To create a training job in NexusOne, you provide a completed dataset. NexusOne then trains an adapter, which is a small add-on file that holds what the model learned, and it sits on top of the base model. The supported adapter types include:
  • LoRA: Trains an adapter on top of the base model at full precision.
  • QLoRA: Compresses the base model while training, which uses less GPU memory.
The portal always uses QLoRA. To choose the adapter type or the number of GPUs, use the API. When the job finishes, NexusOne registers the adapter in MLflow with tags that record the base model, domain, adapter type, and training job ID. At this point, the adapter is only stored in MLflow. It isn’t deployed anywhere yet, so it can’t answer requests until you deploy it, as described in Inferencing.

Training job lifecycle

A training job uses the same statuses as AutoML training, in the following order:
  • ready
  • queued
  • running
  • complete
  • failed
In the portal, a training job in the queued status shows as starting.

Inferencing

Inferencing is how you use a fine-tuned model. A training job produces an adapter, which is a file that contains what the model has learned. An adapter can’t answer requests on its own, so you deploy it. NexusOne then serves the adapter on top of its base model using vLLM, an open source LLM serving engine that exposes an OpenAI-compatible API.

How NexusOne serves a model

To serve a model for inferencing in NexusOne, you create a deployment by providing a completed training job and a name. NexusOne scales the deployment with demand. It runs more or fewer replicas, which are copies of a model server, as the number of requests rises and falls. Once the deployment is ready, NexusOne adds the model to an LLM Router automatically. When you stop or delete the deployment, NexusOne removes the model from the router, so traffic never goes to a model that’s gone.

Inferencing deployment lifecycle

An inferencing deployment moves through the following statuses.

Use cases

These examples show how different industries can use NexusOne’s Fine-tune LLM capabilities:
  • Financial services: Fine-tune a model on your risk and compliance domain, so analysts get answers that use your approved definitions for terms such as exposure, counterparty, and settlement.
  • Retail: Fine-tune a model on your product catalog glossary and merchandising documents, so an internal assistant answers questions about product categories and sales definitions in your company’s own terms.