Skip to main content
You create a dataset to have NexusOne generate training data for fine-tuning a base large language model. You choose a DataHub business domain and a base model, and NexusOne generates the dataset from the knowledge in that domain. When it finishes, the dataset waits for your review.

Prerequisites

Before starting this task, ensure you have the following:
  • Required roles: nx1_finetune_admin, nx1_finetune_user, or nx1_engineer.
  • An existing domain created using DataHub.
  • A base model that your platform administrator made available.

Create a dataset

Perform these steps to create a dataset:
  1. Log in to NexusOne.
  2. On the top navigation bar, hover your mouse over AI and then select AutoML & AutoLLM.
  3. Click Fine-tune LLMs (AutoLLM).
  4. Click Datasets.
  5. Click New dataset.
  6. In the Business domain drop-down, select the DataHub business domain you want NexusOne to read.
  7. In the Base model field, enter the base model the dataset is for. Training works only for base models your platform team has made available.
  8. Optional: Enter a name to identify this dataset. If you leave it empty, then NexusOne names the dataset after the domain.
  9. Optional: Enter extra guidance, such as facts or instructions you want NexusOne to include in the dataset.
  10. Optional: In the Target examples field, enter roughly how many example conversations you want, up to 60. NexusOne treats this number as a goal. If you leave it empty, then NexusOne aims for 60.
  11. Optional: Under Include from the domain, clear the checkbox for each knowledge source you don’t want NexusOne to use. The portal checks all of the following by default:
    • Glossary terms
    • Documents
    • Lineage / relationships
    • Tags
    • Data products
  12. Click Generate. NexusOne shows what the agents are doing while they work.
  13. When the button label changes to Close, click Close.
After you close the dialog, the new dataset appears in the AutoLLM datasets table on the portal, with the running status. When the agents finish, the status changes to pending_approval. To continue, refer to Approve a dataset.

Retry a failed dataset

A dataset gets the failed status when the agents can’t produce a valid dataset.
Perform these steps to run the agents again:
  1. Log in to NexusOne.
  2. On the top navigation bar, hover your mouse over AI and then select AutoML & AutoLLM.
  3. Click Fine-tune LLMs (AutoLLM).
  4. Click Datasets.
  5. Select the dataset with the failed status. The dialog that opens shows the error message.
  6. Click Retry.

Additional resources

For a high-level overview of datasets and how they’re used in NexusOne, refer to Fine-tune LLM.