Prerequisite
Appropriate permission:nx1_ingest, nx1_monitor, nx1_s3_admin, airflow_user, superset_user,
spark_sql, and trino_admin
Download a CSV file
Use a financial transactions dataset from Kaggle containing user data.Upload it to the portal
- Log in to NexusOne.
- On the NexusOne homepage, navigate to Ingest > File.
- In the File Details section, click Upload File > Choose file.
- Header: Use it to define if the first row is the column name. Defaults to
True. - Infer Schema: If NexusOne should guess the data type of each column. Defaults to
True. - Delimeter: To separate each field.
- Quote character: Protects the slicing of strings by a delimeter.
- Date format: Indicates the date format of the CSV file.
- Timestamp format: Indicates the timestamp format of the CSV file.
Add ingest details
Add the following information to the fields:- Name:
csv - Schema:
csv_schema - Table:
csv_table - Schedule:
None - Mode:
append - Tags: Don’t add any tags
None specifies that the DAG on Apache Airflow won’t run.
After adding these details, click Ingest. Wait for a few minutes until you see a success message appear.
Monitor and trigger job
- When you ingest the file, this creates an Airflow job, so a success message with a View Jobs button appears.
- Track the job status by clicking View Jobs or navigate to the NexusOne homepage and click Monitor.
- You should see your job name,
csv, in the list, and its current status. Use the three dots...menu to trigger the job. - If you clicked Trigger job, then click the job’s name to open its DAG details in Airflow’s portal.
Visualize your dataset
Use the following steps to visualize your dataset:- On the NexusOne homepage, click Discover to launch Superset.
- Hover your mouse over SQL and select SQL Lab.
- Enter the following command in the query box:
Visualize your dataset
Additional resources
- To get an overview of what file ingestion is, refer to Data ingestion overview.
- For general instructions about how to ingest a file in NexusOne, refer to How to ingest a file.
- For more information about roles or permissions, refer to Govern Overview.