Skip to main content
The app manager provides a management layer for organizing pipeline files into named, versioned apps.

​
End-to-end flow of the app manager feature

Each created app works like a project with version control built in. Creating one just sets up an empty project. Nothing runs yet. The next step is adding files. But you don’t add them straight to the app. You add them to an app version instead. Every app can have multiple versions and only one version can be active at a time. The rest stay inactive. When you create an app version, you add the files it needs, such as DAGs, notebooks, serving configs, and artifacts. NexusOne tracks each version separately, so your changes stay isolated to that version. Lastly, you make the app version active. So, what does making an app version active do? NexusOne copies the app version files to a new location. That’s the point where your changes actually go live. An app isn’t only about files, though. It also controls who can manage it through roles, and stores any credentials it needs as secrets.

​
App structure

In this feature, an app comprises the following:
  • Components: Files belonging to an app. They’re used in an app version to support pipeline execution, and comprise the following:
    • DAGs: Apache Airflow DAG files
    • Notebooks: Jupyter notebook files
    • Serving configs: YAML or JSON files that define how an app serves model predictions
    • Artifacts: Files such as .csv, .jar, .py, or other file formats
  • Roles: Who can manage an app
  • Secrets: Sensitive values scoped to an app, injected into components at runtime
Together, these previously described items define how NexusOne packages and secures pipeline assets within an app.

​
Where components run

Apache Airflow is the pipeline previously discussed in this page’s introduction. Not every component runs there, though. The supported components that run in a pipeline include:
  • DAGs: The DAG component runs directly as Airflow DAGs.
  • Notebooks: The notebook component runs as a task inside a DAG, if you build it with the Elyra pipeline editor in JupyterHub.
The components that don’t run in a pipeline include:
  • Serving configs: The serving configs component skips the pipeline entirely. It only describes how to serve a model.
  • Artifacts: The artifacts component skips the pipeline for a different reason. Artifact files support a DAG or notebook component, such as a helper script or a config file.
Which of these components an app version can hold depends on the app type you picked when you created it.

​
App types

When you create an app, you choose one of the following app types:
  • Hand authored: Supports the DAG and artifact component types. You write the DAG by hand.
  • Notebook pipeline: Supports the DAG, notebook, and artifact component types. You build the DAG in JupyterHub by chaining notebooks with the Elyra pipeline editor, then upload it via the app manager.
  • Inference service: Supports the serving config and artifact component types. No DAG here, just a config that describes how to serve a model.
  • Full pipeline: Supports the DAG, notebook, serving config, and artifact component types. Everything from Notebook pipeline, plus a serving config to deploy the model.
When you activate a version, NexusOne copies its files into the current/ location. The next section shows that path, along with where every version’s files live regardless of status.

​
Default component locations

You can access the DAGs, notebooks, serving configs, and artifacts in Jupyter Notebook. However, they’re stored in unique directories depending on the app version.
  • For an active app version, you can find the components in the following paths:
    • Artifacts: <bucket_name>/deps/<app_name>/current/
    • DAGs: <bucket_name>/deps/<app_name>/current/dags/
    • Notebooks: <bucket_name>/deps/<app_name>/current/notebooks/, or <bucket_name>/users/<username>/ if linked by reference
    • Serving configs: <bucket_name>/deps/<app_name>/current/serving/
  • For all versions of an app, including the active version, you can find the components in the following paths:
    • Artifacts: <bucket_name>/deps/<app_name>/<version_name>/
    • DAGs: <bucket_name>/deps/<app_name>/<version_name>/dags/
    • Notebooks: <bucket_name>/deps/<app_name>/<version_name>/notebooks/, or <bucket_name>/users/<username>/ if linked by reference
    • Serving configs: <bucket_name>/deps/<app_name>/<version_name>/serving/
Linking a notebook by reference doesn’t copy it into the deps/<app_name>/current/notebooks/ or deps/<app_name>/<version_name>/notebooks/ paths. If you keep editing the notebook in JupyterHub after linking it, the app version’s reference isn’t a live sync. You’d need to re-attach to pick up changes.

​
Use cases

These examples show how different industries can use NexusOne’s app manager:
  • Financial services: Version and manage assets in a trading or reporting pipeline. These assets include notebooks for model training and serving configs to deploy a fraud-detection model.
  • Healthcare: Manage clinical pipeline assets as versioned apps that are explicitly promoted to production when you make updates to the assets.

​
Additional resources

To see all the open source software tools that make up the components of the NexusOne platform, refer to Platform components.