Skip to main content
nx1-deployer image 1.17.0
The October 2026 release for non-web portal users introduces Data APIs v2, which publish a semantic model or table as a versioned API with its contract and context attached. It also adds a dedicated Hive Metastore database, Karpenter autoscaling for Trino and Spark, and new Keycloak roles. This release also enhances several existing features, including Spark 4.2 support alongside Spark 3.5 with an optional Spark Connect frontend. It also adds:
  • An optional centralized Grafana for a fleet-wide view across tenants
  • A per-classification Keycloak role for Ranger’s unmask policy, instead of one shared role for every column
  • A way to route Claude traffic through Claude Platform on AWS
  • New Ranger repository modes
  • Opt-in distributed tracing for Spark and Kyuubi
  • Read-only Tenant Manager roles, scoped to every tenant or to just one
This release also upgrades Airflow, DataHub, Grafana Mimir, Keycloak, Kyuubi, and Trino. It ships hardened or CVE-patched images for more than a dozen components, and fixes a Spark issue where tenants sharing a Ranger repository could reach each other’s S3 scratch paths, along with several authentication and deployment reliability fixes.

Before you upgrade

Check the three changes in the following sub-sections before you deploy the nx1-deployer 1.17.0 image.

Check S3 delete permissions

Deleting or moving an object in the JupyterHub S3 browser, or deleting one with s3Cli rm, now requires Ranger delete permission instead of write. A user who could delete objects with only write permission before can’t after you upgrade. See JupyterHub’s S3 browser use now checks for delete permission.

Mirror the new Spark image names

Every Spark-family image now carries its Spark major version in its name. Mirror the images you need to your registry before you deploy:
  • For Spark 3.5: spark3, airflow-spark3, jupyterhub-spark3, and kyuubi-spark3.
  • For Spark 4.2: spark4, airflow-spark4, jupyterhub-spark4, and kyuubi-spark4.
See Image versions for each image’s tag.

Plan your Ranger policy migration

If you change ranger_repo_mode on an existing tenant, then NexusOne creates a new repository that contains only default policies. Your existing policies don’t carry over. Plan how you’ll migrate them before you switch modes. See Expanded Ranger repository modes.

New features

This section contains new features recently added to the NexusOne platform.

Data APIs v2

Data APIs are now published, versioned APIs. You build one from a deployed semantic model or a table, never from free-form SQL. Each version freezes a contract, so it never changes underneath an integration built against it. A contract covers:
  • Grain and keys, meaning what one row represents
  • Columns and what each one means
  • Filters, sorts, and page limits
  • Worked examples of calling the API
  • An optional pinned context pack
Every API also ships a context envelope alongside its data. It holds the contract, DataHub owners, terms, lineage, links, and properties pinned at publish, plus live state such as freshness, incidents, deprecations, profile, and data quality. You can fetch it in these formats:
  • JSON
  • Markdown, for agents
  • Open Data Contract Standard (ODCS) v3
Each version also has its own OpenAPI spec. Every call runs as the user who made it, so Ranger decides what that user can see:
  • The response drops columns the caller can’t select from both the results and the context.
  • The response names masked columns, so the caller knows they’re masked.
  • You can’t filter or sort sensitive columns.
A Data API refuses the shared platform PSK. Any integration that calls a Data API needs to authenticate as an individual user.
Agents can reach Data APIs through three new read-only MCP tools: dataapi_list, dataapi_context, and dataapi_query. See Data APIs v2 for how to publish and manage an API from the portal.

Dedicated Hive Metastore database

This release adds a new tenant variable, metastore_dbname. Set it to point a tenant’s Hive Metastore to a dedicated Postgres database, instead of the shared hive schema in the core database. Create this database on the Postgres instance set in core tier’s dbhost variable. A tenant inherits that connection automatically, so you don’t set it yourself. If you leave metastore_dbname empty, then NexusOne keeps using the shared hive schema.

Karpenter autoscaling for Trino and Spark

This release adds a new tenant variable, karpenter_enabled. Set it to true to let Karpenter place Trino workers and Spark executors, so a cluster scales its nodes to match query and job load. Defaults to false.

New Keycloak roles

The following roles are now available in NexusOne, and you can assign them to users:
  • Centralized Grafana roles:
    • nx1_grafana_platform_admin: Signs in to the central Grafana with the Admin role
    • nx1_grafana_platform_viewer: Signs in to the central Grafana with the Viewer role
  • Tenant Manager roles:
    • nx1-tenant-reader: Read-only access across every tenant
    • nx1-tenant-reader-<tenant_name>: Read-only access for one tenant
  • nx1_engineer_admin: Grants engineering administrator access
  • nx1_jupyterhub_elyra: Turns on Elyra in JupyterHub for a tenant’s users

Bug fixes

This section contains fixes for issues affecting apps or features on the NexusOne platform.

Deployment reliability

A tenant apply could time out waiting on a PVC that never left the Pending state. In this release, NexusOne no longer waits for tenant PVCs to bind before continuing.

JupyterHub’s S3 browser use now checks for delete permission

Ranger has supported a delete permission, separate from write, for S3 resources for some time, and other NexusOne components already check it. The JupyterHub’s S3 browser didn’t check delete permission. Deleting or moving an object there, or deleting one with s3Cli rm, only checked for Ranger write permission. A user who could write to a bucket could also delete from it. This release brings the JupyterHub’s S3 browser in line with the rest of the platform, and requires delete permission for those actions instead. See Check S3 delete permissions before you upgrade.

Kyuubi’s Keycloak outage-tolerance cache now works

Kyuubi used to rebuild its whole authentication cache from scratch on every single request, not just after a restart. This meant Kyuubi had to refetch Keycloak’s signing keys on every request. A grace cache built to tolerate a Keycloak outage never actually held anything. This release shares the outage-tolerance grace cache across requests, so Kyuubi reuses a signing key it already fetched, instead of refetching it every time.

Spark

This release fixes two Spark issues, such as tenant isolation for scratch and upload paths, and the OpenLineage endpoint.

OpenLineage endpoint

Spark sent OpenLineage events to the wrong endpoint. This release fixes the endpoint.

Scratch and upload paths are now scoped per tenant

Spark uses a scratch space as a temp folder a query writes to while it’s still working, and an upload path to drop off a job’s code so the cluster can grab it and run it. Before this release, when tenants shared a Ranger repository to enforce the same policies and admins across them, Spark’s S3 scratch and upload paths weren’t scoped to each tenant. That meant one tenant could reach another tenant’s scratch and upload data. This release scopes the scratch and upload paths per tenant.

Other fixes

  • AiAPI: Fixed authentication for tokens generated through admin-cli.
  • Airflow: Fixed Keycloak signing-key resolution, so Airflow now retrieves the key on each login.
  • DataHub: Fixed a startup crash. Every DataHub component now receives DATAHUB_SYSTEM_CLIENT_SECRET.
  • Envoy: Fixed 404 responses for router-rewritten paths outside /v1/.
  • Grafana: Fixed Terraform plans that kept showing a diff for alert rule groups on every run.
  • JupyterHub: Fixed the notebook pod’s fsGid on OpenShift when you use shared, static home storage.
  • Ranger sync: Two versions of the sync agent no longer run at the same time during an upgrade.
  • Superset: Fixed initialization on the hardened image, which doesn’t include coreutils.

Enhancements

This section contains enhancements to existing app features on the NexusOne platform.

Airflow DAG bundles and Keycloak sign-in

Airflow now supports dedicated DAG bundles, and you can sign in to Airflow directly through Keycloak.

Airflow parallelism

Airflow caps how many tasks can run at once across your whole tenant, and how many runs of the same DAG can overlap. NexusOne used to leave both at Airflow’s own defaults, 32 and 16. This release increases both to 128.

Connect existing Claude integrations through AWS

NexusOne’s AI features, such as crews and natural-language SQL, could already call Claude through the public Anthropic API. This release lets you reach Claude through Anthropic’s own Claude Platform hosted on AWS instead. This is important if your AI traffic needs to stay inside your own cloud for compliance reasons. Configure it in NexusOne’s Tenant Manager with these new tenant variables:
  • llm_anthropic_base_url: Overrides the base URL for the Anthropic API, so you can reach Claude through your Claude on AWS API, instead of Anthropic’s standard one. Leave empty to use Anthropic’s standard API endpoint.
  • llm_anthropic_workspace_id: Workspace ID for Claude on AWS.
  • llm_extra_headers: If your LLM gateway or backend needs custom authentication headers, then set them here. AiAPI attaches them to every LLM call.
For the AI Control Plane’s new Claude Platform on AWS endpoint preset, see Connect Claude through your AWS account. This release also adds two more AI integrations:
  • The NX1 LLM router now accepts portal JWTs for authentication.
  • A new Trino ai catalog connector.

DataHub

Besides the upgrade to the DataHub 1.7.0.1 image, this release changes DataHub in these ways:
  • datahub-upgrade replaces the legacy setup jobs.
  • DataHub no longer reports telemetry.
  • Kafka topics, new, and existing, now use zstd compression and a 50 MB message limit.
  • DataHub now enables the business attributes and metrics features.
You can also now deploy optional network policies for DataHub. Set create_networkpolicy to true to create them. Defaults to false.

Expanded Ranger repository modes

A new tenant variable, ranger_repo_mode, sets how NexusOne groups a tenant’s Ranger policies into repositories. It accepts three modes:
  • tenant: The tenant gets its own repository.
  • shared: The tenant shares one repository with other tenants.
  • group: The tenant shares a repository with a group of tenants.
The previous single_ranger_repo variable still works as an alias. This release also adds support for running mixed Ranger releases, optional cleanup of URL policies, and cleanup of unused service definitions.
Changing ranger_repo_mode on an existing tenant creates a repository that contains only default policies. See Plan your Ranger policy migration before you switch.

JupyterHub

These JupyterHub settings are now configurable per tenant:
  • Session timeout: Set the tenant tier variable jupyter_session_timeout to how long, in seconds, an idle notebook server and its kernel run before NexusOne shuts them down. Defaults to 3600.
  • Notebook image tags: Choose which notebook image tags a tenant uses.
  • Elyra: Turn on Elyra for a tenant’s users with the new nx1_jupyterhub_elyra Keycloak role.

Kyuubi

This release adds a new way to connect to Kyuubi, and a new way to sign in to it.

NX1 PSK authentication

Kyuubi can now authenticate you with a personal NX1 user PSK or Keycloak JWTs. PSK authentication is on by default. For a personal NX1 user PSK, Kyuubi caches resolved identities, so it doesn’t call back to the NX1 API on every request. You can also now sign in to the Kyuubi Web UI through Keycloak.

Use Spark’s DataFrame API remotely

Kyuubi can now run an optional Spark Connect frontend on port 15002. It lets Spark Connect clients reach a tenant’s Spark without a full Spark install on the client side. Set the tenant variable spark_connect_enabled to true to enable it. Defaults to false. Spark Connect only works on Spark 4, so set spark_version to "4" too.

Observability

This release improves Grafana and distributed tracing across the platform.

Centralized Grafana for the modules tier

Each tenant already has its own Grafana. This release adds an optional, platform-level Grafana in the modules tier. It gives you one view across every tenant’s metrics, logs, and traces, instead of a separate login per tenant. Central Grafana queries each observability-enabled tenant’s existing Mimir, Loki, and Tempo backends directly. It doesn’t change where any tenant stores its telemetry. Enable it in your modules tier config by using the following new variables:
  • central_grafana_enabled: true to deploy the central Grafana. Defaults to false.
  • central_grafana_sync_enabled: true to deploy the datasource sync that discovers tenants. Only used when central_grafana_enabled is true. Defaults to true.
Assign one of the new Grafana Keycloak roles after you first deploy central Grafana. Nobody can sign in through an SSO until someone has one. NexusOne also creates a Fleet Overview dashboard by default, which includes:
  • A per-namespace workload table: Pods running, pending, and failed, plus restarts and containers not ready.
  • CPU and memory requested vs used.
  • A table of Kubernetes events, with a count of warning events in the last hour.
Before you enable central Grafana at a site, redeploy each tenant from Tenant Manager so its NetworkPolicy allows central Grafana through. Until you do, each tenant’s metrics and logs panels appear empty on central Grafana, while its traces still work.

Distributed tracing for Spark and Kyuubi

You can now send OpenTelemetry traces from Spark and Kyuubi to Tempo. This lets you follow a single query from Kyuubi through to the Spark jobs it runs. Set the tenant variable tracing_enabled to true to enable it. Defaults to false. Tracing requires the tenant’s embedded monitoring stack.

Grafana content controls

The tenant tier variable grafana_alerts_enabled, added in the nx1-deployer 1.16.1 image, only turned off NexusOne’s built-in Grafana alert rules for platform components like Spark and Gravitino. This release adds a dedicated Grafana content module, controlled by a broader variable:
  • grafana_content_enabled: Whether NexusOne manages Grafana’s content or not. If you disable it, then you manage that content yourself, while Mimir, Loki, and Tempo keep running unaffected. Defaults to true.
The previous grafana_alerts_enabled variable is now a subset of this new variable, so it’s ignored when grafana_content_enabled is false. This release also adds Grafana dashboard embedding in the portal, and support for collection tiers that span namespaces.

Ranger’s unmask policy now uses a Keycloak role per data classification

Before, every Ranger masking policy used the same nx1_pii_unmask role to unmask a column, no matter what kind of sensitive data it held. A Payment Card Industry (PCI) column and an authentication column shared one role, so granting someone unmask access to one gave them access to both. This release adds two new unmask roles, one per classification, instead of reusing the old shared role for everything. These new roles include:
  • nx1_pci_unmask: Members can unmask a PCI column.
  • nx1_authentication_unmask: Members can unmask an authentication column.
Accepting your first suggestion for a classification creates its role in Keycloak, with no members, so acceptance doesn’t grant access to anyone until you assign members yourself.

S3 endpoint formats

You can now specify an S3 endpoint as either host or scheme://host. If you don’t set a region, then it defaults to the cluster’s region.

Scheduling and ingress

This release makes small scheduling and networking improvements to YuniKorn, Envoy, and Trino:
  • YuniKorn: When you enable YuniKorn, it now schedules the remaining pods too, not just some of them.
  • Envoy: Envoy’s listener configuration now supports targeting a specific Gateway listener by name. It can also configure whether Envoy forces the HTTPS forwarded-proto header for backends, which fixes Keycloak mixed-content and third-party-iframe timeouts.
  • Trino: Trino no longer logs the AWS SDK v1 deprecation notice.

Secrets in deployment metadata

Every Terraform variable that holds a secret is now marked sensitive, and the deployer includes the patched Helm provider 3.3.0. Together, these keep secrets out of deployment metadata. This release also removes the unused Vault provider.

Spark

This release changes how you configure Spark and choose its version.

Choose your version

This release adds a new tenant variable, spark_version. A Spark 4.2 image is now available alongside the existing Spark 3.5 image, and this variable lets you choose between them. Pass "3" for Spark 3.5.6 on Scala 2.12, or "4" for Spark 4.2.0 on Scala 2.13. Defaults to "3". Airflow, JupyterHub, and Kyuubi each match the version you choose automatically, so the whole tenant stays on one consistent Scala version. See Mirror the new Spark image names.

Configuration and catalog discovery

Spark used to get its configuration from copysparkhome and the utils mount. This release replaces both with a new spark-conf module, which delivers Spark’s configuration through a Kubernetes Secret and its own dedicated Keycloak client. Spark also now discovers Iceberg REST catalogs, including Gravitino, at runtime through AiAPI. Before, each catalog needed a static entry in spark-defaults.conf.

SQL access to decisions in Trino and Spark

Before this release, a decisions_workspaces tenant variable required you to list every workspace explicitly, so adding one meant a terraform apply and a restart before Trino or Spark could see it. This release adds a new tenant variable, decisions_enabled. Set it to true to let Trino and Spark discover a tenant’s workspaces automatically and query the business rules they contain through SQL functions. Defaults to true.

Superset

Superset now ships its own MCP service. AiAPI is the backend service behind NexusOne’s AI features, such as chat and crews. AiAPI now proxies Superset’s MCP service through its own existing MCP server, so those features can work with Superset directly.

Switch for required user profile fields

NexusOne now manages the Keycloak realm User Profile, and lets you choose whether users must have an email, first name, and last name. Control this with the new user_profile_required variable:
  • true: Requires email, first name, and last name. This matches Keycloak’s default.
  • false: Makes email, first name, and last name optional.
Defaults to true. Set it in the modules tier for the shared services realm, and in the tenant tier for each tenant’s realm. Every other profile attribute stays the same in both modes. Set it to false if your users sync from LDAP without an email. Those users can then authenticate with a username and password, for example to Kyuubi. Some components use the email to identify users, so users without one can run into issues. For example, when they sign in to DataHub, or when they request an Airflow API token with a username and password.
NexusOne now owns the realm User Profile, so each deployment overwrites changes you made to it in the Keycloak administrator console. This includes fields you made optional there. If you made these fields optional manually, then set user_profile_required to false before you upgrade to keep them optional.

Upgrades

This section contains version upgrades to existing apps on the NexusOne platform.

Security

This release also includes hardened or CVE-patched security updates for the following components:
  • Alloy
  • JupyterHub configurable-http-proxy
  • Keycloak
  • Loki
  • oauth2-proxy
  • OpenSearch
  • postgres-exporter
  • Ranger and ranger-sync
  • Redis
  • Superset
  • The observability router and webhook
  • YuniKorn scheduler

Image versions

The following images changed in this release: Images not listed keep their tags from the previous nx1-deployer image version, 1.16.1.