Non-web portal users

The August 2026 release for non-web portal users adds new API endpoints across the AI router, code scanning, S3 credentials, and data quality. It also adds disaster recovery support and fixes multi-site domain and cache issues in Keycloak. AI API, Metastore, and Keycloak also gain autoscaling, and Kyuubi and S3 Gateway get reliability fixes.

​

New features

This section contains new features recently added to the NexusOne platform.

​

AI router classifier management

The NX1 LLM router now supports classifiers that tag incoming requests with a DataHub domain, so the router’s decision rules can act on that domain. You can manage those classifiers and publish them to the router’s live config through the following new API endpoints:

• Compile enabled classifiers into the live router config
• Create a heuristic classifier
• Delete a classifier
• Get a single classifier
• List governance domains
• List heuristic classifiers
• Update a classifier

​

AutoLLM fine-tuning

AutoLLM is a new pipeline for fine-tuning your own large language models. A CrewAI run first assembles a DataHub domain’s knowledge into a Q/A training dataset, which you review and approve before it trains. Its new API endpoints include the following:

• Approve a dataset proposal and finalize it
• Create a fine-tune training job
• Create a fine-tuning dataset proposal
• Delete a fine-tune training job
• Delete a fine-tuning dataset job
• Get a fine-tune training job
• Get a fine-tuning dataset proposal
• Get the full example set for a dataset job
• List base models available for fine-tuning
• List fine-tune training jobs
• List fine-tuning dataset jobs
• Manually trigger a training DAG
• Re-run the crew on a failed dataset job
• Trainer status writeback

​

Code scanner

The code scanner scans a Spark codebase for changes needed to migrate it to NexusOne, and flags each issue by severity. Its new API endpoints include the following:

• Create a code scan run
• Delete a code scan run
• Get a code scan run
• Get the code scan report
• Lifetime code scan statistics
• List code scan runs
• Poll code scan run status
• Resume a paused scan
• Stop a running scan

​

Data migration

A new Count data migration runs matching filters endpoint returns a count of migration runs matching a filter, without listing them all.

​

Data quality rule suggestions

CrewAI already suggests data quality rules from a table’s sample data. You can now preview a suggestion’s matching data before promoting it to a real rule, or create rules in bulk. The new API endpoints include the following:

• Bulk create data quality rules
• Create a suggested data quality rule
• Preview rule data

​

Inferencing

Deploying a fine-tuned AutoLLM model with vLLM applies its Kubernetes manifests and registers it with the NX1 LLM router automatically. You can do this through the following new API endpoints:

• Delete a stopped vLLM deployment
• Deploy a fine-tuned model with vLLM
• Get a vLLM deployment
• Get Kubernetes events for a vLLM deployment
• List vLLM deployments
• Stop a vLLM deployment

​

Metastore

A new Get unredacted Iceberg REST catalog config endpoint that returns a single Iceberg REST catalog’s full connection config. NexusOne’s Gravitino Iceberg REST server calls it to authenticate to that catalog’s upstream endpoint, such as another tenant’s data shares credential or a third-party catalog like Unity Catalog.

​

S3 credential vending

You can now generate, view, and delete your own S3 access credentials, and vend credentials for a specific bucket or the platform default. The new API endpoints include the following:

• Delete the current user’s S3 credential
• Generate or regenerate the current user’s S3 credential
• Get the current user’s S3 credential information
• Resolve an S3 access key to its owner and secret
• Vend credentials for a bucket
• Vend the default credentials

​

Bug fixes

This section contains fixes for issues affecting apps or features on the NexusOne platform.

​

AI API no longer blocks on Trino during a fresh tenant deployment

AI API is NexusOne’s natural language SQL and data engineering service. Its pod mounts two Trino-managed Kubernetes volumes at startup:

• trino-s3: S3 access config that Trino also uses
• trino-gateway-routing-rules: Routing config that Trino Gateway uses to reach the right Trino cluster

AI API reads from those volumes whenever it calls Trino to run the SQL it generates. In previous releases, Terraform defined those volumes inside the Trino and Trino Gateway modules. But portal, which deploys AI API, had no dependency on either. That meant AI API’s pod could start on a fresh tenant deployment before the volumes even existed, and it would just sit pending. In this release, Terraform moves the two volumes out of the Trino and Trino Gateway modules into standalone top-level resources. The Trino, Trino Gateway, and portal modules now all explicitly depend on them. AI API’s pod no longer waits on Trino during a fresh deployment.

​

Data shares catalog URL now uses a global domain

The data shares credential feature lets one NexusOne tenant share Iceberg datasets with another, through a Gravitino Iceberg REST Catalog (IRC) server. In previous releases, NX1 Portal showed that server’s URL using only the local domain Terraform variable. A consumer tenant on a different cluster can’t reach a local-only URL. The portal now shows the IRC server’s URL at a global_domain Terraform variable by default. The portal still shows the domain URL too, but only as a fallback when global_domain and domain are different.

​

DR support

DataHub’s OpenSearch previously had no way to keep a standby site’s data current. There was no multi-site snapshot support, no security plugin protecting the search API, and no way to point DataHub at the active site. JupyterHub’s and Kyuubi’s database schemas also had no cluster namespacing, so main and standby sites sharing one database would collide. DR support is now built into the shared services core and modules layers. The main site now snapshots OpenSearch to S3 hourly, not every 6 hours. That changes standby and demoted site behavior, covering the following:

• Standby site: Auto-restores the main site’s latest snapshot, using the main site’s snapshot bucket directly instead of its own.
• Demoted site: Deletes its own snapshot policy instead of pausing it, so gaps don’t appear silently after promotion.
• OpenSearch security: A new security plugin adds basic auth and transport TLS, and an optional external route lets DataHub follow whichever site is active.
• Per-cluster JupyterHub schema: JupyterHub’s schema name now includes a new cluster_name Terraform variable. Setting a distinct value per site gives each one its own schema within the same shared database, instead of colliding.
• Per-cluster Kyuubi schema: Kyuubi’s schema name now includes that same cluster_name Terraform variable. It falls back to the workspace name automatically when cluster_name isn’t set.

​

Keycloak

This release includes Keycloak fixes for Active Directory sync, cache staleness on standby-to-main site promotion, and multi-site domain routing.

​

Active Directory sync now runs every 6 hours

Keycloak previously resolved Active Directory group membership with a recursive LDAP query, searching upward through each group’s parents on every sync. It also checked Active Directory for changes only once a day. In this release, Keycloak’s LDAP group sync reads each group’s member attribute directly instead, finding only direct group membership. Active Directory change checks are now every 6 hours instead of every 24.

​

Cache staleness on site promotion

The main and standby NexusOne sites each run their own Keycloak, but the two share a database and previously ran separate caches. Promoting a standby site to main could leave stale realm and user data. Users could see outdated group or role assignments until those cache entries expired on their own. In this release, a new cache-clear job runs on site promotion and clears stale user, realm, and keys caches. That job needs to run in Keycloak’s own image, which was previously hardcoded. The image tag is now the keycloak_tag Terraform variable, default 26.5.6.1, and the keycloak_image output tracks it automatically for the job.

​

Domain routing fixes

NexusOne supports multi-site DR pairs, a main site and a standby site. Some components need a domain that resolves to whichever site is currently active, so users don’t have to know which one is live. In previous releases, a few components used the alternate_domain Terraform variable to provide that active-site domain. Airflow stayed hardcoded to the local domain instead, breaking access through the second domain. This release renames alternate_domain to global_domain, with no backward compatibility. Airflow’s webserver URL and Trino connections now follow that variable when it’s set.

​

S3 Gateway now enforces Ranger policy consistently

The S3 Gateway had the following two authorization issues in previous releases:

• Ranger policy and the bucket’s own legacy policy or Access Control List (ACL) ran separately for every request. They could give conflicting answers for the same request.
• The copy-source read that happens during a copy operation doesn’t route through Ranger’s central authorization hook.

In this release, S3 Gateway makes Ranger the single source of truth for authorization. The gateway no longer checks the legacy bucket policy or ACL path, though read-only and object-lock checks still apply separately. The copy-source read is now explicitly authorized through Ranger too.

​

Session and job recovery for Kyuubi database failover

When you connect to Spark through Kyuubi, Kyuubi creates a session for each connection. Each session gets a Spark engine to run its jobs. MetadataManager stores that session, engine, and job metadata in a managed database, and reads it back when needed. In previous releases, that database could fail over and cause MetadataManager’s reads to fail instead of retrying. This caused a Kyuubi session’s jobs to restart or drop instead of continuing. In this release, Kyuubi now retries failed MetadataManager reads a few times, while waiting longer between each try. When a database failover happens, it no longer interrupts a session or its running jobs. MetadataManager writes are also now protected from concurrent-write conflicts, so job history stays consistent across a failover. Lastly, Kyuubi’s MetadataManager logger no longer logs anything.

​

Superset’s Trino connection now uses a global domain

Superset’s Trino connection config lives in a database that main and standby sites share. In previous releases, that config’s Trino host value used the local domain Terraform variable. Two sites writing their own local Trino host value to the same shared row meant whichever site’s init job ran last would win. The other site was then left with a config pointing at a Trino host it can’t reach. Superset’s Trino host now uses a global_domain Terraform variable instead. That name resolves to whichever site is currently active, so the config stays valid from either cluster.

​

Enhancements

This section contains enhancements to existing app features on the NexusOne platform.

​

AI API now scales automatically with load

AI API previously ran a fixed two replicas, with hardcoded CPU and memory limits. That worked exactly as configured, but it couldn’t grow under heavy load or shrink when idle. In this release, a Kubernetes HorizontalPodAutoscaler scales AI API automatically, using the following new Terraform variables:

• hpa_min_replicas/hpa_max_replicas: Scales between 2 and 6 replicas by default.
• hpa_target_cpu: Adds or removes pods when average CPU usage crosses this target percentage.
• hpa_memory_enabled/hpa_memory_average_value: Turns on memory-based scaling, in addition to CPU. Pods scale up once memory passes the hpa_memory_average_value amount. It never scales down, since memory usage doesn’t reliably drop even when idle.
• hpa_scaledown_stabilization: Seconds to wait before scaling down. Scale-up is immediate. Scale-down removes one pod at a time.
• cpu_request/cpu_limit/memory_request/memory_limit: CPU and memory requests and limits for the pod.

​

DataHub editor and reader roles now available through Keycloak

DataHub has three built-in access levels, Admin, Editor, and Reader. In previous releases, Keycloak could only grant the Admin role. Assigning Editor or Reader access required configuring it manually inside DataHub instead. In this release, two new Keycloak realm roles, datahub-editor and datahub-reader, grant DataHub’s Editor and Reader roles the same way Admin already worked. All three DataHub access levels are now assignable directly through Keycloak.

​

kyuubi-submit can now authenticate with a Keycloak token

The kyuubi-submit command-line tool already supported a --token flag for Bearer-token authentication, as an alternative to --username/--password. In previous releases, there was no way to actually obtain a token, since no Keycloak client existed for it. In this release, users can request a token from Keycloak directly by specifying client_id=kyuubi-cli. That token carries the user’s realm roles as a groups claim, for downstream authorization. Users can then pass that generated token to kyuubi-submit --token instead of a username and password.

​

Spark History Server URL now also shows running jobs

Spark’s built-in job UI shows live progress, stages, and executors. It lives on the job’s own driver pod, though, at a port that changes with every job. The Spark History Server serves a saved copy of that page once a job finishes, at a fixed, familiar URL. But visiting that URL for a still-running job didn’t work, since History Server only has saved copies. In this release, the History Server’s URL routes through a new spark-web-proxy service instead. It checks whether a job is still running and forwards to that job’s live page. If the job already finished, then it forwards to the History Server’s saved copy like before.

​

T-shirt sizing extends to Metastore and Keycloak

Kyuubi already supported t-shirt sizing, with small, medium, large, and xlarge tiers, each setting replicas, CPU, memory, and JVM heap together. Metastore and Keycloak didn’t. Their sizing stayed hardcoded regardless of tenant size. In this release, Metastore reuses the existing kyuubi_sizing variable to size itself alongside Kyuubi, since the two scale together. Keycloak gets its own new tiers through the existing datahub_sizing variable instead. Both changes cover the following:

• Metastore’s replica count now varies by tier:
  • small: 2
  • medium: 3
  • large: 4
  • xlarge: 6
• Keycloak gains real autoscaling for the first time, based on CPU usage.
• Keycloak’s minimum replicas never drops below 2, so its session cache always keeps more than one copy.
• Both now scale their JVM heap size with the tier too. Previously, the heap stayed fixed even after raising the memory limit, so the extra memory went unused.

​

Upgrades

This section contains version upgrades to existing apps on the NexusOne platform.

​

JupyterHub 5.4-nx1.10.1 upgrade

Upgraded JupyterHub from 5.4-nx1.6-slim to 5.4-nx1.10.1. The 5.4-nx1.9 and 5.4-nx1.10 upgrades happened in the middle of this release.

​

Metastore 3.1.7 upgrade

Upgraded Metastore from 3.1.4 to 3.1.7. The 3.1.5 and 3.1.6 upgrades happened in the middle of this release.

​

S3 Gateway v0.17 upgrade

Upgraded S3 Gateway from v0.15 to v0.17. The v0.16 upgrade happened in the middle of this release.