PhoenixAI version 26.2

26.2.0

Release Date: September 21, 2026

Upgrade and compatibility

v26.2 supports an upgrade from v4.1. If an upgrade does not go as planned, you can roll back to the latest v4.1 patch release.

Read this section before upgrading. Six defaults change, one configuration option has been removed, and one table property is no longer accepted on cloud-native (shared-data) Primary Key tables. Each entry names the setting that controls the behavior and its default before and after. Everything here takes effect on upgrade without any action on your part.

Existing tables are not rewritten or redistributed by the upgrade.

• Global late materialization is now on by default.

  Columns that are not used in predicates are read after predicate evaluation on native table scans, rather than alongside them. For a selective query this reads substantially less data; for a query that returns most rows it adds a second pass over the row positions. Query profiles change shape accordingly, so a saved profile from v4.1 will not compare directly. Controlled by the session variable enable_global_late_materialization, whose default moves from false to true, together with enable_global_late_materialization_cost_based, which lets the optimizer decide per query.

• JDBC join pushdown is now on by default.

  A join between JDBC external tables is executed by the source database and only the result is returned, instead of PhoenixAI reading both tables and joining locally. This is usually much faster and transfers far less data. It also moves work onto the source system, which is worth checking if that system is capacity-constrained or shared with transactional traffic. The controlling session variable is enable_jdbc_join_push_down, new in this release and defaulting to true. It is not listed by SHOW VARIABLES, but it can be set per session or globally.

• Avro data is now read by the native reader instead of the JNI reader.

  This affects Avro files in Hive and Iceberg tables. The JNI reader's CHAR handling was aligned with the native reader before the default changed, so results should be identical; performance characteristics differ, and a workload tuned around the JNI reader should be re-measured. The session variable avro_use_jni_reader moves from true to false; set it back to true to restore the previous reader.

• Iceberg scan planning now runs locally by default.

  Planning happens on the coordinator rather than being selected per query. This makes planning time predictable, which matters most for tables with large manifest lists where automatic selection could occasionally choose the slower path. The session variable plan_mode moves from auto to local; set plan_mode = auto to restore the previous behavior.

• Cloud-native Primary Key tables now use the cloud-native persistent index only.

  The LOCAL persistent index is no longer supported for these tables. Existing table metadata is normalized to the cloud-native index automatically when the FE loads after the upgrade, so no migration step is required, but persistent_index_type = LOCAL set explicitly on a cloud-native (shared-data) Primary Key table will no longer be honored, and the setting is rejected on new tables.

• Horizontal compaction no longer fills the local data cache.

  Horizontal compaction reads each input byte exactly once and replaces its input rowsets immediately afterwards, so caching that data mostly evicted query-hot data and added an inline local-disk write on every cache miss. Removing it brings horizontal compaction in line with the other full-scan background paths. The new BE configuration lake_enable_horizontal_compaction_fill_data_cache defaults to false; set it to true to restore the previous behavior. Vertical compaction, which scans its input once per column group, is unchanged.

• Vector index caches are no longer populated when an index is built.

  Building an index no longer warms the cache, so a build does not displace cached data that queries are using. Query-time caching is unchanged, and the cache can load asynchronously on a miss rather than blocking the query. The new BE configuration enable_vector_index_cache_on_build defaults to false.

• enable_experimental_vector has been removed.

  Vector indexing is no longer experimental and the configuration that gated it is gone. Remove it from fe.conf before upgrading — the FE will not recognize it.

• DELETE on a non-Primary Key table now returns an informational notice.

  The statement behaves exactly as before; it now explains what it did rather than succeeding silently.

• Segment index sidecar files are now included in external cluster snapshots.

  Files with the .vi and .idx extensions — which hold vector and other segment-level indexes — were not copied into an external cluster snapshot, so a cluster restored from one came back without them. If you use vector indexes and rely on external cluster snapshots for recovery, take a fresh snapshot after upgrading; snapshots taken before the upgrade are incomplete.

• Stricter implicit-cast checking is available, and off by default.

  The new sql_mode value FORBID_INVALID_IMPLICIT_CAST rejects implicit casts that previously succeeded and produced a surprising value. Enabling it can turn queries that ran in v4.1 into errors, which is the point — consider it for new workloads before existing ones.

New features

Search and AI

• Vector search on shared-data clusters

  Vector indexes were previously available only on tables in shared-nothing clusters, which meant an AI workload could not sit on the same storage architecture as the rest of the warehouse. In v26.2, vector indexes can be built, written and read on cloud-native (shared-data) tables, and an index can be added to an existing table with ALTER TABLE rather than requiring the table to be recreated. Index files are named by their owning tablet and tracked in shared-data metadata, so storage is reclaimed precisely when an index is dropped and indexes survive tablet split and merge on range-distributed tables. Both L2 distance and inner product are available as index backends, chosen when the index is created. ef_search, which controls how much of an HNSW graph a query explores, is now derived from the query rather than fixed globally — it scales with the requested k and the size of the data being searched, so a query asking for many results no longer needs a hand-tuned session variable, and a search that under-fills falls back rather than returning a short result set. A query vector can also be bound as a parameter with CAST(? AS ARRAY<FLOAT>), so an application can use a prepared statement instead of building SQL text with a few hundred floats inlined in it. Vector indexing is no longer gated behind an experimental flag — see Upgrade and compatibility for more information.

• Vector indexes build asynchronously

  Building an HNSW index is CPU- and I/O-intensive, and doing it inline meant a large build could slow the ingestion that triggered it. Index construction now runs as a background task. Build concurrency, the maximum share of CPU a build may consume, and the row-count threshold below which no index is built are all configurable. Builds are scheduled with awareness of compaction so the two do not contend for the same I/O, and they run in the table's own warehouse rather than wherever the write landed — so index maintenance can be isolated from query capacity. information_schema.partitions_meta reports the version each partition's index was built at, which is how you tell whether an index has caught up with recent writes.

• Index quantization, for control over memory versus recall

  SQ4, SQ8 and PQ quantization reduce the memory an index occupies, at some cost in recall. Because a quantized index stores approximate vectors, results can optionally be refined against the original vectors; that refinement is now an option you control and is keyed to the quantization in use, rather than implicit behavior that varied by configuration. The trade-off is therefore made explicitly at index creation instead of being discovered later.

• Filtered vector search

  A nearest-neighbor search combined with a predicate previously had to search first and filter afterwards, which either returned too few rows or forced a much larger search to compensate. v26.2 supports pre-filter strategies, so the predicate constrains the search itself. This matters most for multi-tenant tables, where almost every vector query is scoped to one tenant.

Lakehouse

• Iceberg UPDATE

  v4.1 added DELETE on Iceberg tables. v26.2 completes row-level DML with UPDATE, reusing the same commit path, so an Iceberg table can be corrected in place instead of being rewritten by an external job. Updates are committed as an Iceberg row-delta, so readers of the table — including engines other than PhoenixAI — see a consistent snapshot throughout. Update activity is exposed as metrics.

• Iceberg MERGE INTO

  MERGE INTO applies inserts, updates and deletes from a source query in a single statement, which is the natural shape of a change-data-capture load or a slowly-changing dimension. v26.2 implements the full semantics rather than a subset. If the source contains more than one row matching the same target row, the statement fails instead of applying an arbitrary one — the uniqueness check runs on the join distribution, so it costs no extra shuffle. This makes MERGE INTO safe to run against a source you do not fully control.

• Iceberg v3 deletion vectors

  Deletion vectors are a compact representation of row-level deletes introduced in Iceberg v3. v26.2 both reads and writes them: DELETE produces Puffin-encoded deletion vectors, and deletes that already exist on the table are merged into the vector rather than left behind as a second mechanism the reader has to consult. Deletion vectors are served through the data cache, so repeated reads of a table with many deletes do not re-fetch them from object storage. Write activity is exposed as metrics.

• Iceberg VARIANT shredding

  A VARIANT column holds semi-structured data whose shape is not fixed in the schema; shredding stores frequently accessed fields as separate typed columns in the Parquet file while keeping the rest as an opaque value. v26.2 reads both shredded and non-shredded VARIANT columns. Querying one field no longer means materializing the whole column: field paths are reconstructed from the typed columns, pruning is path-aware, predicates push down using the shredded columns' Parquet statistics, and shredded fields participate in low-cardinality dictionary filtering like ordinary columns. In practice this closes most of the gap between a VARIANT field and a real column for read performance.

• Iceberg metadata maintenance runs automatically.

  An Iceberg table accumulates snapshots, manifests and orphaned files, and left alone this degrades planning time and wastes storage. Maintenance is now scheduled from catalog-level properties that individual tables inherit and can override, so it can be configured once per catalog. Statistics and metrics are collected per task, and a new information_schema.iceberg_maintenance_tasks table shows what ran, when, and what it did. Orphan-file scanning, the most expensive maintenance operation, is parallelized.

Real-time and shared-data engine

• Incremental materialized views on cloud-native tables

  Incremental maintenance updates only the rows that a base-table change affects, instead of recomputing the view — the difference between a refresh proportional to the change and one proportional to the table. v26.2 extends incremental maintenance to shared-data cloud-native tables, including Primary Key base tables, where it previously applied to lake tables. The supported shapes are aggregates, top-level inner and cross joins, top-level UNION ALL, derived-table subqueries, and single-table projection and filter views with retractable maintenance; rollups over BITMAP_UNION, HLL_UNION and PERCENTILE_UNION on aggregate base columns are also supported. Aggregates are recomputed from a single snapshot rather than two, which reduces the work each refresh does. A view now reports which distribution it actually uses, and range distribution is available for incremental views.

• Range distribution is ready for production use.

  With hash distribution the bucket count is fixed when a table is created: guess low and a partition cannot use the available parallelism, guess high and small partitions carry needless metadata, and changing it means rebuilding the table. Range distribution splits and merges tablets as data grows, so a partition's tablet count is not decided in advance and has no fixed ceiling. v4.1 could create range-distributed tables but not maintain them, which made them impractical for a table you intend to keep. v26.2 adds the missing operations — on cloud-native range-distribution tables you can now:

  • change the key and sort key, add or drop a key column, and widen an integer sort-key column
  • create multiple rollups, including a rollup with its own independent sort key
  • change ORDER BY on an asynchronous materialized view online
  • migrate data
  • use range distribution for incremental materialized views

  Tablet pre-splitting, which sizes tablets from a sample instead of waiting for splits to happen, extends to range tables with a rollup index and to INSERT ... FROM into a range table. Vector indexes and lake index deltas both follow tablet split and merge. Range distribution stays opt-in: it is chosen per table with DISTRIBUTED BY, and upgrading does not change how any existing table is distributed.

• Rejected rows are queryable

  When a bulk load rejects rows, finding out which rows and why previously meant retrieving the load's error file. v26.2 records rejected rows in a system table maintained by a background daemon, so they can be queried with SQL — filtered by load, inspected alongside the reason for rejection, and joined against whatever else you need to identify the source. This turns a load failure into something an analyst can diagnose without cluster access.

• VARCHAR up to 2 GiB

  The VARCHAR ceiling rises from roughly 1 MB to 2 GiB minus 1 KiB. This matters for JSON documents, serialized payloads, model inputs and long text that previously had to be split across columns or truncated at load time. Page-encoding overhead for large strings is reduced, so the wider limit does not cost proportionally more storage for values that stay small. FILES(), translate() and the type documentation are updated accordingly.

• Light-weight tablet creation

  Creating a table normally requires every compute node that will hold a tablet to write that tablet's metadata before the statement returns, which dominates create time for a table with many tablets and makes bulk table creation slow. The new light_weight_tablet_creation table property skips that step; metadata is produced on demand, with a fallback on the compute node if it is missing. Schema changes and rollups skip the corresponding replica-creation work. The property is reversible — turning it off backfills the metadata that was not written.

• Explicit read schema for FILES()

  A schema parameter sets the schema explicitly instead of relying on inference, which matters when the first file in a set is not representative or when a column's type must match an existing table exactly.

Query execution

• Finer-grained parallel scan within a tablet

  Scan parallelism was bounded by the number of tablets a query touched, so a table with a few large tablets could not use the cores available. That is an awkward constraint, because larger tablets are otherwise desirable — they mean less metadata and cheaper maintenance. v26.2 parallelizes scanning inside a single tablet, so scan concurrency is determined by the hardware rather than by a distribution decision made when the table was created. The practical effect is that tablet sizing becomes a storage decision instead of a query-performance decision, and a table does not need to be over-bucketed just to keep scans parallel.

• Prepared physical split scan for lake tables

  A scan over a lake table is only as fast as its slowest split, so one oversized or skewed tablet sets the time for the whole scan. v26.2 prepares scan ranges before execution: pruning happens against seed metadata, the read state is built once and reused, and skewed tablets are divided so work is distributed by size rather than by tablet count. Splits are refined before the scan starts instead of being discovered during it. The path is deliberately disabled for duplicate scans of the same table in one query, where preparing twice would cost more than it saves.

• Query queue in ETL mode

  Query queueing was designed around interactive queries, which are short and roughly comparable in cost. ETL statements are neither, and treating them the same way meant either admitting too many and exhausting memory, or admitting too few and leaving the cluster idle. v26.2 adds an ETL mode to the query queue and improves how the slots a statement will occupy are estimated, so admission reflects what a statement will actually hold rather than a uniform assumption. This makes it practical to run ETL and interactive workloads on one cluster under a single admission policy.

• JDBC aggregate pushdown

  Aggregation over a JDBC external table is now executed by the source database, so a GROUP BY over a large remote table returns grouped rows instead of transferring every row to be aggregated locally. Together with the join pushdown default change described above, this makes a JDBC catalog usable for more than small dimension lookups. A schema_resolver catalog property controls how schemas are resolved on the source, which matters for databases whose schema and catalog concepts do not map cleanly. Native query passthrough sends a statement to the source unchanged when you want the source's own dialect and semantics rather than a translation.

Improvements

This release includes several hundred further improvements. The themes most likely to be visible in your workload:

• Low-cardinality dictionary optimization

  Low-cardinality dictionary optimization now covers ARRAY_AGG and its sort columns, array functions with constant arguments, structs, UNION ALL over constant strings, physical filters, CTE operators, and more string functions including ifnull, STARTSWITH and ENDSWITH. The global dictionary cache is bounded by memory rather than entry count, so a workload with many distinct string columns no longer evicts unpredictably.

• Cardinality estimation

  Cardinality estimation improves for compound predicates, OR predicates, range and inequality comparisons, outer join keys and date and time functions, and Iceberg column statistics now feed the optimizer's min/max rule. Better estimates mostly show up as better join orders.

• Compaction and vacuum

  Compaction and vacuum gain parallel compaction for non-overlapping output, full-chain compaction profiles, fair scheduling for automatic vacuum, and a faster path for large Primary Key deletes at publish time.

• Bundled connector libraries

  The Delta Lake kernel moves from a 4.0 release candidate to 4.3.0, and Apache Arrow to 24.0.0.

• Storage internals

  Adaptive string offsets, large binary column support through the aggregate and window paths, delta-offset encoding for string columns, and reusable segment iterators.