Skip to main content
Version: 26.2 Preview

PhoenixAI version 26.2

26.2.1​

Release Date: November 2, 2026

Behavior Changes​

  • The Poco HTTP client for the AWS SDK is now off by default. The BE configuration enable_poco_client_for_aws_sdk moves from true to false, so object-storage access uses the default client again. The request timeouts are also applied to the pooled Poco session. To restore the previous behavior, set enable_poco_client_for_aws_sdk = true in be.conf and restart the BE.
  • Online OPTIMIZE TABLE is now off by default. The FE configuration enable_online_optimize_table moves from true to false. To restore the previous behavior, set it back to true; it can be changed at runtime.
  • Delete-triggered base compaction of cloud-native Primary Key tablets is now opt-in. Previously a scheduled compaction switched to base compaction on its own once a tablet's deletes crossed lake_pk_compaction_base_delete_ratio_threshold (0.5) or lake_pk_compaction_base_delete_rows_threshold (10000000). On a large tablet whose existing keys keep being updated or deleted this repeated after every base compaction and rewrote a large share of the tablet's data. The new BE configuration lake_pk_compaction_base_enable_delete_trigger defaults to false, so both thresholds are now ignored. Set it to true to restore the previous behavior. A manual ALTER TABLE ... COMPACT still runs base compaction.
  • jemalloc settings changed. The default of the BE configuration jemalloc_conf changes oversize_threshold from 0 to 134217728, so allocations of 128 MiB or more go to a dedicated arena that is purged eagerly. To restore the previous behavior, set oversize_threshold:0 in jemalloc_conf. The new mutable BE configuration enable_jemalloc_decay_under_rss_pressure defaults to true and tightens jemalloc's decay as the resident size approaches mem_limit; set it to false to turn that off.
  • Vector index BE defaults changed. vector_index_cache_expire_sec moves from 900 to 1800, vector_index_cache_loading_wait_timeout_ms from 5000 to 60000, and config_vector_index_default_build_threshold from 10000 to 1. All three are mutable; set them back to the old values to restore the previous behavior.
  • PostgreSQL uuid columns in JDBC catalogs are now mapped to VARCHAR(36). The default of the catalog property postgresql.uuid.mapping is varchar; the value is the same 36-character lowercase text and is now typed as text. Set the property to varbinary to restore the previous mapping.
  • REFRESH ASYNC without EVERY is now rejected for materialized views. The statement fails with an error that points to REFRESH ON_CHANGE. Add an EVERY clause to keep time-based refresh, or use REFRESH ON_CHANGE for refresh triggered by base-table changes.
  • Some statements that previously ran are now rejected. DISTINCT aggregation over a PERCENTILE value, an ambiguous ADD COLUMN on an aggregate table, ALTER of a column on an incrementally maintained materialized view, and auto_refresh_partitions_limit on an incrementally maintained materialized view now return an error.
  • Results that were wrong now change. months_diff and years_diff returned -1 for two values in the same month or year; they now return the correct value, so reports that used them will change. A LARGEINT above the BIGINT maximum cast to JSON returned NULL; it is now a JSON unsigned 64-bit number. A JSON unsigned integer above INT64_MAX was read as a rounded DOUBLE; it is now an exact DECIMAL.
  • UDFs created with a cloud storage configuration before this release must be recreated. The function's storage configuration used to be saved without its settings, so it could not be restored after an FE restart. It is now saved with the function. A function created earlier, or whose saved configuration cannot be restored, loads normally but fails when called with an error that asks you to recreate the function. Drop and recreate the function to fix it.

Improvements​

  • Added the session variable enable_jdbc_runtime_filter_push_down, default true, which renders a join runtime filter into the query sent to a JDBC source. It is visible in SHOW VARIABLES.
  • Added the FE configuration files_require_explicit_credentials, default false. When true, FILES() accepts only storage credentials written in its own properties and rejects instance-profile, assumed-role, default-credential-chain, managed-identity and impersonation authentication, and hdfs:// paths.
  • Added SHOW CREATE USER and ALTER GROUP PROVIDER for updating a group provider's properties in place.
  • LDAP and Active Directory user and group names are now case-insensitive, and groups can be resolved from the user's memberOf attribute.
  • Added the ON_CHANGE refresh mode for materialized views, which refreshes when base tables change.
  • PostgreSQL JDBC catalogs now read table and column statistics, string arrays and more array element types, and push down constant array subscripts, collated string comparisons and safe TopN. An unconstrained numeric column maps to DECIMAL(38,18).
  • Vector and full-text search: array_contains and NOT array_contains are pushed down to the storage layer for ANN and BM25 scans, the BM25 top-k limit is pushed through scalar conjuncts, a shared-data vector index loads in parallel ranges, and a prepared vector search with equality filters reuses its plan.
  • Broker Load and INSERT ... SELECT from a table support tablet pre-split in more cases, including CSV, partition columns read from the path, column lists, and Parquet or ORC footers.
  • Shared-data vacuum can run in bounded incremental rounds, and the shared-data publish version timeout is configurable.
  • label_clean_interval_second can be changed at runtime, and the TaskRun dispatch interval is a mutable FE configuration (task_runs_dispatch_interval_ms, default 1000).
  • Query errors are recorded in the audit log, and a query is re-planned and retried when a lake metadata version is missing.
  • Added jemalloc dirty and muzzy page metrics, the jemalloc_page_size BE configuration, and lake compaction observability metrics.
  • Upgraded StarOS, starlet and starcache to v4.2-rc5 and tenann to v0.5.1-rc3. JDK 17 is again the recommended JDK for 26.2.
  • Updated tenann to v0.5.1-rc3.1, which fixes a build problem in v0.5.1-rc3.

Security​

  • Dependency upgrades for security: guava 33.7.2, snappy-java 1.1.10.9, lz4-java 1.11.4, BouncyCastle 1.86, jackson 2.21.7, zstd-jni 1.5.7-15, parquet 1.18.1, ZooKeeper 3.8.7 and 3.9.6, Apache HttpClient5 5.6.4, broker Tomcat 9.0.121, and a pprof update.
  • Reading a query profile now requires a privilege check.
  • LDAP security-integration users are now authenticated on non-MySQL channels.
  • TRUNCATE now checks the privilege against the catalog named in the statement.

Bug Fixes​

  • Fixed wrong results when an inverted index answered <> or NOT IN, which returned rows with NULL values, and when array_contains_all had a constant target and the row contained NULL elements.
  • Fixed list partition pruning for multi-value partitions that contain NULL.
  • Fixed a BE abort on a JSON integer above int64 max, and a BE crash when a slot was missing from the tablet schema.
  • Fixed a compaction crash (division by zero) when a vector-index rowset reported a negative data size, and a division by zero when a replication tablet had no replica.
  • Fixed an error when a vector index was queried before its asynchronous build finished; the query now falls back to brute force.
  • Fixed primary key partial updates that omitted or touched a vector-indexed column.
  • Fixed MATCH predicates being rewritten through a global dictionary, and a crash from a mistyped predicate in that rewrite.
  • Fixed materialized view rewrite for ANN and BM25 score() queries.
  • Fixed prepared statements that bound parameters by position in the AND tree instead of by column.
  • Fixed FULL OUTER JOIN ... USING columns that lost their own type or merged with the wrong side.
  • Fixed SELECT * EXCLUDE columns that were not backquoted when the statement was rewritten, and statistics collection for column names that contain dots.
  • Fixed the SHOW ALTER TABLE ROLLUP and MATERIALIZED VIEW output ignoring WHERE, ORDER BY and LIMIT, and FinishedTime predicates being rejected.
  • Fixed information_schema.tasks failing when a materialized view's warehouse was dropped, and task_runs reporting a migrating task run twice.
  • Fixed a runtime change of sys_log_level not taking effect, and an invalid sys_log_level that disabled BE logging.
  • Fixed stale reads while a journal was stuck, and Iceberg time travel being ignored when the SQL dialect is Trino.
  • Fixed many FE metadata-lock stalls caused by external-catalog I/O held under the lock.
  • Fixed several lake publish, schema change, reshard and persistent-index issues, including stuck alter jobs after failover, delete-vector checksum failures, and lake partition versions that could move backwards.
  • Fixed an FE that could fail to load its image or replay its edit log, or lose the storage configuration of a UDF, when the UDF was created with a cloud storage configuration. The configuration is now persisted, and a malformed one no longer blocks the FE from starting.

26.2.0​

Release Date: September 21, 2026

Upgrade and compatibility​

v26.2 supports an upgrade from v4.1. If an upgrade does not go as planned, you can roll back to the latest v4.1 patch release.

Read this section before upgrading. Six defaults change, one configuration option has been removed, and one table property is no longer accepted on cloud-native (shared-data) Primary Key tables. Each entry names the setting that controls the behavior and its default before and after. Everything here takes effect on upgrade without any action on your part.

Existing tables are not rewritten or redistributed by the upgrade.

  • Global late materialization is now on by default.

    Columns that are not used in predicates are read after predicate evaluation on native table scans, rather than alongside them. For a selective query this reads substantially less data; for a query that returns most rows it adds a second pass over the row positions. Query profiles change shape accordingly, so a saved profile from v4.1 will not compare directly. Controlled by the session variable enable_global_late_materialization, whose default moves from false to true, together with enable_global_late_materialization_cost_based, which lets the optimizer decide per query.

  • JDBC join pushdown is now on by default.

    A join between JDBC external tables is executed by the source database and only the result is returned, instead of PhoenixAI reading both tables and joining locally. This is usually much faster and transfers far less data. It also moves work onto the source system, which is worth checking if that system is capacity-constrained or shared with transactional traffic. The controlling session variable is enable_jdbc_join_push_down, new in this release and defaulting to true. It is not listed by SHOW VARIABLES, but it can be set per session or globally.

  • Avro data is now read by the native reader instead of the JNI reader.

    This affects Avro files in Hive and Iceberg tables. The JNI reader's CHAR handling was aligned with the native reader before the default changed, so results should be identical; performance characteristics differ, and a workload tuned around the JNI reader should be re-measured. The session variable avro_use_jni_reader moves from true to false; set it back to true to restore the previous reader.

  • Iceberg scan planning now runs locally by default.

    Planning happens on the coordinator rather than being selected per query. This makes planning time predictable, which matters most for tables with large manifest lists where automatic selection could occasionally choose the slower path. The session variable plan_mode moves from auto to local; set plan_mode = auto to restore the previous behavior.

  • Cloud-native Primary Key tables now use the cloud-native persistent index only.

    The LOCAL persistent index is no longer supported for these tables. Existing table metadata is normalized to the cloud-native index automatically when the FE loads after the upgrade, so no migration step is required, but persistent_index_type = LOCAL set explicitly on a cloud-native (shared-data) Primary Key table will no longer be honored, and the setting is rejected on new tables.

  • Horizontal compaction no longer fills the local data cache.

    Horizontal compaction reads each input byte exactly once and replaces its input rowsets immediately afterwards, so caching that data mostly evicted query-hot data and added an inline local-disk write on every cache miss. Removing it brings horizontal compaction in line with the other full-scan background paths. The new BE configuration lake_enable_horizontal_compaction_fill_data_cache defaults to false; set it to true to restore the previous behavior. Vertical compaction, which scans its input once per column group, is unchanged.

  • Vector index caches are no longer populated when an index is built.

    Building an index no longer warms the cache, so a build does not displace cached data that queries are using. Query-time caching is unchanged, and the cache can load asynchronously on a miss rather than blocking the query. The new BE configuration enable_vector_index_cache_on_build defaults to false.

  • enable_experimental_vector has been removed.

    Vector indexing is no longer experimental and the configuration that gated it is gone. Remove it from fe.conf before upgrading — the FE will not recognize it.

  • DELETE on a non-Primary Key table now returns an informational notice.

    The statement behaves exactly as before; it now explains what it did rather than succeeding silently.

  • Segment index sidecar files are now included in external cluster snapshots.

    Files with the .vi and .idx extensions — which hold vector and other segment-level indexes — were not copied into an external cluster snapshot, so a cluster restored from one came back without them. If you use vector indexes and rely on external cluster snapshots for recovery, take a fresh snapshot after upgrading; snapshots taken before the upgrade are incomplete.

  • Stricter implicit-cast checking is available, and off by default.

    The new sql_mode value FORBID_INVALID_IMPLICIT_CAST rejects implicit casts that previously succeeded and produced a surprising value. Enabling it can turn queries that ran in v4.1 into errors, which is the point — consider it for new workloads before existing ones.

New features​

Search and AI​

  • Vector search on shared-data clusters

    Vector indexes were previously available only on tables in shared-nothing clusters, which meant an AI workload could not sit on the same storage architecture as the rest of the warehouse. In v26.2, vector indexes can be built, written and read on cloud-native (shared-data) tables, and an index can be added to an existing table with ALTER TABLE rather than requiring the table to be recreated. Index files are named by their owning tablet and tracked in shared-data metadata, so storage is reclaimed precisely when an index is dropped and indexes survive tablet split and merge on range-distributed tables. Both L2 distance and inner product are available as index backends, chosen when the index is created. ef_search, which controls how much of an HNSW graph a query explores, is now derived from the query rather than fixed globally — it scales with the requested k and the size of the data being searched, so a query asking for many results no longer needs a hand-tuned session variable, and a search that under-fills falls back rather than returning a short result set. A query vector can also be bound as a parameter with CAST(? AS ARRAY<FLOAT>), so an application can use a prepared statement instead of building SQL text with a few hundred floats inlined in it. Vector indexing is no longer gated behind an experimental flag — see Upgrade and compatibility for more information.

  • Vector indexes build asynchronously

    Building an HNSW index is CPU- and I/O-intensive, and doing it inline meant a large build could slow the ingestion that triggered it. Index construction now runs as a background task. Build concurrency, the maximum share of CPU a build may consume, and the row-count threshold below which no index is built are all configurable. Builds are scheduled with awareness of compaction so the two do not contend for the same I/O, and they run in the table's own warehouse rather than wherever the write landed — so index maintenance can be isolated from query capacity. information_schema.partitions_meta reports the version each partition's index was built at, which is how you tell whether an index has caught up with recent writes.

  • Index quantization, for control over memory versus recall

    SQ4, SQ8 and PQ quantization reduce the memory an index occupies, at some cost in recall. Because a quantized index stores approximate vectors, results can optionally be refined against the original vectors; that refinement is now an option you control and is keyed to the quantization in use, rather than implicit behavior that varied by configuration. The trade-off is therefore made explicitly at index creation instead of being discovered later.

  • Filtered vector search

    A nearest-neighbor search combined with a predicate previously had to search first and filter afterwards, which either returned too few rows or forced a much larger search to compensate. v26.2 supports pre-filter strategies, so the predicate constrains the search itself. This matters most for multi-tenant tables, where almost every vector query is scoped to one tenant.

Lakehouse​

  • Iceberg UPDATE

    v4.1 added DELETE on Iceberg tables. v26.2 completes row-level DML with UPDATE, reusing the same commit path, so an Iceberg table can be corrected in place instead of being rewritten by an external job. Updates are committed as an Iceberg row-delta, so readers of the table — including engines other than PhoenixAI — see a consistent snapshot throughout. Update activity is exposed as metrics.

  • Iceberg MERGE INTO

    MERGE INTO applies inserts, updates and deletes from a source query in a single statement, which is the natural shape of a change-data-capture load or a slowly-changing dimension. v26.2 implements the full semantics rather than a subset. If the source contains more than one row matching the same target row, the statement fails instead of applying an arbitrary one — the uniqueness check runs on the join distribution, so it costs no extra shuffle. This makes MERGE INTO safe to run against a source you do not fully control.

  • Iceberg v3 deletion vectors

    Deletion vectors are a compact representation of row-level deletes introduced in Iceberg v3. v26.2 both reads and writes them: DELETE produces Puffin-encoded deletion vectors, and deletes that already exist on the table are merged into the vector rather than left behind as a second mechanism the reader has to consult. Deletion vectors are served through the data cache, so repeated reads of a table with many deletes do not re-fetch them from object storage. Write activity is exposed as metrics.

  • Iceberg VARIANT shredding

    A VARIANT column holds semi-structured data whose shape is not fixed in the schema; shredding stores frequently accessed fields as separate typed columns in the Parquet file while keeping the rest as an opaque value. v26.2 reads both shredded and non-shredded VARIANT columns. Querying one field no longer means materializing the whole column: field paths are reconstructed from the typed columns, pruning is path-aware, predicates push down using the shredded columns' Parquet statistics, and shredded fields participate in low-cardinality dictionary filtering like ordinary columns. In practice this closes most of the gap between a VARIANT field and a real column for read performance.

  • Iceberg metadata maintenance runs automatically.

    An Iceberg table accumulates snapshots, manifests and orphaned files, and left alone this degrades planning time and wastes storage. Maintenance is now scheduled from catalog-level properties that individual tables inherit and can override, so it can be configured once per catalog. Statistics and metrics are collected per task, and a new information_schema.iceberg_maintenance_tasks table shows what ran, when, and what it did. Orphan-file scanning, the most expensive maintenance operation, is parallelized.

Real-time and shared-data engine​

  • Incremental materialized views on cloud-native tables

    Incremental maintenance updates only the rows that a base-table change affects, instead of recomputing the view — the difference between a refresh proportional to the change and one proportional to the table. v26.2 extends incremental maintenance to shared-data cloud-native tables, including Primary Key base tables, where it previously applied to lake tables. The supported shapes are aggregates, top-level inner and cross joins, top-level UNION ALL, derived-table subqueries, and single-table projection and filter views with retractable maintenance; rollups over BITMAP_UNION, HLL_UNION and PERCENTILE_UNION on aggregate base columns are also supported. Aggregates are recomputed from a single snapshot rather than two, which reduces the work each refresh does. A view now reports which distribution it actually uses, and range distribution is available for incremental views.

  • Range distribution is ready for production use.

    With hash distribution the bucket count is fixed when a table is created: guess low and a partition cannot use the available parallelism, guess high and small partitions carry needless metadata, and changing it means rebuilding the table. Range distribution splits and merges tablets as data grows, so a partition's tablet count is not decided in advance and has no fixed ceiling. v4.1 could create range-distributed tables but not maintain them, which made them impractical for a table you intend to keep. v26.2 adds the missing operations — on cloud-native range-distribution tables you can now:

    • change the key and sort key, add or drop a key column, and widen an integer sort-key column
    • create multiple rollups, including a rollup with its own independent sort key
    • change ORDER BY on an asynchronous materialized view online
    • migrate data
    • use range distribution for incremental materialized views

    Tablet pre-splitting, which sizes tablets from a sample instead of waiting for splits to happen, extends to range tables with a rollup index and to INSERT ... FROM into a range table. Vector indexes and lake index deltas both follow tablet split and merge. Range distribution stays opt-in: it is chosen per table with DISTRIBUTED BY, and upgrading does not change how any existing table is distributed.

  • Rejected rows are queryable

    When a bulk load rejects rows, finding out which rows and why previously meant retrieving the load's error file. v26.2 records rejected rows in a system table maintained by a background daemon, so they can be queried with SQL — filtered by load, inspected alongside the reason for rejection, and joined against whatever else you need to identify the source. This turns a load failure into something an analyst can diagnose without cluster access.

  • VARCHAR up to 2 GiB

    The VARCHAR ceiling rises from roughly 1 MB to 2 GiB minus 1 KiB. This matters for JSON documents, serialized payloads, model inputs and long text that previously had to be split across columns or truncated at load time. Page-encoding overhead for large strings is reduced, so the wider limit does not cost proportionally more storage for values that stay small. FILES(), translate() and the type documentation are updated accordingly.

  • Light-weight tablet creation

    Creating a table normally requires every compute node that will hold a tablet to write that tablet's metadata before the statement returns, which dominates create time for a table with many tablets and makes bulk table creation slow. The new light_weight_tablet_creation table property skips that step; metadata is produced on demand, with a fallback on the compute node if it is missing. Schema changes and rollups skip the corresponding replica-creation work. The property is reversible — turning it off backfills the metadata that was not written.

  • Explicit read schema for FILES()

    A schema parameter sets the schema explicitly instead of relying on inference, which matters when the first file in a set is not representative or when a column's type must match an existing table exactly.

Query execution​

  • Finer-grained parallel scan within a tablet

    Scan parallelism was bounded by the number of tablets a query touched, so a table with a few large tablets could not use the cores available. That is an awkward constraint, because larger tablets are otherwise desirable — they mean less metadata and cheaper maintenance. v26.2 parallelizes scanning inside a single tablet, so scan concurrency is determined by the hardware rather than by a distribution decision made when the table was created. The practical effect is that tablet sizing becomes a storage decision instead of a query-performance decision, and a table does not need to be over-bucketed just to keep scans parallel.

  • Prepared physical split scan for lake tables

    A scan over a lake table is only as fast as its slowest split, so one oversized or skewed tablet sets the time for the whole scan. v26.2 prepares scan ranges before execution: pruning happens against seed metadata, the read state is built once and reused, and skewed tablets are divided so work is distributed by size rather than by tablet count. Splits are refined before the scan starts instead of being discovered during it. The path is deliberately disabled for duplicate scans of the same table in one query, where preparing twice would cost more than it saves.

  • Query queue in ETL mode

    Query queueing was designed around interactive queries, which are short and roughly comparable in cost. ETL statements are neither, and treating them the same way meant either admitting too many and exhausting memory, or admitting too few and leaving the cluster idle. v26.2 adds an ETL mode to the query queue and improves how the slots a statement will occupy are estimated, so admission reflects what a statement will actually hold rather than a uniform assumption. This makes it practical to run ETL and interactive workloads on one cluster under a single admission policy.

  • JDBC aggregate pushdown

    Aggregation over a JDBC external table is now executed by the source database, so a GROUP BY over a large remote table returns grouped rows instead of transferring every row to be aggregated locally. Together with the join pushdown default change described above, this makes a JDBC catalog usable for more than small dimension lookups. A schema_resolver catalog property controls how schemas are resolved on the source, which matters for databases whose schema and catalog concepts do not map cleanly. Native query passthrough sends a statement to the source unchanged when you want the source's own dialect and semantics rather than a translation.

Improvements​

This release includes several hundred further improvements. The themes most likely to be visible in your workload:

  • Low-cardinality dictionary optimization

    Low-cardinality dictionary optimization now covers ARRAY_AGG and its sort columns, array functions with constant arguments, structs, UNION ALL over constant strings, physical filters, CTE operators, and more string functions including ifnull, STARTSWITH and ENDSWITH. The global dictionary cache is bounded by memory rather than entry count, so a workload with many distinct string columns no longer evicts unpredictably.

  • Cardinality estimation

    Cardinality estimation improves for compound predicates, OR predicates, range and inequality comparisons, outer join keys and date and time functions, and Iceberg column statistics now feed the optimizer's min/max rule. Better estimates mostly show up as better join orders.

  • Compaction and vacuum

    Compaction and vacuum gain parallel compaction for non-overlapping output, full-chain compaction profiles, fair scheduling for automatic vacuum, and a faster path for large Primary Key deletes at publish time.

  • Bundled connector libraries

    The Delta Lake kernel moves from a 4.0 release candidate to 4.3.0, and Apache Arrow to 24.0.0.

  • Storage internals

    Adaptive string offsets, large binary column support through the aggregate and window paths, delta-offset encoding for string columns, and reusable segment iterators.