メインコンテンツまでスキップ
バージョン: 4.0 Stable

PhoenixAI version 4.0

v4.0 is a long-term support release. It was first released on October 17, 2025, and has been maintained continuously since; the most recent patch, 4.0.16, was released on September 30, 2026.

This page describes the 4.0 line as a whole rather than patch by patch. The Upgrade and compatibility section separates what changes when you move to 4.0 from what changed later within the line, because a cluster that has been on 4.0 since launch and a cluster arriving on 4.0.16 today face different lists. Everything else is organized by capability: a feature that arrived in 4.0.0 and was extended in a later patch appears once, in its finished form.

Upgrade and compatibility​

Before you upgrade to 4.0​

Downgrade is restricted. After upgrading to v4.0, do not downgrade directly to v3.5.0 or v3.5.1 — the metadata is incompatible and the FE will crash. Downgrade to v3.5.2 or later instead. Separately, before downgrading from v4.0.2 or later to v4.0.1, v4.0.0 or v3.5.2–v3.5.10, run SET GLOBAL enable_rewrite_simple_agg_to_meta_scan = false;, and set it back to true after upgrading again.

Data Cache configuration is consolidated. datacache_mem_size and datacache_disk_size now take effect; storage_page_cache_limit, block_cache_mem_size and block_cache_disk_size are deprecated. BE metadata caching and data caching are unified behind one adaptive sizing strategy rather than two independently tuned pools, so a cluster that was tuned by setting the deprecated items should be re-tuned against the two that remain.

Shared-data clusters on Azure Blob Storage or Data Lake Storage Gen2 lose their Data Cache on upgrade. The cache is reloaded automatically, but the first queries after the upgrade read from object storage. Plan the upgrade window accordingly.

Defaults that change on upgrade to 4.0​

These take effect without any action on your part.

  • Flat JSON is on by default. JSON columns are stored with their frequently accessed fields extracted into typed sub-columns. Queries that read individual JSON fields get substantially faster; the trade-off is extra work at write time and a different storage layout, so storage figures and write throughput for JSON-heavy tables will not compare directly with 3.x.

  • Aggregation pushdown for materialized view query rewrite is on by default. enable_materialized_view_agg_pushdown_rewrite moves from false to true. More queries are rewritten to use materialized views; plans for those queries change shape.

  • The default bucket size for automatic-bucket tables is 1 GB, down from 4 GB. New tables get more, smaller tablets, which improves parallelism and resource utilization. Existing tables are not redistributed.

  • Hive and Iceberg metadata caches are bounded by memory, with a 24-hour TTL. New catalog properties cap the memory each cache may use: remote_file_cache_memory_ratio for Hive, and iceberg_data_file_cache_memory_usage_ratio and iceberg_delete_file_cache_memory_usage_ratio for Iceberg, all defaulting to 0.1 (10%). A deployment that relied on an effectively unbounded metadata cache may see more cache misses; raise the ratio to compensate.

  • The Partial Update mode is chosen by the system. It is derived from the relevant session variable together with the number of columns in the INSERT statement, rather than being fixed by configuration alone.

  • split_part returns NULL when the delimiter does not match, rather than returning a value derived from a non-match.

  • CTAS and CREATE MATERIALIZED VIEW produce STRING rather than fixed-length CHAR. Deducing a fixed length from the source expression could get the length wrong, which showed up later as materialized view refresh failures. Column types in tables and views created this way will differ from 3.x.

  • SHOW DATA DISTRIBUTION reports per materialized index. It no longer merges the statistics of all materialized indexes that share a bucket sequence number. Output that was previously one row per bucket is now one row per index per bucket.

  • information_schema.fe_tablet_schedules is restructured. TABLET_STATUS is renamed to SCHEDULE_REASON, CLONE_SRC to SRC_BE_ID, and CLONE_DEST to DEST_BE_ID; CREATE_TIME, SCHEDULE_TIME and FINISH_TIME change from DOUBLE to DATETIME. Any monitoring query against this view needs updating.

  • Some column types in information_schema.materialized_views change to match the data they carry. The is_leader label is added to several FE metrics, which changes the label set a metrics scrape returns.

  • auto_partition_refresh_number now limits the number of partitions refreshed in both automatic and manual materialized view refreshes, where it previously applied to automatic refresh only.

Changes within the 4.0 line that can alter query results​

Each of these corrects behavior that was wrong. The previous results were incorrect, so a query whose output changes was returning the wrong answer before — but the change is still visible, and a report or downstream job built on the old output needs checking.

  • Parquet INT64 timestamps written as timezone-naive are no longer shifted (4.0.12). When isAdjustedToUTC=false, SELECT FROM FILES() and broker or stream loads previously shifted values by the session timezone offset, so the data drifted whenever the session timezone was not UTC. Such timestamps are now read as wall-clock values, consistent with Trino, Spark and Impala.

  • LIKE escaping on constant operands follows MySQL 8 (4.0.13). SELECT 'a\\b' LIKE 'a\\\\b' now returns 1 and SELECT 'a\\b' LIKE 'a\\b' returns 0. A query written against the previous non-MySQL escaping semantics will return different rows.

  • A division expression with a non-constant divisor is no longer treated as monotonic (4.0.14). Zone Map pruning could drop rows that satisfied a predicate such as 10 DIV c. Such queries may now return more rows. Monotonic forms such as c DIV 10 still benefit from the pruning.

  • NOT MATCH on a GIN inverted index no longer returns NULL rows (4.0.14), which brings it in line with SQL three-valued logic. Affected queries return fewer rows.

  • List partition pruning no longer discards a partition because its value list contains NULL (4.0.16). !=, NOT IN and IS NOT NULL predicates — including the IS NOT NULL a planner derives on a join key — previously dropped an entire multi-value partition if NULL appeared among its values. Those non-null rows are returned again.

  • A Parquet null_count that under-reports NULLs is no longer trusted (4.0.16). Files written by older parquet-mr versions could return corrupted values or silently drop rows during decoding and pruning.

  • Queries against insert-only ACID Hive tables are rejected rather than answered (4.0.10). INSERT OVERWRITE on such tables was not recognized, so a query could silently return more rows than were actually visible. Affected tables now return an explicit error.

  • VARBINARY values nested inside ARRAY, MAP or STRUCT are binary-encoded in MySQL result sets (4.0.9). Raw bytes were previously emitted directly, which broke text-protocol parsing on null bytes and non-printable characters. Clients and tools that parse VARBINARY inside nested types may need adjusting.

  • VARCHAR(N) length is preserved through CTAS (4.0.12) when the source carries an explicit user length — a catalog column reference, CAST AS VARCHAR(N), or a string literal — instead of being widened to VARCHAR(1048576). This keeps the length constraint enforceable and aligns generated DDL with dbt schema contracts. Materialized view materialization still widens columns as before.

Changes within the 4.0 line that can cause a statement to fail​

  • Complex scalar expressions can now be rejected at analysis (4.0.3). max_scalar_operator_flat_children (default 10000) previously limited only expressions built by CASE WHEN; it now limits every scalar expression the optimizer rewrites, and the node count is recalculated after each rewrite rather than cached. A query that ran before can fail with Expression too complex. Current nodes: N, Limit: M., typically when inlining a view, a CTE or a subquery expands the expression tree. Simplify the expression, or raise the limit, which is dynamically modifiable.

  • ARRAY and MAP constructors fail rather than corrupt (4.0.14). A constructor whose flattened result exceeds the addressable byte limit for one chunk now raises CapacityLimitExceed instead of silently returning corrupted values that a CTAS, INSERT or materialized view refresh could persist. Reduce chunk_size or split the expression.

  • sql_mode is enforced where it previously was not (4.0.8). With DIVISION_BY_ZERO or FAIL_PARSE_DATE set, division by zero and str_to_date/str2date parse failures return an error instead of being ignored. With FORBID_INVALID_DATE set, invalid dates in INSERT ... VALUES are rejected instead of being bypassed.

  • A static partition clause in an INSERT into an Iceberg or Hive table is validated even when a target column list is given (4.0.14). A clause naming a non-partition column is rejected with a clear error instead of being silently ignored. Every INSERT shape accepted before is still accepted.

  • Routine Load pauses on non-retryable errors (4.0.9) — for example a row that exceeds the Primary Key size limit — instead of retrying indefinitely. A job that previously appeared to be running while making no progress now stops and reports.

  • pipeline_enable_large_column_checker is enabled by default (4.0.11), so a query producing an oversized column is caught rather than proceeding.

  • Creating a materialized view on an Iceberg view is disallowed (4.0.7).

Other operational changes within the line​

  • The AWS SDK's own HTTP client is used again (4.0.16). enable_poco_client_for_aws_sdk defaults to false; set it to true to restore the previous client.
  • jemalloc's dedicated oversize arena is enabled by default (4.0.16), so large allocations no longer land in the calling thread's per-CPU arena. This improves reuse of large buffers and reduces fragmentation, and changes how BE and CN memory is distributed across arenas.
  • HDFS files are no longer hsynced before close (4.0.16). The hsync worked around a crash that a separate change had already fixed, and forced an fsync() on every DataNode replica on each close.
  • Iceberg REST catalogs with vended credentials use the table metadata cache again (4.0.13). An earlier cache bypass sent every getTable() to the REST catalog and caused AWS Lake Formation Rate exceeded failures. Cached tables renew their credentials on each refresh cycle, and the table cache expiry for REST catalogs is capped at 3000 seconds.
  • A table with many stale partitions no longer reports a statistics health pinned near 1.0 (4.0.16), so auto-ANALYZE triggers when it should. Tables previously considered healthy may be re-analyzed after the upgrade.
  • paimon.option.cache-enabled = false disables the Paimon metadata cache again (4.0.16), and the background metadata refresh for a Paimon catalog visits only recently accessed, changed tables instead of enumerating the whole warehouse each cycle.

New features​

Enterprise-only capabilities​

These are available in PhoenixAI and not in open-source StarRocks. They are the part of 4.0 that has not previously been covered by a release note.

  • Manual cluster snapshots

    Before 4.0 a cluster could keep one automated snapshot, taken on the system's schedule. That is enough to recover from a failure, but not enough to protect a planned change: if you were about to run a migration or a bulk delete, there was no way to pin a known-good point you could return to, and the next automated snapshot would overwrite the one you wanted to keep. From 4.0 you can create and drop cluster snapshots manually, and keep several at once alongside the automated one. The number of manual snapshots a cluster may hold is capped.

    The automated side gained its own controls in the same line. The automated snapshot schedule takes an interval rather than running on a fixed cadence, ADMIN SHOW AUTOMATED CLUSTER SNAPSHOT reports the current status and schedule so you can confirm that protection is actually running, and each snapshot now carries a snapshot_meta.json marker that is validated before a restore begins — so an incomplete snapshot is detected up front rather than part-way through a recovery.

  • Compute Node Groups

    From 4.0.0 a warehouse can be divided into Compute Node Groups. The group sits between the warehouse and its Compute Nodes, adding a management layer that isolates compute resources within a warehouse more finely than the warehouse boundary allows, which is what improves the warehouse's availability. A dedicated metric set reports on CN Groups for monitoring and capacity work.

  • Databricks Unity Catalog: Service Principal authentication and vended credentials

    A Databricks Unity Catalog could previously be reached only with a Personal Access Token — a credential tied to a person, which is awkward to rotate and wrong for a service. From 4.0 a Service Principal (OAuth machine-to-machine) can be used instead.

    Vended credentials are also supported from 4.0, and are enabled by default: the catalog supplies the credentials for the underlying object store, so PhoenixAI does not need standing storage credentials of its own. For AWS S3 you specify only aws.s3.region; for Azure Blob Storage, Azure Data Lake Storage Gen2 and Google Cloud Storage you specify no storage credentials at all.

  • Warehouse operations

    Several operational gaps in multi-warehouse clusters are closed across the line: CURRENT_WAREHOUSE() reports the warehouse a session is actually using; db_admin can set warehouse properties without a cluster-level role; a Compute Node or Backend can be dropped without naming its warehouse; Backend resource statistics and pipeline parallelism (DOP) are computed per warehouse rather than cluster-wide, so one warehouse's load no longer skews another's parallelism; materialized view metrics carry a warehouse_name label; information_schema queries no longer keep a warehouse awake during the idle check; and a suspended warehouse no longer fills the log with Backend heartbeat failures.

    On licensing, the scale-out grace period starts when capacity actually grows rather than when the change is requested, and the FE metrics endpoint reports the number of days remaining on the license.

Lakehouse​

  • Iceberg table maintenance and layout control

    Compaction is available through the rewrite_data_files procedure, so a table whose writers produce many small files can be consolidated from SQL rather than by an external job. Hidden partitions are supported end to end — creating, writing and reading such tables — and sort keys can be set when a table is created. Partition specs can be altered in place with ALTER TABLE ADD/DROP PARTITION COLUMN, partition columns may be declared at any position in the column list rather than only at the end, and partition transforms accept parentheses, as in PARTITION BY (bucket(k1, 3)).

  • Iceberg write performance

    The Iceberg sink gained spilling for large operators, global shuffle and local sorting, which together address the small-file problem that a wide parallel write otherwise creates. Global shuffle can be enabled dynamically based on the data distribution rather than configured up front, and host-level sorting is available through connector_sink_sort_scope to organize the data layout for faster subsequent reads. Bucket-aware execution improves concurrency on bucketed tables.

  • Iceberg read and planning performance

    COUNT, MIN and MAX queries are answered from Iceberg metadata without scanning data files, which is a large difference on wide partitioned tables. Statistics metadata files are parsed once rather than repeatedly, foreground row-count estimation is computed from manifest metadata instead of enumerating every data file, and reads of tables carrying position deletes are faster. Statistics for Iceberg and Delta Lake tables are collected from metadata by default, including for tables using bucket and truncate partition transforms.

  • Unified page and data cache for BE metadata

    BE metadata caching and data caching are served from one pool with an adaptive sizing strategy, rather than two pools sized independently against the same memory budget. See Upgrade and compatibility for the configuration items this replaces.

  • Paimon

    The TIME data type is supported, as are ARRAY, MAP and STRUCT columns — querying a complex-typed Paimon column could previously crash the BE. SHOW CREATE TABLE and DESC show Primary Key columns for Paimon external tables, connector_max_split_size is honored when calculating scan splits so scan parallelism can be tuned, and the bundled Paimon version moves to 1.3.1.

  • Iceberg 1.10.0 is bundled, and Iceberg views can be created with properties, which are shown in SHOW CREATE VIEW output.

Storage, ingestion and cluster management​

  • File Bundling for shared-data clusters

    A shared-data cluster writes many small files — one set per load, compaction and publish — and each one costs an API call against the object store. At high write frequency those calls, not the bytes, become the dominant cost. File Bundling combines the files produced by these operations before they are written. It is enabled by default for tables created in 4.0 or later; existing tables are unaffected. Later patches in the line reduced remote storage access during vacuum once bundling is on, cached the latest bundled tablet metadata, and moved bundle writes onto a tablet-local aggregator so the path no longer needs cross-node shard lookups.

  • Multi-table write-write transactions

    INSERT, UPDATE and DELETE across several tables can be committed atomically, through both Stream Load and INSERT INTO. This is what an ETL job or a CDC pipeline needs in order to keep a set of related tables consistent with each other: previously each table committed independently, so a reader could observe one table updated and another not. Multi-statement Stream Load dispatches all per-table channels before waiting on them, which shortens commit time for pipelines writing many tables under one label.

  • Checksum protection for shared-data metadata

    Tablet metadata and transaction logs in shared-data clusters carry checksums, so corruption is detected when the metadata is read rather than surfacing later as an unexplained query failure.

  • Resource group memory pools

    The mem_pool resource group attribute lets several resource groups share one memory pool and be governed by a joint limit, instead of each group holding a reservation that the others cannot use. The change is backward compatible: a group without mem_pool uses default_mem_pool.

  • Routine Load supports Kafka 4.0.

  • Full-text inverted indexes on Primary Key tables in shared-nothing clusters.

  • Aggregate keys of an Aggregate table can be modified without recreating the table.

  • Case-insensitive handling of catalog, database, table, view and materialized view names can be enabled.

  • Compute Nodes can be blacklisted in shared-data clusters, as Backends already could.

  • Global connection ID, so a connection is identifiable across the cluster rather than per FE.

  • The recyclebin_catalogs view in Information Schema shows deleted metadata that is still recoverable.

Query execution and performance​

  • DECIMAL256

    DECIMAL precision rises from 38 to 76 digits. DECIMAL128 overflows in large aggregations and higher-order arithmetic — a sum over a long financial series, or a product of several high-precision terms — and the usual workarounds are to lose precision deliberately or to move the calculation out of the database. A 256-bit decimal removes that ceiling for financial and scientific workloads.

  • SQL Plan Manager (Preview)

    A query plan can be bound to a query, so the plan does not change when the data or the statistics behind it do. The purpose is predictability rather than speed: a query whose plan is stable will not regress overnight because an ANALYZE landed or a partition grew. It is a preview feature in 4.0.

  • Flat JSON V2

    Flat JSON gains page- and segment-level indexes (ZoneMaps and Bloom filters), predicate pushdown with late materialization, dictionary encoding, and integration with the low-cardinality global dictionary, while retaining the V1 mechanism. It can be configured per table rather than only globally. Together these bring a JSON field much closer to a real column for query purposes. Flat JSON is on by default in 4.0 — see Upgrade and compatibility.

  • Partition-wise spillable aggregation

    Spillable aggregate and distinct operators are partition-wise, replacing an implementation based on sorted aggregation. For a high-cardinality GROUP BY that does not fit in memory, this substantially reduces the read and write overhead of spilling.

  • Fast Schema Evolution v2

    Schema changes that only alter metadata complete without rewriting data. In 4.0 the FE serves schemas to Compute Nodes on demand through a TableSchemaService, so a shared-data cluster does not have to propagate every schema everywhere; historical schemas are retained until all ingestion jobs depending on them finish, which closes a window in which an in-flight load could reference a schema that had already been discarded. Materialized views support Fast Schema Evolution from 4.0.16.

  • Query scheduling

    Operators and drivers are prepared in parallel, and a single-node query deploys its fragments in one batch. Java UDAF instances are loaded once per query and reused across pipeline drivers, removing an overhead that grew linearly with pipeline_dop.

  • Optimizer

    Join and aggregation operators are faster, as are the basic operators beneath them. Grouping-sets push-down extends to AVG and COUNT alongside MAX, MIN and SUM, which speeds up ROLLUP, CUBE and GROUPING SETS queries with more than three levels. COUNT, SUM and AVG over DISTINCT are supported within framed windows. Partial TopN is pushed into the pre-aggregation phase to cut network traffic. Materialized view rewrite takes the base table's distribution and sort keys into account when choosing among candidate views.

  • Adaptive ZoneMap index creation for STRING columns, so a ZoneMap is built where it will help rather than uniformly.

  • Table-level query timeout through table_query_timeout, resolved in the order session, then table, then cluster.

Security and authentication​

  • JWT identity passes through to Iceberg

    When JWT authentication is used with an Iceberg REST Catalog, the user's login information is forwarded to Iceberg through the REST Session Catalog, so data access is authorized as that user at the Iceberg layer rather than under a single shared service identity.

  • Vended credentials for Iceberg catalogs, so storage access uses short-lived scoped credentials issued per table instead of standing credentials held by the cluster.

  • Internal roles can be granted to external groups obtained through a Group Provider, which lets an existing directory group map onto a PhoenixAI role rather than requiring membership to be maintained twice.

  • A REFRESH privilege on external tables, separating the right to refresh a table's metadata from the right to read or alter it.

  • LDAP direct bind through a DN pattern removes the need for an admin search account in single-tenant LDAP deployments — one fewer privileged credential to hold and rotate.

  • OAuth2 and JWT authentication for Iceberg REST catalogs is extended.

  • Cloud storage credentials are redacted in INSERT INTO FILES errors, so a secret cannot reach an error log or SHOW LOAD output.

SQL and functions​

  • New functions: bitmap_hash64, bool_or, strpos, to_datetime and to_datetime_ntz, regexp_count, tokenize, format_bytes, encode_sort_key, column_size and column_compressed_size.
  • Syntax: EXCLUDE in SELECT; FILTER in aggregate functions, which makes a conditional aggregation both readable and cheaper than the CASE WHEN form; IF NOT EXISTS in CREATE ANALYZE FULL TABLE; ORDER BY on STRUCT values; FULL OUTER JOIN ... USING with SQL-standard semantics.
  • Java UDFs: STRUCT arguments and return types for UDAF and UDTF, STRUCT arguments for scalar UDFs, DATE and DATETIME types, and nested ARRAY/MAP types.
  • INSERT INTO FILES CSV export supports csv.enclose and csv.escape.
  • ngram_search accepts a non-constant needle, so the search term can be a column expression.
  • VARBINARY is supported by COUNT DISTINCT-style aggregates; map_agg is supported in the Trino dialect; EXPLAIN and EXPLAIN ANALYZE work for INSERT in the Trino dialect.

Observability​

  • EXPLAIN ANALYZE output is grouped by operator and by execution group, which makes a large plan readable rather than a flat wall of counters.
  • Query profiles can be retrieved across all FEs, and QueryDetailActionV2 and QueryProfileActionV2 return JSON — so a profile can be fetched without knowing which FE ran the query.
  • Audit logs carry the tables and views each query referenced, and the data cache hit ratio.
  • SHOW PROCESSLIST includes catalog and query ID; the query queue reports running and pending states.
  • Data Cache monitoring gains memory and disk quota and usage metrics, page cache statistics on the api/datacache/stat endpoint, and hit-rate statistics for native tables.
  • FE memory can be inspected at /api/memory_usage, and FE memory estimation counts the internal overhead of maps and collections rather than only sampled contents, so weight-bounded caches report their true footprint.
  • Server-level spill metrics aggregate spill activity across all spillable operators, broken down by local and remote backend.
  • max_journal_replay_lag on the leader FE reports how far the furthest-behind follower or observer trails in replayed journal IDs — a direct answer to "is any FE falling behind".
  • jemalloc internals are observable and partly tunable at runtime: dirty and muzzy page metrics, an extent-metadata and rtree breakdown, stats.mapped and stats.retained in be.INFO, per-arena and size-class detail through /memz, the startup options visible through jemalloc_conf, and mutable options such as dirty_decay_ms and muzzy_decay_ms changeable without a restart.
  • Query plans are dumped when a query hits an exception or times out, and the pipeline state is dumped on timeout.
  • Iceberg metadata-table queries and time-travel queries have their own metric categories; starrocks_fe_tablet_num is reported for shared-data clusters to match the shared-nothing metric set; new metrics cover catalog recycle bin size, vacuum batch size and retries, FE maximum pending-publish time, and StarOS shard-info cache fallbacks.

Improvements​

Sixteen patch releases carry several hundred further improvements. The themes most likely to be visible in your workload:

  • FE lock contention

    Database-level locks are replaced by table-scoped locks across LocalMetastore, InformationSchemaDataSource and the frontend service; locks held during INSERT OVERWRITE are relaxed or shortened per phase; tablet deletion during DDL is batched so the tablet-metadata write lock is taken once per batch; the routine-load broker RPC is moved out of the per-job write lock; shared-data schema-change job locks are scoped to the table; and external-table metadata refresh triggered by INSERT ... SELECT no longer runs while FE metadata locks are held. The effect is most visible on clusters with many concurrent DDL and metadata operations.

  • Optimizer cost and statistics

    DP join reorder prunes expensive candidate plans earlier, enumerates partitions with fewer allocations, and caps the atom count; statistics estimation avoids repeatedly rebuilding Statistics objects and skips predicate column collection; per-bucket distinct counts for histograms are estimated with HyperLogLog or sampling, giving better NDV for predicates and joins; histogram MCV propagates through semantics-safe expressions such as cast(k as bigint) + 10, improving skew detection; and skew handling gains histogram- and NULL-based strategies. Better estimates mostly show up as better join orders.

  • Shared-data metadata and vacuum

    Tablet location lookups are batched during physical planning instead of issuing one StarOS RPC per tablet in the planner's critical section; lake segment metadata cache lookups no longer pass through a process-wide lock; load spill files move to a flat directory layout keyed by transaction ID and are reclaimed by a txn-id-based vacuum pass, which takes bulk deletes off the write path and lets vacuum clean up after a BE crash; vacuum retries get decorrelated jitter to avoid retry storms; and compaction and replication drop the never-read delete_predicate when archiving a rowset, which reclaims tablet-metadata bloat at its source.

  • Memory

    FE memory estimation counts collection overhead rather than sampled contents only, so caches bounded by a memory ratio evict at their true footprint. The object storage client cache size becomes a mutable BE configuration item rather than a hard-coded capacity. An expired external scan context left behind by a dead Spark or Flink connector reader now cancels its pipeline fragment when reaped, releasing the buffered scan memory within the keep-alive window instead of holding it until the query timeout. On arm64, a 4K-page jemalloc variant is built alongside the 64K one and selected at runtime, avoiding the memory overhead of 64K page granularity on 4K-page kernels.

  • Compression and storage layout

    Under "compression" = "zstd", flat JSON sub-columns and the synthetic null and offset sub-columns of ARRAY, MAP and STRUCT columns are compressed; they were previously written as raw pages, which could make a ZSTD table larger on disk than the same table under LZ4. Only segments written after the upgrade are affected, so existing tables shrink gradually as their segments are rewritten.

  • Security dependency upgrades

    Across the line: Thrift to 0.24.0, Netty to 4.1.136.Final, the PostgreSQL JDBC driver to 42.7.12, Tomcat to 9.0.118, jackson-databind to 2.21.4, hadoop-client to 3.4.3 and lz4 to 1.10.0; and removal of vulnerable transitive dependencies — bcprov-jdk15on, the end-of-life OkHttp 2.x line, avro-ipc (which bundles jQuery 1.4.2), the Jetty client and security jars, and org.jline:jline from the Hadoop dependency tree.