Skip to main content
Version: 26.2 Preview

BE Configuration - Shared-data, Data Lake, and Others

Some BE configuration items are dynamic parameters which you can set interactively when BE nodes are online. The rest of them are static parameters. You can only set the static parameters of a BE node by changing them in the PhoenixAI Cloud console.

  1. Sign in to the PhoenixAI Cloud console.
  2. On the Clusters page, click the classic cluster that you want to configure.
  3. On the cluster details page, click the Cluster parameters tab.
  4. In the BE Node static configuration section, click View all parameter list.
  5. In the dialog box that appears, click New parameter.
  6. Enter the name of the static parameter you want to configure in the Parameter key field, and the parameter value you want to set in the Value field. For the details of the parameters you can configure, see Usage notes.
  7. Click the save button next to the Value field to save the record of the parameter change.
  8. After configuring all the parameters you want to change, click Save changes to save the changes.
  9. To allow the changes to take effect, you can either manually suspend and resume the cluster whenever you find suitable, or click Apply to all nodes in the BE Node static configuration section to restart the cluster instantly.

Follow these steps to configure Compute Node parameters of a warehouse in the PhoenixAI Cloud console:

  1. Sign in to the PhoenixAI Cloud console.
  2. On the Clusters page, click the elastic cluster that you want to configure.
  3. On the cluster details page, click the Warehouses tab.
  4. On the Warehouses tab, click the warehouse that you want to configure.
  5. On the warehouse details page, click the Warehouse parameters tab.
  6. In the Compute Node static configuration section, click View all parameter list.
  7. In the dialog box that appears, click New parameter.
  8. Enter the name of the parameter you want to configure in the Parameter key field, and the parameter value you want to set in the Value field.
    note

    PhoenixAI provides validation checks on parameter keys and value types. Only valid parameter keys and values can be applied. For the details of the parameters you can configure, see Usage notes.

  9. Click the save button next to the Value field to save the record of the parameter change.
  10. After configuring all the parameters you want to change, click Save changes to save the changes.
  11. To allow the changes to take effect, you can either manually suspend and resume the warehouse whenever you find it suitable, or click Apply to all nodes in the Compute Node static configuration section to restart the warehouse instantly. For default_warehouse, you can only restart the cluster.

View BE configuration items​

You can view the BE configuration items using the following command:

SELECT * FROM information_schema.be_configs [WHERE NAME LIKE "%<name_pattern>%"]

Configure BE parameters​

Configure BE dynamic parameters​

You can configure a dynamic parameter of a BE node by using the curl command.

curl -XPOST http://be_host:http_port/api/update_config?<configuration_item>=<value>

Configure BE static parameters​

You can only set the static parameters of a BE by changing them in the PhoenixAI Cloud console, and restarting the BE to allow the changes to take effect.


This topic introduces the following types of BE configurations:

Shared-data​

cloud_native_pk_index_rebuild_files_threshold​

  • Default: 50
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The maximum number of segment files that need to be rebuilt in cloud-native Primary Key index. If the number of files that need to be rebuilt during index recovery exceeds this threshold, StarRocks will flush the in-memory MemTable immediately to reduce the number of segments that must be replayed. Set to 0 to disable this early-flush strategy.
  • Introduced in: -

cloud_native_pk_index_rebuild_rows_threshold​

  • Default: 10000000
  • Type: Long
  • Unit: Rows
  • Is mutable: Yes
  • Description: The maximum number of rows that need to be rebuilt in cloud-native Primary Key index. If the number of rows that need to be rebuilt during index recovery exceeds this threshold, StarRocks will flush the in-memory MemTable immediately to reduce the rebuild overhead. Set to 0 to disable this early-flush strategy. Works in conjunction with cloud_native_pk_index_rebuild_files_threshold; a flush is triggered if either threshold is exceeded. The row count includes segment rows plus the tombstone (delete) rows recorded in del files, so a delete-heavy workload that produces a few large del files also counts toward this threshold; del files written by older versions that did not record a row count contribute 0.
  • Introduced in: -

cloud_native_tablet_metadata_ancestors_recorded​

  • Default: 5
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: Number of historical tablet-metadata ancestor versions retained per tablet; sets the trackable horizon for CHANGES queries. A larger value lets CHANGES reach further-back base versions at the cost of metadata storage.
  • Introduced in: -

download_buffer_size​

  • Default: 4194304
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: Size (in bytes) of the in-memory copy buffer used when downloading snapshot files. SnapshotLoader::download passes this value to fs::copy as the per-transfer chunk size when reading from the remote sequential file into the local writable file. Larger values can improve throughput on high-bandwidth links by reducing syscall/IO overhead; smaller values reduce peak memory use per active transfer. Note: this parameter controls buffer size per stream, not the number of download threads—total memory consumption = download_buffer_size * number_of_concurrent_downloads.
  • Introduced in: v3.2.13

graceful_exit_wait_for_frontend_heartbeat​

  • Default: false
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Determines whether to await at least one frontend heartbeat response indicating SHUTDOWN status before completing graceful exit. When enabled, the graceful shutdown process remains active until a SHUTDOWN confirmation is responded via heartbeat RPC, ensuring the frontend has sufficient time to detect the termination state between two regular heartbeat intervals.
  • Introduced in: v3.4.5

lake_compaction_stream_buffer_size_bytes​

  • Default: 1048576
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The reader's remote I/O buffer size for cloud-native table compaction in a shared-data cluster. The default value is 1MB. You can increase this value to accelerate compaction process.
  • Introduced in: v3.2.3

lake_dump_tablet_metadata_per_request_memory_limit_bytes​

  • Default: 268435456
  • Type: Long
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The tracked memory budget for synchronous allocations made while processing one /api/cloudnative/dump_tablet_metadata request on a CN. It uses the standard MemTracker accounting granularity rather than enforcing a byte-exact ceiling. It covers metadata inspection, sensitive-field redaction, and JSON serialization, but not the metadata already stored in the CN metadata cache or the asynchronous HTTP output buffer. Each request uses the value captured when it is admitted. If this value is less than or equal to 0, new requests fail closed.
  • Introduced in: v26.2

lake_dump_tablet_metadata_per_request_json_size_limit_bytes​

  • Default: 33554432
  • Type: Long
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The maximum size of the complete JSON response for one /api/cloudnative/dump_tablet_metadata request. The size is checked before the response is handed to the HTTP output buffer. Each request uses the value captured when it is admitted. If this value is less than or equal to 0, new requests fail closed.
  • Introduced in: v26.2

lake_dump_tablet_metadata_max_concurrency​

  • Default: 1
  • Type: Int
  • Unit: Requests
  • Is mutable: Yes
  • Description: The maximum number of /api/cloudnative/dump_tablet_metadata requests admitted concurrently on one CN. A request continues to occupy one slot until its HTTP request is released. Increasing the value applies immediately to new requests. Decreasing it does not cancel admitted requests, and new requests are rejected until the active count is below the new limit. If this value is less than or equal to 0, new requests fail closed.
  • Introduced in: v26.2

lake_enable_del_file_crc_check​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to verify a Primary Key delete file (.del) of a shared-data cluster against the CRC32C recorded in its metadata when the file is read back during transaction publish or Primary Key index rebuild. When a mismatch is detected, the operation fails with a corruption error instead of erasing the wrong primary keys. Delete files written before the checksum existed carry none and are always accepted, so this item has no effect on data written by an earlier version. Writing the checksum is not controlled by this item; this item only controls verification.
  • Introduced in: v4.2

lake_enable_pk_preserve_txn_delete_order​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to preserve the in-transaction upsert/delete order for Primary Key tables in a shared-data cluster. When a single load transaction contains both a DELETE and a later re-UPSERT of the same key, enabling this makes the re-upsert win (consistent with shared-nothing clusters). It is enabled by default. For downgrade safety, set it to false before rolling back to (or running a mixed cluster with) a BE version without this fix: when enabled, a load can persist on-disk metadata that a pre-fix BE would misinterpret, potentially producing duplicate primary keys. When disabled, deletes fall back to the legacy behavior (applied after all upserts in the transaction): if a load deletes a key and then upserts it again, and the delete and the upsert are flushed separately (for example, in a large load), the key ends up deleted and the upsert is lost. A Primary Key table whose sort key (ORDER BY) differs from its primary key always preserves the order, whatever this item is set to.
  • Introduced in: v4.1.4

lake_enable_protobuf_file_checksum​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to write tablet metadata and transaction log files of a shared-data cluster with an Adler-32 checksum, so that corruption of these files can be detected when they are read. Regardless of this item, readers always detect and verify the checksum automatically when it is present; this item only controls the write format. Set it to false only while the cluster may still be downgraded to a version that predates the checksummed format. During a rolling upgrade or a downgrade, an earlier BE or CN uses the legacy reader and cannot parse files written in the new format.
  • Introduced in: v4.2

lake_enable_publish_partial_meta​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether a publish of a shared-data table with file bundling keeps the result of each tablet that the aggregator may not have received. A tablet whose publish finishes more than the publish RPC timeout plus 3 seconds after the request arrived, or whose publish is a retry, keeps its result in the local Data Cache of the node, never in object storage. A later retry on that node reads it back instead of applying the transactions again, so tablets that already finished are not redone on every retry of a slow partition. The entries are evicted like any other Data Cache entry. Takes effect only when Data Cache is enabled. Set the same value on all BE and CN nodes.
  • Introduced in: v4.2

lake_enable_txn_vlog_group​

  • Default: true

  • Type: Boolean

  • Unit: -

  • Is mutable: Yes

  • Description: Whether schema-change publish in a shared-data cluster writes an ordered statement group into a separate {tablet}_{version}.group.vlog file for a multi-statement transaction. Legacy single logs use {tablet}_{version}.vlog. Readers in this version always support both legacy single-log files and statement groups, regardless of this setting. With the default true, each statement keeps its own schema, segments, rewrite metadata, and delete positions, so a DELETE followed by an INSERT in one transaction no longer causes mismatched partial-update metadata during shadow publish. Setting it to false restores the legacy merged-log writer, including that known failure. Changing the setting does not convert versioned transaction logs that have already been written.

    CNs of versions without group readers cannot read statement groups. When you upgrade from an earlier version in which this setting does not exist or defaults to false, set lake_enable_txn_vlog_group = false in cn.conf before you start each upgraded CN, and keep it false until every CN that may perform publish, replay, or cleanup has been upgraded, including background workers and images used for replacement nodes. Then enable it as described below. If the earlier version cannot read statement groups, this step is required: enabling group writes while older CNs remain can silently lose committed statements. After a group writer deletes some source logs, an older CN can retry using only the remaining sources and write an incomplete legacy log, which replay prefers over the group file.

    Before changing this setting in either direction, pause the relevant schema-change and multi-statement publish work, drain in-flight requests, update all affected CNs consistently, and then resume the work. The setting does not negotiate reader capabilities or provide an atomic cluster-wide switch.

    Setting it to false only stops subsequent group writes; it does not convert, drain, or remove existing grouped logs. Before downgrading to a version without group readers, list every affected storage root's txnlog directory and verify that no *.group.vlog files remain, including unused group files next to valid legacy logs: older tablet cleanup also scans these files and cannot decode them. ALTER reaching FINISHED, transactions becoming VISIBLE, or routine vacuum completing does not establish this. With file bundling, successful replay leaves these logs in place, and routine vacuum does not reclaim version logs. If any remain, keep group-capable readers until safe cleanup has been completed. Disabling this setting alone does not make a downgrade safe.

  • Introduced in: -

lake_pk_compaction_max_input_rowsets​

  • Default: 500
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The maximum number of input rowsets allowed in a Primary Key table compaction task in a shared-data cluster. The default value of this parameter is changed from 5 to 1000 since v3.2.4 and v3.1.10, and to 500 since v3.3.1 and v3.2.9. After the Sized-tiered Compaction policy is enabled for Primary Key tables (by setting enable_pk_size_tiered_compaction_strategy to true), StarRocks does not need to limit the number of rowsets for each compaction to reduce write amplification. Therefore, the default value of this parameter is increased.
  • Introduced in: v3.1.8, v3.2.3

lake_pk_compaction_max_rowsets_per_level​

  • Default: 20
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The number of rowsets that still need compaction at which a size level of a Primary Key table in a shared-data cluster is merged even if it does not have the highest score. The Size-tiered Compaction policy merges only the level with the highest score in each round. While loads keep arriving, the level of the newest and smallest rowsets has the highest score in every round, so a level of larger rowsets (outputs of earlier compactions and medium-sized loads) keeps growing. This increases the number of segments that queries read, and shows up as a spike in the compaction score right after a compaction finishes. When a level that is not picked holds this many rowsets that still need compaction, and these rowsets fit in the same compaction task as the picked ones, the level is merged in that compaction task. Set it to 0 to disable this behavior.
  • Introduced in: -

lake_pk_compaction_base_enable_delete_trigger​

  • Default: false
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether automatic compaction of a shared-data Primary Key tablet can select Base Compaction when its delete ratio or delete-row count reaches a threshold. This switch only controls automatic Base Compaction triggered by deletes and does not affect ordinary Cumulative Compaction. It is disabled by default because large tablets under continuous updates or deletes can frequently reach the absolute delete-row threshold and repeatedly rewrite data. Manual ALTER TABLE ... COMPACT requests bypass this switch and both delete thresholds; tablets without deleted rows fall back to ordinary compaction selection. Neither automatic nor manual requests guarantee a full-tablet merge in one execution.
  • Introduced in: v4.1.7, v26.2.1

lake_pk_compaction_base_delete_ratio_threshold​

  • Default: 0.5
  • Type: Double
  • Unit: -
  • Is mutable: Yes
  • Description: One of the two thresholds for automatically selecting Base Compaction by deletes for a shared-data Primary Key tablet. Base Compaction can be selected when the aggregate delete ratio across all current Rowsets, sum(num_dels) / sum(num_rows), reaches this value, or when the total delete-row count reaches lake_pk_compaction_base_delete_rows_threshold. In v4.1.4–v4.1.6, both thresholds are checked directly. From v4.1.7, lake_pk_compaction_base_enable_delete_trigger = true is also required. Manual ALTER TABLE ... COMPACT requests bypass both thresholds. Reaching a threshold does not guarantee immediate FE scheduling.
  • Introduced in: v4.1.4

lake_pk_compaction_base_delete_rows_threshold​

  • Default: 10000000
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The absolute delete-row threshold for automatically selecting Base Compaction by deletes for a shared-data Primary Key tablet. It is combined with lake_pk_compaction_base_delete_ratio_threshold using OR. The delete-row count is sum(num_dels) across all current Rowsets in the tablet. In v4.1.4–v4.1.6, this threshold is checked directly. From v4.1.7, lake_pk_compaction_base_enable_delete_trigger = true is also required. Manual requests bypass this threshold. This condition covers tablets with a low aggregate delete ratio but many delete marks in delete vectors. Raise it to make automatic Base Compaction less frequent, or lower it to select delete reclamation sooner.
  • Introduced in: v4.1.4

enable_lake_pk_compaction_score_gate​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Master switch for the Primary Key size-tiered compaction score gate in a shared-data cluster. When enabled (the default), low-value sparse mid-tier compaction picks are skipped: a picked level whose compaction score is below lake_pk_compaction_min_level_score, and which trips none of the override conditions, is left uncompacted to avoid rewriting large amounts of base data for a negligible reduction in file count. Set to false to turn off the whole gate in one step; every picked level then compacts unconditionally (the behavior before the gate existed), and the threshold parameters below have no effect.
  • Introduced in: v4.2

lake_pk_compaction_min_level_score​

  • Default: 2.0
  • Type: Double
  • Unit: -
  • Is mutable: Yes
  • Description: Score threshold of the Primary Key compaction score gate (see enable_lake_pk_compaction_score_gate). When the size-tiered selector picks a level whose total compaction score is below this value and no override condition fires, the compaction is skipped. The per-rowset score is io_count * 1MB / read_bytes summed across the level: levels with many small or overlapped rowsets score high (useful compaction), while a few large non-overlapped rowsets score low (a near-pure base rewrite). Set to 0 to make the gate a no-op, because a level's score is always greater than or equal to 0.
  • Introduced in: v4.2

lake_pk_compaction_min_benefit_cost_ratio​

  • Default: 0.005
  • Type: Double
  • Unit: -
  • Is mutable: Yes
  • Description: An override of the Primary Key compaction score gate. The minimum benefit/cost ratio (segments saved per MB rewritten) at which a below-threshold level is still compacted. The benefit folds in delete-vector cleanup pressure (see lake_pk_compaction_delvec_benefit_weight). Set to 0 to disable this override so that only lake_pk_compaction_min_level_score applies.
  • Introduced in: v4.2

lake_pk_compaction_emergency_score​

  • Default: 50.0
  • Type: Double
  • Unit: -
  • Is mutable: Yes
  • Description: An override of the Primary Key compaction score gate. When the tablet-wide read-pressure score (the sum of all rowset scores) reaches this value, compaction proceeds even if the picked level is below lake_pk_compaction_min_level_score, so that hot tablets with heavy read amplification are not starved. Set to 0 to disable this override.
  • Introduced in: v4.2

lake_pk_compaction_delvec_benefit_weight​

  • Default: 12.0
  • Type: Double
  • Unit: -
  • Is mutable: Yes
  • Description: The weight that converts a level's delete pressure into segment-equivalent benefit units in the benefit/cost ratio used by lake_pk_compaction_min_benefit_cost_ratio, computed as benefit = real_benefit_segments + delete_ratio * input_segments * weight. A higher value makes compaction more aggressive at cleaning up delete vectors; a lower value favors reducing write amplification. Set to 0 to drop the delete contribution so that the ratio reflects segment-count benefit only.
  • Introduced in: v4.2

lake_pk_compaction_size_overflow_ratio​

  • Default: 2.0
  • Type: Double
  • Unit: -
  • Is mutable: Yes
  • Description: An override of the Primary Key compaction score gate. When a below-threshold level's total bytes exceed ratio * largest_rowset_bytes * size_tiered_level_multiple (that is, ratio times the natural next-tier promotion target), compaction is forced to bound long-tail mid-tier accumulation. The default 2.0 tolerates twice the natural promotion threshold before forcing a merge. Set to 0 to disable this override, so that there is no size cap.
  • Introduced in: v4.2

enable_lake_prepared_split_pre_refinement​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether a Prepared Physical Split Lake scan (see the session variable enable_lake_prepared_physical_split_scan) issues an extra coarse-range morsel over an un-pruned segment range while the seed page-pruning is still running, so otherwise-idle drivers stay busy until the refined ranges land. Disabling it never drops data (the coarse range is always a superset that the refined ranges subtract from); it only trades early parallelism for less redundant coarse scanning.
  • Introduced in: v4.2

lake_prepared_split_max_splitted_scan_rows​

  • Default: 262144
  • Type: Int
  • Unit: Rows
  • Is mutable: Yes
  • Description: The upper bound on splitted_scan_rows (the number of rows scanned per split morsel) applied only when the Prepared Physical Split Lake Scan is enabled (see the session variable enable_lake_prepared_physical_split_scan). The effective bound is min(tablet_internal_parallel_max_splitted_scan_rows, this), so it can only make split morsels finer -- cutting a large tablet into more sub-range morsels that fill otherwise-idle drivers -- never coarser. Takes effect only in a shared-data cluster.
  • Introduced in: v4.2

lake_put_txn_log_timeout_guard_ms​

  • Default: -1
  • Type: Int64
  • Unit: Milliseconds
  • Is mutable: Yes
  • Description: Timeout guard for writing a transaction log to object storage in a shared-data cluster (the put_txn_log and put_combined_txn_log paths). If writing a transaction log takes longer than this value, StarRocks dumps the stack trace of the slow thread to the BE log to help diagnose slow object-storage writes. Disabled by default (a value less than or equal to 0 disables the guard); set it to a positive value such as 4000 (4 seconds) to enable.
  • Introduced in: -

lake_rows_mapper_read_parallelism​

  • Default: 32
  • Type: Int
  • Unit: sub-chunks
  • Is mutable: Yes
  • Description: Maximum number of in-flight .lcrm (lake compaction rows-mapper) sub-chunk reads kept by RowsMapperIterator during light Primary Key compaction publish in a shared-data cluster. Each sub-chunk is lake_rows_mapper_sub_chunk_bytes in size and never crosses a segment boundary; the iterator submits up to this many reads to the PK index execution thread pool and pipelines them against the caller's per-segment processing. Memory bound is lake_rows_mapper_read_parallelism * lake_rows_mapper_sub_chunk_bytes. Set to 1 to disable pipelining and fall back to sequential reads.

lake_rows_mapper_sub_chunk_bytes​

  • Default: 4194304
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: Sub-chunk granularity for RowsMapperIterator pipelined reads of .lcrm files during light Primary Key compaction publish in a shared-data cluster. Each output segment is split into ceil(segment_bytes / lake_rows_mapper_sub_chunk_bytes) sub-chunks pipelined independently. Smaller values raise the achievable parallelism for few-but-large output segments at the cost of more range reads and an extra memcpy on consume. Defaults to 4 MiB to align with the starcache disk-tier block size.

lake_segment_metadata_cache_limit_percent​

  • Default: 10
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The capacity of the segment metadata cache in a shared-data cluster, as a percentage of the BE process memory limit. This cache holds opened segments (their footers, indexes, and column readers), delete vectors, and tablet schemas. It is separate from the tablet metadata cache (see lake_tablet_metadata_cache_limit_percent), so loading many segments does not evict tablet metadata or transaction logs. Values outside the range 0–100 are treated as 0 or 100. Set this item to 0 to disable the cache. Together with lake_tablet_metadata_cache_limit_percent, this item replaces lake_metadata_cache_limit. This value is the largest capacity of the cache: the cache shrinks below it when the BE process runs short of memory (see lake_tablet_metadata_cache_min_limit).
  • Introduced in: -

lake_segment_metadata_cache_min_limit​

  • Default: 1073741824
  • Type: Long
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The capacity below which the segment metadata cache in a shared-data cluster does not shrink when it gives memory back under memory pressure (see lake_tablet_metadata_cache_min_limit). If this value is larger than the capacity set by lake_segment_metadata_cache_limit_percent, the cache stays at that capacity.
  • Introduced in: -

lake_shard_info_cache_ttl_sec​

  • Default: 60
  • Type: Int
  • Unit: Seconds
  • Is mutable: Yes
  • Description: How long a BE or CN in a shared-data cluster keeps the storage location of a tablet it does not own, so that touching the tablet again within this time does not request the location from StarMgr. A node keeps two kinds of locations. The first kind is what it requested from StarMgr, counted in starrocks_be_staros_shard_info_fallback_total, when it read or wrote the files of a tablet assigned to another node, for example when several nodes write the same tablet in one load, or when a vacuum reads the metadata of every tablet in a partition; lookups answered by them are counted in staros_shard_info_fallback_cache_hits_total. The second kind is what the nodes that own the tablets hand over to the node that coordinates a load into a table with file_bundling enabled, together with their transaction logs, so that the coordinating node writes the combined transaction log of each partition without requesting the location; lookups answered by them are counted in lake_tablet_location_handoff_hits_total. The node that owns a tablet receives storage volume updates pushed by StarMgr, while a kept location picks them up only after it expires, so this item also bounds how long such a node can keep using outdated storage volume settings, such as rotated credentials. The kept locations include the credentials of the storage volume. 0 disables both kinds: nothing is handed over or kept, and every such lookup requests the location from StarMgr.
  • Introduced in: -

lake_tablet_metadata_cache_limit_percent​

  • Default: 5
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The capacity of the tablet metadata cache in a shared-data cluster, as a percentage of the BE process memory limit. This cache holds tablet metadata and transaction logs. It is separate from the segment metadata cache (see lake_segment_metadata_cache_limit_percent), so loading many segments does not evict them. Values outside the range 0–100 are treated as 0 or 100. Set this item to 0 to disable the cache. Together with lake_segment_metadata_cache_limit_percent, this item replaces lake_metadata_cache_limit. This value is the largest capacity of the cache: the cache shrinks below it when the BE process runs short of memory (see lake_tablet_metadata_cache_min_limit).
  • Introduced in: -

lake_tablet_metadata_cache_min_limit​

  • Default: 1073741824
  • Type: Long
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The capacity below which the tablet metadata cache in a shared-data cluster does not shrink. When the BE process memory usage exceeds memory_high_level, the tablet metadata cache and the segment metadata cache lower their capacities below their current usage and evict their least recently used entries. The memory to give back is shared between the two caches in proportion to how far each cache's usage is above its floor (this item and lake_segment_metadata_cache_min_limit). The usage above memory_high_level is released gradually over about 50 seconds. If the usage also exceeds memory_urgent_level, the part above memory_urgent_level is released at once. After the usage drops below memory_high_level, the capacities grow back over about 200 seconds to the values set by lake_tablet_metadata_cache_limit_percent and lake_segment_metadata_cache_limit_percent. The check runs every 10 seconds. If this value is larger than the capacity set by lake_tablet_metadata_cache_limit_percent, the cache stays at that capacity.
  • Introduced in: -

lake_vacuum_enable_task_timeout​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether Vacuum tasks honor the timeout carried in the request (VacuumRequest.timeout_ms) and abort themselves once it elapses. Set this item to false to let Vacuum tasks always run to completion no matter how long the FE caller waits.

lake_vacuum_min_batch_delete_size​

  • Default: 200
  • Type: Int64
  • Unit: Number of files
  • Is mutable: Yes
  • Description: The number of stale files Vacuum batches into a single DeleteObjects request on a shared-data cluster. A larger batch amortizes per-call HTTP / auth / signing overhead and reduces the prefix-level request rate against the object store, at the cost of higher single-call latency and a larger replay cost when a transient error retries. Users running on AWS S3 are encouraged to raise this further (up to the protocol cap of 1000) where per-request server time is nearly batch-size insensitive.

loop_count_wait_fragments_finish​

  • Default: 2
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The number of loops to be waited when the BE/CN process exits. Each loop is a fixed interval of 10 seconds. You can set it to 0 to disable the loop wait. From v3.4 onwards, this item is changed to mutable and its default value is changed from 0 to 2.
  • Introduced in: v2.5

max_client_cache_size_per_host​

  • Default: 10
  • Type: Int
  • Unit: entries (cached client instances) per host
  • Is mutable: No
  • Description: The maximum number of cached client instances retained for each remote host by BE-wide client caches. This single setting is used when creating BackendServiceClientCache, FrontendServiceClientCache, and BrokerServiceClientCache during ExecEnv initialization, so it limits the number of client stubs/connections kept per host across those caches. Raising this value reduces reconnects and stub creation overhead at the cost of increased memory and file-descriptor usage; lowering it saves resources but may increase connection churn. The value is read at startup and cannot be changed at runtime. Currently one shared setting controls all client cache types; separate per-cache configuration may be introduced later.
  • Introduced in: v3.2.0

primary_key_index_memory_high_percent​

  • Default: 75
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The level to which a background thread periodically shrinks the primary key index cache in a shared-data cluster, expressed as a percentage of the cache capacity. The cache capacity equals the memory limit for updates (see update_memory_limit_percent). When the cache holds more than this percentage, the least recently used primary key indexes that are not in use are evicted until the cache is within this percentage. Indexes in use are never evicted. Set it to a value lower than primary_key_index_memory_urgent_percent.
  • Introduced in: -

primary_key_index_memory_middle_percent​

  • Default: 50
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The level, expressed as a percentage of the cache capacity, to which the primary key index cache is shrunk when a publish of a Primary Key table in a shared-data cluster fails because it exceeds the memory limit. The primary key indexes that are not in use are evicted until the cache is within this percentage, which frees memory for the retried publish. Set it to a value lower than primary_key_index_memory_high_percent.
  • Introduced in: -

primary_key_index_memory_urgent_percent​

  • Default: 85
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The upper limit of the primary key index cache in a shared-data cluster, expressed as a percentage of the cache capacity. It is checked after every publish that does not fail because it exceeds the memory limit: when the cache holds more than this percentage, the least recently used primary key indexes that are not in use are evicted until the cache is within this percentage. It is a backstop for when the cache grows faster than the background thread brings it down to primary_key_index_memory_high_percent, so set it to a value higher than that item.
  • Introduced in: -

starlet_filesystem_instance_cache_capacity​

  • Default: 10000
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The cache capacity of starlet filesystem instances.
  • Introduced in: v3.2.16, v3.3.11, v3.4.1

starlet_filesystem_instance_cache_ttl_sec​

  • Default: 86400
  • Type: Int
  • Unit: Seconds
  • Is mutable: Yes
  • Description: The cache expiration time of starlet filesystem instances.
  • Introduced in: v3.3.15, 3.4.5

starlet_fslib_azure_storage_max_single_part_size​

  • Default: 104857600
  • Type: Int64
  • Unit: Bytes
  • Is mutable: Yes
  • Description: In a shared-data cluster, the object size above which files written to Azure Blob Storage or ADLS Gen2 use a multipart upload instead of a single upload request. Both Azure filesystems share this item. BE buffers up to this many bytes in memory per output stream before switching to multipart, so raising it raises memory usage, multiplied by the number of concurrent output streams. Raising it high enough to avoid multipart entirely can also fail uploads outright if it exceeds the object store's single-request upload limit; the upload then fails with the backend's own error. Changing it takes effect on uploads already in progress. The value must be greater than 0; a non-positive value is rejected. A valid value here overrides an equivalent --fslib_* gflag passed on the BE command line. A value rejected at BE startup is not applied at all: BE keeps the previously effective value and logs a warning, so this item can report a value that is not the one in effect.
  • Introduced in: v4.1.5, v4.2

starlet_fslib_azure_storage_min_upload_part_size​

  • Default: 5242880
  • Type: Int64
  • Unit: Bytes
  • Is mutable: Yes
  • Description: In a shared-data cluster, the block size used for multipart uploads to Azure Blob Storage or ADLS Gen2. Both Azure filesystems share this item. BE buffers up to this many bytes in memory per output stream between block uploads, so raising it raises memory usage, multiplied by the number of concurrent output streams. Setting it very low increases request count and can exceed the block limits of the storage service. Changing it takes effect on uploads already in progress. The value must be greater than 0; a non-positive value is rejected. A valid value here overrides an equivalent --fslib_* gflag passed on the BE command line. A value rejected at BE startup is not applied at all: BE keeps the previously effective value and logs a warning, so this item can report a value that is not the one in effect.
  • Introduced in: v4.1.5, v4.2

starlet_fslib_gcs_max_single_part_size​

  • Default: 104857600
  • Type: Int64
  • Unit: Bytes
  • Is mutable: Yes
  • Description: In a shared-data cluster, the object size above which files written to Google Cloud Storage use a streaming upload instead of a single insert request. BE buffers up to this many bytes in memory per output stream before switching to streaming, so raising it raises memory usage, multiplied by the number of concurrent output streams. Raising it high enough to avoid streaming entirely can also fail uploads outright if it exceeds the object store's single-request upload limit; the upload then fails with the backend's own error. Changing it takes effect on uploads already in progress. The value must be greater than 0; a non-positive value is rejected. A valid value here overrides an equivalent --fslib_* gflag passed on the BE command line. A value rejected at BE startup is not applied at all: BE keeps the previously effective value and logs a warning, so this item can report a value that is not the one in effect.
  • Introduced in: v4.1.5, v4.2

starlet_fslib_s3_max_single_part_size​

  • Default: 104857600
  • Type: Int64
  • Unit: Bytes
  • Is mutable: Yes
  • Description: In a shared-data cluster, the object size above which files written to S3 or an S3-compatible object store use a multipart upload instead of a single PutObject request. BE buffers up to this many bytes in memory per output stream before switching to multipart, so raising it raises memory usage, multiplied by the number of concurrent output streams. Raising it too high to avoid multipart entirely can also fail uploads outright: AWS S3 rejects a single PutObject larger than 5 GiB, and the limits of S3-compatible object stores differ. Changing it takes effect on uploads already in progress. The value must be greater than 0; a non-positive value is rejected. A valid value here overrides an equivalent --fslib_* gflag passed on the BE command line. A value rejected at BE startup is not applied at all: BE keeps the previously effective value and logs a warning, so this item can report a value that is not the one in effect. This item controls starlet uploads in a shared-data cluster and is unrelated to the experimental_s3_* items.
  • Introduced in: v4.1.5, v4.2

starlet_fslib_s3_min_upload_part_size​

  • Default: 5242880
  • Type: Int64
  • Unit: Bytes
  • Is mutable: Yes
  • Description: In a shared-data cluster, the part size used for multipart uploads to S3 or an S3-compatible object store. BE buffers up to this many bytes in memory per output stream between part uploads, so raising it raises memory usage, multiplied by the number of concurrent output streams. Setting it very low increases request count and can exceed the 10,000-part limit or fall below the 5 MB minimum part size that AWS S3 enforces; the limits of other S3-compatible stores differ. Changing it takes effect on uploads already in progress. The value must be greater than 0; a non-positive value is rejected. A valid value here overrides an equivalent --fslib_* gflag passed on the BE command line. A value rejected at BE startup is not applied at all: BE keeps the previously effective value and logs a warning, so this item can report a value that is not the one in effect. This item controls starlet uploads in a shared-data cluster and is unrelated to the experimental_s3_* items.
  • Introduced in: v4.1.5, v4.2

starlet_port​

  • Default: 9070
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: An extra agent service port for BE and CN.
  • Introduced in: -

starlet_star_cache_disk_size_percent​

  • Default: 80
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The percentage of disk capacity that Data Cache can use at most in a shared-data cluster. Only takes effect when datacache_unified_instance_enable is false.
  • Introduced in: v3.1

starlet_star_cache_skip_fresh_block_checksum_verification​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: No
  • Description: In a shared-data cluster, whether a Data Cache disk block that this CN process wrote and has not evicted since skips the first full-block checksum read when a later request touches it. That verification re-reads the whole block from disk to check data that the process has just written and still tracks in memory, which costs one extra disk read on the first access to every freshly cached block. Set this item to false to verify every block on first access, which is only worth its cost where the cache disk is suspected of corrupting data at rest. Blocks inherited from a previous run of the process are always verified regardless of this item, because nothing in the current process witnessed their write. This item is read once when Data Cache initializes.
  • Introduced in: v4.2.0

starlet_star_cache_write_overwrite​

  • Default: false
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: In a shared-data cluster, whether a Data Cache write replaces an object that is already cached under the same key. When this item is false, the existing object is kept and the write is reported as successful, which suits the read-through fill path, where the bytes being written are the ones already cached. Set this item to true to let a re-read refresh a block that is suspected to be stale or damaged. Data Cache reads this item on every write, so a change takes effect on the next write without re-initializing the cache.
  • Introduced in: -

starlet_starmgr_client_compression_type​

  • Default: none
  • Type: String
  • Unit: -
  • Is mutable: Yes
  • Description: In a shared-data cluster, the compression applied to the worker heartbeat that this CN sends to StarMgr. The heartbeat carries one entry per tablet on the CN, which makes it the largest recurring request in the cluster. Valid values: none and zstd. zstd compresses the payload on the heartbeat thread, within the same RPC timeout budget, and the FE decompresses it. none sends the heartbeat uncompressed. The FE must be able to decompress the payload before any CN is switched, because an FE that predates this mechanism discards the compressed heartbeat, so upgrade the FEs first and only then set this item on the CNs. An unrecognized value is reported and falls back to none instead of preventing the CN from starting.
  • Introduced in: v4.2.0

starlet_starmgr_client_rpc_timeout_ms​

  • Default: 5000
  • Type: Int
  • Unit: Milliseconds
  • Is mutable: Yes
  • Description: In a shared-data cluster, the timeout of each RPC that this CN sends to StarMgr, including GetShard, the worker heartbeat, and the tablet replica information update. The CN sends GetShard when it accesses a tablet that it does not hold locally, for example when a single CN vacuums all tablets of a partition in a table with file bundling enabled. The value must be greater than 0. A dynamic update to a non-positive value is rejected. At startup, a non-positive value is not applied and a warning is logged, and the default value of 5000 remains in effect.
  • Introduced in: v4.2.0

starlet_use_star_cache​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to enable Data Cache in a shared-data cluster. true indicates enabling this feature and false indicates disabling it. The default value is set from false to true from v3.2.3 onwards.
  • Introduced in: v3.1

starlet_write_file_with_tag​

  • Default: false
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: In a shared-data cluster, whether to tag files written to object storage with object storage tags for convenient custom file management.
  • Introduced in: v3.5.3

table_schema_service_max_retries​

  • Default: 3
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The maximum number of retries for Table Schema Service requests.
  • Introduced in: v4.1

Data Lake​

datacache_block_buffer_enable​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: No
  • Description: Whether to enable Block Buffer to optimize Data Cache efficiency. When Block Buffer is enabled, the system reads the Block data from the Data Cache and caches it in a temporary buffer, thus reducing the extra overhead caused by frequent cache reads.
  • Introduced in: v3.2.0

datacache_disk_adjust_interval_seconds​

  • Default: 10
  • Type: Int
  • Unit: Seconds
  • Is mutable: Yes
  • Description: The interval of Data Cache automatic capacity scaling. At regular intervals, the system checks the cache disk usage, and triggers Automatic Scaling when necessary.
  • Introduced in: v3.3.0

datacache_disk_idle_seconds_for_expansion​

  • Default: 180
  • Type: Int
  • Unit: Seconds
  • Is mutable: Yes
  • Description: The minimum wait time for Data Cache automatic expansion. Automatic scaling up is triggered only if the disk usage remains below datacache_disk_low_level for longer than this duration.
  • Introduced in: v3.3.0

datacache_disk_size​

  • Default: 100%
  • Type: String
  • Unit: -
  • Is mutable: Yes
  • Description: The maximum amount of data that can be cached on a single disk. You can set it as a percentage (for example, 80%) or a physical limit (for example, 2T, 500G). For example, if you use two disks and set the value of the datacache_disk_size parameter as 21474836480 (20 GB), a maximum of 40 GB data can be cached on these two disks. A percentage is relative to the capacity of the disk that holds each storage_root_path. The default value is 100%. When enable_datacache_disk_auto_adjust is true (default), the system adjusts the actual disk cache quota based on disk usage, and this item only sets the upper limit of that quota. 0 disables the disk cache.
  • Introduced in: -

datacache_enable​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: No
  • Description: Whether to enable Data Cache. true indicates Data Cache is enabled, and false indicates Data Cache is disabled. The default value is changed to true from v3.3.
  • Introduced in: -

datacache_eviction_policy​

  • Default: slru
  • Type: String
  • Unit: -
  • Is mutable: No
  • Description: The eviction policy of Data Cache. Valid values: lru (least recently used) and slru (Segmented LRU).
  • Introduced in: v3.4.0

datacache_inline_item_count_limit​

  • Default: 130172
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The maximum number of inline cache items in Data Cache. For some particularly small cache blocks, Data Cache stores them in inline mode, which caches the block data and metadata together in memory.
  • Introduced in: v3.4.0

datacache_lru_shard_evict_fallback_enable​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether an LRU shard of Data Cache can evict entries from the other shards when it is being refilled or cannot supply the whole charge by itself. false restores the behavior that a shard only evicts its own entries, and datacache_lru_shard_refill_low_watermark_percent and datacache_lru_shard_refill_high_watermark_percent are ignored.
  • Introduced in: -

datacache_lru_shard_refill_low_watermark_percent​

  • Default: 50
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: An LRU shard of Data Cache is refilled from the largest shards instead of evicting its own entries once its usage drops below this percentage of its capacity, until it reaches datacache_lru_shard_refill_high_watermark_percent. The value must not exceed datacache_lru_shard_refill_high_watermark_percent. Setting both parameters to 0 disables the refill.
  • Introduced in: -

datacache_lru_shard_refill_high_watermark_percent​

  • Default: 70
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The percentage of the shard capacity at which the refill of an LRU shard of Data Cache stops. Valid values: datacache_lru_shard_refill_low_watermark_percent to 100. Setting both parameters to 0 disables the refill.
  • Introduced in: -

datacache_mem_size​

  • Default: 20%
  • Type: String
  • Unit: -
  • Is mutable: Yes
  • Description: The maximum amount of data that can be cached in memory. You can set it as a percentage (for example, 10%) or a physical limit (for example, 10G, 21474836480).
  • Introduced in: -

datacache_min_disk_quota_for_adjustment​

  • Default: 10737418240
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The minimum effective capacity for Data Cache Automatic Scaling. If the system tries to adjust the cache capacity to less than this value, the cache capacity will be directly set to 0 to prevent suboptimal performance caused by frequent cache fills and evictions due to insufficient cache capacity.
  • Introduced in: v3.3.0

disk_high_level​

  • Default: 90
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The upper limit of disk usage (in percentage) that triggers the automatic scaling up of the cache capacity. When the disk usage exceeds this value, the system automatically evicts cache data from the Data Cache. From v3.4.0 onwards, the default value is changed from 80 to 90. This item is renamed from datacache_disk_high_level to disk_high_level from v4.0 onwards.
  • Introduced in: v3.3.0

disk_low_level​

  • Default: 60
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The lower limit of disk usage (in percentage) that triggers the automatic scaling down of the cache capacity. When the disk usage remains below this value for the period specified in datacache_disk_idle_seconds_for_expansion, and the space allocated for Data Cache is fully utilized, the system will automatically expand the cache capacity by increasing the upper limit. This item is renamed from datacache_disk_low_level to disk_low_level from v4.0 onwards.
  • Introduced in: v3.3.0

disk_safe_level​

  • Default: 80
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The safe level of disk usage (in percentage) for Data Cache. When Data Cache performs automatic scaling, the system adjusts the cache capacity with the goal of maintaining disk usage as close to this value as possible. From v3.4.0 onwards, the default value is changed from 70 to 80. This item is renamed from datacache_disk_safe_level to disk_safe_level from v4.0 onwards.
  • Introduced in: v3.3.0

enable_connector_sink_spill​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to enable Spilling for writes to external tables. Enabling this feature prevents the generation of a large number of small files as a result of writing to an external table when memory is insufficient. Currently, this feature only supports writing to Iceberg tables.
  • Introduced in: v4.0.0

enable_datacache_disk_auto_adjust​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to enable Automatic Scaling for Data Cache disk capacity. When it is enabled, the system dynamically adjusts the cache capacity based on the current disk usage rate. This item is renamed from datacache_auto_adjust_enable to enable_datacache_disk_auto_adjust from v4.0 onwards.
  • Introduced in: v3.3.0

datacache_unified_instance_enable​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: No
  • Description: Whether to use a unified Data Cache instance to manage data caching for both internal catalog and external catalog in a shared-data cluster.
  • Introduced in: v3.4.0

external_table_clear_corrupted_cache_data​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to allow the system to clear the corrupted data cache of external table files (for example, Hive and Iceberg tables). When a scan fails to decode a file whose content was read from the data cache, the system drops the cached blocks it read, so that the next read of the file recovers from the remote storage. For Parquet files, if no row has been returned yet, the system first reads the file again bypassing the data cache, and drops the cached blocks only if this read succeeds. This item does not affect the internal tables of shared-data clusters, which are controlled by lake_clear_corrupted_cache_data.
  • Introduced in: -

jdbc_connection_idle_timeout_ms​

  • Default: 600000
  • Type: Int
  • Unit: Milliseconds
  • Is mutable: No
  • Description: The length of time after which an idle connection in the JDBC connection pool expires. If the connection idle time in the JDBC connection pool exceeds this value, the connection pool closes idle connections beyond the number specified in the configuration item jdbc_minimum_idle_connections. If no query uses a connection pool for this length of time, the BE closes the whole connection pool, including the connections kept by jdbc_minimum_idle_connections, and creates a new one on the next query. This releases the connection pools of dropped or altered JDBC catalogs and resources.
  • Introduced in: -

jdbc_connection_pool_size​

  • Default: 8
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The JDBC connection pool size. On each BE node, queries that access the external table with the same jdbc_url share the same connection pool.
  • Introduced in: -

jdbc_minimum_idle_connections​

  • Default: 1
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The minimum number of idle connections in the JDBC connection pool.
  • Introduced in: -

jdbc_connection_max_lifetime_ms​

  • Default: 300000
  • Type: Long
  • Unit: Milliseconds
  • Is mutable: No
  • Description: Maximum lifetime of a connection in the JDBC connection pool. Connections are recycled before this timeout to prevent stale connections. Minimum allowed value is 30000 (30 seconds).
  • Introduced in: -

jdbc_connection_keepalive_time_ms​

  • Default: 30000
  • Type: Long
  • Unit: Milliseconds
  • Is mutable: No
  • Description: Keepalive interval for idle JDBC connections. Idle connections are tested at this interval to detect stale connections proactively. Set to 0 to disable keepalive probing. When enabled, must be >= 30000 and less than jdbc_connection_max_lifetime_ms. Invalid enabled values are silently disabled (reset to 0).
  • Introduced in: -

lake_clear_corrupted_cache_data​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to allow the system to clear the corrupted data cache in a shared-data cluster.
  • Introduced in: v3.4

lake_dump_corrupted_cache​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to dump the locally cached blocks of a file before an automatic corruption-recovery path drops them in a shared-data cluster. The dump preserves the exact bytes the local cache was serving, so the corruption can be analyzed offline while the drop still lets the read recover from remote storage. Blocks are read from the local cache only, so a dump never contains bytes fetched from remote storage and never changes the cache. A dump that cannot be taken is logged and never blocks the drop. Only meaningful together with lake_clear_corrupted_cache_meta and lake_clear_corrupted_cache_data: nothing is preserved when nothing is dropped. Dumps are written to lake_cache_dump_dir and bounded by lake_cache_dump_interval_seconds, lake_cache_dump_max_bytes_per_event, and lake_cache_dump_max_dumps.
  • Introduced in: v4.2.0

lake_cache_dump_dir​

  • Default: none
  • Type: String
  • Unit: -
  • Is mutable: Yes
  • Description: The directory that holds local cache dumps. An empty value means <the first path in storage_root_path>/cache_dump, because a dump is data and belongs on a storage disk rather than on the log disk; one dump root is used even when several storage paths are configured. Each dump creates one <file name>.<unix milliseconds> subdirectory holding manifest.json, one <block id>.blk per cached block, and the object's copy of the same byte range under the file's own name. Only the directory this item resolves to is garbage collected: after you change it, the dumps under the previous root stay on disk, and a dump_dir passed to the /api/cloudnative/cache API is never swept.
  • Introduced in: v4.2.0

lake_cache_dump_interval_seconds​

  • Default: 600
  • Type: Int
  • Unit: Seconds
  • Is mutable: Yes
  • Description: The minimum interval between two automatic cache dumps. The limit is process-wide and independent of which file failed, because a failing disk makes every file a first offender. Dumps requested through the /api/cloudnative/cache API are not subject to it.
  • Introduced in: v4.2.0

lake_cache_dump_max_bytes_per_event​

  • Default: 134217728
  • Type: Long
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The upper bound on everything a single automatic cache dump writes. A detection site that knows which page failed dumps only the cache blocks covering that page, usually one block, so this bound only clips whole-file dumps, which are then marked "truncated": true in the manifest. Dumps requested through the /api/cloudnative/cache API are not clipped. A dump saves the range twice -- once as the cache served it, once as the object holds it -- so the range it covers is half this value.
  • Introduced in: v4.2.0

lake_cache_dump_max_dumps​

  • Default: 50
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: How many cache dumps are kept under lake_cache_dump_dir. Before a dump is taken, the oldest dumps are removed until there is room for it, so the limit holds the most recent dumps rather than the first ones to arrive: one file that keeps failing must not make the next corruption invisible. A count rather than a byte budget: what an operator needs is the last N corruption events, and the disk footprint of the automatic dumps is already bounded by lake_cache_dump_max_bytes_per_event. Dumps written to an operator-supplied dump_dir through the API are never swept, so a dump worth keeping can be put out of reach of this limit. Worst case on disk is this count times lake_cache_dump_max_bytes_per_event.
  • Introduced in: v4.2.0

lake_cache_dump_retention_seconds​

  • Default: 172800
  • Type: Int
  • Unit: Seconds
  • Is mutable: Yes
  • Description: How long a cache dump is kept before it is removed. Expired dumps are swept when a new dump is taken.
  • Introduced in: v4.2.0

lake_pk_index_sst_verify_checksum​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to verify sstable block checksums when reading the cloud-native Primary Key index (point lookups and compaction merges) in a shared-data cluster. When enabled, corrupted bytes (usually from a corrupted local cache copy) deterministically fail as a Corruption error and trigger the corrupted-cache cleanup, instead of being misparsed or silently returning wrong index values. Note that the automatic cleanup of the corrupted cache additionally requires lake_clear_corrupted_cache_data to be enabled.
  • Introduced in: v4.1

lake_clear_corrupted_cache_meta​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to allow the system to clear the corrupted metadata cache in a shared-data cluster. This covers the local cache of tablet metadata files and transaction log files. When a read of such a file fails with a corruption error, the system drops the cached copy and reads the file again from remote storage once.
  • Introduced in: v3.3

lake_enable_horizontal_compaction_fill_data_cache​

  • Default: false
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to allow horizontal compaction tasks to cache the input data they read on local disks in a shared-data cluster. Horizontal compaction reads each input segment exactly once and its input rowsets are replaced right after the task finishes, so caching that data mainly evicts query-hot data from the local cache. Set this item to true to restore the previous behavior of always caching.
  • Introduced in: v4.2

lake_enable_vertical_compaction_fill_data_cache​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: Whether to allow vertical compaction tasks to cache data on local disks in a shared-data cluster.
  • Introduced in: v3.1.7, v3.2.3

lake_replication_read_buffer_size​

  • Default: 16777216
  • Type: Long
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The read buffer size used when downloading lake segment files during lake replication. This value determines the per-read allocation for reading remote files; the implementation uses the larger of this setting and a 1 MB minimum. A larger value reduces the number of read calls and can improve throughput but increases memory used per concurrent download; a smaller value lowers memory usage at the cost of more I/O calls. Tune according to network bandwidth, storage I/O characteristics, and the number of parallel replication threads.
  • Introduced in: v4.1.2

lake_replication_max_file_copy_retry​

  • Default: 3
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: Maximum number of retry attempts for non-segment file copy (.sst, .delvec, .del, .cols) during lake-to-lake (shared-data) cross-cluster replication. Each attempt verifies the copied file size matches the source to detect truncated copies caused by transient object storage issues. Increase this value if experiencing intermittent file corruption during replication over unreliable storage.
  • Introduced in: v4.1.2

lake_replication_file_copy_threads​

  • Default: 16
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: Number of threads in the CN-wide dedicated thread pool used by lake-to-lake (shared-data) cross-cluster replication for per-file copy. The default caps concurrent copy tasks and their read buffers independently of CPU count. 0 means cpu_cores * 4; negative values mean -value * cpu_cores. The pool is built once at startup, so CN restart is required to change its size.
  • Introduced in: v4.1.2

lake_replication_max_parallel_files_per_tablet​

  • Default: 4
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: Maximum number of files copied concurrently for one tablet during lake-to-lake (shared-data) cross-cluster replication. All tablets share the CN-wide lake_replication_file_copy_threads pool, so this setting prevents one tablet from monopolizing the pool. Values less than or equal to 1 serialize file copy within each tablet.
  • Introduced in: v4.1.4

lake_replication_parallel_copy_min_file_count​

  • Default: 2
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: Minimum number of files in a tablet required to copy that tablet's files concurrently during lake-to-lake (shared-data) cross-cluster replication. Set this parameter to 0 to disable the dedicated file copy pool and execute file copy in the calling replication task.
  • Introduced in: v4.1.2

lake_service_max_concurrency​

  • Default: 0
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The maximum concurrency of RPC requests in a shared-data cluster. Incoming requests will be rejected when this threshold is reached. When this item is set to 0, no limit is imposed on the concurrency.
  • Introduced in: -

max_hdfs_scanner_num​

  • Default: 50
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: Limits the maximum number of concurrently running connector (HDFS/remote) scanners that a ConnectorScanNode can have. During scan startup the node computes an estimated concurrency (based on memory, chunk size and scanner_row_num) and then caps it with this value to determine how many scanners and chunks to reserve and how many scanner threads to start. It is also consulted when scheduling pending scanners at runtime (to avoid oversubscription) and when deciding how many pending scanners can be re-submitted considering file-handle limits. Lowering this reduces threads, memory and open-file pressure at the cost of potential throughput; increasing it raises concurrency and resource usage.
  • Introduced in: v3.2.0

query_max_memory_limit_percent​

  • Default: 90
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The maximum memory that the Query Pool can use. It is expressed as a percentage of the Process memory limit.
  • Introduced in: v3.1.0

rocksdb_max_write_buffer_memory_bytes​

  • Default: 1073741824
  • Type: Int64
  • Unit: -
  • Is mutable: No
  • Description: It is the max size of the write buffer for meta in rocksdb. Default is 1GB.
  • Introduced in: v3.5.0

rocksdb_write_buffer_memory_percent​

  • Default: 5
  • Type: Int64
  • Unit: -
  • Is mutable: No
  • Description: It is the memory percent of write buffer for meta in rocksdb. default is 5% of system memory. However, aside from this, the final calculated size of the write buffer memory will not be less than 64MB nor exceed 1G (rocksdb_max_write_buffer_memory_bytes)
  • Introduced in: v3.5.0

Other​

default_mv_resource_group_concurrency_limit​

  • Default: 0
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The maximum concurrency (per BE node) of the materialized view refresh tasks in the resource group default_mv_wg. The default value 0 indicates no limits. Deprecated: from v3.4 onwards, this item no longer takes effect because the FE stores default_mv_wg and sends it to the BEs. Use ALTER RESOURCE GROUP default_mv_wg WITH (...) instead.
  • Introduced in: v3.1

default_mv_resource_group_cpu_limit​

  • Default: 1
  • Type: Int
  • Unit: -
  • Is mutable: No
  • Description: The maximum number of CPU cores (per BE node) that can be used by the materialized view refresh tasks in the resource group default_mv_wg. Deprecated: from v3.4 onwards, this item no longer takes effect because the FE stores default_mv_wg and sends it to the BEs. Use ALTER RESOURCE GROUP default_mv_wg WITH (...) instead.
  • Introduced in: v3.1

default_mv_resource_group_memory_limit​

  • Default: 0.8
  • Type: Double
  • Unit:
  • Is mutable: No
  • Description: The maximum memory proportion (per BE node) that can be used by the materialized view refresh tasks in the resource group default_mv_wg. The default value indicates 80% of the memory. Deprecated: from v3.4 onwards, this item no longer takes effect because the FE stores default_mv_wg and sends it to the BEs. Use ALTER RESOURCE GROUP default_mv_wg WITH (...) instead.
  • Introduced in: v3.1

default_mv_resource_group_spill_mem_limit_threshold​

  • Default: 0.8
  • Type: Double
  • Unit: -
  • Is mutable: No
  • Description: The memory usage threshold before a materialized view refresh task in the resource group default_mv_wg triggers intermediate result spilling. The default value indicates 80% of the memory. Deprecated: from v3.4 onwards, this item no longer takes effect because the FE stores default_mv_wg and sends it to the BEs. Use ALTER RESOURCE GROUP default_mv_wg WITH (...) instead.
  • Introduced in: v3.1

enable_resolve_hostname_to_ip_in_load_error_url​

  • Default: false
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: For error_urls debugging, whether to allow operators to choose between using original hostnames from FE heartbeat or forcing resolution to IP addresses based on their environment needs.
    • true: Resolve hostnames to IPs.
    • false (Default): Keeps the original hostname in the error URL.
  • Introduced in: v4.0.1

enable_retry_apply​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: When enabled, Tablet apply failures that are classified as retryable (for example transient memory-limit errors) are rescheduled for retry instead of immediately marking the tablet in error. The retry path in TabletUpdates schedules the next attempt using retry_apply_interval_second multiplied by the current failure count and clamped to a 600s maximum, so backoff grows with successive failures. Explicitly non-retryable errors (for example corruption) bypass retries and cause the apply process to enter the error state immediately. Retries continue until an overall timeout/terminal condition is reached, after which the apply will enter the error state. Turning this off disables automatic rescheduling of failed apply tasks and causes failed applies to transition to error state without retries.
  • Introduced in: v3.2.9

enable_token_check​

  • Default: true
  • Type: Boolean
  • Unit: -
  • Is mutable: Yes
  • Description: A boolean value to control whether to enable the token check. true indicates enabling the token check, and false indicates disabling it.
  • Introduced in: -

es_scroll_keepalive​

  • Default: 5m
  • Type: String
  • Unit: Minutes (string with suffix, e.g. "5m")
  • Is mutable: No
  • Description: The keep-alive duration sent to Elasticsearch for scroll search contexts. The value is used verbatim (for example "5m") when building the initial scroll URL (?scroll=<value>) and when sending subsequent scroll requests (via ESScrollQueryBuilder). This controls how long the ES search context is retained before garbage collection on the ES side; setting it longer keeps scroll contexts alive for more time but prolongs resource usage on the ES cluster. The value is read at startup by the ES scan reader and is not changeable at runtime.
  • Introduced in: v3.2.0

jemalloc_page_size​

  • Default: auto
  • Type: String
  • Unit: -
  • Is mutable: No
  • Description: Forces bin/start_backend.sh to load a specific jemalloc page-size variant on arm64 (aarch64) hosts instead of auto-detecting the host's page size via getconf PAGESIZE. Accepted values are auto (default, auto-detect), 4k/4096 (forces the 4K-page build), and 64k/65536 (forces the 64K-page build); any other value falls back to auto-detection with a warning logged. A forced 4k is still rejected in favor of the 64K build (with a warning) if a getconf PAGESIZE check succeeds and reports a size other than 4096, since jemalloc aborts at startup when the runtime page size is larger than its build-time one; a failed page-size check does not block the forced value. If 4k is requested but the 4K-page jemalloc build was not packaged, BE falls back to the 64K build with a warning instead of failing to start. This value is read directly from be.conf/cn.conf by bin/start_backend.sh before the BE process starts (to select which jemalloc .so to load via LD_LIBRARY_PATH), so it does not reconfigure jemalloc at runtime and changing it requires a BE restart. Ignored on non-arm64 hosts, where only one jemalloc build is produced.
  • Introduced in: -

load_replica_status_check_interval_ms_on_failure​

  • Default: 2000
  • Type: Int
  • Unit: Milliseconds
  • Is mutable: Yes
  • Description: The interval that the secondary replica checks it's status on the primary replica if the last check rpc fails.
  • Introduced in: v3.5.1

load_replica_status_check_interval_ms_on_success​

  • Default: 15000
  • Type: Int
  • Unit: Milliseconds
  • Is mutable: Yes
  • Description: The interval that the secondary replica checks it's status on the primary replica if the last check rpc successes.
  • Introduced in: v3.5.1

max_length_for_bitmap_function​

  • Default: 1000000
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The maximum length of input values for bitmap functions.
  • Introduced in: -

max_length_for_to_base64​

  • Default: 200000
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: The maximum length of input values for the to_base64() function.
  • Introduced in: -

memory_high_level​

  • Default: 75
  • Type: Long
  • Unit: Percent
  • Is mutable: Yes
  • Description: High water memory threshold expressed as a percentage of the process memory limit. When total memory consumption rises above this percentage, BE begins to free memory gradually (currently by evicting data cache and update cache, and by shrinking the shared-data metadata caches) to relieve pressure. The monitor uses this value to compute memory_high = mem_limit * memory_high_level / 100 and, if consumption > memory_high, performs controlled eviction guided by the GC advisor; if consumption exceeds memory_urgent_level (a separate config), more aggressive immediate reductions occur. This value is also consulted to disable certain memory‑intensive operations (for example, primary-key preload) when the threshold is exceeded. Must satisfy validation with memory_urgent_level (memory_urgent_level > memory_high_level, memory_high_level >= 1, memory_urgent_level <= 100).
  • Introduced in: v3.2.0

report_exec_rpc_request_retry_num​

  • Default: 10
  • Type: Int
  • Unit: -
  • Is mutable: Yes
  • Description: The retry times of rpc request to report exec rpc request to FE. The default value is 10, which means that the rpc request will be retried 10 times if it fails only if it's fragment instatnce finish rpc. Report exec rpc request is important for load job, if one fragment instance finish report failed, the load job will be hang until timeout.
  • Introduced in: -

sleep_one_second​

  • Default: 1
  • Type: Int
  • Unit: Seconds
  • Is mutable: No
  • Description: A small, global sleep interval (in seconds) used by BE agent worker threads as a one-second pause when the master address/heartbeat is not yet available or when a short retry/backoff is needed. In the codebase it is referenced by several report worker pools (e.g., ReportDiskStateTaskWorkerPool, ReportOlapTableTaskWorkerPool, ReportWorkgroupTaskWorkerPool) to avoid busy-waiting and reduce CPU consumption while retrying. Increasing this value slows the retry frequency and responsiveness to master availability; reducing it increases polling rate and CPU usage. Adjust only with awareness of the trade-off between responsiveness and resource use.
  • Introduced in: v3.2.0

small_file_dir​

  • Default: ${STARROCKS_HOME}/lib/small_file/
  • Type: String
  • Unit: -
  • Is mutable: No
  • Description: The directory used to store the files downloaded by the file manager.
  • Introduced in: -

upload_buffer_size​

  • Default: 4194304
  • Type: Int
  • Unit: Bytes
  • Is mutable: Yes
  • Description: Buffer size (in bytes) used by file copy operations when uploading snapshot files to remote storage (broker or direct FileSystem). In the upload path (snapshot_loader.cpp) this value is passed to fs::copy as the read/write chunk size for each upload stream. The default is 4 MiB. Increasing this value can improve throughput on high-latency or high-bandwidth links but increases memory usage per concurrent upload; decreasing it reduces per-stream memory but may reduce transfer efficiency. Tune together with upload_worker_count and overall available memory.
  • Introduced in: v3.2.13

user_function_dir​

  • Default: ${STARROCKS_HOME}/lib/udf
  • Type: String
  • Unit: -
  • Is mutable: No
  • Description: The directory used to store User-defined Functions (UDFs).
  • Introduced in: -

web_log_bytes​

  • Default: 1048576 (1 MB)
  • Type: long
  • Unit: Bytes
  • Is mutable: No
  • Description: Maximum number of bytes to read from the INFO logfile and show on the BE debug webserver's log page. The handler uses this value to compute a seek offset (showing the last N bytes) to avoid reading or serving very large log files. If the logfile is smaller than this value the whole file is shown. Note: in the current implementation the code that reads and serves the INFO log is commented out and the handler reports that the INFO log file couldn't be opened, so this parameter may have no effect unless the log-serving code is enabled.
  • Introduced in: v3.2.0