warehouse_auto_scaling_policy

note

The resource's API may change in subsequent versions to simplify user experience.

Generates a warehouse Auto Scaling policy in the JSON format.

This policy will automatically scale the number of Compute Nodes (CN) based on CPU utilization of the warehouse. For more information, see Enable Auto Scaling for your warehouse.

Scaling Unit

A warehouse's size is its cngroup count multiplied by its compute nodes per cngroup. auto_scaling_unit picks which of the two the policy scales, and the unit that min_size, max_size and step_size are counted in:

• Node (default): compute nodes per cngroup. The cngroup count is left as it is, so the warehouse holds min_size * cngroup count to max_size * cngroup count compute nodes. On a warehouse with a single cngroup, this is the same as the total compute node count.
• Group: the number of cngroups. The compute nodes per cngroup are left as they are.

note

On a multi_az warehouse, Node bounds used to count the warehouse's total compute nodes and had to be multiples of the AZ count. They now count compute nodes per cngroup, and CelerData has converted existing policies by dividing them by the AZ count. If such a warehouse takes its policy from Terraform, divide min_size, max_size and every step_size by len(specified_azs) in your configuration. Otherwise the next terraform plan shows a change to auto_scaling_policy, and applying it sets the old totals as per-cngroup bounds, multiplying the warehouse's range by the AZ count.

Example Usage

CPU-based Auto Scaling Policy

Supported Metrics

Scale Out Metrics:

• AVERAGE_CPU_UTILIZATION

Scale In Metric:

• AVERAGE_CPU_UTILIZATION

Scaling Trigger Mechanism

Scale Out (Expansion)

Trigger Conditions

• AVERAGE_CPU_UTILIZATION > 85.75
  (Resource utilization above 85.75%)
• Must persist for 300 seconds (5 minutes)

Behavior:

• Adds 2 nodes per scaling action (step_size = 2)

Scale In (Contraction)

Trigger Condition:

• AVERAGE_CPU_UTILIZATION < 5.25
  (Resource utilization below 5.25%)
• Must persist for 600 seconds (10 minutes)

Behavior:

• Removes 1 node per scaling action (step_size = 1)

Configuration Example

resource "celerdatabyoc_auto_scaling_policy" "policy_1" {
  min_size = <max_compute_node_count>
  max_size = <min_compute_node_count>
  policy_item {
    step_size = 2
    type      = "SCALE_OUT"
    condition {
      duration_seconds = 300
      type             = "AVERAGE_CPU_UTILIZATION"
      value            = 85.75
    }
  }
  policy_item {
    type      = "SCALE_IN"
    step_size = 1
    condition {
      duration_seconds = 600
      type             = "AVERAGE_CPU_UTILIZATION"
      value            = 5.25
    }
  }
}

resource "celerdatabyoc_elastic_cluster_v2" "multi_az_cluster" {
  ...
  default_warehouse {
    ...
    auto_scaling_policy = celerdatabyoc_auto_scaling_policy.policy_1.policy_json
  }
  warehouse {
    ...
    auto_scaling_policy = celerdatabyoc_auto_scaling_policy.policy_1.policy_json
  }
}

Queue-based Auto Scaling Policy

Supported Metrics

Scale Out Metrics:

• QUERY_QUEUE_LENGTH
• EARLIEST_QUERY_PENDING_TIME

Scale In Metric:

• WAREHOUSE_RESOURCE_UTILIZATION

Scaling Trigger Mechanism

Scale Out (Expansion)

Trigger Conditions (OR logic - either condition will trigger):

1. QUERY_QUEUE_LENGTH > 50
   (Backlogged queries exceed 50)
2. EARLIEST_QUERY_PENDING_TIME > 200
   (Earliest query pending time exceeds 200s)

Behavior:

• Adds 2 nodes/groups per scaling action (step_size = 2)

Scale In (Contraction)

Trigger Condition:

• WAREHOUSE_RESOURCE_UTILIZATION < 5
  (Resource utilization below 5%)
• Must persist for 300 seconds (5 minutes)

Behavior:

• Removes 1 node/group per scaling action (step_size = 1)

Configuration Example

resource "celerdatabyoc_auto_scaling_policy" "policy_1" {
  min_size = <max_compute_node_count|max_compute_node_group_count>
  max_size = <min_compute_node_count|min_compute_node_group_count>
  auto_scaling_unit = <"Node"|"Group">
  policy_item {
    step_size = 2
    type      = "SCALE_OUT"
    condition {
      type             = "QUERY_QUEUE_LENGTH"
      value            = 50
    }
  }
  policy_item {
    step_size = 2
    type      = "SCALE_OUT"
    condition {
      type             = "EARLIEST_QUERY_PENDING_TIME"
      value            = 200
    }
  }
  policy_item {
    type      = "SCALE_IN"
    step_size = 1
    condition {
      duration_seconds = 300
      type             = "WAREHOUSE_RESOURCE_UTILIZATION"
      value            = 5
    }
  }
}

resource "celerdatabyoc_elastic_cluster_v2" "multi_az_cluster" {
  ...
  default_warehouse {
    ...
    auto_scaling_policy = celerdatabyoc_auto_scaling_policy.policy_1.policy_json
  }
  warehouse {
    ...
    auto_scaling_policy = celerdatabyoc_auto_scaling_policy.policy_1.policy_json
  }
}

Attribute Reference

This resource exports the following attribute:

• id: The ID of this resource.
• policy_json: The JSON-formatted Auto Scaling policy.

See Also

• celerdatabyoc_elastic_cluster_v2