Terraform Cheatsheet

How Terraform turns configuration into infrastructure changes, and how to keep that safe: state and backends, the plan/apply lifecycle, count vs for_each, modules, import and moved blocks, drift, CI workflows and testing, plus step-by-step playbooks for stuck state locks, unexpected destroy/recreate, dependency cycles and slow plans. Each entry gives the problem, how it works, configuration run on a real Terraform CLI, real-world notes and trade-offs.

Updated: 2026-10-10

29 entries

Quick tables

Reference and comparison tables for quick scanning.

Terraform CLI by task

The commands that answer a specific question or do a specific job, with behaviour checked on Terraform 1.16.5 using the local backend and the random, local and terraform_data providers. Remote-backend behaviour is described from the docs.

Terraform CLI by task
TaskCommandNotes
Install providers, modules and the backend`terraform init`Rerun after changing providers, modules or the backend. Creates `.terraform/` and `.terraform.lock.hcl`. A non-interactive run that needs to answer a migration question fails with `Error asking for confirmation: EOF`.
Upgrade providers inside their constraints`terraform init -upgrade`Ignores the locked version, installs the newest allowed one and rewrites the lock file. Without it a mismatch fails with `must use terraform init -upgrade`.
Move state to a new backend location`terraform init -migrate-state`Add `-force-copy` to skip the question. `-reconfigure` switches without copying, leaving an empty state at the new place.
Pass backend settings at init`terraform init -backend-config=dev.local.tfbackend`File of `key = "value"` lines or one `key=value`. Resolved values are cached in `.terraform/terraform.tfstate` in plain text.
Check formatting`terraform fmt -check -diff`Prints the names of files that would change and a diff; exit code 3 when something needs formatting, 0 when clean. `fmt` alone rewrites the files; add `-recursive` for subdirectories.
Check syntax and references`terraform validate`No API calls and no state. Catches undeclared references and cycles (`Error: Cycle:`). `-json` gives `valid`, `error_count` and `diagnostics`. Needs `init` first.
Preview and keep the plan`terraform plan -out=tfplan`Writes a zip holding plan, state at plan time, configuration and lock file. Treat it as a secret; it contains sensitive values.
Apply exactly the reviewed plan`terraform apply tfplan`No prompt, no new plan. Fails with `Error: Saved plan is stale` if state moved. Passing `-var` with a plan file was rejected on 1.16.5.
Gate a pipeline on changes`terraform plan -detailed-exitcode`Exit 0 no changes, 1 error, 2 changes present.
Run without prompts`terraform apply -auto-approve -input=false``-input=false` turns a missing required variable into `Error: No value for required variable` instead of a prompt.
Set variables`-var 'env=prod'`, `-var-file=prod.tfvars`, `TF_VAR_env=prod`A missing `-var-file` gives `Failed to read variables file`. Later sources override earlier ones; `*.auto.tfvars` and `terraform.tfvars` load without flags.
Recreate one resource`terraform apply -replace=random_id.x`Shown as `will be replaced, as requested`. Replaces the deprecated `terraform taint`.
Limit a run to some addresses`terraform apply -target=random_pet.a`Always warns `Resource targeting is in effect` and `Applied changes may be incomplete`; meant for recovery, not routine use.
See drift without changing anything`terraform plan -refresh-only``apply -refresh-only` records the findings in state (a deleted object leaves state). `-refresh=false` skips the read step.
Preview and perform destroy`terraform plan -destroy`, `terraform destroy``destroy` is `apply -destroy`: same graph in reverse order.
Change concurrency`terraform apply -parallelism=1`Default 10 concurrent operations. Three independent 2 s resources took 2.6 s by default and 6.6 s with 1.
Evaluate an expression against state`echo 'upper(random_pet.a.id)' | terraform console`Prints `"NATIVE-SKUNK"` style quoted results. Reads the current state and configuration.
Draw the dependency graph`terraform graph | dot -Tsvg > graph.svg`Prints DOT; `dot` is Graphviz (not run here). Arrows point from dependent to dependency.
Read outputs`terraform output -json`, `terraform output -raw name`Plain `terraform output` hides sensitive values as `<sensitive>`; asking for one name by name prints the value.
Inspect state or a plan`terraform state list`, `terraform show -json [tfplan]``state list` fails with `No state file was found!` when state is empty. `show -json` is the documented machine format.
List and pin providers`terraform providers`, `terraform providers lock -platform=linux_amd64``providers` prints the requirement tree. `lock` adds checksums for other platforms to `.terraform.lock.hcl`.
Release a stuck remote lock`terraform force-unlock <LOCK_ID>`Use the ID from the error. On the local backend it only answers `Failed to unlock state: LocalState not locked`.
Keep separate state per workspace`terraform workspace new|select|list|show`Local backend keeps non-default state in `terraform.tfstate.d/<name>/`. `new` also switches to it.
Debug a run`TF_LOG=INFO TF_LOG_PATH=tf.log terraform plan`Levels TRACE, DEBUG, INFO, WARN, ERROR. Logs may contain secrets; do not attach them unreviewed.
Run in another directory`terraform -chdir=infra plan`A global option placed before the subcommand; relative paths resolve inside that directory.

`validate`, `fmt`, `graph`, `providers` and `show` do not change infrastructure; `plan` only reads it (and takes the state lock). `apply`, `destroy`, `import`, `force-unlock` and `state` subcommands that write can change state or real resources.

Version constraints and the lock file

Operators accepted by `required_version`, provider `version` and module `version`. The `required_version` rows were checked against Terraform 1.16.5; provider rows against hashicorp/random 3.5.1 to 3.9.1.

Version constraints and the lock file
FormExampleAllowsTypical use and notes
`=` (or bare version)`= 1.16.5`Exactly that versionAccepted on 1.16.5. Rare for providers; sometimes used for `required_version` in a locked CI image.
`!=``!= 1.16.5`Everything except that versionRejected on 1.16.5 with `Unsupported Terraform Core version`. Use inside a range to skip one broken release, e.g. `>= 1.5, != 1.7.0`.
`>``> 1.16.5`Strictly newer versionsRejected on 1.16.5. Rarely useful alone because it has no upper bound.
`>=``>= 1.5`That version or newerGood for a minimum feature level (for example a feature added in 1.5), but it allows next-major releases unless paired with `<`.
`<``< 2.0`Strictly older versionsCombine with `>=` to cap the major line: `>= 1.5, < 2.0`. `>= 1.5, < 1.16` was rejected on 1.16.5.
`<=``<= 1.16.5`That version or olderAccepted on 1.16.5. Usually clearer written as `<`.
`~>` with three parts`~> 1.16.0``>= 1.16.0, < 1.17.0`: patch releases onlyAccepted on 1.16.5, `~> 1.17` was rejected. For providers `~> 3.5.0` selected 3.5.1.
`~>` with two parts`~> 1.15``>= 1.15, < 2.0`: any newer minor in the same majorAccepted on 1.16.5. The rightmost component is the one allowed to rise; for providers `~> 3.5` lets 3.9.1 in.
Several constraints`>= 1.5, < 2.0`All must hold (comma means AND)There is no OR operator. Quote the whole string: `required_version = ">= 1.5, < 2.0"`.
`required_version``terraform { required_version = "~> 1.5" }`Constrains the Terraform CLIChecked by `validate`, `plan` and `apply` before any work. A range with a floor and the next-major cap is the common choice.
Provider `version``required_providers { random = { source = "hashicorp/random", version = "~> 3.5" } }`Constrains the provider pluginUse `~>` on the major or minor. Combined with the lock file it gives a reproducible plugin.
Module `version``module "x" { source = "hashicorp/consul/aws", version = "~> 0.1" }`Constrains a registry moduleThe `version` argument applies to registry sources only; Git sources are pinned with `?ref=` instead (per the docs, not run here).
Lock file after `init``.terraform.lock.hcl`Pins provider `version`, `constraints` and `hashes`Reused while it satisfies the constraints, even after loosening them. Commit it. Does not cover modules or the CLI.
Lock file mismatchlock 3.5.1, constraint `~> 3.6``init` fails`Error: Failed to query available provider packages` ... `must use terraform init -upgrade`. `init -upgrade` then selects 3.9.1.
Other platforms`terraform providers lock -platform=darwin_arm64 -platform=windows_amd64`Adds `h1:` checksums per platformAfter adding two platforms the file held three `h1:` entries. Run it whenever dev and CI use different OS or CPU.
Pre-release versions`1.17.0-beta1`Typically only selected by an exact matchFrom the docs (not run here): range operators do not select pre-releases, so pin one with `=` if you must test it.

Version constraint strings are compared with semantic-version ordering, not text ordering: `1.10` is newer than `1.9`.

Meta-arguments

The arguments Terraform itself interprets on resource, data and module blocks, whatever the provider is. Notes record behaviour checked on Terraform 1.16.5.

Meta-arguments
Meta-argumentAllowed onEffectGotcha
`count`resource, module, dataCreates N instances addressed `[0]` to `[N-1]`; `count.index` is the position.Indexes shift when the list changes. The value must be known at plan time (`Invalid count argument`). With `count` set, `res.attr` is an error: use `res[0].attr`, `res[*].attr` or `one(res[*].attr)`.
`for_each`resource, module, dataCreates one instance per key of a map or per member of a set of strings, addressed `["key"]`; `each.key` and `each.value`.Mutually exclusive with `count`. A list is rejected (`value of type tuple`). Keys must be known at plan time and not sensitive; map values may be unknown.
`depends_on`resource, module, data (and `output`)Adds an explicit ordering edge to the dependency graph for dependencies Terraform cannot see in expressions.A data block that depends on an object with pending changes is read during apply, not plan (`will be read during apply`). Prefer a real reference when one exists.
`provider`resource, dataSelects a non-default provider configuration, for example `provider = random.b` for an aliased `provider "random"`.Modules do not accept `provider`; they take a `providers = { ... }` map in the module call instead. The reference is a static address, not an expression.
`lifecycle { create_before_destroy }`resourceOn replacement, creates the new object before destroying the old one.Needs a name or identity that can exist twice at once. The setting propagates to the resources a replaced object depends on.
`lifecycle { prevent_destroy }`resource (not data)Any plan that would destroy the instance fails with `Instance cannot be destroyed`.Deleting the whole resource block removes the setting with it: the plan then destroyed the object without error. `terraform destroy` is blocked while the block is present. Rejected on data blocks.
`lifecycle { ignore_changes }`resourceExcludes the listed attributes (or `all`) from the diff once the object exists.Hides drift and intended changes alike: with `ignore_changes = [input]` a new variable value produced no diff. Does not apply on creation.
`lifecycle { replace_triggered_by }`resourceReplaces this resource when a referenced resource (or one of its attributes) changes.The plan reads `will be replaced due to changes in replace_triggered_by`. Referenced resource instance changes cascade, so a count or for_each target can replace many. Terraform 1.2+.
`lifecycle { precondition }`resource, data (inside `lifecycle`); output (a bare `precondition` block)Checks an assumption before the object is planned or applied; failure stops with your `error_message`.The condition must refer to some other object in the configuration, otherwise: `The condition expression must refer to at least one object from elsewhere in the configuration`. Terraform 1.2+.
`lifecycle { postcondition }`resource, dataChecks the result after the object is read or applied; use `self` to inspect it.Fails the run with `Resource postcondition failed`, and dependents are not evaluated. On a data block it is the only way to assert on what was read. Terraform 1.2+.
`lifecycle` on a module callmodule (not supported)None: module blocks do not take `lifecycle`.A `lifecycle` block in a module call fails with `Reserved block type name in module block`. Put the rules on the resources inside the module.

`create_before_destroy` propagation and `provider` on data blocks follow the Terraform documentation and were not exercised here. `dynamic` blocks cannot generate `lifecycle` or `provisioner` blocks (verified).

count / for_each changes and their state impact

What the plan does when you change how a resource repeats, with the address before and after and the fix. Plans were produced with `terraform_data` and `triggers_replace`, so an attribute that changes with the item shows as a replacement; a plain in-place attribute would show as an update instead.

count / for_each changes and their state impact
ChangeAddress before -> afterWhat the plan doesFix
`count`: remove the middle of `[a,b,c]``item[1]`=b, `item[2]`=c -> `item[1]`=c`item[1] must be replaced`, `item[2] will be destroyed (because index [2] is out of range for count)`. `Plan: 1 to add, 0 to change, 2 to destroy`.Move to `for_each` keyed by a stable name, or accept the index shift for truly interchangeable copies.
`count`: reorder to `[c,b,a]``item[0]`=a -> c, `item[2]`=c -> a`item[0]` and `item[2]` must be replaced. `Plan: 2 to add, 0 to change, 2 to destroy`.Keep the list order stable (append only), or switch to `for_each`.
`count`: insert `x` at the front`item[0..2]` shift to `item[1..3]`All three existing instances must be replaced and `item[3]` is created. `Plan: 4 to add, 0 to change, 3 to destroy`.Append instead of inserting, or use `for_each`.
`count`: toggle `1` to `0` (`var.enabled`)`feature[0]` -> none`feature[0] will be destroyed (because index [0] is out of range for count)`; `one(feature[*].id)` becomes `null`.Reference it as `feature[*]` or `one(...)`, never `feature[0]`, so the `0` case does not break consumers.
`for_each`: remove key `b``item["b"]` -> noneOnly `item["b"] will be destroyed (because key ["b"] is not in for_each map)`. `Plan: 0 to add, 0 to change, 1 to destroy`.Intended. To keep the real object, forget it first with `terraform state rm 'terraform_data.item["b"]'`; a `removed` block cannot name a single instance (`Resource instance keys not allowed`, verified).
`for_each`: reorder keys`item["a"]`, `item["b"]`, `item["c"]` unchanged`No changes. Your infrastructure matches the configuration.`None needed.
`for_each`: rename key `b` to `bb``item["b"]` -> `item["bb"]`Destroy `item["b"]` and create `item["bb"]`. `Plan: 1 to add, 0 to change, 1 to destroy`.Add `moved { from = terraform_data.item["b"] to = terraform_data.item["bb"] }`: the plan says `has moved` and keeps the object, but attributes computed from the key (here `triggers_replace`) still change and can replace it.
No repetition -> `count = 1``x` -> `x[0]`Terraform maps it itself: `x has moved to x[0]`, `Plan: 0 to add, 0 to change, 0 to destroy`.No `moved` block needed (verified).
No repetition -> `for_each``x` -> `x["a"]``x will be destroyed (because resource uses count or for_each)` and `x["a"]` is created.Add `moved { from = terraform_data.x to = terraform_data.x["a"] }`: the plan then reports `has moved`.
`count` -> `for_each``item[0..2]` -> `item["a"]`, `item["b"]`, `item["c"]`Each `item[n]` is destroyed (`because resource does not use count`) and each `item["k"]` is created. `Plan: 3 to add, 0 to change, 3 to destroy`.One `moved` block per instance (Terraform 1.1+). The result is a plan with moves only: `Plan: 0 to add, 0 to change, 0 to destroy`, but `plan -detailed-exitcode` exits `2`.

The `for_each` rename was run with `moved` and showed `must be replaced (moved from ...)` because `triggers_replace` equals the key. A resource whose attributes do not depend on the key keeps its object and shows only the move.

Resource address syntax

The same address grammar is used by `terraform state`, `-target`, `-replace`, `taint`, and the `to`/`from` arguments of `import`, `moved` and `removed` blocks. Notes record behaviour checked on Terraform 1.16.5 with the random, local and terraform_data providers.

Resource address syntax
FormExampleRefers toWhere it works, and gotchas
Resource`random_pet.web`A resource without `count` or `for_each`: `TYPE.NAME`.Accepted by every command. `state show` prints it as `# random_pet.web:`. As the `from` of a `removed` block it must be this form.
Data source`data.local_file.in`A data source instance: `data.TYPE.NAME`.`state list` includes it and `state show data.local_file.in` works. `-replace` and `taint` apply to managed resources.
count instance`random_id.worker[0]`One instance of a resource that uses `count`, by number.Needed by `state show`, `state mv`, `taint` and `-replace`. After removing a middle item the numbers shift, which is why plans show replacements.
for_each instance`random_id.by_key["us-east-1"]`One instance of a resource that uses `for_each`, by string key in double quotes.Keys are quoted even when they look simple. In `moved` blocks: `from = random_id.worker[0]`, `to = random_id.worker["a"]`.
All instances`random_id.worker`Every instance of a counted or keyed resource: the address without an index.`state list` and `state rm` match all instances (`Removed random_id.worker[0]`, `Removed random_id.worker[5]`). `-target` matches all. `state show` fails with `No instance found for the given address!`, `taint` with `No such resource instance`, and `-replace` only warns `Incompletely-matched force-replace resource instance` and plans nothing.
Module`module.m`A whole module call, including everything below it.`state list module.m` lists its resources, `state rm module.m` removes them from state, and `removed { from = module.pet }` drops them. `-target=module.m` limits a plan to the module.
Resource in module`module.m.random_id.inner`A resource inside a module call: module path first, then the resource address.`moved { to = module.pet.random_pet.this }` moves a root resource into a module that declares it. `state mv random_pet.app module.m.random_pet.app` writes it without checking the module code.
Module with count`module.c[0].random_id.inner`A module call that uses `count`: the index follows the module name.A path such as `module.c[0].module.deep.random_id.leaf` is valid; all instances of the call match `module.c`.
Module with for_each`module.m["eu"].random_id.inner`A module call that uses `for_each`, keyed by string.`state list 'module.m["eu"]'` lists that instance only. Adding or removing a key creates or destroys that whole module instance.
Nested modules`module.m["eu"].module.deep.random_id.leaf`A chain of module calls, each with its own optional key, ending in a resource.Accepted by `state show`, `-replace` and `-target` (`-replace='module.m["eu"].module.deep.random_id.leaf'` planned exactly one replacement).
Shell quoting`'random_id.by_key["a"]'`Any address containing `[` or `"`: wrap it in single quotes in bash or zsh.Unquoted, the shell strips the double quotes and Terraform fails with `Error parsing instance address: module.m[eu].module.deep.random_id.leaf`. Quoting rules for PowerShell and cmd differ (not verified here).
Not an address`random_id.worker[*].id`Expression syntax: splats, `each.key`, `count.index` and `var.x` are for configuration only.Commands take addresses, not expressions. Use `terraform state list` to copy the exact spelling.

When unsure of the spelling, copy it from `terraform state list` output; that is exactly what the state commands expect.

terraform state and related commands

What each state-touching command reads or writes, and what it leaves behind. Observed on Terraform 1.16.5 with the local backend; remote backends add their own locking and versioning (not verified here).

terraform state and related commands
CommandEffectWrites state?Backup and safetyRisk
`terraform state list [ADDR] [-id=ID]`Prints one address per line; filters by module, resource or `id` attribute.NoRead only. Unknown resource: `Error: Unknown resource`.Low. Output is the spelling to use in other commands.
`terraform state show [-json] ADDR`Prints one instance's attributes; `-json` (new in 1.16.0) gives an object with `format_version` and a `resource` object (`address`, `values`).NoRead only. Needs one instance: `Error: No instance found for the given address!` otherwise.Low, but sensitive values can appear in plain text in the output.
`terraform state mv [-dry-run] SRC DEST`Renames or relocates addresses in state, including into a module.YesWrites `terraform.tfstate.<unix>.backup` each run; `-backup=PATH` redirects it; `-dry-run` prints `Would move ...`.Medium. The configuration is not changed, so the next plan can still destroy and create.
`terraform state rm [-dry-run] ADDR`Forgets instances; the real objects stay.YesSame backup behaviour. A resource address removes all its instances; `-dry-run` prints `Would remove ...`.Medium. Next plan wants to create the object again, and the unmanaged copy keeps existing.
`terraform state pull`Prints the current state as JSON (version 4 with `serial` and `lineage`) to stdout.NoRead only. Works on any backend, so `> snapshot.json` is a portable backup.Low. The file contains every attribute, including secrets, in plain text.
`terraform state push [-force] FILE`Replaces the state with a local file.YesRefuses a lower serial (`cannot import state with serial 19 over newer state with serial 20`) or a different lineage; `-force` overrides. No backup file was written.High. A stale file silently discards newer changes when forced.
`terraform state replace-provider [-auto-approve] FROM TO`Rewrites the provider source address on all matching resources.YesShows `~ Updating provider:` and `Changing N resources:` first and asks for confirmation; writes a backup.Medium. If the configuration still requires the old source, the next plan fails with `Error: Missing required provider`.
`terraform import ADDR ID`Adds an existing object to state at ADDR; needs the `resource` block first.YesWrites state immediately and leaves `terraform.tfstate.backup`. An already-managed address fails with `Error: Resource already managed by Terraform`.Medium. No plan or review; prefer an `import` block, which plans first.
`terraform apply -refresh-only`Updates state to match the real objects after a plan you confirm.YesNormal apply flow: plan, confirmation (or `-auto-approve`), and `terraform.tfstate.backup` is rewritten.Medium. Accepting drift can remove an object from state, as with a deleted `local_file`.
`terraform taint` / `untaint ADDR`Marks one instance so the next plan replaces it (`is tainted, so must be replaced`), or clears the mark.YesWrites state (backup behaviour not checked). Needs an instance address: a counted resource without an index fails with `No such resource instance`.Low to medium. The documentation recommends `-replace=ADDR` on plan and apply instead.
`terraform show -json`Prints the whole state (or a saved plan file) as JSON for tools.NoRead only. The alternative to scripting `state show` for every address.Low, with the same plain-text sensitivity as `state pull`.

The built-in help states that every state command that modifies state creates a timestamped backup. In 1.16.5, `-backup=-` did not suppress it for `state rm`, and `state push` wrote no file; keep your own `state pull` snapshot before risky work. `-state`, `-state-out` and `-backup` are documented as legacy local-backend options.

Fundamentals

Terraform does not run files top to bottom: it builds a graph from references and `depends_on`, then walks it with up to 10 concurrent operations and destroys in reverse order.

The problem

A team splits infrastructure across files and assumes the order of resources in the file, or the file name, controls the order of creation. It works until a resource relies on a side effect that no expression mentions, such as a file written by a provisioner or a permission that has to exist first, and apply fails on some runs and not on others because two independent resources were started at the same moment. Terraform has no script order: every resource is a node in a graph, and only references between nodes decide what waits for what.

How it works

  • The everyday loop is `terraform init` (install providers, configure the backend, fetch modules), `terraform fmt` and `terraform validate` (no API calls, no state needed), `terraform plan` (compute a diff) and `terraform apply` (execute it). `init` must be rerun after you add a provider, change the backend or add a module source.
  • All `*.tf` files in the working directory form one module and are merged, so file names and the order of blocks carry no meaning. Subdirectories are not read unless a `module` block points at them. A stray `bad.tf` next to `main.tf` is planned like any other file (verified).
  • Edges come from references. `content = "hello ${random_pet.name.id}"` makes `local_file.greeting` depend on `random_pet.name`, and `terraform graph` prints it as `"local_file.greeting" -> "random_pet.name"`: the arrow points from the dependent to its dependency (verified).
  • `depends_on = [terraform_data.uses_name]` adds an edge without reading any attribute. Use it only for hidden dependencies, because an explicit edge also forces data sources behind it to be read at apply time instead of plan time and makes the dependent wait for the whole target rather than one attribute.
  • Independent nodes run concurrently: `-parallelism=10` is the default. Three independent resources with `sleep 2` provisioners finished in 2.6 s by default and 6.6 s with `-parallelism=1` (verified). Because of this, the order of the `Creating...` lines for independent resources is not stable between runs.
  • Destroy walks the same graph in reverse: dependents go first. In the verified run `terraform_data.hidden` was destroyed before `terraform_data.uses_name`, and `random_pet.name` last, mirroring the create order backwards.
  • A reference loop cannot be walked, so it is rejected before any API call: `terraform validate` fails with `Error: Cycle:` followed by the addresses involved (verified with two `terraform_data` resources that reference each other).
  • `terraform graph` prints Graphviz DOT, by default for the plan walk. Pipe it to `dot -Tsvg` (Graphviz, not run here) for a picture; `-type=plan-destroy` shows the destroy walk. The graph also contains providers, variables, locals and outputs, not only resources.

Examples

One real edge, three independent nodes: the graph and what -parallelism does to wall-clock time.

bash
mkdir graph-demo && cd graph-demo
cat > main.tf <<'HCL'
resource "random_pet" "name" {
  length = 2
}

resource "local_file" "greeting" {
  filename = "greeting.txt"
  content  = "hello ${random_pet.name.id}\n"
}

resource "terraform_data" "a" {
  provisioner "local-exec" {
    command = "sleep 2"
  }
}

resource "terraform_data" "b" {
  provisioner "local-exec" {
    command = "sleep 2"
  }
}

resource "terraform_data" "c" {
  provisioner "local-exec" {
    command = "sleep 2"
  }
}
HCL

terraform init
terraform fmt -check
terraform validate

# the reference random_pet.name.id is the only edge; a, b and c are independent
terraform graph | grep -- '->'

# independent nodes run concurrently (default -parallelism=10): about 2 s, not 6 s
time terraform apply -auto-approve
terraform destroy -auto-approve
time terraform apply -auto-approve -parallelism=1

Implicit versus explicit edges, and destroy running the create order backwards.

bash
mkdir order-demo && cd order-demo
cat > main.tf <<'HCL'
resource "random_pet" "name" {}

resource "terraform_data" "uses_name" {
  input = random_pet.name.id # implicit edge: reads an attribute
}

resource "terraform_data" "hidden" {
  depends_on = [terraform_data.uses_name] # explicit edge: no attribute is read
}
HCL

terraform init > /dev/null
terraform graph | grep -- '->'
terraform apply -auto-approve | grep -E 'Creating|Apply'
terraform destroy -auto-approve | grep -E 'Destroying|Destroy complete'

In practice

  • Terraform CLI — Builds the graph in the core, independent of providers; providers only implement per-resource create, read, update and delete.
  • Graphviz — Renders the DOT output of `terraform graph` into SVG or PNG for reviews and onboarding diagrams.
  • Terraform Registry modules — A module call is a node too; outputs referenced by other modules create cross-module edges.
  • Atlantis — Runs plain `init`, `plan` and `apply` per directory and workspace, so the same graph rules apply to pull-request automation.
  • Make — The comparable mental model: targets and prerequisites, not a script; unlike Make, Terraform derives most prerequisites from expressions.

Trade-offs

Pros

  • Order is derived from data flow, so refactoring files or moving blocks never changes behaviour
  • Independent resources are created in parallel without any extra configuration
  • Cycles and unknown references are caught by `validate` before anything is touched
  • `terraform graph` makes the real dependencies inspectable

Cons

  • Hidden dependencies (timing, permissions, side effects) are invisible to the graph and need `depends_on` or a data flow workaround
  • Parallelism can exceed API rate limits and surfaces as intermittent throttling errors
  • A broad `depends_on` serialises work and defers data reads to apply time
  • Apply logs are not in a stable order, which makes diffing two runs noisy

The state file maps each resource address to the real object it created and records attributes, a `serial` and a `lineage`; Terraform cannot plan without it, and it holds secrets in plain text.

The problem

Someone deletes `terraform.tfstate` because it is "just generated", or commits it to git with the database password inside. In the first case the next `plan` proposes to create everything again, because Terraform has no record that those objects already exist. In the second the password is readable by everyone with repository access, even though the output was marked `sensitive`. The state file is neither a cache nor a log: it is the only link between configuration addresses and real objects.

How it works

  • State maps `resource address -> real object`. Each instance stores the full attribute set the provider returned (including `id`), the provider that owns it and its `schema_version`. Plan compares configuration with this record (after refreshing it) and never discovers objects by name or tag.
  • The local file is JSON with a top-level `version` (4 in 1.16.5), `terraform_version`, `serial`, `lineage`, `outputs`, `resources` and `check_results` (verified). `version` is the file format, not the Terraform release.
  • `serial` increases with every write of a changed state, and `lineage` is a UUID fixed when the state is first created. `terraform state push` uses them to refuse an overwrite: pushing an older file failed with `cannot import state with serial 1 over newer state with serial 4`, and a different lineage with `cannot import state with lineage "other" over unrelated state ...` (verified on the local backend; remote backends typically apply the same checks). In the verified run a second apply moved `serial` from 3 to 5, and a no-change apply did not write at all (serial stayed 3).
  • Before overwriting an existing state, the local backend copies the previous one to `terraform.tfstate.backup`. The very first write has no predecessor, so no backup exists yet (verified: only `terraform.tfstate` after the first apply; the backup showed `serial` 3 against 5 after the second).
  • Sensitive values are hidden only in CLI output. `terraform output` prints `db_password = <sensitive>`, but `terraform output db_password` and `terraform output -raw db_password` print the value, and the state JSON holds `"result": "..."` in plain text (verified). Treat the state, and any saved plan file, as a secret.
  • Read state through commands: `terraform state list` (addresses), `terraform state show <addr>` (attributes of one instance), `terraform state pull` (raw JSON of the current workspace), `terraform show -json` (the documented machine format, with its own `format_version`). Hand-editing the JSON bypasses the serial and lineage checks.
  • Plan refreshes state first by reading each real object. A deleted object is dropped from the working state, so the plan proposes to recreate it; `terraform apply -refresh-only` records that fact in state without changing anything else, after which `terraform state list` no longer shows the resource (verified).
  • Each workspace has its own state. On the local backend the `default` workspace stays in `terraform.tfstate` and named workspaces live in `terraform.tfstate.d/<name>/` (verified); other backends use their own layout.

Examples

The fields that guard the file, plain-text secrets, and when the .backup file appears.

bash
mkdir state-demo && cd state-demo
cat > main.tf <<'HCL'
resource "random_pet" "name" {}

resource "random_password" "db" {
  length = 12
}

output "db_password" {
  value     = random_password.db.result
  sensitive = true
}
HCL

terraform init > /dev/null
terraform apply -auto-approve > /dev/null

# the three fields that guard the file
grep -E '^  "(version|serial|lineage)"' terraform.tfstate

# mapping from address to real object, readable without opening the JSON
terraform state list
terraform state show random_pet.name

# "sensitive" only hides values in CLI output; the state keeps them in plain text
terraform output db_password
terraform output -raw db_password; echo
grep '"result"' terraform.tfstate

# the first write creates no backup; every later change keeps the previous version
ls terraform.tfstate*
cat >> main.tf <<'HCL'

resource "random_id" "extra" {
  byte_length = 2
}
HCL
terraform apply -auto-approve > /dev/null
ls terraform.tfstate*
grep -E '^  "serial"' terraform.tfstate terraform.tfstate.backup

Reading state through supported commands, and one state file per workspace.

bash
mkdir state-read-demo && cd state-read-demo
cat > main.tf <<'HCL'
resource "random_pet" "name" {}

output "pet" {
  value = random_pet.name.id
}
HCL

terraform init > /dev/null
terraform apply -auto-approve > /dev/null

# read state through supported commands, not by parsing the file
terraform state pull | grep -E '^  "(version|terraform_version)"'
terraform show -json | grep -o '"format_version":"[0-9.]*"'
terraform output -json pet

# each workspace has its own state file; the default one stays at terraform.tfstate
terraform workspace new staging > /dev/null
terraform apply -auto-approve > /dev/null
ls terraform.tfstate.d/staging
terraform workspace select default > /dev/null
terraform state list

In practice

  • Terraform CLI — Local backend writes `terraform.tfstate` and `terraform.tfstate.backup`; `terraform_version` is recorded in the file and `version` was 4 in the checked 1.16.5 release.
  • Amazon S3 with versioning — A common remote home for state; the docs recommend enabling bucket versioning so earlier state objects can be recovered.
  • HCP Terraform — Hosts state per workspace with history and access control, so the state file never sits on laptops.
  • git-secrets and gitleaks — Secret scanners that catch a committed `terraform.tfstate` containing credentials.
  • OpenTofu — Documents built-in encryption of state and plan files at rest (opentofu.org, `encryption` block; not run here); Terraform relies on backend-side encryption.

Trade-offs

Pros

  • Exact mapping to real objects, so plans are fast and do not guess by name
  • Stored attributes allow diffs without calling every API for every plan field
  • Serial and lineage make lost updates and wrong-file mistakes detectable on remote backends
  • Machine-readable output via `show -json` feeds policy and cost tools

Cons

  • Contains secrets in plain text and needs access control plus encryption at rest
  • Losing or corrupting it leaves orphaned real objects that Terraform will try to recreate
  • One shared file per workspace is a serialisation point (locking) for every change
  • Hand edits are easy to get wrong; supported commands cover most needs

A backend decides where state lives and whether it is locked during writes: the local backend uses an OS file lock, remote backends add their own locks, and `-lock=false`, `-lock-timeout`, `-backend-config` and `init -migrate-state` control the edges.

The problem

Two engineers run `terraform apply` against the same state on the same afternoon, or a CI job and a laptop do. Without a lock, both read serial 12, both write their own result, and one change silently disappears from state while the real objects exist. With a lock the second run stops with `Error acquiring the state lock`, but then someone's laptop dies mid-apply and nobody knows whether it is safe to force the lock open. Knowing what a backend can and cannot guarantee decides how a team stores state.

How it works

  • A backend is chosen in a `terraform { backend "<type>" { ... } }` block; at most one is allowed, and without it Terraform uses the `local` backend and `terraform.tfstate`. The block cannot use variables or other named values: `path = var.p` fails at `init` with `Error: Variables not allowed` (verified), which is why partial configuration exists.
  • The local backend locks with an operating-system file lock on the state file and writes a hint file `.terraform.tfstate.lock.info` holding the holder's `ID`, `Operation`, `Who`, `Version`, `Created` and `Path` (verified). The hint file is removed when the lock is released. Contention fails immediately: a `terraform plan` during a running apply exited 1 with `Error: Error acquiring the state lock`, `Error message: resource temporarily unavailable` and a `Lock Info:` block listing `ID`, `Path`, `Operation: OperationTypeApply`, `Who`, `Version` and `Created` (verified).
  • `-lock-timeout=30s` makes a command print `Acquiring state lock. This may take a few moments...` and keep retrying until the holder releases or the timeout passes; the verified run waited out a 15 s apply and then planned normally. The default is `0s`, so no waiting. `-lock=false` skips locking completely: in the verified run it let a `plan` read state while an apply held the lock, which can show half-written data and must never be used for `apply`.
  • A killed process does not leave a stale local lock. After `kill -9` of an apply the `.lock.info` file remained, but the OS lock was gone, so the next `plan` ran and the file disappeared (verified). `terraform force-unlock -force <ID>` on a local backend only answers `Failed to unlock state: LocalState not locked` (verified).
  • Remote backends hold locks as remote records (a lease, an object, a table item) and these can outlive a dead client; that is the case `terraform force-unlock <LOCK_ID>` exists for, using the ID printed in the error. Confirm the holder is really gone first: unlocking under a live apply invites exactly the lost update the lock prevents.
  • Partial configuration keeps environment-specific values out of the `.tf` files: write `backend "local" {}` and pass `-backend-config=dev.local.tfbackend` (a file of `key = "value"` lines) or `-backend-config="path=state/dev.tfstate"` at `init`. The resolved settings are cached in plain text in `.terraform/terraform.tfstate`, so do not pass credentials this way if that directory is shared (verified path and contents).
  • Changing a backend is an explicit act. A plain `init` asks `Do you want to copy existing state to the new backend?` and with no terminal fails with `Error asking for confirmation: EOF`. `-migrate-state` (add `-force-copy` to skip the question) copies state to the new place; `-reconfigure` switches without copying, so `terraform state list` afterwards printed `No state file was found!` (all verified). Changing the settings of an already initialized backend without either flag gives `Error: Backend configuration changed`.
  • Remote backends from the documentation (not run here, so verify against your version): S3 takes `bucket`, `key`, `region` and locks either through an object `<key>.tflock` with `use_lockfile = true` (S3-native locking was added in 1.10; 1.11 deprecated the DynamoDB arguments in its favour) or through the deprecated `dynamodb_table`; GCS stores one object per workspace as `<prefix>/<name>.tfstate`; `azurerm` stores a blob in a storage account container with native locking; the `cloud` block uses HCP Terraform. Docs for S3 and GCS recommend enabling object versioning.

Examples

A second command hits the local lock: the error, the hint file, -lock=false and -lock-timeout.

bash
mkdir lock-demo && cd lock-demo
cat > main.tf <<'HCL'
resource "terraform_data" "slow" {
  provisioner "local-exec" {
    command = "sleep 15"
  }
}
HCL

terraform init > /dev/null

# terminal 1 (simulated): a long apply holds the lock
terraform apply -auto-approve > apply.log 2>&1 &
sleep 3

# terminal 2: any command that locks fails at once with exit code 1
terraform plan 2>&1 | grep -E 'Error acquiring|Error message|Operation:|Path:'
echo "exit=${PIPESTATUS[0]}"

# the local lock is an OS file lock plus a hint file with the holder
cat .terraform.tfstate.lock.info; echo

# -lock=false skips locking: the plan runs, and can read state while it is being written
terraform plan -lock=false | grep -E '^Plan:'

# wait for the lock instead of failing (retries until the timeout)
terraform plan -lock-timeout=30s | grep -E 'Acquiring|^No changes'

wait
ls -a | grep lock.info || echo "lock info file removed after the apply finished"

Partial backend configuration, -migrate-state, and -reconfigure leaving the state behind.

bash
mkdir backend-demo && cd backend-demo
cat > main.tf <<'HCL'
resource "terraform_data" "x" {
  input = "demo"
}
HCL

# 1. start with the implicit local backend
terraform init > /dev/null
terraform apply -auto-approve > /dev/null
ls

# 2. declare a backend with no arguments: values arrive at init time
cat > backend.tf <<'HCL'
terraform {
  backend "local" {}
}
HCL
echo 'path = "state/dev.tfstate"' > dev.local.tfbackend

# a plain init would ask "Do you want to copy existing state to the new backend?"; with no terminal it fails
terraform init -backend-config=dev.local.tfbackend 2>&1 | grep -E '^Error'

# 3. -migrate-state copies the existing state into the new location
terraform init -backend-config=dev.local.tfbackend -migrate-state -force-copy > /dev/null
ls state
terraform plan -detailed-exitcode > /dev/null; echo "plan exit=$?"

# the chosen settings are cached here, not in your .tf files
grep -A3 '"config"' .terraform/terraform.tfstate

# -reconfigure points at a different location WITHOUT copying state
terraform init -backend-config="path=state/other.tfstate" -reconfigure > /dev/null
terraform state list 2>&1 | head -1 | sed 's/\x1b\[[0-9;]*m//g'

In practice

  • Amazon S3 — The most common remote backend; docs describe lock objects named `<key>.tflock` via `use_lockfile` and recommend bucket versioning (not run here).
  • Google Cloud Storage — Stores one `<prefix>/<workspace>.tfstate` object per workspace and supports customer-supplied or Cloud KMS keys per the docs (not run here).
  • Azure Blob Storage — The `azurerm` backend stores state in a container blob with the platform's native locking per the docs (not run here).
  • HCP Terraform — The `cloud` block stores and locks state in a workspace and can run plan and apply remotely.
  • OpenTofu — Typically uses the same backend block syntax; its S3 backend documents `use_lockfile` and still supports `dynamodb_table` (no deprecation planned per its docs). Details of newer features may differ (not run here).

Trade-offs

Pros

  • A lock turns concurrent applies into a clear error instead of a lost update
  • `-lock-timeout` lets CI queue behind a running job without custom scripting
  • Partial configuration keeps one codebase usable across environments
  • Remote backends with versioning give recovery points for state

Cons

  • Remote locks can go stale after a crash and need a human decision to force-unlock
  • `-lock=false` and `force-unlock` remove the only protection against lost updates
  • Backend settings cannot use variables, so initialisation scripts carry `-backend-config` arguments
  • Backend-specific features (lock mechanism, encryption) differ and are only partly covered by the common docs

Providers are plugins downloaded by `init`; `required_providers` and `required_version` constrain what is acceptable, and `.terraform.lock.hcl` pins the exact provider build and checksums until `init -upgrade`.

The problem

A pipeline that worked on Friday fails on Monday: a new minor release of a provider changed a default, and the plan now wants to replace a database. Nobody changed the code, because `version = ">= 3.0"` accepted whatever was newest at `init` time on a fresh CI runner. Separately, a developer on an Apple laptop commits a lock file with checksums for macOS only, and Linux CI refuses to install the provider. Version pinning in Terraform is two files with different jobs.

How it works

  • A provider is a separate plugin binary, addressed by a source such as `hashicorp/random` (full form `registry.terraform.io/hashicorp/random`). `required_providers` maps a local name to a `source` and a `version` constraint; `terraform init` downloads the matching build into `.terraform/providers` (verified: `Installing hashicorp/random v3.5.1...`).
  • `version` takes comma-separated constraints with `=`, `!=`, `>`, `>=`, `<`, `<=` and `~>`. `~> 3.5.0` means `>= 3.5.0, < 3.6.0` and `~> 3.5` means `>= 3.5, < 4.0`; with `~> 3.5.0` `init` selected 3.5.1 (verified). See the version-constraints table for the whole list.
  • `.terraform.lock.hcl` records for each provider the exact `version`, the `constraints` that were in force when it was selected, and a set of `hashes` (`h1:` and `zh:` entries) (verified). It is created by `init`, is meant to be committed, and applies to providers only: modules and the Terraform CLI itself are not locked.
  • While a locked version still satisfies the constraint, `init` reuses it: after loosening `~> 3.5.0` to `~> 3.5` the output said `Reusing previous version of hashicorp/random from the dependency lock file` and the lock stayed at 3.5.1, with the old `constraints` text unchanged (verified).
  • When the lock and the constraint disagree, `init` stops: with the lock at 3.5.1 and the constraint `~> 3.6` it printed `Error: Failed to query available provider packages` ... `locked provider registry.terraform.io/hashicorp/random 3.5.1 does not match configured version constraint ~> 3.6; must use terraform init -upgrade to allow selection of new versions` (verified). `terraform init -upgrade` then installed 3.9.1 and rewrote `version` and `constraints` in the lock file.
  • Hashes cover platforms. `terraform providers lock -platform=linux_amd64 -platform=darwin_arm64 -platform=windows_amd64` fetched the checksums for each, and the file ended up with 3 `h1:` entries (verified). Without that, a lock file created on one platform typically lacks the `h1:` hash another platform needs and the install there fails the checksum check.
  • `required_version` constrains the Terraform CLI. On 1.16.5 `~> 1.16.0`, `~> 1.15` and `= 1.16.5` were accepted; `~> 1.17`, `>= 1.5, < 1.16` and `!= 1.16.5` were rejected with `Error: Unsupported Terraform Core version` and `This configuration does not support Terraform version 1.16.5` (verified).
  • Credentials and settings do not belong in `required_providers`. A `provider "<name>" { ... }` block (or environment variables the provider reads) configures an instance, and `terraform providers` shows which module requires which provider and constraint (verified output: `provider[registry.terraform.io/hashicorp/random] ~> 3.6`). `TF_PLUGIN_CACHE_DIR` shares downloads between directories; several `init` runs at once into the same cache failed once with `text file busy` while I tested (observed, not guaranteed).

Examples

Lock file lifecycle: install, reuse, mismatch error, -upgrade, and multi-platform checksums.

bash
mkdir providers-demo && cd providers-demo
cat > main.tf <<'HCL'
terraform {
  required_version = ">= 1.5, < 2.0"

  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 3.5.0"
    }
  }
}

resource "random_pet" "p" {}
HCL

terraform init | grep -E 'Installing|Installed'
grep -E 'version|constraints' .terraform.lock.hcl
terraform version | grep provider

# loosen the constraint: the lock file still wins until you ask for an upgrade
sed -i 's/~> 3.5.0/~> 3.5/' main.tf
terraform init | grep -E 'Reusing|Installing'
grep -E 'version|constraints' .terraform.lock.hcl

# an incompatible constraint fails, and says how to proceed
sed -i 's/~> 3.5"/~> 3.6"/' main.tf
terraform init 2>&1 | grep -E 'does not match|init -upgrade' || true
terraform init -upgrade | grep -E 'Installed'
grep -E 'version|constraints' .terraform.lock.hcl

# record checksums for the platforms your teammates and CI run on
terraform providers lock -platform=linux_amd64 -platform=darwin_arm64 -platform=windows_amd64 | grep -E 'Obtained|Success'
grep -c 'h1:' .terraform.lock.hcl

required_version checked against the running CLI, with the rejection message.

bash
mkdir constraint-demo && cd constraint-demo
terraform version | head -1

# required_version gates the CLI itself; check how each constraint treats this CLI
for c in '~> 1.16.0' '~> 1.15' '~> 1.17' '>= 1.5, < 1.16' '!= 1.16.5' '= 1.16.5'; do
  printf 'terraform {\n  required_version = "%s"\n}\n' "$c" > main.tf
  if terraform validate > /dev/null 2>&1; then
    echo "ok       $c"
  else
    echo "rejected $c"
  fi
done

# the error names the file, line and the running version
printf 'terraform {\n  required_version = "~> 1.17"\n}\n' > main.tf
terraform validate 2>&1 | grep -E '^Error|required_version|does not support'

In practice

  • Terraform Registry — Default source for `hashicorp/*` and community providers; `init` queries it for versions and checksums.
  • Dependabot and Renovate — Open pull requests that bump provider constraints and refresh `.terraform.lock.hcl`, turning upgrades into reviewed diffs.
  • Terraform CLI — `terraform providers mirror` and `network_mirror` or `filesystem_mirror` in the CLI config serve providers in air-gapped networks.
  • tfenv and mise — Version managers that select a CLI release per directory (tfenv reads `.terraform-version`), so the CLI can match `required_version`.
  • OpenTofu — Typically resolves providers from its own registry and also writes `.terraform.lock.hcl` (both per opentofu.org docs); registry and hash details may differ (not run here).

Trade-offs

Pros

  • Committed lock file gives every machine the same provider build until someone upgrades on purpose
  • Checksums let `init` detect a tampered or swapped plugin package
  • `required_version` turns "works on my CLI" into an explicit error
  • Pessimistic `~>` constraints allow patch fixes while blocking surprises in the next minor line

Cons

  • Lock files lack hashes for platforms nobody ran `providers lock` for, which breaks CI on another OS or architecture
  • Upgrades are manual: staleness and missed security fixes are your job to track
  • Provider releases can change defaults inside an allowed range, so plans still deserve review
  • Modules and the CLI have no lock file, so they need explicit pins

Plan refreshes state, diffs it against configuration and shows `+ - ~ -/+ <=` actions; apply executes a plan, and a saved plan file pins exactly what runs and fails with `Saved plan is stale` if state moved.

The problem

A reviewer approves a pull request after reading `terraform plan` output in the CI log, and the deploy job then runs `terraform apply` by itself. Between the two, someone changed a resource by hand, and the apply does something the reviewer never saw: it replaces a database instead of updating a tag. A plan is a computation made at one moment; what actually runs depends on whether the plan is saved and on how the later command is invoked.

How it works

  • `plan` has three steps: refresh (read each real object and update the in-memory state), diff (compare configuration with that state, provider by provider) and order (use the dependency graph). `apply` without a plan file runs the same computation and asks for confirmation; `-auto-approve` skips the question.
  • Symbols seen in the verified runs: `+` create, `-` destroy, `~ update in-place`, `-/+ destroy and then create replacement` and `+/- create replacement and then destroy` (the latter with `create_before_destroy = true`), and `<=` for a data resource read during apply. The summary is `Plan: 1 to add, 1 to change, 1 to destroy.` or `No changes. Your infrastructure matches the configuration.`
  • Replacement is explained in the diff: the resource header says `must be replaced` and the attribute that caused it carries `# forces replacement` (verified for `local_file.content` and `random_pet.length`). `-replace=ADDR` forces it on purpose and labels it `will be replaced, as requested`. Attributes not yet computed show as `(known after apply)`. A data source whose arguments are known is read during plan (`data.local_file.cfg: Reading...`); if its arguments depend on something that does not exist yet, or it has `depends_on` on a changing resource, the read is deferred and the plan shows `<= read (data resources)` and `will be read during apply` (verified).
  • `plan -detailed-exitcode` returns 0 for an empty plan, 1 for an error and 2 when changes are present (verified: first plan 2, after apply 0). CI uses it to detect drift or to gate a manual approval step.
  • `plan -out=tfplan` writes a binary file that is a zip holding the plan, the state at plan time (`tfstate`, `tfstate-prev`), the configuration (`tfconfig/`) and the lock file (verified). `terraform apply tfplan` applies exactly that, with no prompt and no new plan. If state changed since, it exits 1 with `Error: Saved plan is stale` (verified). The file contains sensitive values, so treat it like state.
  • A saved plan fixes the inputs: `terraform apply -var env=x t3` was rejected on 1.16.5 (`Error: Can't change variable when applying a saved plan` when the variable is declared and the value differs from the saved one; `Error: Value for undeclared variable` when the configuration does not declare it). Pass variables when planning. `terraform show -json tfplan` gives `resource_changes[].change.actions` such as `["update"]` and `["delete","create"]` for policy checks (verified).
  • Narrowing flags change what plan considers. `-refresh-only` records drift in state without changing infrastructure (verified: a deleted file showed `has been deleted`, and after applying it `terraform state list` was empty). `-refresh=false` skips the read step. `-target=ADDR` limits the plan and always prints `Warning: Resource targeting is in effect` and, on apply, `Warning: Applied changes may be incomplete` (verified). `plan -destroy` previews `terraform destroy`.
  • Apply is not atomic. When one resource fails, the ones already created stay in state: after a `local-exec` provisioner exited 3, `random_pet.ok` was in state, `terraform_data.boom` was recorded too, and the next plan said `terraform_data.boom is tainted, so must be replaced` (verified). Fix the cause and run again instead of editing state.

Examples

terraform plan with exit codes, a saved plan, the symbols for update and replace, and a stale plan.

bash
mkdir plan-demo && cd plan-demo
cat > main.tf <<'HCL'
variable "tag" {
  type    = string
  default = "v1"
}

resource "terraform_data" "app" {
  input = var.tag
}

resource "terraform_data" "worker" {
  triggers_replace = var.tag
}
HCL

terraform init > /dev/null

# exit codes with -detailed-exitcode: 0 = empty plan, 1 = error, 2 = changes present
terraform plan -detailed-exitcode > /dev/null || echo "first plan exit=$?"

# save the plan, then apply exactly that plan (no prompt, no new plan)
terraform plan -out=tfplan | grep -E '^Plan:'
terraform show tfplan | grep -E 'will be created'
terraform apply tfplan | grep -E '^Apply complete'
terraform plan -detailed-exitcode > /dev/null && echo "after apply exit=$?"

# change the input: one update in place (~), one replacement (-/+)
terraform plan -var tag=v2 | grep -E '^  # |^Plan:|forces replacement|^  ~ |^-/\+'

# a saved plan is only valid for the state it was made from
terraform plan -var tag=v2 -out=old.tfplan > /dev/null
terraform apply -auto-approve -var tag=v3 > /dev/null
terraform apply old.tfplan 2>&1 | grep -E '^Error'; echo "stale apply exit=${PIPESTATUS[0]}"

# the plan file is JSON-inspectable for policy checks
terraform plan -var tag=v4 -out=new.tfplan > /dev/null
terraform show -json new.tfplan | grep -o '"actions":\[[^]]*\]'

Drift: -refresh-only records a deleted object in state, and the next plan proposes to recreate it.

bash
mkdir drift-demo && cd drift-demo
cat > main.tf <<'HCL'
resource "local_file" "note" {
  filename = "note.txt"
  content  = "managed by terraform\n"
}
HCL

terraform init > /dev/null
terraform apply -auto-approve > /dev/null
terraform state list

# someone deletes the real object behind Terraform's back
rm note.txt

# state still claims it exists until a refresh reads reality
terraform state list
terraform plan -refresh-only | grep -E 'has been deleted|Objects have changed'

# accept reality into state only: no resource is created or destroyed
terraform apply -refresh-only -auto-approve | grep -E 'Apply complete'
terraform state list || true

# a normal plan now proposes to create the file again
terraform plan | grep -E '^Plan:'

In practice

  • Atlantis — Comments the plan on the pull request and runs `atlantis apply` against the saved plan file, so the reviewed plan is what runs.
  • GitHub Actions with setup-terraform — Typical pipeline: `plan -out=tfplan -detailed-exitcode`, upload the plan as an artifact, then `apply tfplan` in a gated job.
  • HCP Terraform — Runs plan remotely, stores it, and applies the same plan after approval.
  • Open Policy Agent (conftest) — Evaluates `terraform show -json tfplan` and fails the pipeline on forbidden actions such as deleting a database.
  • Infracost — Reads the JSON plan to estimate the cost difference before apply.

Trade-offs

Pros

  • Reviewing a plan shows creates, in-place updates and replacements before anything runs
  • A saved plan makes the executed change identical to the reviewed one, and detects state movement
  • `-detailed-exitcode` and `show -json` make plans machine-checkable
  • `-refresh-only` separates recording drift from changing infrastructure

Cons

  • A plan is a snapshot: unsaved plans can differ from what a later apply does
  • Saved plan files hold sensitive values and tie to one state version and one set of variables
  • `-target` and `-refresh=false` give partial pictures and are easy to leave in scripts
  • Failed applies leave partial state, including tainted resources, which needs follow-up

Language

How input variables are typed, validated and layered (tfvars, -var, TF_VAR_), what sensitive really hides, and how locals and outputs wire modules together.

The problem

A module takes `env` as a free string, someone passes `prod ` with a trailing space, and the plan quietly builds resources named `prod -app`. Another team marks a database password `sensitive = true`, sees `(sensitive value)` everywhere in the plan, and concludes it is safe, then finds it in clear text in `terraform.tfstate` and in `terraform output -raw`. Input variables, locals and outputs look like plumbing, but they are where types are enforced, where values are layered, and where secrets leak.

How it works

  • A `variable` block declares an input of a module: `type`, `default`, `description`, `sensitive`, `nullable` and `validation`. A variable with no default and no supplied value fails in non-interactive runs with `Error: No value for required variable` (verified with `-input=false`); interactively Terraform prompts instead.
  • Values are layered, and later sources win: `TF_VAR_name` environment variables, then `terraform.tfvars` (and `.json`), then `*.auto.tfvars` files in lexical order, then `-var` and `-var-file` in command-line order. Verified: with `TF_VAR_env=prod` and `terraform.tfvars` setting `staging` the plan used `staging`; adding `override.auto.tfvars` with `prod` made it `prod`; `-var env=dev` beat all of them.
  • A type constraint is checked and converted on input. Object types can mark attributes `optional(type, default)` (since Terraform 1.3), and a caller that sets only `{ name = "web" }` gets `replicas = 2` and `ports = [80]` filled in (verified in the plan output). Without `optional()`, a missing attribute is a type error.
  • `validation` blocks run an arbitrary `condition` and print `error_message` on failure, together with `This was checked by the validation rule at main.tf:4,3-13`. Since Terraform 1.9 a condition may refer to other variables and data sources; before that it could only mention the variable being validated. Use `can(...)` to turn a failing function call (for example `cidrhost`) into `false` instead of an error.
  • Validation is not the first check. A root output that uses a sensitive value without `sensitive = true` stops the plan with `Error: Output refers to sensitive values` before variable validation runs (verified), so a validation failure can be hidden behind it until the output is fixed.
  • `sensitive = true` only redacts display: `(sensitive value)` in plans, `<sensitive>` in `terraform output`. The value stays in clear text in state (`grep -c` found it on three lines of `terraform.tfstate`), and `terraform output -raw conn` and `terraform output -json` print it unredacted. Since Terraform 1.10 a variable, or an output of a child module (not a root output), can instead be `ephemeral = true`, which keeps it out of plan and state files (not run here; see the output block reference).
  • `locals` are named expressions with no input of their own: they cannot be overridden by the caller, are evaluated as part of the dependency graph, and give a computed value a single definition (`local.name_prefix`). Anything that references a local inherits the dependencies of the expressions inside it.
  • Outputs are the only way a child module exposes values to its caller and the only values `terraform output` and `terraform_remote_state` can read from a root module. Outputs can carry `description`, `sensitive`, `depends_on` and `precondition`. They are recorded in state, so a root output holding a secret copies that secret into every consumer's state too.

Examples

A validated variable, a sensitive value flowing through a local into an output, and what `terraform output` and the state file still reveal.

bash
mkdir -p demo-vars && cd demo-vars
cat > main.tf <<'HCL'
variable "env" {
  type        = string
  description = "Deployment environment"
  validation {
    condition     = contains(["dev", "staging", "prod"], var.env)
    error_message = "env must be one of dev, staging, prod."
  }
}

variable "db_password" {
  type      = string
  sensitive = true
  default   = "hunter2"
}

locals {
  name_prefix = "${var.env}-app"
  conn        = "postgres://app:${var.db_password}@db/${local.name_prefix}"
}

resource "terraform_data" "db" {
  input = local.conn
}

output "name_prefix" {
  value = local.name_prefix
}

output "conn" {
  value     = local.conn
  sensitive = true
}
HCL

terraform init
terraform plan -var env=qa        # fails: Invalid value for variable
terraform apply -auto-approve -var env=dev

terraform output                  # conn = <sensitive>
terraform output -raw conn        # prints the real value
grep -c hunter2 terraform.tfstate # the state file still holds it in clear text

An object type with `optional()` defaults, and the precedence of `TF_VAR_`, `terraform.tfvars`, `*.auto.tfvars` and `-var`.

bash
mkdir -p demo-types && cd demo-types
cat > main.tf <<'HCL'
variable "env" {
  type    = string
  default = "dev"
}

variable "service" {
  type = object({
    name     = string
    replicas = optional(number, 2)
    ports    = optional(list(number), [80])
  })
}

output "env" {
  value = var.env
}

output "service" {
  value = var.service
}
HCL

cat > service.auto.tfvars <<'TFVARS'
service = { name = "web" }
TFVARS

terraform init
terraform plan | grep -A6 'service = '                # optional() defaults filled in
echo 'env = "staging"' > terraform.tfvars
TF_VAR_env=prod terraform plan | grep 'env  *='       # terraform.tfvars beats TF_VAR_env
echo 'env = "prod"' > override.auto.tfvars
terraform plan | grep 'env  *='                       # *.auto.tfvars beats terraform.tfvars
terraform plan -var env=dev | grep 'env  *='          # -var beats everything

Cross-variable validation (Terraform 1.9+) and the order in which the sensitive-output check and validation fire.

bash
mkdir -p demo-guard && cd demo-guard
cat > main.tf <<'HCL'
variable "min_nodes" {
  type    = number
  default = 3
}

variable "max_nodes" {
  type    = number
  default = 2
  validation {
    # Referring to another variable needs Terraform 1.9 or later
    condition     = var.max_nodes >= var.min_nodes
    error_message = "max_nodes must be >= min_nodes."
  }
}

variable "token" {
  type      = string
  default   = "abc123"
  sensitive = true
}

output "token_hint" {
  value = "${substr(var.token, 0, 2)}****"
}
HCL

terraform init
terraform plan                    # Error: Output refers to sensitive values (checked first)
sed -i 's/^output "token_hint" {/&\n  sensitive = true/' main.tf
terraform plan                    # now: Invalid value for variable (max_nodes)
terraform plan -var max_nodes=5   # passes; token_hint = (sensitive value)

In practice

  • Terraform CLI — Implements the layering, type conversion and validation described here; the sensitive-output check is a static error raised before planning completes.
  • terraform-docs — Generates input and output tables for module READMEs directly from `variable` and `output` blocks, which is why `description` and `type` are worth writing.
  • TFLint — Rules such as `terraform_typed_variables` and `terraform_unused_declarations` flag untyped and unused variables and locals.
  • HCP Terraform — Typically stores per-workspace variable values and marks them sensitive in its UI; the values still reach the run as ordinary variables.
  • Terraform Registry modules — A published module's contract is its variables and outputs, so changing a type or removing an output is a breaking change for callers.

Trade-offs

Pros

  • Types and validation reject bad input at plan time with a message you wrote, before any resource is touched
  • The layering rules let the same code run with defaults, a per-environment tfvars file and a one-off `-var`
  • `optional()` defaults keep module interfaces small without forcing callers to spell out every attribute
  • Outputs and `description` fields give a module a documented, tool-readable interface

Cons

  • `sensitive` is display-only: state, `-raw` and `-json` output, and downstream states still hold the value
  • Many sources with a fixed precedence make it hard to tell where a value came from; `*.auto.tfvars` loads silently
  • Validation can only check what is known at plan time and cannot see the real world
  • Every root output copies its value into state, and a changed output type can break every `terraform_remote_state` consumer

Why count identifies instances by position and for_each by key, what each does to the plan when you remove, reorder or rename items, and how to switch without destroying anything.

The problem

A list of three subnet names drives `count = length(var.names)`. Someone deletes the middle name, and the plan wants to change the second subnet into the third and destroy the last one, because Terraform tracks `subnet[1]` and `subnet[2]`, not "b" and "c". On a resource whose differing attribute is immutable, that is a replacement of infrastructure other things depend on. The same refactor with `for_each` would have destroyed only `"b"`. Choosing between the two is really choosing what identifies an instance.

How it works

  • `count = N` creates instances addressed `[0]` to `[N-1]`; `for_each` over a map or a set of strings creates instances addressed `["key"]`. Setting both on one block is rejected at validation with `Invalid combination of "count" and "for_each"` (verified). Both work on resources and on modules (a `for_each` module call applied cleanly in the check).
  • The address is the identity stored in state. With `count`, removing `b` from `["a","b","c"]` yields `item[1] must be replaced` and `item[2] will be destroyed (because index [2] is out of range for count)`, `Plan: 1 to add, 0 to change, 2 to destroy` (verified with `terraform_data` and `triggers_replace`). Whether `item[1]` is replaced or updated in place depends on the changed attribute: a plain `input` change on `terraform_data` is an in-place update.
  • Reordering a `count` list swaps what each index means: `["c","b","a"]` replaced `item[0]` and `item[2]`, and inserting `x` at the front replaced all three existing instances and created `item[3]` (`Plan: 4 to add, 0 to change, 3 to destroy`, verified).
  • `for_each` takes a map or a set of strings; a list is an error: `Invalid for_each argument ... you have provided a value of type tuple` (verified), so wrap it in `toset()` or build a map with a `for` expression. Removing one key only destroys that instance (`because key ["b"] is not in for_each map`), reordering gives `No changes`, and renaming `b` to `bb` is one destroy plus one create.
  • Keys and the count must be known at plan time. `for_each = toset([random_pet.p.id])` fails with `Invalid for_each argument ... random_pet.p.id is a string, known only after apply`, and an unknown `count` fails with `Invalid count argument ... To work around this, use the -target argument` (both verified). A map such as `{ name = random_pet.p.id }` works because the key is static and only the value is unknown. Sensitive values are rejected as keys: `var.db_password has a sensitive value`.
  • Inside the block, `count.index` is the position, and `each.key` / `each.value` are the map entry (for a set both are the string). From outside, `res[*].attr` is the splat for `count`; for `for_each` use `{ for k, v in res : k => v.attr }`. With `count` set, `terraform_data.feature.id` is an error (`Missing resource instance key`, verified), and `one(terraform_data.feature[*].id)` returns the single id or `null` for a `count = 0` toggle.
  • Changing the style of repetition changes every address. Adding `count` to a block that had none is mapped for you: the plan says `terraform_data.x has moved to terraform_data.x[0]` with no `moved` block (verified). Adding `for_each` is not: it destroys `x` (`because resource uses count or for_each`) and creates `x["a"]`, and switching `count` to `for_each` destroys and creates every instance (`because resource does not use count`) unless you add `moved` blocks (Terraform 1.1+).
  • A plan that contains only moves shows `Plan: 0 to add, 0 to change, 0 to destroy` yet `terraform plan -detailed-exitcode` exits `2`, because the state still has to be rewritten (verified). After `apply` the moves are done and the `moved` blocks can stay or be removed once every state has applied them.

Examples

`count`: removing the middle item, swapping two items and inserting at the front each rewrite instances by position.

bash
mkdir -p demo-count && cd demo-count
cat > main.tf <<'HCL'
variable "names" {
  type    = list(string)
  default = ["a", "b", "c"]
}

resource "terraform_data" "item" {
  count            = length(var.names)
  triggers_replace = var.names[count.index]
}
HCL

terraform init
terraform apply -auto-approve
terraform state list                                # item[0] item[1] item[2]

# Remove the middle element: "c" slides into index 1
terraform plan -var='names=["a","c"]' | grep -E '^  #|^Plan'
# Same set, different order: index 0 and 2 swap
terraform plan -var='names=["c","b","a"]' | grep -E '^  #|^Plan'
# Insert at the front: every existing index shifts
terraform plan -var='names=["x","a","b","c"]' | grep -E '^  #|^Plan'

`for_each` over a set: only the removed key is destroyed, reordering is a no-op, a rename is destroy plus create, and a list is rejected.

bash
mkdir -p demo-foreach && cd demo-foreach
cat > main.tf <<'HCL'
variable "names" {
  type    = set(string)
  default = ["a", "b", "c"]
}

resource "terraform_data" "item" {
  for_each         = var.names
  triggers_replace = each.key
}
HCL

terraform init
terraform apply -auto-approve
terraform state list                                  # item["a"] item["b"] item["c"]

terraform plan -var='names=["a","c"]' | grep -E '^  #|^Plan'      # only "b" goes away
terraform plan -var='names=["c","b","a"]'  | grep -E 'No changes'   # order is irrelevant
terraform plan -var='names=["a","bb","c"]' | grep -E '^  #|^Plan'   # rename = destroy + create

# A list is not accepted directly
cat > bad.tf <<'HCL'
resource "terraform_data" "bad" {
  for_each = ["x", "y"]
}
HCL
terraform plan 2>&1 | grep -A5 'Error:'
rm bad.tf

Switching `count` to `for_each` with `moved` blocks: the plan reports moves only, and `-detailed-exitcode` returns 2.

bash
mkdir -p demo-switch && cd demo-switch
cat > main.tf <<'HCL'
variable "names" {
  type    = list(string)
  default = ["a", "b", "c"]
}

resource "terraform_data" "item" {
  count            = length(var.names)
  triggers_replace = var.names[count.index]
}
HCL
terraform init
terraform apply -auto-approve

# Switch to for_each. Without moved blocks the plan is 3 destroys + 3 creates.
cat > main.tf <<'HCL'
variable "names" {
  type    = list(string)
  default = ["a", "b", "c"]
}

resource "terraform_data" "item" {
  for_each         = toset(var.names)
  triggers_replace = each.key
}

moved {
  from = terraform_data.item[0]
  to   = terraform_data.item["a"]
}
moved {
  from = terraform_data.item[1]
  to   = terraform_data.item["b"]
}
moved {
  from = terraform_data.item[2]
  to   = terraform_data.item["c"]
}
HCL
terraform plan                                       # "has moved to", Plan: 0 to add, 0 to change, 0 to destroy
terraform plan -detailed-exitcode > /dev/null; echo "exit=$?"   # 2: the moves are pending state changes
terraform apply -auto-approve
terraform state list

In practice

  • Terraform CLI — Prints the reason next to each destroy (`because index [2] is out of range for count`), which is the quickest way to spot an index shift in review.
  • terraform-aws-modules — Community modules typically gate optional resources with a create flag through `count = var.create ? 1 : 0` and key repeated objects with `for_each`.
  • Cloud Posse modules — Typically use an `enabled` input that becomes `count = enabled ? 1 : 0`, the conditional-resource pattern behind `one(res[*].attr)` outputs.
  • Terraform `moved` block — The supported way to rename an instance or change `count` to `for_each` without destroying objects; it is recorded in configuration and reviewed like code.
  • OpenTofu — Typically follows the same address rules, `moved` semantics and error messages for `count` and `for_each` (not run here; check the OpenTofu docs for your version).

Trade-offs

Pros

  • `count` is the simplest way to toggle a resource on or off (`0` or `1`) or to make N interchangeable copies
  • `for_each` keys survive removal, insertion and reordering, so unrelated instances stay untouched
  • Keys are readable in plans, state and `terraform state` commands: `item["web"]` says more than `item[2]`
  • `moved` blocks let you change the repetition style and keep the existing objects

Cons

  • `count` indexes shift when the list changes, which can replace working infrastructure
  • `for_each` keys must be known at plan time and cannot be sensitive, which forces static keys or a staged `-target` apply
  • The two cannot be combined on one block; a count-of-for_each needs a module or a flattened map
  • References change shape (`res[0].id` vs `res["k"].id`), so switching breaks every consumer in the configuration

The expression toolbox in practice: for expressions, splats, conditionals, type conversion and the functions you reach for most, tested interactively with terraform console.

The problem

A security group module takes a list of server objects and needs a map by name, a flat list of ports, a default region when the caller leaves it out, and a subnet CIDR carved from a parent block. Teams write it with trial and error, running `plan` against real infrastructure to see what `merge` or a conditional returned, and then get surprised: `true ? 1 : "a"` is the string `"1"`, `try()` hides a typo as a silent fallback, and `merge` does not merge nested maps. The language is small, but its edge cases are only visible when you evaluate it.

How it works

  • `terraform console` loads the configuration in the current directory and evaluates expressions against it (`local.*`, `var.*`, and state values for resources). Resource attributes that are not in state yet print as `(known after apply)`. Piping several lines evaluates them but printed only the last result in the check (`printf '1+1\n2+2\n' | terraform console` printed `4`), so script it one expression per call.
  • A `for` expression builds a collection from another: `[for s in servers : s.name]` is a tuple, `{ for s in servers : s.name => length(s.ports) }` is an object, and a trailing `if` filters (`[for s in servers : s.name if length(s.ports) > 1]` returned `["web"]`). Duplicate keys in the object form are an error (`Duplicate object key`, verified) unless you add `...` after the value to group them into lists.
  • Splat is shorthand for a simple `for`: `local.servers[*].name` equals `[for s in local.servers : s.name]`. On a single non-list value it wraps it in a list (`local.defaults[*]` returned a one-element list) and on `null` it yields `[]`. For nested lists use `flatten`: `flatten([for s in local.servers : s.ports])` returned `[80, 443, 5432]`.
  • A conditional `cond ? a : b` must produce one type, so Terraform converts both branches to a common type instead of failing: `true ? 1 : "a"` returned `"1"` and `true ? [1] : ["a"]` returned `["1"]` (verified). Only the selected branch's errors surface: `false ? tonumber("z") : 2` returned `2` (verified).
  • `try(expr, fallback)` returns the first argument that evaluates without error, and `can(expr)` turns the same into a boolean. Both also swallow mistakes: `try(local.cfg.regoin, "eu")` returned `"eu"` for a misspelled attribute. Use them for genuinely optional data, not to silence errors during development.
  • Selection functions differ in what they treat as missing: `lookup(map, key, default)` falls back only for an absent key and errors without a default (`the given object has no attribute "b"`), `coalesce` skips `null` and `""` and errors when nothing is left (`no non-null, non-empty-string arguments`), `index` errors with `item not found`, and `one([])` returns `null` while `one` of two or more elements is an error.
  • `merge(a, b)` is shallow: later keys replace earlier ones whole, so `merge({ a = { x = 1 } }, { a = { y = 2 } })` returned `{ a = { y = 2 } }` (verified). Deep merging needs a `for` expression or a module. `cidrsubnet("10.0.0.0/16", 8, 3)` returned `10.0.3.0/24` and `cidrsubnets("10.0.0.0/16", 8, 8, 4)` allocated `10.0.0.0/24`, `10.0.1.0/24`, `10.0.16.0/20` in order without overlap.
  • `templatefile(path, vars)` renders a file with `${...}` interpolation and `%{ for }` / `%{ if }` directives; a trailing `~` strips the adjacent newline. It reads the file at plan time and number lists passed to `join` are converted to strings. Since Terraform 1.9, `templatestring` renders a template held in a string value, for example one read from a data source. The console prints list types as `tolist([...])` and a tuple as a bare `[...]`, which shows what type a `for` expression really built.

Examples

Evaluating the everyday functions and `for` / splat expressions with `terraform console`, one expression per call.

bash
mkdir -p demo-expr && cd demo-expr
cat > main.tf <<'HCL'
locals {
  defaults  = { tier = "std", region = "eu" }
  overrides = { tier = "pro" }
  servers = [
    { name = "web", ports = [80, 443] },
    { name = "db", ports = [5432] },
  ]
}
HCL
terraform init

# terraform console evaluates stdin; with several piped lines only the last result is
# printed, so run one expression per call
tfx() { echo "> $1"; echo "$1" | terraform console; }
tfx 'merge(local.defaults, local.overrides)'
tfx 'lookup(local.defaults, "zone", "none")'
tfx 'try(local.defaults.zone, "fallback")'
tfx 'coalesce("", null, "x")'
tfx 'cidrsubnet("10.0.0.0/16", 8, 3)'
tfx 'cidrsubnets("10.0.0.0/16", 8, 8, 4)'
tfx 'flatten([for s in local.servers : s.ports])'
tfx '[for s in local.servers : s.name if length(s.ports) > 1]'
tfx '{ for s in local.servers : s.name => length(s.ports) }'
tfx 'local.servers[*].name'

`templatefile` with a `%{ for }` directive, a `for` expression into an object, and a splat, as real outputs.

bash
mkdir -p demo-template && cd demo-template
cat > servers.tftpl <<'TPL'
%{ for s in servers ~}
server ${s.name} listens on ${join(",", s.ports)}
%{ endfor ~}
TPL
cat > main.tf <<'HCL'
locals {
  servers = [
    { name = "web", ports = [80, 443] },
    { name = "db", ports = [5432] },
  ]
}

output "rendered" {
  value = templatefile("${path.module}/servers.tftpl", { servers = local.servers })
}

output "by_name" {
  value = { for s in local.servers : s.name => s.ports }
}

output "names" {
  value = local.servers[*].name
}
HCL

terraform init
terraform apply -auto-approve
terraform output -raw rendered
terraform output -json by_name

Edge cases worth knowing: unified conditional types, `try()` hiding a typo, shallow `merge`, and the errors of `coalesce`, `lookup` and `index`.

bash
mkdir -p demo-gotchas && cd demo-gotchas
cat > main.tf <<'HCL'
locals {
  cfg = { region = "eu" }
}
HCL
terraform init

tfx() { echo "> $1"; echo "$1" | terraform console 2>&1 | head -${2:-6}; }

tfx 'true ? 1 : "a"'                  # the branches are unified: the number becomes "1"
tfx 'true ? [1] : ["a"]'
tfx 'try(local.cfg.regoin, "eu")'     # try() also swallows typos
tfx 'merge({ a = { x = 1 } }, { a = { y = 2 } })'   # merge is shallow
tfx 'one([])'
tfx 'one(["x"])'
tfx 'coalesce("", "")' 5
tfx 'lookup({ a = 1 }, "b")' 8
tfx 'index(["a"], "c")' 6

In practice

  • terraform console — The REPL for the language itself: evaluates against the loaded configuration and state, which makes it the fastest way to check a function before it is baked into a plan.
  • HashiCorp HCL and go-cty — Terraform's expression syntax and its type system (tuple vs list, object vs map) come from these libraries, which is where the conversion rules above originate.
  • cloud-init — `templatefile` is the usual way to render cloud-init user data or config files with per-instance values before handing them to a compute resource.
  • AWS IAM — Policy documents are typically built with `jsonencode({...})` or the `aws_iam_policy_document` data source instead of hand-written JSON strings.
  • TFLint — Flags deprecated syntax and unused declarations in expressions, a cheap check next to console-based experimentation.

Trade-offs

Pros

  • A small, declarative set of expressions that can be evaluated offline with `terraform console` before touching infrastructure
  • `for` expressions, splats and `merge` remove most loops and copy-paste from configuration
  • Automatic type conversion lets numbers, strings and lists flow between modules without casts
  • Functions like `cidrsubnet` and `templatefile` keep derived values and rendered files in one place

Cons

  • Silent conversions (`true ? 1 : "a"` becomes `"1"`) and permissive helpers such as `try()` can hide real mistakes
  • No user-defined functions in the language: complex logic ends up as long nested expressions or locals
  • Values that are unknown at plan time (resource attributes) cannot be explored in `console` until they are in state
  • Shallow `merge` and tuple vs list typing surprise people who expect general-purpose-language semantics

How dynamic generates repeated nested blocks from a collection, the iterator and content rules, the optional-block idiom, and where it should not be used.

The problem

A firewall resource needs one `ingress` block per allowed port, and the list of ports comes from a variable. Copy-pasting five blocks works until the sixth port arrives; wrapping them in `for_each` does not work, because `for_each` repeats a whole resource, not a block inside it. The `dynamic` block fills that gap, but over-used it turns a readable resource into a template nobody can follow, and its errors are reported from inside a `content` body that validation cannot always check.

How it works

  • `dynamic "name" { for_each = ...; content { ... } }` generates one nested block of type `name` per element of the collection. The label must be a block type the resource or data source declares; `dynamic "foo"` on `terraform_data` fails with `Blocks of type "foo" are not expected here` (verified).
  • Inside `content`, an iterator object named after the block label exposes `.key` and `.value`: with `dynamic "source"` you write `source.key` and `source.value`. `iterator = f` renames it (verified: `f.key`, `f.value`). A map gives string keys, a list gives the index as the key.
  • `each.key` and `each.value` are not available in `content` unless the enclosing resource itself uses `for_each`; using them there failed with `Reference to "each" in context without for_each` (verified). `terraform validate` accepts that mistake (it reported `Success!`) and the error only appears when a plan evaluates the body.
  • Zero elements means zero blocks, which is how the optional-block idiom works: `for_each = var.extra == null ? [] : [var.extra]` produced no block when `extra` was `null` and one block when it was set (verified, with a static `source` block next to the dynamic one). Whether zero blocks is valid is up to the provider: `archive_file` with none failed with `Missing Attribute Configuration`.
  • Expansion happens during evaluation, so the collection can be an expression of variables, locals and data; reordering the keys of a map did not change the plan (`No changes`, verified). With a list the generated order follows the list, and for a nested block type that the provider models as an ordered list, a reordered input is typically a real diff on the next plan.
  • Not everything can be generated. `dynamic` cannot create meta-argument blocks: `dynamic "provisioner"` and `dynamic "lifecycle"` both failed with `Blocks of type ... are not expected here` (verified). It also cannot produce arguments or nested attributes that are not blocks in the provider schema; for those, use a `for` expression to build the list or map value.
  • The block is data-driven, so the same rules apply as anywhere else for unknown values. A set containing a value that is unknown at plan time still produced a block in the check, and a `data "archive_file"` that depended on it was reported as `will be read during apply` with `filename = (known after apply)`. Check what your own provider's plan shows before relying on that: this was only exercised on a data source.
  • `dynamic` is applied at the block level of any resource, data source or provider configuration, so it composes with module inputs: a module can accept `rules = map(object({...}))` and expand it, letting callers add an entry instead of editing the module. The cost is readability, and the usual guideline is to prefer literal blocks when there are one to three of them.

Examples

One nested `source` block per map entry on `data "archive_file"` (hashicorp/archive, no credentials): the zip contains both files, and key order does not change the plan.

bash
mkdir -p demo-dynamic && cd demo-dynamic
cat > main.tf <<'HCL'
terraform {
  required_providers {
    archive = { source = "hashicorp/archive" }
  }
}

variable "files" {
  type = map(string)
  default = {
    "a.txt" = "alpha"
    "b.txt" = "beta"
  }
}

data "archive_file" "bundle" {
  type        = "zip"
  output_path = "${path.module}/bundle.zip"

  # One nested "source" block per map entry
  dynamic "source" {
    for_each = var.files
    content {
      filename = source.key
      content  = source.value
    }
  }
}

output "zip_size" {
  value = data.archive_file.bundle.output_size
}
HCL

terraform init
terraform apply -auto-approve
unzip -l bundle.zip
terraform plan -var='files={"b.txt"="beta","a.txt"="alpha"}' | grep 'No changes'
terraform plan -var='files={"a.txt"="alpha","b.txt"="beta2"}' | grep -A1 'Changes to Outputs'

`iterator`, the provider error for zero generated blocks, and the `each` reference error inside `content`.

bash
mkdir -p demo-dynamic-iter && cd demo-dynamic-iter
cat > main.tf <<'HCL'
terraform {
  required_providers {
    archive = { source = "hashicorp/archive" }
  }
}

variable "files" {
  type    = map(string)
  default = { "a.txt" = "alpha" }
}

data "archive_file" "bundle" {
  type        = "zip"
  output_path = "${path.module}/bundle.zip"

  dynamic "source" {
    for_each = var.files
    iterator = f # defaults to the block label, "source"
    content {
      filename = f.key
      content  = f.value
    }
  }
}
HCL

terraform init
terraform plan | grep -E 'Reading|Read complete|Plan|No changes|Changes'

# Zero generated blocks: the provider's own validation reports the problem
terraform plan -var='files={}' 2>&1 | grep -A3 'Error:'

# Inside content, each.* is not the dynamic iterator
sed -i 's/filename = f.key/filename = each.key/' main.tf
terraform plan 2>&1 | grep -A4 'Error:'

The optional-block idiom: a list with zero or one element next to a static block.

bash
mkdir -p demo-dynamic-optional && cd demo-dynamic-optional
cat > main.tf <<'HCL'
terraform {
  required_providers {
    archive = { source = "hashicorp/archive" }
  }
}

variable "extra" {
  type = object({
    name = string
    body = string
  })
  default = null
}

data "archive_file" "bundle" {
  type        = "zip"
  output_path = "${path.module}/bundle.zip"

  source {
    filename = "main.txt"
    content  = "always present"
  }

  # Optional block: a list with zero or one element
  dynamic "source" {
    for_each = var.extra == null ? [] : [var.extra]
    content {
      filename = source.value.name
      content  = source.value.body
    }
  }
}
HCL

terraform init
terraform apply -auto-approve && unzip -Z1 bundle.zip
terraform apply -auto-approve -var='extra={name="extra.txt",body="more"}' && unzip -Z1 bundle.zip

In practice

  • hashicorp/archive provider — `archive_file` takes repeatable `source` blocks, a convenient credential-free target for experimenting with `dynamic` (used in the snippets).
  • AWS provider — Resources such as `aws_security_group` typically have repeatable `ingress` and `egress` blocks, the classic case where rules from a variable become dynamic blocks (not run here).
  • Kubernetes provider — Nested pod and container specs (for example `env` and `volume_mount` blocks) are commonly generated with `dynamic` from a map of settings (not run here).
  • terraform-aws-modules — Community modules typically expose rule maps and expand them into dynamic blocks, so callers add an entry instead of forking the module.
  • HashiCorp HCL (dynblock) — The `dynamic` expansion is implemented by the dynblock extension of the HCL library and applies to blocks declared in the schema being decoded, which `lifecycle` and `provisioner` are not.

Trade-offs

Pros

  • One block definition covers any number of rules, so adding an entry to a variable is a data change, not a code change
  • The optional-block idiom replaces `null` handling and duplicated resources with a single expression
  • Keys in a map-driven `for_each` keep the plan stable when entries are reordered
  • Modules can expose a small, typed map as their interface instead of dozens of boolean toggles

Cons

  • Readability drops quickly: `dynamic` plus `content` plus iterator references obscure what the resource looks like
  • Errors surface at plan time from inside `content`, and `terraform validate` cannot always check the generated body (an `each.key` mistake validated cleanly before failing in plan)
  • It cannot generate `lifecycle` or `provisioner` blocks, or attributes that are not blocks
  • Provider-specific rules about zero blocks, ordering and unknown counts are not visible in the configuration

When a data block is read during plan and when it is deferred to apply, what depends_on and conditions change, and the trade-offs of terraform_remote_state and external.

The problem

A configuration looks up a value with a `data` block and uses it in a resource, and the value is stale: the data source printed the old file contents because it was read during plan, before the resource that rewrites the file ran. Another team reads a platform team's VPC id with `terraform_remote_state` and discovers they now hold the platform team's database password in their own state file. Data sources are read-only, but when they run and what they copy decides whether plans are trustworthy.

How it works

  • A `data` block reads existing information through a provider and exposes it as `data.<type>.<name>.<attr>`. It never creates or destroys anything, but its result is stored in state (`terraform state list` shows `data.local_file.cfg` and `data.external.whoami` after apply) and is read again on every plan and refresh.
  • When all arguments are known at plan time the read happens during plan: the log shows `data.local_file.before: Reading...` and `Read complete`, and any output derived from it has its real value in the plan (verified).
  • When an argument depends on something not yet known, or `depends_on` points at an object with pending changes, the read is deferred to apply. The plan marks it with `<= data "local_file" "after"` and the reason: `will be read during apply (depends on a resource or a module with changes pending)`, or `(config refers to values not yet known)`; the attributes are `(known after apply)` (verified).
  • Deferral is per plan, not permanent. After the first apply, the same data block with `depends_on = [local_file.cfg]` was read during the next plan, because nothing was pending any more. A data source that is not connected to the resource that changes a file reads the old content: the check applied a new `app.conf` and the `before` output stayed `"port=1\n"` while `after` held `"port=8080\n"`.
  • Data blocks accept `count`, `for_each`, `depends_on` and `provider`, but their `lifecycle` supports only `precondition` and `postcondition`; `prevent_destroy` is rejected with `The lifecycle argument "prevent_destroy" is defined only for managed resources` (verified). A `postcondition` with `self` checks what was read and stops the plan with `Resource postcondition failed` (`self.content is "port=8080\n"`, verified).
  • `data "external"` runs a program on the machine running Terraform at plan time, every plan. It must print a JSON object whose keys and values are all strings: nested JSON fails with `Program output must be a JSON encoded map of string keys and string values`, and a non-zero exit fails with `External Program Execution Failed` and the stderr text (both verified). The id in state is `-`.
  • `terraform_remote_state` reads the root-module outputs stored in another configuration's state, here with `backend = "local"` and a path (other backends need their own credentials and were not run). Only outputs are visible: asking for one that is missing failed with `This object does not have an attribute named "nope"`.
  • Remote state copies values. A sensitive output in the producer stayed redacted in the consumer's plan (`pw = (sensitive value)`), but the consumer's own `terraform.tfstate` contained the secret on two lines (verified). Giving a team read access to a state for one output therefore gives it every output, and the state of the consumer needs the same protection as the producer's.

Examples

One data source read at plan time, another deferred to apply by `depends_on`, and the stale value that results when the read happens before the file is rewritten.

bash
mkdir -p demo-data && cd demo-data
echo 'port=1' > app.conf     # a file that already exists before the first plan
cat > main.tf <<'HCL'
resource "local_file" "cfg" {
  filename = "${path.module}/app.conf"
  content  = "port=8080\n"
}

# Arguments are all known at plan time: read during plan
data "local_file" "before" {
  filename = "${path.module}/app.conf"
}

# Depends on a resource with pending changes: read during apply
data "local_file" "after" {
  filename   = local_file.cfg.filename
  depends_on = [local_file.cfg]
}

output "before" {
  value = data.local_file.before.content
}

output "after" {
  value = data.local_file.after.content
}
HCL

terraform init
terraform plan | grep -E 'Reading|Read complete|will be read|\(depends on|\+ (before|after)'
terraform apply -auto-approve | grep -E 'Reading|Read complete|^(before|after)'
terraform output -json                     # before = "port=1\n" (stale), after = "port=8080\n"
terraform plan | grep -E 'Reading|Read complete'   # nothing pending: both are read at plan time, and the stale output is refreshed

A `postcondition` on a data source and `data "external"` (hashicorp/external): both run during plan, and both are stored in state.

bash
mkdir -p demo-data-check && cd demo-data-check
echo 'port=8080' > app.conf
cat > main.tf <<'HCL'
data "local_file" "cfg" {
  filename = "${path.module}/app.conf"

  lifecycle {
    postcondition {
      condition     = strcontains(self.content, "port=9")
      error_message = "app.conf must set a port in the 9xxx range."
    }
  }
}

data "external" "whoami" {
  program = ["sh", "-c", "printf '{\"user\":\"%s\"}' \"$(id -un)\""]
}

output "user" {
  value = data.external.whoami.result.user
}
HCL

terraform init
terraform plan                     # Error: Resource postcondition failed
echo 'port=9090' > app.conf
terraform plan | grep -E 'Reading|Read complete|\+ user'
terraform apply -auto-approve | tail -3
terraform state list               # data sources are stored in state too

`terraform_remote_state` with a local backend: the consumer reads a sensitive output and ends up with the secret in its own state.

bash
mkdir -p demo-remote-state/net demo-remote-state/app

cd demo-remote-state/net
cat > main.tf <<'HCL'
terraform {
  backend "local" {
    path = "net.tfstate"
  }
}

resource "terraform_data" "vpc" {
  input = "vpc-demo"
}

output "vpc_id" {
  value = terraform_data.vpc.output
}

output "db_password" {
  value     = "s3cret"
  sensitive = true
}
HCL
terraform init
terraform apply -auto-approve | tail -3

cd ../app
cat > main.tf <<'HCL'
data "terraform_remote_state" "net" {
  backend = "local"
  config = {
    path = "${path.module}/../net/net.tfstate"
  }
}

output "vpc" {
  value = data.terraform_remote_state.net.outputs.vpc_id
}

output "pw" {
  value     = data.terraform_remote_state.net.outputs.db_password
  sensitive = true
}
HCL
terraform init
terraform apply -auto-approve | tail -4
grep -c s3cret terraform.tfstate        # the secret now also sits in this state file

In practice

  • Terraform CLI — Decides per data block whether to read at plan or defer to apply, and says why in the plan (`will be read during apply`).
  • AWS provider — Data sources such as `aws_ami` and `aws_caller_identity` typically look up image ids and account ids that other resources consume (not run here).
  • HashiCorp Vault provider — Secret-reading data sources such as `vault_kv_secret_v2` typically write the fetched secret into state, one reason state is treated as sensitive (not run here).
  • hashicorp/external provider — The escape hatch for facts no provider exposes, at the price of running local code on every plan and requiring a string-only JSON contract.
  • HCP Terraform — Has a workspace setting for sharing state with other workspaces (not run here); a consumer that is granted access typically can read every output of the producer.

Trade-offs

Pros

  • Configuration can reuse existing infrastructure and facts (ids, AMIs, accounts) without hard-coding them
  • Reads during plan give real values in the plan output and let `for_each` keys and `count` depend on them
  • `precondition` / `postcondition` turn assumptions about what was read into early, readable failures
  • `terraform_remote_state` is a simple, dependency-free way to share outputs between configurations

Cons

  • A data block that depends on pending changes is deferred to apply, so everything derived from it is unknown in the plan and cannot be used in `count` or `for_each` keys
  • Reads that are not tied to the writer by a dependency return stale values
  • `external` runs arbitrary local programs on each plan and only exchanges string maps
  • `terraform_remote_state` couples configurations through output names and copies secrets into the consumer's state

Modules

A module is a directory with a narrow interface: variables in, outputs out, providers only when handed over, and a new address prefix for every resource inside it.

The problem

A team copies a 200-line block of resources into three environments, then tidies up by moving it into a `modules/` folder. Two weeks later `terraform plan` wants to destroy and recreate everything that used to live in the root, a child module complains that it needs a provider configuration nobody passed, and someone asks why `module.app.random_pet.this.id` cannot be read from the root. A module is a boundary with a narrow door: values enter as variables, results leave as outputs, providers cross only when handed over, and every resource inside gets a new address.

How it works

  • A module is a directory of `.tf` files; the working directory is the root module and a `module` block calls another directory through `source`. The child sees only the variables the caller passes and the caller sees only declared outputs: `module.identity.secret` fails with `Unsupported attribute` and `module.identity is object with 1 attribute "name"` (verified).
  • Modules are installed, not just read. `terraform validate` before `init` fails with `Module not installed` and `Run "terraform init" to install all modules`, and `init` prints one `- app in modules/name` line per call (verified). Adding or renaming a `module` block needs `init` or `terraform get` again; `.terraform/modules/modules.json` records Key, Source and Dir per call, and the old key stayed in the file after a rename.
  • Local paths are referenced, not copied: `modules.json` shows `"Dir":"modules/name"`, the source directory itself, so editing a local module is picked up by the next plan without another `init` (verified: a prefix edit planned `must be replaced` for all three instances). Registry and git sources are downloaded under `.terraform/modules/<key>` instead.
  • Every resource address gains the module path: `module.app.random_pet.this`, `module.worker["a"].random_pet.this`, `module.pool[0].random_pet.this`. The address is the resource's identity in state, so moving a resource into a module without a `moved` block plans `random_pet.app will be destroyed` and `module.app.random_pet.this will be created` with `Plan: 1 to add, 0 to change, 1 to destroy` (verified).
  • A `moved` block turns that into a pure address change: `random_pet.app has moved to module.app.random_pet.this` and `Plan: 0 to add, 0 to change, 0 to destroy`. The same works for renaming a call (`module.app` to `module.web`) and for adding `for_each` or `count` (`module.web` to `module.web["x"]` or `module.web[0]`, all verified). A `moved` whose `from` is still declared fails with `Moved object still exists`.
  • `count` and `for_each` work on `module` blocks, so `module.worker` becomes a map of objects: collect outputs with `{ for k, m in module.worker : k => m.id }` and from a counted module with `module.pool[*].name` (both verified). `depends_on` on a module call makes everything inside it wait for the listed targets, which is a blunt tool covered in the composition entry.
  • Providers cross the boundary only by inheritance or explicit hand-over. A child inherits the caller's default, un-aliased configuration implicitly; extra ones are declared with `configuration_aliases` in the child and passed with `providers = { time.primary = time, time.secondary = time.other }`. Leaving the argument out fails `init` with `Missing required provider configuration` and `with the local name "time.primary"`, once per alias per module call (verified).
  • Keep `provider` blocks out of reusable modules. An empty one produces `Warning: Redundant empty provider block` (verified). Terraform's documentation also says that a module containing its own provider configurations is incompatible with `count`, `for_each` and `depends_on`, because removing an instance would remove the provider needed to destroy its resources; I could not reproduce that failure with the credential-free providers, so treat it as documented behaviour.

Examples

A local module called once and with for_each: the not-installed error, init output, modules.json, addresses, and a local edit picked up without init.

bash
mkdir -p modules/name
cat > modules/name/main.tf <<'EOF'
terraform {
  required_providers {
    random = { source = "hashicorp/random" }
  }
}

variable "prefix" {
  type = string
}

resource "random_pet" "this" {
  prefix = var.prefix
}

output "id" {
  value = random_pet.this.id
}
EOF
cat > main.tf <<'EOF'
module "app" {
  source = "./modules/name"
  prefix = "app"
}

module "worker" {
  source   = "./modules/name"
  for_each = toset(["a", "b"])
  prefix   = "w-${each.key}"
}

output "worker_ids" {
  value = { for k, m in module.worker : k => m.id }
}
EOF
terraform validate              # Error: Module not installed
terraform init                  # Initializing modules... - app in modules/name
cat .terraform/modules/modules.json
terraform apply -auto-approve
terraform state list
sed -i 's/prefix = var.prefix/prefix = "${var.prefix}-x"/' modules/name/main.tf
terraform plan                  # a local module edit needs no new init

Passing two provider configurations into a module with configuration_aliases, using count on the call, and the init error when providers is omitted.

bash
mkdir -p modules/stamp
cat > modules/stamp/main.tf <<'EOF'
terraform {
  required_providers {
    time = {
      source                = "hashicorp/time"
      configuration_aliases = [time.primary, time.secondary]
    }
  }
}

resource "time_static" "a" {
  provider = time.primary
}

resource "time_static" "b" {
  provider = time.secondary
}
EOF
cat > main.tf <<'EOF'
terraform {
  required_providers {
    time = { source = "hashicorp/time" }
  }
}

provider "time" {}

provider "time" {
  alias = "other"
}

module "stamp" {
  source = "./modules/stamp"
  providers = {
    time.primary   = time
    time.secondary = time.other
  }
}

module "counted" {
  source     = "./modules/stamp"
  count      = 2
  depends_on = [module.stamp]
  providers = {
    time.primary   = time
    time.secondary = time.other
  }
}
EOF
terraform init
terraform apply -auto-approve
terraform state list
# drop the providers argument: init refuses
sed -i '/^  providers = {/,/^  }/d' main.tf
terraform init   # Error: Missing required provider configuration

Moving a resource into a module: destroy and create without a moved block, a pure address change with one, then renaming the call and adding for_each.

bash
mkdir -p modules/pet
cat > modules/pet/main.tf <<'EOF'
terraform {
  required_providers {
    random = { source = "hashicorp/random" }
  }
}

resource "random_pet" "this" {
  length = 2
}
EOF
cat > main.tf <<'EOF'
resource "random_pet" "app" {
  length = 2
}
EOF
terraform init && terraform apply -auto-approve
# 1) move the resource into a module WITHOUT a moved block
cat > main.tf <<'EOF'
module "app" {
  source = "./modules/pet"
}
EOF
terraform init
terraform plan    # random_pet.app will be destroyed, module.app...this will be created
# 2) with a moved block it is a pure address change
cat >> main.tf <<'EOF'

moved {
  from = random_pet.app
  to   = module.app.random_pet.this
}
EOF
terraform plan    # random_pet.app has moved to module.app.random_pet.this
terraform apply -auto-approve
# 3) renaming the module call also needs init, then a moved block
cat > main.tf <<'EOF'
module "web" {
  source = "./modules/pet"
}

moved {
  from = module.app
  to   = module.web
}
EOF
terraform init
terraform plan    # module.app.random_pet.this has moved to module.web.random_pet.this
terraform apply -auto-approve
# 4) adding for_each to a module call changes every address; a moved block carries the state across
cat > main.tf <<'EOF'
module "web" {
  source   = "./modules/pet"
  for_each = toset(["x"])
}

moved {
  from = module.web
  to   = module.web["x"]
}
EOF
terraform init
terraform plan    # module.web.random_pet.this has moved to module.web["x"].random_pet.this

In practice

  • Terraform Registry — Publishes modules from repositories named `terraform-<PROVIDER>-<NAME>` and expects the standard layout of `main.tf`, `variables.tf` and `outputs.tf`, so the variables and outputs are the whole public interface.
  • terraform-aws-modules (vpc, eks) — Community modules that expose ids such as `vpc_id` and `private_subnets` as outputs for other modules to consume instead of letting callers reach into their resources.
  • Azure Verified Modules — Microsoft's module programme for Azure; its modules typically document their interface (variables and outputs) as a contract that callers rely on.
  • Terraform CLI — `terraform init` and `terraform get` install module sources into `.terraform/modules`, and `terraform state mv` or `moved` blocks relocate addresses when a module is introduced.
  • OpenTofu — Uses the same `module` block, `providers` argument and `moved` blocks, so module code is typically portable between the two tools (UNVERIFIED here, OpenTofu was not run).

Trade-offs

Pros

  • Variables and outputs make the interface explicit and reviewable, so internals can change without touching callers
  • `count` and `for_each` on a module call repeat a whole group of resources with one block
  • A `moved` block makes refactoring into modules a zero-change plan instead of a destroy and create
  • Local modules need no publishing step and edits apply immediately

Cons

  • Every resource address changes when it enters a module, so refactors need `moved` blocks and a plan review
  • Anything not exposed as an output is unreachable, so callers keep asking for one more output or variable
  • Provider wiring (`configuration_aliases`, `providers`) is easy to get wrong and fails only at `init` or `validate` time
  • `modules.json` is a local cache that can go stale after renames, so CI should always run `init`

Registry modules take a version constraint, git modules are pinned by ?ref=, nothing locks either one in a lock file, and child modules add their own Terraform and provider constraints to yours.

The problem

A module from the public registry is declared with `version = "~> 1.0"`, CI is green for weeks, and then a fresh laptop clone plans a diff nobody committed because a new 1.x release changed a default. A teammate pins a git module to `?ref=main` and the same thing happens silently. Terraform locks provider versions in `.terraform.lock.hcl`, but modules have no lock file, so the only thing standing between your code and an unreviewed upgrade is how tightly you wrote the constraint.

How it works

  • `version` is valid only for registry sources. On a local path `init` fails with `Invalid registry module source address` and `can't use local directory "./modules/pet" as a module registry address` (verified). Git, HTTP and local sources are pinned by the source address itself, for example `?ref=v1.0.1`.
  • Registry constraints use the same operators as providers: `~> 1.0.0` allows only 1.0.x, `~> 1.0` allows any 1.x, and `>= 1.0.0, < 1.0.2` is a range. With no `version`, `init` takes the newest release. A first `init` against `hashicorp/dir/template` printed `Downloading registry.terraform.io/hashicorp/dir/template 1.0.2 for files...` (verified).
  • There is no lock file for modules. `.terraform.lock.hcl` records providers only, and in my run no lock file was created at all because the configuration had no providers. The resolved module version lives in `.terraform/modules/modules.json` as `"Version":"1.0.2"`, a local cache that is normally git-ignored, so a fresh clone resolves again.
  • `init` re-resolves lazily. After narrowing the range to `>= 1.0.0, < 1.0.2` it re-downloaded 1.0.1; after widening to `~> 1.0` it kept 1.0.1 without any Downloading line; only `terraform init -upgrade` printed `Upgrading modules...` and fetched 1.0.2 (verified). Two machines with identical code can therefore run different module versions.
  • A constraint that nothing satisfies fails with `Unresolvable module version constraint` and `The newest available version is 1.0.2.` (verified), which is also the quickest way to learn what the registry currently offers.
  • Git sources: `git::https://github.com/hashicorp/terraform-template-dir.git?ref=v1.0.1` clones into `.terraform/modules/<key>` and leaves a real `.git` directory there (verified). A ref that does not exist fails with `Failed to download module` and `invalid ref: "v9.9.9"`, and changing the address string triggers a fresh `Downloading git::...?ref=v1.0.0`. A tag can be moved by its maintainer and a branch moves by design; a full commit SHA cannot. Whether a branch ref is re-fetched without `terraform get -update` was not reproduced here, so rely on the documentation for that.
  • Terraform 1.15.0 added variables and locals in `source` and `version` (changelog: "Terraform now supports variables and locals in module source and version attributes"; the local 1.16.5 behaves as described): a variable declared with `const = true` or a local built from such values works, while a normal variable fails with `Unknown module version` and `Only literal values and const variables can be evaluated during init.` (verified). Releases before 1.15 require literals. OpenTofu's changelog lists variables and locals in module sources from 1.8 and a `const = true` variable option from 1.12, but I did not run it (UNVERIFIED).
  • A module brings its own constraints and they are intersected with yours. A root `random >= 3.0` plus a child `~> 2.0` fails with `no available releases match the given constraints ~> 2.0, >= 3.0.0`, and a child `required_version = ">= 99.0"` fails with `Unsupported Terraform Core version` and `Module module.identity (from ./modules/identity) does not support Terraform version 1.16.5` (both verified). `terraform providers` prints the constraint of each module so you can see who asks for what.

Examples

A registry module: first download, modules.json, narrowing and widening the range, init -upgrade, and an unsatisfiable constraint.

bash
mkdir -p tpl && echo 'hello' > tpl/a.txt
cat > main.tf <<'EOF'
module "files" {
  source  = "hashicorp/dir/template"
  version = "~> 1.0.0"

  base_dir = "${path.module}/tpl"
}
EOF
terraform init            # Downloading registry.terraform.io/hashicorp/dir/template 1.0.2 for files...
cat .terraform/modules/modules.json; echo
ls .terraform.lock.hcl    # only providers are locked; no module entry exists
# narrow the range to exclude the installed 1.0.2
sed -i 's/~> 1.0.0/>= 1.0.0, < 1.0.2/' main.tf
terraform init            # re-downloads: 1.0.1
# widen it again: init keeps the installed 1.0.1 because it still satisfies
sed -i 's/>= 1.0.0, < 1.0.2/~> 1.0/' main.tf
terraform init
terraform init -upgrade   # Upgrading modules... 1.0.2
# a range nothing satisfies
sed -i 's/~> 1.0/~> 1.1/' main.tf
terraform init            # Error: Unresolvable module version constraint

A git module pinned with ?ref=, a missing ref, changing the ref, and a const variable used as the registry version.

bash
mkdir -p tpl && echo 'hello' > tpl/a.txt
cat > main.tf <<'EOF'
module "files" {
  source   = "git::https://github.com/hashicorp/terraform-template-dir.git?ref=v1.0.1"
  base_dir = "${path.module}/tpl"
}
EOF
terraform init
grep -o '"Source":"[^"]*"' .terraform/modules/modules.json
ls -d .terraform/modules/files/.git       # a real clone: init-time download cached here
sed -i 's/v1.0.1/v9.9.9/' main.tf
terraform init            # Error: Failed to download module
sed -i 's/v9.9.9/v1.0.0/' main.tf
terraform init            # a changed address string is re-downloaded
cat > main.tf <<'EOF'
variable "tpl_version" {
  type    = string
  default = "1.0.0"
  const   = true
}

module "files" {
  source   = "hashicorp/dir/template"
  version  = var.tpl_version
  base_dir = "${path.module}/tpl"
}
EOF
terraform init            # Terraform 1.15+: const variables allowed in source/version

Provider constraints from root and child are intersected, and a child required_version that the CLI cannot meet.

bash
mkdir -p modules/identity
cat > modules/identity/main.tf <<'EOF'
terraform {
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 2.0"
    }
  }
}

resource "random_pet" "this" {}
EOF
cat > main.tf <<'EOF'
terraform {
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = ">= 3.0"
    }
  }
}

module "identity" {
  source = "./modules/identity"
}
EOF
terraform init -upgrade   # constraints from root and child are intersected
terraform providers
sed -i 's/~> 2.0/>= 3.0/' modules/identity/main.tf
terraform init -upgrade
terraform providers       # one random provider, both constraints listed
sed -i 's/^terraform {/terraform {\n  required_version = ">= 99.0"/' modules/identity/main.tf
terraform validate        # Error: Unsupported Terraform Core version ... module.identity

In practice

  • Terraform Registry — Builds the module version list from semantic-version git tags such as `v1.0.2` in the module repository, which is what `version` constraints resolve against.
  • Renovate and Dependabot — Both can open pull requests that bump registry module versions and git `?ref=` tags in `.tf` files, which substitutes for a module lock file.
  • HCP Terraform private registry — Hosts internal modules with the same `version` constraint syntax; one-line mention only, not covered here.
  • terraform-aws-modules (vpc) — Publishes major versions with upgrade notes, which is why callers usually pin the major with `~>` and bump it deliberately.
  • OpenTofu — Has its own registry and typically resolves the same `version` constraints; its changelog lists variables and locals in module sources since 1.8 (UNVERIFIED here, not run).

Trade-offs

Pros

  • An exact `=` or tight `~>` pin plus a reviewed bump pull request gives reproducible module code
  • Git refs work for private modules without running a registry
  • A commit SHA ref is immutable, unlike a tag or branch
  • Child `required_providers` and `required_version` fail early, at `init` or `validate`, instead of during apply

Cons

  • No lock file: a loose constraint silently resolves differently on each fresh clone
  • `init` keeps an installed version that still satisfies the range, so machines drift until someone runs `-upgrade`
  • Registry modules can only be pinned by version and git modules only by ref, so you cannot mix both styles under one constraint
  • A child's provider constraint can make the whole configuration unsolvable, and the error names only the intersection

Prefer small modules wired together in the root over a one-resource wrapper or one giant module: the plan shows where depends_on defers reads, where sensitive values leak, and why modules do not split state.

The problem

A platform team publishes `company/bucket/aws`, a module that mirrors one resource and its arguments one-to-one, and within a month it has 40 variables and a pull request for every attribute somebody needs. Elsewhere a single `app` module holds networking, database and compute; changing a tag plans 120 lines, `depends_on` on it turns every data source inside into `(known after apply)`, and a password passed in comes out of an output in plain text. Boundaries help only when they hide a decision and the root, not the module, decides how pieces connect.

How it works

  • The wrapper anti-pattern: a module that wraps one resource and re-exposes its arguments as variables adds an address segment, a version to manage and a variable per attribute, but no abstraction. Every missing argument means editing the module first: `separator = "_"` on a wrapper without that variable fails with `Unsupported argument` and `An argument named "separator" is not expected here.` (verified). A module earns its place when it hides a decision or a group of resources that change together.
  • Composition keeps the wiring in the root: `owner = module.identity.name` passes one module's output into another's input. `terraform graph` draws a cluster per module but no node for the module itself, and the edge runs between resources: `"module.config.terraform_data.config" -> "module.identity.random_pet.this";` (verified).
  • Dependencies pull in what they need: `terraform apply -auto-approve -target=module.config` also created `module.identity.random_pet.this`, and printed `Warning: Resource targeting is in effect` and `Warning: Applied changes may be incomplete` (verified). `-target` is for recovery, not routine use.
  • References are what order reads, and a known value is not a dependency on a pending change. A data source inside a module that reads a file path taken from `local_file.src.filename` failed during the plan step of `apply`, with `Read local file data source error` and `open ./src.txt: no such file or directory`, because the filename is known before the file exists (verified).
  • `depends_on = [local_file.src]` on the module call fixes it by deferring the module's data sources: the plan shows `<= read (data resources)` and `module.reader.data.local_file.f will be read during apply` with `(depends on a resource or a module with changes pending)` (verified). Once nothing is pending the plan is quiet, with `-detailed-exitcode` returning 0.
  • The price of that deferral is unknowns. While `local_file.src` has a pending change, every output of the module is `(known after apply)` and shows as `~ seen = "v1" -> (known after apply)`; a `for_each` or `count` that depends on such a value cannot be planned at all. A module-level `depends_on` is the widest hammer, so prefer a real reference when one exists.
  • Sensitivity propagates through module outputs, and the root insists on acknowledgment: a module output built from a `sensitive = true` variable is sensitive in the caller, and a root output that uses it fails with `Output refers to sensitive values` until you add `sensitive = true` (verified). `nonsensitive()` inside the module removes the mark and prints plain: `hint = "hu"` (verified).
  • Marks are display-only. `terraform output -json connection` printed `postgres://app:hunter2hunter2@db/app` in full, and `terraform.tfstate` contained the password three times, in the resource's `input` and `output` attributes and in the root output (verified). Also, a module boundary does not split state: all modules called from one root share one state file and one lock, as `terraform state list` showed `module.config...` and `module.identity...` side by side, so a real blast-radius split needs separate root configurations.

Examples

A one-resource wrapper that rejects a new argument, then two small modules composed in the root: graph edges, and -target pulling in the dependency.

bash
mkdir -p modules/pet_wrapper modules/identity modules/config
cat > modules/pet_wrapper/main.tf <<'EOF'
terraform {
  required_providers {
    random = { source = "hashicorp/random" }
  }
}

variable "length" {
  type    = number
  default = 2
}

resource "random_pet" "this" {
  length = var.length
}
EOF
cat > modules/identity/main.tf <<'EOF'
terraform {
  required_providers {
    random = { source = "hashicorp/random" }
  }
}

variable "env" {
  type = string
}

resource "random_pet" "this" {
  prefix = var.env
}

output "name" {
  value = random_pet.this.id
}
EOF
cat > modules/config/main.tf <<'EOF'
variable "owner" {
  type = string
}

resource "terraform_data" "config" {
  input = { owner = var.owner }
}
EOF
# Anti-pattern: a wrapper that mirrors one resource. Every new attribute needs a new variable.
cat > main.tf <<'EOF'
module "wrapped" {
  source    = "./modules/pet_wrapper"
  separator = "_"
}
EOF
terraform init
terraform validate   # Error: Unsupported argument ... "separator" is not expected here
# Composition: the root wires small modules together through outputs
cat > main.tf <<'EOF'
module "identity" {
  source = "./modules/identity"
  env    = "prod"
}

module "config" {
  source = "./modules/config"
  owner  = module.identity.name
}
EOF
terraform init
terraform validate
terraform graph | grep -- '->'
terraform apply -auto-approve -target=module.config   # pulls in module.identity only
terraform state list

A data source inside a module read too early, fixed with depends_on on the module call, and the unknown outputs that deferral produces.

bash
mkdir -p modules/reader
cat > modules/reader/main.tf <<'EOF'
variable "path" {
  type = string
}

data "local_file" "f" {
  filename = var.path
}

output "content" {
  value = data.local_file.f.content
}
EOF
cat > main.tf <<'EOF'
resource "local_file" "src" {
  filename = "${path.module}/src.txt"
  content  = "v1"
}

module "reader" {
  source = "./modules/reader"
  path   = local_file.src.filename
}

output "seen" {
  value = module.reader.content
}
EOF
terraform init
terraform apply -auto-approve   # Error: Read local file data source error (read at plan time)
# depends_on on the module call defers its data sources to apply time
sed -i 's|path   = local_file.src.filename|path   = "${path.module}/src.txt"\n  depends_on = [local_file.src]|' main.tf
terraform apply -auto-approve
terraform plan -detailed-exitcode ; echo "exit=$?"   # 0: steady state is quiet
sed -i 's/"v1"/"v2"/' main.tf
terraform plan   # <= read (data resources): module.reader.data.local_file.f will be read during apply

A sensitive variable passing through a module: the required sensitive flag on the root output, nonsensitive() leaking a fragment, and plaintext in state.

bash
mkdir -p modules/vault
cat > modules/vault/main.tf <<'EOF'
variable "password" {
  type      = string
  sensitive = true
}

resource "terraform_data" "db" {
  input = var.password
}

output "connection" {
  value = "postgres://app:${var.password}@db/app"
}

output "hint" {
  value = nonsensitive(substr(var.password, 0, 2))
}
EOF
cat > main.tf <<'EOF'
module "vault" {
  source   = "./modules/vault"
  password = "hunter2hunter2"
}

output "connection" {
  value = module.vault.connection
}

output "hint" {
  value = module.vault.hint
}
EOF
terraform init
terraform plan        # Error: Output refers to sensitive values (root output "connection")
sed -i 's|value = module.vault.connection|value     = module.vault.connection\n  sensitive = true|' main.tf
terraform apply -auto-approve   # connection = <sensitive>, hint = "hu"
terraform output -json connection
grep -c hunter2hunter2 terraform.tfstate

In practice

  • terraform-aws-modules (vpc) — Exposes outputs such as `vpc_id` and `private_subnets` that callers pass to separate modules (for example eks), which is composition in the root rather than a mega-module.
  • Graphviz — `terraform graph | dot -Tsvg` renders the resource graph shown here, with one cluster per module.
  • Atlantis — Runs plan and apply per directory or workspace, so splitting a monolith into several root configurations also splits its pull request checks and locks (typically; UNVERIFIED here).
  • Terraform CLI — `terraform output -json` and `terraform state pull` expose values that `sensitive = true` only hides from console output.
  • OpenTofu — Behaves the same for module wiring and sensitive marking in the cases I know of; OpenTofu was not run here (UNVERIFIED).

Trade-offs

Pros

  • Small modules with real interfaces are reusable, and the root shows how they connect
  • Plans stay short and reviewable because each module hides one decision
  • Output-to-input wiring creates graph edges automatically, with no `depends_on` needed
  • Separate root configurations shrink blast radius, refresh time and lock contention

Cons

  • More modules mean more variables and outputs to maintain, and the root becomes the place with all the wiring
  • Module-level `depends_on` defers data sources and makes outputs unknown, which can break `for_each` downstream
  • `sensitive = true` hides values from the console only: JSON output and state still hold them in plaintext
  • Splitting into several roots trades one plan for cross-state data passing and apply ordering you must manage yourself

State operations

How a configuration-driven `import` block adopts an existing object in a normal plan, what `-generate-config-out` writes, how it differs from the `terraform import` command, and what fails.

The problem

A bucket, a DNS zone or a database was created by hand during an incident, and now it has to be managed as code. Writing a `resource` block that matches it attribute by attribute is slow, and the old `terraform import` command writes straight into state with no plan and no review, so a wrong address or a half-matching block shows up later as a surprise replacement. Terraform 1.5 made import a planned action: you declare an `import` block, read the plan, and apply it like any other change.

How it works

  • An `import` block has two required arguments: `to` (the resource address that will own the object) and `id` (a provider-specific identifier string). It is processed in a normal `terraform plan`: each target shows `# random_id.web will be imported` and the summary reads `Plan: 3 to import, 0 to add, 0 to change, 0 to destroy.` Apply prints `Importing...`, `Import complete [id=...]` and `Apply complete! Resources: 3 imported, 0 added, 0 changed, 0 destroyed.` (verified on 1.16.5; import blocks and `-generate-config-out` arrived in 1.5).
  • The target needs configuration. If the `to` resource has no `resource` block, plan fails with `Error: Configuration for import target does not exist` and the message suggests `terraform plan -generate-config-out=generated.tf`. A plan that contains only imports still exits with code 2 under `-detailed-exitcode`, because there is a change to apply.
  • `-generate-config-out=PATH` writes HCL for import targets that have no configuration yet, starting with `# __generated__ by Terraform` and one `# __generated__ by Terraform from "<id>"` comment per resource. It prints `Warning: Config generation is experimental`. In the verified run `terraform_data` came out as `input = null` and `triggers_replace = null`, and `random_id` as `byte_length = 6`, `keepers = null`, `prefix = null`. Treat the file as a draft: for real cloud resources it typically also contains computed or default attributes that need pruning, and it should be moved into your own files.
  • The output file must not exist: a second run fails with `Error: Target generated file already exists`. The file is also written when the plan then fails on a different import (verified: it held the importable resources even though `random_pet` errored), so a partial `generated.tf` can be left behind after an error.
  • Support depends on the provider. `random_pet` has no importer, and the plan stops with `Error: Resource Import Not Implemented` / `This resource does not support import.` The `id` format is defined by each resource's documentation: `random_id` takes its `b64_url` value (`Zm9vYmFy`) and rejects `!!!` with the provider error `illegal base64 data at input byte 0`, `time_static` takes an RFC 3339 timestamp, and `terraform_data` accepts any string.
  • Import is read-then-compare, not a rewrite of the real object. The plan prints `Preparing import... [id=...]`, then `Refreshing state...`, and then diffs the imported values against your configuration. After importing a `random_id` with `byte_length = 4` and then changing the block to `8`, the plan showed `~ byte_length = 4 -> 8 # forces replacement` and `-/+ destroy and then create replacement`. An import is only safe when the following plan is empty.
  • `for_each` on the block (since 1.7) imports several instances: `for_each = local.ids`, `to = terraform_data.item[each.key]`, `id = each.value` produced `terraform_data.item["a"]` and `terraform_data.item["b"]`. The `to` address can reach into a module from the root configuration (`to = module.m.terraform_data.this` worked); an `import` block inside the child module itself also planned an import on 1.16.5 (`module.m.terraform_data.x will be imported`), but the 1.16.0 changelog lists "`import` blocks inside modules are now supported" as new, so on older releases declare them in the root module.
  • A finished block is idempotent: with the objects already in state, the next plan said `No changes.` with the `import` blocks still present, so removing them is housekeeping, not a requirement. The legacy command `terraform import ADDR ID` still exists: it needs the `resource` block first, writes state immediately (leaving `terraform.tfstate.backup`), prints `Import successful!`, and refuses an address that is already tracked with `Error: Resource already managed by Terraform`.

Examples

Adopt three objects with import blocks: the failing plan, the generated draft, then a plan that says 3 to import.

bash
mkdir demo-import && cd demo-import
cat > main.tf <<'EOF'
terraform {
  required_providers {
    random = { source = "hashicorp/random" }
    time   = { source = "hashicorp/time" }
  }
}

import {
  to = random_id.web
  id = "Zm9vYmFy"
}

import {
  to = terraform_data.cfg
  id = "abc-123"
}

import {
  to = time_static.t
  id = "2024-01-02T03:04:05Z"
}
EOF
terraform init

# No resource blocks yet: Error: Configuration for import target does not exist
terraform plan

# Draft the configuration, review it, then plan and apply as usual
terraform plan -generate-config-out=generated.tf
cat generated.tf
terraform plan
terraform apply -auto-approve
terraform plan -detailed-exitcode; echo "exit=$?"

One import block expands to one import per key (Terraform 1.7 and later).

bash
mkdir demo-import-each && cd demo-import-each
cat > main.tf <<'EOF'
locals {
  ids = { a = "id-a", b = "id-b" }
}

import {
  for_each = local.ids
  to       = terraform_data.item[each.key]
  id       = each.value
}

resource "terraform_data" "item" {
  for_each = local.ids
}
EOF
terraform init
terraform apply -auto-approve
terraform state list

The legacy command: no plan, state is written at once, and a second import of the same address is refused.

bash
mkdir demo-import-cli && cd demo-import-cli
cat > main.tf <<'EOF'
resource "random_id" "api" {
  byte_length = 4
}
EOF
terraform init
terraform import random_id.api YWJjZA     # writes state immediately
terraform import random_id.api YWJjZA     # Error: Resource already managed by Terraform
terraform plan                            # No changes: 4 random bytes decode from YWJjZA

In practice

  • Terraform CLI 1.5+ — Plans imports together with other changes, so a pull-request plan can show which hand-made objects will be adopted.
  • Terraformer — A separate tool that reads existing cloud resources and emits HCL plus state; `-generate-config-out` covers part of the same ground inside the CLI.
  • aztfexport (Azure Export for Terraform) — Microsoft's exporter that produces Terraform configuration and state for existing Azure resources; it is the Azure-specific equivalent of hand-writing import blocks.
  • Atlantis — Runs `terraform plan` on pull requests, which makes import blocks reviewable in the same comment as the rest of the change.
  • OpenTofu — UNVERIFIED here: it typically supports `import` blocks and `tofu plan -generate-config-out` with the same shape, so check its docs for version-specific differences.

Trade-offs

Pros

  • Reviewable: the import appears in plan output and in CI before anything is written to state
  • Repeatable and version-controlled, unlike a one-off shell command in someone's terminal
  • `for_each` makes bulk adoption a few lines instead of dozens of commands
  • `-generate-config-out` produces a starting configuration for resources with many attributes

Cons

  • Generated configuration is marked experimental and usually needs hand editing before it is clean
  • Not every resource type implements import, and the `id` format must be looked up per resource
  • Import adopts the object but does not make the configuration correct: a mismatch is a diff on the next plan, possibly a replacement
  • Blocks left in the repository after the import add noise until someone deletes them

Refactor addresses (rename, move into a module, count to for_each) without destroying anything, and drop an object from state while leaving the real one alone.

The problem

Someone renames `random_pet.web` to `random_pet.app` for readability, or turns `count = 2` into a `for_each`, and the plan says one destroy and one create per object. To Terraform a new address is a new object, so the database behind the old address would be deleted and rebuilt. The opposite case hurts too: deleting a resource block from the configuration destroys the real object, even when the intent was only to hand it to another team's state. `moved` and `removed` blocks put those intentions in code, where a plan can show them.

How it works

  • Without help, a rename is a replacement: the verified plan shows `# random_pet.app will be created`, `# random_pet.web will be destroyed` with `(because random_pet.web is not in configuration)`, and `Plan: 1 to add, 0 to change, 1 to destroy.`
  • `moved { from = random_pet.web to = random_pet.app }` (since 1.1) turns that into `# random_pet.web has moved to random_pet.app` and `Plan: 0 to add, 0 to change, 0 to destroy.` The plan still exits 2 under `-detailed-exitcode` because the state will change. Apply rewrites the address in state without recreating the object.
  • Moves cover whole resources, single instances and index changes. For `count` to `for_each`, one block per instance (`from = random_id.worker[0]`, `to = random_id.worker["a"]`) changed `2 to add, 2 to destroy` into `0 to add, 0 to change, 0 to destroy`. Moving a resource into a child module takes `to = module.pet.random_pet.this`, and the module must already declare that resource.
  • Terraform validates the statements. A `from` that is still declared fails with `Error: Moved object still exists` (`This statement declares a move from random_id.worker["a"], but that resource instance is still declared`), and a loop such as `a -> b` plus `b -> a` fails with `Error: Cyclic dependency in move statements`. Chains (`a -> b`, then `b -> c`) are followed to the end.
  • Keep the block after applying. A re-plan with it present says `No changes.` Other workspaces, environments and consumers of a shared module still hold the old address and only move when they next apply, so module authors typically leave `moved` blocks in place across releases rather than deleting them.
  • `removed { from = ADDRESS }` (since 1.7) is the opposite refactor. With `lifecycle { destroy = false }` the verified plan prints `will no longer be managed by Terraform, but will not be destroyed` and `(destroy = false is set in the configuration)`, plus `Warning: Some objects will no longer be managed by Terraform` listing every instance that is dropped from state. Without that nested block the default is to destroy the object, as before.
  • `removed` has its own rules. The `from` address must be a resource or module, not an instance: `random_id.zz["a"]` fails with `Error: Resource instance keys not allowed`. If the resource is still in the configuration the plan fails with `Error: Removed resource still exists`. `from = module.pet` removed all resources in the module from state. The documentation says only destroy-time provisioners are supported inside a `removed` block.
  • The imperative equivalents are `terraform state mv` and `terraform state rm`, which act immediately on state with no plan. Prefer the blocks when the change is reviewed by others or applied in several environments, and the commands for a one-off repair. Moving between different resource types is typically restricted to cases the provider supports (UNVERIFIED here, since the stand-in providers have no such pair).

Examples

The same edit as a replacement plan and as a move: three moved blocks cover a rename and a count to for_each switch.

bash
mkdir demo-moved && cd demo-moved
cat > main.tf <<'EOF'
resource "random_pet" "web" {
  length = 2
}

resource "random_id" "worker" {
  count       = 2
  byte_length = 2
}
EOF
terraform init && terraform apply -auto-approve

# Rename web -> app and count -> for_each, with no moved blocks yet
cat > main.tf <<'EOF'
resource "random_pet" "app" {
  length = 2
}

resource "random_id" "worker" {
  for_each    = toset(["a", "b"])
  byte_length = 2
}
EOF
terraform plan          # Plan: 3 to add, 0 to change, 3 to destroy.

cat > moved.tf <<'EOF'
moved {
  from = random_pet.web
  to   = random_pet.app
}

moved {
  from = random_id.worker[0]
  to   = random_id.worker["a"]
}

moved {
  from = random_id.worker[1]
  to   = random_id.worker["b"]
}
EOF
terraform plan          # "has moved to" x3, Plan: 0 to add, 0 to change, 0 to destroy.
terraform apply -auto-approve
terraform state list
terraform plan          # No changes. (moved.tf can stay)

Moving a resource into a module, and the error for a cyclic move chain.

bash
# Continue in demo-moved: move random_pet.app into a child module
mkdir -p modules/pet
cat > modules/pet/main.tf <<'EOF'
resource "random_pet" "this" {
  length = 2
}
EOF
cat > main.tf <<'EOF'
module "pet" {
  source = "./modules/pet"
}

resource "random_id" "worker" {
  for_each    = toset(["a", "b"])
  byte_length = 2
}
EOF
cat > moved.tf <<'EOF'
moved {
  from = random_pet.app
  to   = module.pet.random_pet.this
}
EOF
terraform init
terraform plan          # random_pet.app has moved to module.pet.random_pet.this
terraform apply -auto-approve

# Validation errors: a loop of moves
cat > moved.tf <<'EOF'
moved {
  from = random_id.a
  to   = random_id.b
}

moved {
  from = random_id.b
  to   = random_id.a
}
EOF
terraform plan          # Error: Cyclic dependency in move statements

A removed block with destroy = false: both instances leave state and nothing is deleted.

bash
mkdir demo-removed && cd demo-removed
cat > main.tf <<'EOF'
resource "random_id" "legacy" {
  count       = 2
  byte_length = 2
}

resource "random_id" "kept" {
  byte_length = 2
}
EOF
terraform init && terraform apply -auto-approve

# Hand legacy to someone else: delete its block, record the intent
cat > main.tf <<'EOF'
resource "random_id" "kept" {
  byte_length = 2
}
EOF
cat > removed.tf <<'EOF'
removed {
  from = random_id.legacy

  lifecycle {
    destroy = false
  }
}
EOF
terraform plan          # "will no longer be managed by Terraform, but will not be destroyed"
terraform apply -auto-approve
terraform state list    # random_id.kept only

In practice

  • Terraform AWS modules (terraform-aws-modules) — Public modules typically add `moved` blocks in releases that rename internal resources, so existing users can upgrade without replacement (check each module's changelog).
  • HashiCorp Terraform provider upgrade guides — Major-version upgrade guides typically cover renamed resources and point to refactoring with `moved` blocks or state commands instead of recreating them.
  • Terraform CLI — Evaluates moves during planning and applies them to state; the verified plan output lists each move before any provider call.
  • Spacelift and env0 — Hosted runners that run `terraform plan` on pull requests, which is where `moved` and `removed` intent is reviewed before apply.
  • OpenTofu — UNVERIFIED here (docs and changelogs read, OpenTofu not run): it supports `moved` and `removed` blocks, and its 1.12 changelog adds a `lifecycle { destroy = false }` option on resources; Terraform added the same option in 1.16.0, so check the version you run.

Trade-offs

Pros

  • A refactor becomes a reviewed plan with zero destroys instead of an out-of-band state edit
  • Works for every environment and workspace that applies the same code, each at its own pace
  • Terraform rejects loops and still-declared sources before touching anything
  • `removed` with `destroy = false` is the supported way to stop managing an object without deleting it

Cons

  • Stale `moved` blocks accumulate in long-lived modules and must be kept until every consumer has applied
  • One block per instance is needed when keys change, which is verbose for large `count` lists
  • A wrong `to` address still produces a plan with destroys, so the plan has to be read
  • `removed` leaves an unmanaged real object: it keeps costing money and nobody is tracking it

What `state list/show/mv/rm/pull/push/replace-provider` read or write, the backup files they leave behind, and the serial and lineage checks that guard `state push`.

The problem

A refactor went wrong, a provider namespace changed, or an object has to leave one state and enter another, and the only tool left is editing the state directly. `terraform state mv` and `terraform state rm` rewrite the file that decides what Terraform believes exists, and they take effect immediately: there is no plan, no confirmation, and the next apply acts on the new belief. Knowing which subcommands only read, which write, and which backup file they leave is what separates a repair from an outage.

How it works

  • `state list` and `state show` only read. `state list` prints one address per line (`random_id.by_key["a"]`, `module.m.random_id.inner`, `data.local_file.in`) and takes an address filter: `module.m` lists the module, a resource name lists all its instances, and `-id=VALUE` filters by the `id` attribute. An unknown resource fails with `Error: Unknown resource`.
  • `state show ADDRESS` prints one instance's attributes as HCL-like text (`# random_pet.web:` followed by `resource "random_pet" "web" {`). A resource with several instances fails with `Error: No instance found for the given address!` and `This command requires that the address references one specific instance.` `state show -json` (new in 1.16.0 per the changelog; checked on 1.16.5) returned an object with `format_version` and a `resource` object holding `address`, `mode`, `provider_name` and `values`; `terraform show -json` prints the whole state in a similar shape.
  • `state mv SOURCE DEST` rewrites addresses in state, for single instances (`random_id.worker[1]` to `random_id.worker[5]`), whole resources, and into modules (`module.m.random_pet.app`). Add `-dry-run` to get `Would move "random_pet.app" to "random_pet.web"` without writing. An unmatched source fails with `Error: Invalid source address` / `Cannot move random_pet.zzz: does not match anything in the current state.`
  • The configuration has the final say. After `state mv random_pet.web random_pet.app` with the `resource` block still named `web`, the verified plan showed `random_pet.app will be destroyed` (`because random_pet.app is not in configuration`) and `random_pet.web will be created`. Change the code and the state together, or use a `moved` block so the plan shows the move.
  • `state rm ADDRESS` forgets objects without touching the real ones. A resource address removes every instance (`Removed random_id.worker[0]`, `Removed random_id.worker[5]`, `Successfully removed 2 resource instance(s).`), `module.m` removes the module's resources, and `-dry-run` prints `Would remove ...`. The next plan wants to create the object again (`will be created`), and an address that matches nothing fails with `Error: Invalid target address`.
  • `state pull` prints the current state as JSON (format `version` 4 with `terraform_version`, `serial`, `lineage`, `outputs`, `resources`) to stdout, so `terraform state pull > backup.json` is a safe snapshot from any backend. `state push FILE` uploads one; it refuses a lower serial (`Failed to write state: cannot import state with serial 19 over newer state with serial 20`) and a different lineage (`cannot import state with lineage "other" over unrelated state with lineage "..."`), and `-force` overrides these checks.
  • `state replace-provider FROM TO` rewrites the provider address on every matching resource. It first prints `~ Updating provider:`, the old and new source, and `Changing 4 resources:`; `-auto-approve` skips the prompt, and it ends with `Successfully replaced provider for 4 resources.` If the configuration still requires the old provider, the next plan fails with `Error: Missing required provider` and `This state requires provider registry.terraform.io/example/random, but that provider isn't available.`
  • Backups depend on the command, although the built-in help says every state command that modifies state creates a timestamped backup. Every `state mv`, `state rm` and `state replace-provider` run left a new `terraform.tfstate.<unix-seconds>.backup` next to the state (five of them in one session), `-backup=PATH` redirects it (verified with `mybak.tfstate`), and `-backup=-` did not suppress it for `state rm` in 1.16.5. A normal `apply` rewrites `terraform.tfstate.backup` with the previous state, and `state push` left no file. Remote backends keep no local backup; object versioning on the bucket is the usual safety net (UNVERIFIED here). The writers also take the state lock, with `-lock=false` and `-lock-timeout=DURATION` to change that.

Examples

Read-only inspection: list with filters, show one instance, and the multi-instance error.

bash
mkdir demo-state && cd demo-state
echo hi > in.txt
cat > main.tf <<'EOF'
resource "random_pet" "web" {
  length = 2
}

resource "random_id" "worker" {
  count       = 2
  byte_length = 2
}

resource "random_id" "by_key" {
  for_each    = toset(["a", "b"])
  byte_length = 2
}

data "local_file" "in" {
  filename = "in.txt"
}

module "m" {
  source = "./m"
}
EOF
mkdir m
cat > m/main.tf <<'EOF'
resource "random_id" "inner" {
  byte_length = 1
}
EOF
terraform init && terraform apply -auto-approve

terraform state list
terraform state list module.m
terraform state list random_id.by_key
terraform state show random_pet.web
terraform state show 'random_id.by_key["a"]'      # quote the brackets for the shell
terraform state show random_id.worker             # Error: No instance found for the given address!
terraform state show -json random_pet.web | head -12

Writers: -dry-run first, a timestamped terraform.tfstate.<n>.backup per run, and the plan that still follows the configuration.

bash
# Continue in demo-state
terraform state mv -dry-run random_pet.web random_pet.app   # Would move ...
terraform state mv random_pet.web random_pet.app            # writes state + a timestamped backup
ls
terraform plan                       # random_pet.web will be created, random_pet.app destroyed

terraform state mv 'random_id.worker[1]' 'random_id.worker[5]'
terraform state rm -dry-run random_id.worker
terraform state rm random_id.worker                         # removes both instances
terraform state rm -backup=before-rm.tfstate 'random_id.by_key["a"]'
terraform state list

pull as a snapshot, the serial guard on push, and replace-provider (undone again here).

bash
# Continue in demo-state: snapshot, guarded push, provider address rewrite
terraform state pull > snapshot.json
terraform state rm 'random_id.by_key["b"]'
terraform state push snapshot.json   # Failed to write state: cannot import state with serial N over newer state with serial N+1
terraform state push -force snapshot.json

terraform state replace-provider -auto-approve registry.terraform.io/hashicorp/random registry.terraform.io/example/random
terraform plan                       # Error: Missing required provider
terraform state replace-provider -auto-approve registry.terraform.io/example/random registry.terraform.io/hashicorp/random

In practice

  • Terraform CLI — All of these subcommands go through the configured backend, so the same commands work against local files and remote state.
  • Amazon S3 backend with versioning — UNVERIFIED here: a versioned bucket is the usual way to recover a state object after a bad `state push` or `state rm`.
  • Terraform provider forks and namespace moves — `state replace-provider` is the documented way to move existing resources to a provider published under a different registry address.
  • Atlantis — Runs plan and apply against the same backend and commonly locks a directory per pull request, so manual `state mv` calls should go through the same lock.
  • OpenTofu — UNVERIFIED here: `tofu state` subcommands typically mirror these, and OpenTofu's state encryption affects what `state pull` returns, so check its docs.

Trade-offs

Pros

  • The only way to repair a state that the configuration cannot describe, such as a wrong provider address
  • `-dry-run` on `mv` and `rm` shows the exact objects before anything is written
  • `pull` works against any backend and gives a plain JSON snapshot
  • A timestamped backup is written locally without any flag for mv, rm and replace-provider

Cons

  • Changes take effect at once with no plan, so a typo in an address is already in the state
  • Moving or removing state does not change the code, so the next plan can still plan destroys
  • `push -force` and `-lock=false` bypass the protections that stop two writers corrupting a state
  • Backups are local files that may hold secrets in plain text and are easy to commit by accident

How a plan detects changes made outside Terraform, why a changed local file can look like a deletion, and when to use `-refresh-only`, `-refresh=false` and `-replace`.

The problem

An engineer fixes production by hand at 2 a.m., or a provider silently normalises a value, and the state file now describes an object that no longer exists in that form. The next `terraform plan` quietly reads the real object first, so the diff it prints mixes your code changes with changes nobody in the pull request made. Teams need to separate three actions that look alike: letting the plan undo the drift, recording the drift in state, and skipping the check to save time.

How it works

  • A normal `plan` and `apply` start by refreshing: each resource prints `Refreshing state... [id=...]` and the provider reads the real object. If the result differs from state, the plan opens with `Note: Objects have changed outside of Terraform` and lists them (`# local_file.motd has been deleted`) before the planned actions, so a reader can tell drift from your own edits.
  • What counts as drift is up to the provider. The `local_file` resource reads the file and compares its hash: overwriting `motd.txt` with other text, or deleting it, both produced `# local_file.motd has been deleted` followed by `# local_file.motd will be created` and `Plan: 1 to add, 0 to change, 0 to destroy.` An edit is reported as a deletion here, and the apply wrote the configured `hello` back.
  • `terraform plan -detailed-exitcode` returned 2 for the drifted file, so a scheduled plan with that flag is a simple drift detector. Exit code 0 means no differences and 1 means an error.
  • `terraform plan -refresh-only` shows only the drift and states `This is a refresh-only plan, so Terraform will not take any actions to undo these.` It also lists output changes (`~ motd_sha = "5891..." -> (known after apply)`), and it exits 2 when something would be recorded.
  • `terraform apply -refresh-only` records the drift into state without changing any real object. In the verified run it removed `local_file.motd` from state, `terraform output` then printed `motd_sha = null`, and the next normal plan wanted to create the file again. Accepting drift therefore makes state match reality, not the configuration; fix the code or the object separately.
  • `terraform refresh` is the older command that does the same state update with no review step. It also dropped the drifted resource from state in the verified run, which is why `-refresh-only` with its plan-then-confirm flow is the one to use.
  • `-refresh=false` skips reading real objects. After overwriting `motd.txt`, `terraform plan -refresh=false -detailed-exitcode` printed `No changes.` and exited 0, because the plan only compared configuration with the stale state. It is a speed tool for large states (and the reason a plan can miss drift), not a way to ignore it.
  • `-replace=ADDRESS` forces a resource to be destroyed and created again in the same run: `# local_file.motd will be replaced, as requested`, `-/+ destroy and then create replacement`, and `Plan: 1 to add, 0 to change, 1 to destroy.` It works on `plan` and `apply`, and an address that matches nothing was silently ignored (`No changes.`), so read the plan. `terraform taint` marks the same intent in state (`local_file.motd is tainted, so must be replaced`) and `terraform untaint` clears it; the documentation recommends `-replace` instead.

Examples

Drift from an edited file, detected by plan, then recorded with -refresh-only.

bash
mkdir demo-drift && cd demo-drift
cat > main.tf <<'EOF'
resource "local_file" "motd" {
  filename = "motd.txt"
  content  = "hello\n"
}

output "motd_sha" {
  value = local_file.motd.content_sha256
}
EOF
terraform init && terraform apply -auto-approve

echo tampered > motd.txt                             # drift made outside Terraform
terraform plan                                       # Note: Objects have changed outside of Terraform
terraform plan -detailed-exitcode > /dev/null; echo "exit=$?"   # 2

terraform plan -refresh-only                         # shows the drift, takes no action
terraform apply -refresh-only -auto-approve          # records it in state
terraform state list                                 # local_file.motd is gone
terraform output                                     # motd_sha = null
terraform plan                                       # local_file.motd will be created

-refresh=false hides drift, deleting the file is the same signal, and -replace and taint force recreation.

bash
# Continue in demo-drift
terraform apply -auto-approve                        # writes hello back
echo tampered > motd.txt
terraform plan -refresh=false -detailed-exitcode; echo "exit=$?"   # No changes. exit=0

rm motd.txt
terraform plan                                       # has been deleted, then will be created
terraform apply -auto-approve

terraform plan -replace=local_file.motd              # will be replaced, as requested
terraform plan -replace=local_file.nope              # silently: No changes.
terraform apply -replace=local_file.motd -auto-approve

terraform taint local_file.motd
terraform plan                                       # is tainted, so must be replaced
terraform untaint local_file.motd

In practice

  • Terraform CLI — `plan -detailed-exitcode` and `-refresh-only` are the two primitives that scheduled drift checks are built from.
  • HCP Terraform — One-line mention: its health assessments run scheduled checks for drift on workspaces; the feature set depends on the plan tier (UNVERIFIED here).
  • Atlantis — Plans on pull requests; drift appears in the same plan comment as the code change unless a separate scheduled plan job is added.
  • driftctl (maintenance mode) — A standalone drift scanner that compared cloud resources with Terraform state; its README says the project is in maintenance mode, which is why built-in refresh-only plans are the usual route now.
  • AWS Config and cloud audit logs — The source for finding who made the out-of-band change; Terraform only reports that the object differs, not who changed it.

Trade-offs

Pros

  • A normal plan shows drift separately from code changes, so reviewers can tell them apart
  • `-refresh-only` records reality in state without changing infrastructure, after a plan you can read
  • `-detailed-exitcode` turns a plan into a cron-friendly drift check
  • `-replace` expresses a one-off rebuild in the command line instead of editing state

Cons

  • Detection depends on what the provider reads back, and some attributes never report drift
  • Accepting drift with `-refresh-only` can drop an object from state, as the local file case shows
  • Refreshing every object makes large plans slow, and `-refresh=false` trades that speed for blindness
  • Terraform reports that something changed but not who changed it or why

`prevent_destroy`, `create_before_destroy`, `ignore_changes`, `replace_triggered_by`, `precondition` and `postcondition`: what each changes in the plan and where each backfires.

The problem

A load balancer must never be deleted by accident, an autoscaler keeps changing a field Terraform also sets, and a config file has to be replaced without a gap. Each is a case where the default order (update in place, or destroy then create) is wrong. The `lifecycle` block changes what the plan does with an object, and each setting also has a way to backfire: `create_before_destroy` can delete the thing it just created, `ignore_changes` can hide a real change, and `prevent_destroy` stops protecting the moment its block is deleted.

How it works

  • `prevent_destroy = true` turns any plan that would destroy the object into an error: `Error: Instance cannot be destroyed` / `Resource terraform_data.guarded has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed.` It triggers on `terraform destroy` and on a replacement caused by a forced-new change (changing `triggers_replace` gave `-/+` and the same error), and the message names `-target` as a way to reduce the scope.
  • `prevent_destroy` only protects while its block exists. When the whole `resource` block was deleted from the configuration, the plan said `Plan: 0 to add, 0 to change, 2 to destroy.` with no error. The argument must be a literal: `prevent_destroy = var.protect` fails with `Error: Variables not allowed`. Combine it with review rules on changes to the block itself.
  • `ignore_changes = [attr]` removes an attribute from the diff. With `ignore_changes = [input]`, `plan -var tag=v2` printed `No changes.` `ignore_changes = all` ignores every argument. A map key can be named (`keepers["env"]`): changing only `env` gave `No changes.`, but when `owner` also changed and forced a replacement, the plan still showed `"env" = "dev" -> "prod"`, because a replacement is created from the configuration. Use it for fields another system owns, not to hide drift you do not understand.
  • `create_before_destroy = true` swaps the order. The plan symbol becomes `+/-` with `create replacement and then destroy`, and the apply log showed `Creating...`, then the dependent's `Modifying...`, then `terraform_data.cbd (deposed object d5698102): Destroying...`. The old object is kept as a deposed object until the new one exists, which needs the name or address to be unique at the same time.
  • The gotcha: when old and new share a unique identifier, the old object's destroy removes the new one. A `local_file` with a fixed `filename` and `create_before_destroy = true` wrote `app.conf`, and after changing `content` the apply ended with `(deposed object ...) Destruction complete` and no `app.conf` on disk; the next plan said `local_file.cfg will be created`. Give replaceable objects generated names (for example with `name_prefix` or a `random_id`) before using this.
  • `replace_triggered_by = [terraform_data.image]` (since 1.2) replaces this resource whenever the referenced one changes. With `input` changed on `image`, the plan showed `terraform_data.image will be updated in-place` and `terraform_data.server will be replaced due to changes in replace_triggered_by`. A reference to an attribute (`terraform_data.image.output`) behaved the same. It is the explicit alternative to hiding the dependency in a `triggers` value.
  • `precondition` (check before the resource is evaluated) and `postcondition` (check against `self` after apply) arrived in 1.2 and carry a custom message. A failing precondition stops the plan: `Error: Resource precondition failed`, the line `var.size is 8` from the expression, then `size must be at least 12, got 8.`. A failing postcondition can only be detected after the object exists: apply printed `Error: Resource postcondition failed`, yet `random_password.db` was already in state, and a further plan reported the same error until the condition held.
  • `create_before_destroy`, `prevent_destroy`, `ignore_changes` and `replace_triggered_by` are meta-arguments in the `lifecycle` block of a `resource`, evaluated while the configuration is loaded, so they cannot use variables or other resources (only `replace_triggered_by` and the condition blocks reference other objects). Use `-replace=ADDRESS` for a one-time rebuild instead of adding a permanent rule.

Examples

prevent_destroy blocks a replacement and a destroy, and disappears with the resource block.

bash
mkdir demo-lifecycle && cd demo-lifecycle
cat > main.tf <<'EOF'
variable "rev" {
  type    = string
  default = "1"
}

resource "terraform_data" "guarded" {
  triggers_replace = var.rev

  lifecycle {
    prevent_destroy = true
  }
}
EOF
terraform init && terraform apply -auto-approve
terraform plan -var rev=2     # -/+ then: Error: Instance cannot be destroyed
terraform destroy -auto-approve   # same error

# prevent_destroy protects only while the block exists
echo "" > main.tf
terraform plan                # Plan: 0 to add, 0 to change, 1 to destroy.

create_before_destroy with a fixed filename: the deposed object's destroy deletes the new file.

bash
mkdir demo-cbd && cd demo-cbd
cat > main.tf <<'EOF'
variable "msg" {
  default = "v1"
}

resource "local_file" "cfg" {
  filename = "app.conf"
  content  = var.msg

  lifecycle {
    create_before_destroy = true
  }
}
EOF
terraform init && terraform apply -auto-approve
cat app.conf; echo
terraform apply -auto-approve -var msg=v2   # +/- create replacement and then destroy
ls app.conf                                  # No such file: the old object's destroy removed it
terraform plan -var msg=v2                   # local_file.cfg will be created

replace_triggered_by, plus a precondition that stops the plan and a postcondition that fails after creation.

bash
mkdir demo-triggers && cd demo-triggers
cat > main.tf <<'EOF'
variable "ami" {
  default = "img-1"
}

variable "size" {
  type    = number
  default = 8
}

resource "terraform_data" "image" {
  input = var.ami
}

resource "terraform_data" "server" {
  input = "web"

  lifecycle {
    replace_triggered_by = [terraform_data.image]
  }
}

resource "random_password" "db" {
  length = var.size

  lifecycle {
    precondition {
      condition     = var.size >= 12
      error_message = "size must be at least 12, got ${var.size}."
    }
    postcondition {
      condition     = length(self.result) == 16
      error_message = "expected a 16 character password."
    }
  }
}
EOF
terraform init
terraform plan                                    # Error: Resource precondition failed
terraform apply -auto-approve -var size=16        # passes both conditions
terraform plan -var size=16 -var ami=img-2        # server will be replaced due to changes in replace_triggered_by
terraform apply -auto-approve -var size=12        # apply: Error: Resource postcondition failed

In practice

  • Terraform AWS provider (aws_db_instance, aws_s3_bucket) — Documentation and community modules commonly put `prevent_destroy = true` on stateful resources such as databases and buckets.
  • Amazon EC2 Auto Scaling — A group's `desired_capacity` is changed by the autoscaler itself, which is the standard reason to list it in `ignore_changes`.
  • Kubernetes Horizontal Pod Autoscaler — Deployment `replicas` managed by an HPA is the same pattern: let the controller own the field and ignore it in the configuration.
  • terraform-aws-modules — Public modules use `create_before_destroy` on resources such as security groups and launch templates, paired with `name_prefix` so old and new names do not collide.
  • OpenTofu — UNVERIFIED here: it typically supports the same lifecycle arguments, and its 1.12 changelog adds a `destroy` option and allows references to input variables in `prevent_destroy` (Terraform 1.16.0 also added `destroy = false`, while `prevent_destroy` stays literal-only there); check the release notes of the version you use.

Trade-offs

Pros

  • `prevent_destroy` turns a risky plan into a hard error before anything is touched
  • `ignore_changes` lets another system own a field without a permanent diff
  • `create_before_destroy` avoids a gap for objects that need a replacement to be live first
  • Conditions put assumptions next to the resource with an actionable message

Cons

  • `prevent_destroy` disappears with the block and also blocks legitimate replacements and `terraform destroy`
  • `ignore_changes` can hide real changes and the ignored attribute still reappears when the object is replaced
  • `create_before_destroy` needs unique names and may propagate to dependencies, and the fixed-name case deletes the new object
  • A postcondition failure arrives after the object exists, so the state already contains it

What `terraform workspace` actually splits (only the state), where the files go, and why most teams separate environments by directory and backend instead.

The problem

The same code has to run for staging and prod, and `terraform workspace new prod` looks like the built-in answer. It splits the state and nothing else: the backend, the credentials, the provider versions and the code are shared, and the only signal of where you are is a name stored in a hidden file. A teammate who forgets `terraform workspace select` runs an apply against the wrong environment with perfectly valid credentials. Many teams therefore keep one directory (and one backend) per environment and share code through modules.

How it works

  • A workspace is a named state inside one backend and one working directory. `terraform workspace list` prints `* default` style output, `workspace new NAME` creates and selects (`Created and switched to workspace "staging"!` followed by `You're now on a new, empty workspace.`), `workspace select NAME` switches, and `workspace show` prints the current name.
  • The state lives in a different file per workspace. With the local backend the `default` workspace uses `terraform.tfstate` in the working directory, and every other one uses `terraform.tfstate.d/<name>/terraform.tfstate` (verified for `staging` and `prod`). The selected name is kept in `.terraform/environment` (it contained `prod`), so it belongs to that working copy, not to the repository.
  • The name is available in the configuration as `terraform.workspace`. A `locals` map with `lookup(local.sizes, terraform.workspace, 1)` as the `count` created 1, 2 and 4 instances in `default`, `staging` and `prod`. This is the only thing that changes between workspaces: it is the same code, the same provider block and the same credentials.
  • A new workspace starts empty, so `terraform plan` there sees none of another workspace's objects and plans to create everything. `terraform workspace new -state=PATH NAME` seeds it from an existing state file instead. Names must be URL-safe: `"bad name!"` failed with `The workspace name "bad name!" is not allowed.`
  • Deleting has two guards. The active workspace cannot be deleted (`You cannot delete the currently active workspace.`), and a non-empty one fails with `Error: Workspace is not empty` listing `random_pet.name[0]` and the other instances. `workspace delete -force` skips the second check and prints `WARNING: "prod" was non-empty` because the state is gone while the real objects remain.
  • Selecting or creating by mistake is cheap in both directions: `workspace select nope` fails with `Workspace "nope" doesn't exist.` and a hint to use `-or-create`, `workspace new` on an existing name fails with `Workspace "staging" already exists`, and `workspace select -or-create dev` creates it when missing. Scripts should always pass the name explicitly and print `terraform workspace show` before applying.
  • A directory-per-environment layout has one root module per environment, each with its own `backend` block and state: `envs/prod/main.tf` with `backend "local" { path = "prod.tfstate" }` called a shared module with `source = "../../modules/app"` and `replicas = 3`, and produced `prod.tfstate` next to it. Remote backends use a different bucket or key (or an account per environment), so access control, provider versions and a failed apply are isolated.
  • Workspaces fit short-lived copies of the same stack (a preview per feature branch, a test cluster) where the code and the permissions really are identical. Separate directories fit long-lived environments with different approvers, credentials, versions or shapes. How remote backends name per-workspace state (for example a key prefix in S3) is typically backend-specific and not verified here, and HCP Terraform workspaces are a different, per-run-environment concept despite the shared name.

Examples

Workspaces in practice: terraform.workspace in config, state files under terraform.tfstate.d, and the delete guards.

bash
mkdir demo-ws && cd demo-ws
cat > main.tf <<'EOF'
locals {
  sizes = { default = 1, staging = 2, prod = 4 }
}

resource "random_pet" "name" {
  count  = lookup(local.sizes, terraform.workspace, 1)
  prefix = terraform.workspace
}

output "workspace" {
  value = terraform.workspace
}
EOF
terraform init
terraform workspace show                       # default
terraform apply -auto-approve                  # 1 pet, state in terraform.tfstate

terraform workspace new staging                # new, empty workspace
terraform apply -auto-approve                  # 2 pets
terraform workspace new prod
terraform apply -auto-approve                  # 4 pets

terraform workspace list
find terraform.tfstate.d -name '*.tfstate'
cat .terraform/environment

terraform workspace delete prod                # Error: active workspace
terraform workspace select default
terraform workspace delete prod                # Error: Workspace is not empty
terraform workspace delete -force prod         # WARNING: "prod" was non-empty
terraform workspace select nope                # Workspace "nope" doesn't exist.
terraform workspace select -or-create dev

One directory and one backend per environment, sharing a module: each environment has its own state file.

bash
mkdir -p demo-dirs/modules/app demo-dirs/envs/prod demo-dirs/envs/staging && cd demo-dirs
cat > modules/app/main.tf <<'EOF'
variable "replicas" {
  type = number
}

resource "random_pet" "name" {
  count = var.replicas
}

output "names" {
  value = random_pet.name[*].id
}
EOF
cat > envs/prod/main.tf <<'EOF'
terraform {
  backend "local" {
    path = "prod.tfstate"
  }
}

module "app" {
  source   = "../../modules/app"
  replicas = 3
}
EOF
cat > envs/staging/main.tf <<'EOF'
terraform {
  backend "local" {
    path = "staging.tfstate"
  }
}

module "app" {
  source   = "../../modules/app"
  replicas = 1
}
EOF
(cd envs/prod && terraform init && terraform apply -auto-approve)
(cd envs/staging && terraform init && terraform apply -auto-approve)
ls envs/prod envs/staging

In practice

  • Terraform CLI workspaces — Documented as a way to keep several states for one configuration, with the caveat that they do not separate credentials or approvals.
  • Amazon S3 backend — UNVERIFIED here: it typically stores each non-default workspace under a key prefix (`env:`) in the same bucket, so the bucket policy covers every environment.
  • HCP Terraform — One-line mention: its workspaces are separate units with their own state, variables and permissions, which is closer to directory-per-environment than to CLI workspaces.
  • Atlantis — Supports both layouts through `atlantis.yaml` projects, one per directory or per directory and workspace pair.
  • OpenTofu — UNVERIFIED here: `tofu workspace` typically behaves the same, and its state encryption can be configured so that dev and prod use different keys.

Trade-offs

Pros

  • Workspaces add an environment with one command and no code duplication
  • `terraform.workspace` makes small per-environment differences a lookup
  • Directories give each environment its own backend, credentials, versions and approvers
  • A directory name in the path and in the CI job is visible; a workspace name is not

Cons

  • Workspaces share backend, credentials and code, so a wrong `select` hits prod with valid access
  • The active workspace is local hidden state in `.terraform/environment`
  • Directories repeat backend and module wiring, which drifts unless generated or tested
  • Environments that differ in shape end up full of `terraform.workspace` conditionals in workspace mode

Teams & safety

How a non-interactive pipeline plans once, applies exactly that saved plan, reads `plan -detailed-exitcode` and `show -json`, and serializes runs with state locks.

The problem

A plan is reviewed in a pull request, but the apply job runs a bare `terraform apply`, which plans again against a state that may have changed in the meantime, so what ships is not what was reviewed. In another pipeline a job hangs for an hour on `var.token` waiting for input nobody can type, and two pipelines for the same state collide. A CI run has to be non-interactive, serialized per state, and apply exactly the plan that was approved.

How it works

  • `terraform plan -out=tfplan` writes a binary plan file and `terraform apply tfplan` applies exactly that plan with no confirmation prompt, because the saved plan counts as the approval (verified: no prompt and no `-auto-approve` needed). A bare `terraform apply` plans again against the current state and asks for `yes` unless `-auto-approve` is given.
  • Saved plans go stale. If the state changed after the plan was written, `terraform apply tfplan` fails with `Error: Saved plan is stale` ("the state was changed by another operation after the plan was created") and changes nothing (verified). That failure is the safety property: the pipeline must plan and get approval again.
  • `terraform plan -detailed-exitcode` exits 0 for no changes, 2 when changes are present and 1 on error (verified: 2 with a pending create or update, 0 after apply, 1 for a missing variable). Without the flag, plan exits 0 whether or not there are changes, so drift-check and "is there anything to apply" jobs need it.
  • `-input=false` (or `TF_INPUT=0`) turns every would-be prompt into an error: an unset required variable fails at once with `Error: No value for required variable` instead of hanging (verified for `-input=false`). Setting `TF_IN_AUTOMATION` to any non-empty value removes the "To perform exactly these actions, run the following command to apply" hint printed after `-out` (verified), so logs do not suggest commands a pipeline will never type.
  • Run cheap checks first. `terraform fmt -check -recursive` lists the unformatted files and exits 3 (verified; add `-diff` to see the change), and `terraform validate` exits 1 with `Error: Missing required argument`-style messages (verified; `-json` returns `{"valid":false,"error_count":1,...}`). `validate` needs providers installed, so a pull-request job that must not touch state can use `terraform init -backend=false` first (verified: validate then passes); without any init it fails with `Error: Missing required provider`.
  • `terraform show -json tfplan` prints the plan as JSON (`format_version` "1.2" on 1.16.5, with keys such as `resource_changes`, `planned_values`, `prior_state`, `configuration`, `variables`). A replacement shows `change.actions` as `["delete","create"]` (verified), so pipelines gate on `jq` or a policy engine instead of grepping human-readable text. The same JSON contains variable values in clear, see the secrets entry.
  • Concurrency is handled by state locking. With `-lock-timeout=3s` a second plan waited three seconds and then failed with `Error acquiring the state lock` plus a Lock Info block (ID, Path, Operation, Who, Version, Created) while another apply held the lock (verified with the local backend; remote backends use their own lock mechanisms and are UNVERIFIED here). `-lock=false` skips locking: a plan with it ran while the apply held the lock (verified), acceptable for a speculative plan, never for apply.
  • A saved plan is tied to its inputs. It is a zip archive holding `tfplan`, `tfstate`, `tfconfig/` and `.terraform.lock.hcl` (verified with `unzip -l`), `terraform apply -var env=x tfplan` is rejected (verified: `Error: Can't change variable when applying a saved plan` when the variable is declared and the value differs from the saved one; `Error: Value for undeclared variable` when the configuration does not declare it), and it typically has to be applied by the same Terraform version with the same providers installed by `init`. Pass it between jobs as an artifact, and treat it as a secret because it can contain variable values.

Examples

A platform-neutral pipeline: format and validate gates, a read-only lock file, `-detailed-exitcode` handling, a `jq` gate on the plan JSON, then apply of the saved plan (run end to end).

bash
cat > main.tf <<'HCL'
terraform {
  required_version = ">= 1.6"
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

variable "env" {
  type = string
}

resource "random_pet" "app" {
  length  = 2
  keepers = { env = var.env }
}
HCL

terraform init -input=false >/dev/null   # developer, once: creates .terraform.lock.hcl, commit it

# Platform-neutral pipeline sketch: any CI runner that can run bash
set -euo pipefail
export TF_IN_AUTOMATION=1 TF_INPUT=0

terraform fmt -check -recursive              # exit 3 and lists files if unformatted
terraform init -lockfile=readonly            # fail instead of rewriting .terraform.lock.hcl
terraform validate

rc=0
terraform plan -detailed-exitcode -lock-timeout=60s -var env=dev -out=tfplan || rc=$?
case "$rc" in
  0) echo "no changes"; exit 0 ;;
  2) echo "changes present" ;;
  *) echo "plan failed ($rc)"; exit 1 ;;
esac

# Gate on the machine-readable plan, not on text
terraform show -json tfplan | jq -r '.resource_changes[] | select(.change.actions | index("delete")) | "DESTROYS \(.address)"'

# After human or policy approval, in the same job or with tfplan passed as an artifact
terraform apply tfplan

A stale saved plan is refused, exit codes 0 and 2, and the plan as JSON.

bash
cat > main.tf <<'HCL'
resource "terraform_data" "app" {
  input = "v1"
}
HCL

terraform init -input=false -no-color >/dev/null
terraform plan -input=false -out=tfplan -no-color >/dev/null
terraform apply -input=false tfplan -no-color | tail -1

# Plan, then let someone else change the state before the saved plan is applied
sed -i 's/v1/v2/' main.tf
terraform plan -input=false -out=tfplan -no-color >/dev/null
terraform apply -input=false -auto-approve -no-color -replace=terraform_data.app >/dev/null
terraform apply -input=false tfplan -no-color 2>&1 | grep -A3 'Error:'

# Exit codes: 0 no changes, 1 error, 2 changes present
terraform plan -input=false -detailed-exitcode -no-color >/dev/null; echo "after apply: $?"
sed -i 's/v2/v3/' main.tf
terraform plan -input=false -detailed-exitcode -no-color >/dev/null; echo "pending change: $?"

# The saved plan as data
terraform plan -input=false -out=tfplan -no-color >/dev/null
terraform show -json tfplan | jq -r '.resource_changes[] | "\(.address) \(.change.actions | join(","))"'
terraform show -json tfplan | jq -c '{format_version, terraform_version}'

In practice

  • Atlantis — Runs `plan` when a pull request changes Terraform files, posts the output as a comment and applies the saved plan when a reviewer comments `atlantis apply`.
  • HCP Terraform — Hosted runs with built-in state locking and a plan/apply split with approval; the CLI-side flags above describe what such systems do for you.
  • Spacelift — Hosted pipeline that separates planning from approval and apply and can attach policy checks to the plan stage (product behaviour UNVERIFIED here).
  • OpenTofu — `tofu plan -out`, `tofu apply` and `-detailed-exitcode` typically behave like their Terraform counterparts (UNVERIFIED: OpenTofu is not installed here).

Trade-offs

Pros

  • The applied change is byte-for-byte the reviewed one, or the apply fails loudly as stale
  • Exit codes and plan JSON let pipelines decide without parsing text
  • `-input=false` and lock timeouts turn hangs and collisions into fast, explicit failures
  • The same four commands work in any runner, so the pipeline logic stays portable

Cons

  • On a busy state the saved plan goes stale quickly and the approval has to be repeated
  • The plan file may contain secrets and must travel as a restricted, short-lived artifact
  • Plan and apply jobs must use the same Terraform and provider versions, which needs pinning
  • Plan jobs need read access to real infrastructure, so credentials and fork pull requests need care

What `sensitive = true` really hides, why secrets still sit in plaintext in state and saved plans, and what ephemeral values and write-only attributes change.

The problem

A database password is passed in as a `sensitive` variable and a `random_password` feeds the API key, so the team assumes the secret is safe. Then the state file in a shared bucket, a `tfplan` uploaded as a CI artifact and a `terraform output -json` pasted into a job log all contain it in plain text. `sensitive = true` is a display filter in the CLI, not encryption, and every resource attribute Terraform manages ends up in state.

How it works

  • `sensitive = true` on a variable, output or provider attribute only redacts what the CLI prints: the plan shows `(sensitive value)`, and apply and a bare `terraform output` show `<sensitive>` (verified). The state file stores the value in plain JSON, and `jq -r '.resources[0].instances[0].attributes.result' terraform.tfstate` printed the generated password (verified), next to a `sensitive_attributes` list of paths that only marks it for display.
  • Sensitivity propagates through expressions, and Terraform makes you acknowledge it: a root output built from a sensitive variable without `sensitive = true` fails with `Error: Output refers to sensitive values` (verified). `nonsensitive()` removes the mark and `nonsensitive(length(var.db_password))` printed `6` (verified), so use it only for values that are genuinely not secret.
  • Explicit retrieval bypasses the masking. `terraform output api_key` prints the raw value and `terraform output -json` returns `{"sensitive":true,"type":"string","value":"..."}` with the value included (verified), so a script that logs the command output leaks it, and any consumer of the state through `terraform_remote_state` can read it.
  • A saved plan carries secrets as well. `terraform show -json tfplan` printed `{"db_password":{"value":"hunter2-demo"}}` under `variables` (verified), and the file itself is an unencrypted zip with an embedded state and configuration. Upload it only as a short-lived, access-controlled CI artifact, never to a public build log or cache.
  • Where the secret comes from decides what else leaks. `TF_VAR_db_password` or a git-ignored `-var-file` keep it out of the repository, but once a resource argument uses it, it lands in state and plan anyway. Resources that generate secrets (`random_password`, `tls_private_key`, database `password` arguments) always persist them. The safer pattern is to keep the secret in a secrets manager and let Terraform handle only a name or ARN that the application resolves at runtime.
  • Ephemeral values keep data out of plan and state. Terraform 1.10 added ephemeral input variables, ephemeral module outputs and `ephemeral` resources, and 1.11 added write-only attributes, which accept ephemeral values and are never stored (both per the release changelogs). On 1.16.5 an `ephemeral = true` variable or `ephemeral "random_password"` assigned to an ordinary attribute fails with `Error: Invalid use of ephemeral value` and "not a write-only attribute and must be persisted to state" (verified).
  • Ephemeral support depends on the provider. `hashicorp/random` 3.9.1 offers the ephemeral resources `random_password` and `random_bytes` and `hashicorp/local` offers `local_command` (verified from `terraform providers schema -json`), but none of the credential-free providers used here had a write-only attribute to receive the value, so the full flow is UNVERIFIED: check the provider documentation for the write-only argument of the resource you manage (often named with a `_wo` suffix, which is a provider convention to confirm).
  • Protect state like a secret store: a private, versioned, encrypted backend with narrow access and no copy in git (add `terraform.tfstate*`, `*.tfplan` and `.terraform/` to `.gitignore`). How a given backend encrypts at rest is backend-specific and UNVERIFIED here. OpenTofu typically offers client-side state and plan encryption through an `encryption` block with key providers such as `pbkdf2`, `aws_kms`, `gcp_kms` and `openbao` (UNVERIFIED locally, from the OpenTofu documentation); Terraform has no equivalent built-in feature.

Examples

Masked in plan and `terraform output`, plain in the state file, in `output <name>` and in the saved plan's variables.

bash
cat > main.tf <<'HCL'
terraform {
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

variable "db_password" {
  type      = string
  sensitive = true
}

resource "random_password" "api" {
  length = 16
}

output "api_key" {
  value     = random_password.api.result
  sensitive = true
}

output "conn" {
  value     = format("postgres://app:%s@db.internal/app", var.db_password)
  sensitive = true
}
HCL

export TF_VAR_db_password='hunter2-demo'
terraform init -input=false -no-color >/dev/null
terraform plan -input=false -out=tfplan -no-color | grep -E 'result|bcrypt'
terraform apply -input=false tfplan -no-color | tail -4

# Masked in plan and apply output and in a bare `terraform output`, plain everywhere else
terraform output
terraform output api_key
jq -r '.resources[0].instances[0].attributes.result' terraform.tfstate
jq -c '.resources[0].instances[0].sensitive_attributes' terraform.tfstate
jq -c '.outputs.conn' terraform.tfstate

# The saved plan carries the variable values too
terraform show -json tfplan | jq -c '.variables'

Derived sensitive values must be marked, and an ephemeral variable cannot feed an ordinary attribute.

bash
cat > main.tf <<'HCL'
variable "db_password" {
  type      = string
  sensitive = true
}

output "conn" {
  value = format("postgres://app:%s@db.internal/app", var.db_password)
}
HCL

terraform init -input=false -no-color >/dev/null
# 1. Derived sensitive values must be marked, or the plan fails
TF_VAR_db_password=s3cret terraform plan -input=false -no-color 2>&1 | grep -E '^Error'

# 2. Ephemeral values (Terraform 1.10+) cannot be written to ordinary attributes
cat > main.tf <<'HCL'
variable "token" {
  type      = string
  ephemeral = true
}

resource "terraform_data" "demo" {
  input = var.token
}
HCL
TF_VAR_token=abc terraform plan -input=false -no-color 2>&1 | grep -A5 '^Error'

List the ephemeral resources each provider offers, and see an ephemeral resource rejected by a non-write-only attribute.

bash
cat > main.tf <<'HCL'
terraform {
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 3.7"
    }
  }
}

ephemeral "random_password" "db" {
  length = 16
}

resource "terraform_data" "demo" {
  input = ephemeral.random_password.db.result
}
HCL

terraform init -input=false -no-color >/dev/null

# Which ephemeral resources does each installed provider offer?
terraform providers schema -json | jq -c '.provider_schemas | map_values(.ephemeral_resource_schemas // {} | keys)'

# An ordinary attribute must persist to state, so it cannot take an ephemeral value
terraform validate -no-color 2>&1 | grep -A8 '^Error'

In practice

  • HashiCorp Vault — A common place for the secret to live instead of Terraform state; Terraform then handles only the path or reference.
  • AWS Secrets Manager — Terraform can create the secret container while the value is set out of band or read by the application at runtime.
  • OpenTofu — Documents a client-side `encryption` block for state and plan files (UNVERIFIED locally).
  • Checkov — Flags hard-coded secrets and unencrypted state storage patterns in the source before they reach a plan (rule coverage UNVERIFIED here).
  • git-secrets — Pre-commit style scanning that keeps `terraform.tfvars` or state files with credentials out of the repository.

Trade-offs

Pros

  • `sensitive` prevents accidental disclosure in normal CLI output and in plan review comments
  • Propagation rules force an explicit decision before a secret becomes a root output
  • Ephemeral values and write-only attributes can keep a secret out of state entirely, where the provider supports them
  • Treating state as a secret store works with any backend that has encryption and access control

Cons

  • Anyone who can read state or a saved plan can read every secret in it
  • Many resources have no write-only variant yet, so the secret still lands in state
  • `terraform output NAME` and `-json` print sensitive values, so careless logging leaks them
  • Rotating a secret that sits in state means touching state and every consumer of it

Which checks run at which layer: fmt and validate, static scanners (tflint, checkov), native `precondition` and `check` blocks, and policy-as-code on the plan JSON (OPA, Sentinel).

The problem

A security group rule open to `0.0.0.0/0` on port 22 passes `terraform validate`, plans cleanly and is applied on a Friday, because "valid" only means the configuration is well-formed. Teams then bolt on three kinds of checks without knowing which layer catches what: linters that read source, scanners with rule catalogues, and policies that read the plan. The cost is false confidence from a check that cannot see the thing it is supposed to catch.

How it works

  • Think in layers of increasing cost and context. `terraform fmt -check` and `terraform validate` need no credentials and only judge syntax and types; static tools (tflint, checkov, trivy) read the `.tf` source; native `variable` `validation`, `precondition`, `postcondition` and `check` blocks run inside plan and apply with real values; policy-as-code (OPA/conftest, Sentinel) evaluates the plan JSON. A violation that only exists in a computed value is invisible to the static layer.
  • Scanners with rule catalogues read source only. Checkov 3.3.26 (installed with pip) scanned an `aws_security_group_rule` open to `0.0.0.0/0` on port 22 without `terraform init` or any credentials, reported `CKV_AWS_24` as FAILED and exited 1 (verified). `--check CKV_AWS_24` restricts the run to one rule, and `--soft-fail` makes the exit code 0 (verified), which is how teams introduce a scanner in report-only mode first.
  • Exceptions live next to the code and show up in the report. Adding `#checkov:skip=CKV_AWS_24:bastion only, ticket OPS-123` inside the resource turned the result into `Skipped checks: 1` and exit 0 (verified). Review skips like code, require the reason text, and search for them periodically; other tools have their own suppression syntax (UNVERIFIED for trivy and tflint here).
  • tflint is a linter that understands Terraform and, with plugins, provider schemas. TFLint 0.64.0 with the bundled terraform ruleset (installed with `go install`) reported a missing `required_version`, a missing provider version constraint and an unused variable, and exited 2 when it found issues (verified). Cloud-specific rules such as invalid instance types come from provider plugins fetched by `tflint --init` (UNVERIFIED here, needs a download).
  • Native checks use real values. A failing `precondition` inside `lifecycle` stops the plan: `Error: Resource precondition failed` with the evaluated `var.env is "prod"` and exit 1 (verified). A `check` block only warns: `Warning: Check block assertion failed`, and the plan still printed `Plan: 1 to add` and exited 0 (verified), so use `check` for monitoring-style assertions and `precondition` when the run must stop.
  • Plan-level policy reads `terraform show -json tfplan`. An OPA 1.0.0 Rego rule `deny contains msg if { some rc in input.resource_changes; "delete" in rc.change.actions ... }` returned `["terraform_data.svc: plan destroys or replaces this resource"]` for a replacement, and `opa eval --fail-defined` exited 1 when the result was non-empty (verified). Conftest wraps the same pattern for files (UNVERIFIED, not installed). Values that are `(known after apply)` are flagged in `after_unknown` (for example `{"id":true}`, verified) and missing from `after`, so rules about computed values need to handle that.
  • Sentinel is HashiCorp's policy language for HCP Terraform and Terraform Enterprise, run after the plan with the plan data as imports and with advisory, soft-mandatory or hard-mandatory enforcement levels (UNVERIFIED here, from the product documentation). It is a one-line mention because it only runs inside those products; OPA works with the open-source CLI.
  • A check only protects what it gates. A local `terraform apply`, a `-target` run or a second pipeline that skips the scan bypasses all of it, so enforce at the path that holds the credentials: the apply job should consume only a plan that passed the scans, and humans should not hold the same credentials. Tune rules to keep false positives low, or teams will mass-skip them.

Examples

Native checks with real values: `precondition` stops the plan, `check` only warns.

bash
cat > main.tf <<'HCL'
variable "env" {
  type    = string
  default = "prod"
}

resource "terraform_data" "svc" {
  input = var.env

  lifecycle {
    precondition {
      condition     = var.env != "prod"
      error_message = "prod must not be changed from this pipeline."
    }
  }
}

check "env_is_known" {
  assert {
    condition     = contains(["dev", "prod", "staging"], var.env)
    error_message = "env is not one of dev, prod, staging."
  }
}
HCL

terraform init -input=false -no-color >/dev/null

# precondition: a hard stop, plan exits 1
terraform plan -input=false -no-color 2>&1 | grep -B1 -A6 'Resource precondition failed'; echo "exit=${PIPESTATUS[0]}"

# check block: a warning only, plan still exits 0
terraform plan -input=false -no-color -var env=qa 2>&1 | grep -A4 'Warning'; echo "exit=${PIPESTATUS[0]}"

A Rego policy over `terraform show -json` that denies destroy and replace actions, with a failing exit code.

bash
cat > main.tf <<'HCL'
variable "version_tag" {
  type    = string
  default = "v1"
}

resource "terraform_data" "svc" {
  triggers_replace = var.version_tag
}
HCL

cat > deny.rego <<'REGO'
package terraform.plan

import rego.v1

deny contains msg if {
  some rc in input.resource_changes
  "delete" in rc.change.actions
  msg := sprintf("%s: plan destroys or replaces this resource", [rc.address])
}
REGO

terraform init -input=false -no-color >/dev/null
terraform apply -input=false -auto-approve -no-color | tail -1
terraform plan -input=false -var version_tag=v2 -out=tfplan -no-color >/dev/null
terraform show -json tfplan > plan.json

opa eval -f pretty -d deny.rego -i plan.json 'data.terraform.plan.deny'
opa eval --fail-defined -d deny.rego -i plan.json 'data.terraform.plan.deny[_]' >/dev/null; echo "opa exit=$?"

Static scanning with checkov (fail, then a documented skip) and tflint, no init or credentials needed.

bash
cat > main.tf <<'HCL'
resource "aws_security_group_rule" "ssh" {
  type              = "ingress"
  from_port         = 22
  to_port           = 22
  protocol          = "tcp"
  cidr_blocks       = ["0.0.0.0/0"]
  security_group_id = "sg-12345678"
}
HCL

# Static scan: no terraform init, no credentials, no plan needed
checkov -d . --framework terraform --check CKV_AWS_24 --quiet --compact; echo "checkov exit=$?"

# A reviewed exception lives next to the code, with a reason
sed -i '1a\  #checkov:skip=CKV_AWS_24:bastion only, ticket OPS-123' main.tf
checkov -d . --framework terraform --check CKV_AWS_24 --quiet --compact; echo "checkov exit=$?"

printf 'rule "terraform_unused_declarations" {\n  enabled = true\n}\n' > .tflint.hcl
printf 'variable "unused" {\n  type = string\n}\n' > vars.tf
tflint; echo "tflint exit=$?"

In practice

  • TFLint — Terraform linter with a bundled ruleset and provider plugins for cloud-specific checks.
  • Checkov — Rule-catalogue scanner for Terraform source and plan JSON; supports inline skip comments and soft-fail mode.
  • Trivy — Scans Terraform for misconfigurations with `trivy config`; it absorbed the tfsec rules (UNVERIFIED here, not installed).
  • Open Policy Agent, Conftest — Rego policies evaluated against `terraform show -json` output, enforced in CI.
  • HashiCorp Sentinel — Policy-as-code inside HCP Terraform and Terraform Enterprise, between plan and apply.

Trade-offs

Pros

  • Static scanners need no credentials or state, so they run on every pull request in seconds
  • Native `precondition` and `check` blocks travel with the module and use real values
  • Plan JSON policies see exactly what will change, including deletes and replacements
  • Inline skips with reasons make exceptions reviewable instead of silent

Cons

  • Static tools cannot see computed values or what modules resolve to at plan time
  • Rule catalogues produce false positives and are cloud-specific, so they need tuning and upkeep
  • Policies are only as strong as the path that enforces them; local applies bypass them
  • Three tool families mean three configuration formats and three places to look at failures

How `.tftest.hcl` files run plan or apply, assert on results, expect failures, and swap providers or resources with `mock_provider` and `override_resource`.

The problem

A module is changed and the only test is applying it to a sandbox account and looking at the result, which takes ten minutes, costs money and cannot be run for every pull request. Variable validations, conditional resources and output shapes are exactly the logic that breaks silently and could be checked in seconds. `terraform test` runs assertions against a plan or a throwaway apply, and with mocks it can run with no credentials at all.

How it works

  • `terraform test` discovers `*.tftest.hcl` (and `*.tftest.json`) files in the module directory and in `tests/`; `-test-directory` changes the folder, and files in `tests/` were found in the verified runs. Each file is a series of `run` blocks executed in order. Native tests were introduced in Terraform 1.6.0 (per the 1.6 changelog), mocking was added in 1.7.0, and `-junit-xml` became generally available in 1.11.
  • A `run` block uses `command = apply` by default or `command = plan`. Apply runs create real resources through the configured providers, share one state per test file, and are destroyed in reverse order at the end, which the output shows as `tests/basic.tftest.hcl... tearing down` (verified). A failed teardown can leave orphaned resources, so real-provider tests belong in a disposable account.
  • `assert { condition, error_message }` blocks check results. A failure prints `Error: Test assertion failed`, the file and line, a `Diff:` block for `==` comparisons and then the message, ends with `Failure! 0 passed, 1 failed.` and exits 1 (verified); success prints `Success! 3 passed, 0 failed.` and exits 0. Variables come from a file-level `variables {}` block, overridden by a `variables {}` block inside a `run`.
  • `command = plan` cannot assert on values that are unknown until apply. Reading `random_pet.app[0].id` in a plan run failed with `Error: Unknown condition value` and the hint to use an apply run or `override_during = plan` (verified), while `length(random_pet.app)` worked because the count is known at plan time. Test plan-time logic (counts, conditions, names built from inputs) with plan runs and computed results with apply runs or mocks.
  • `expect_failures = [var.env]` inverts the expectation for a variable `validation`, `precondition`, `postcondition` or `check`: the run passes only if that object fails, and any other failure still fails the run (verified with `env = "staging"` against a `contains(["dev", "prod"], ...)` validation). This is how you test that guard rails actually reject bad input.
  • `mock_provider "random" {}` replaces the real provider, so an `apply` run needs no credentials or network, and computed strings get generated placeholders (the verified run produced ids such as `ticqt4vj`). `mock_resource` with `defaults` sets fixed values (`id = "dev-mocked-pet"` made `output.names` assertable), and `override_resource` targets one resource address. `override_data` and `override_module` exist as well (all per the 1.7 changelog).
  • Overrides and mocks have sharp edges. By default an override applies during apply, and `override_during = plan` (added in 1.11) also applies it in plan runs, which made the plan assertion on `random_pet.app[0].id` pass (verified). Overriding only `id` with `override_during = plan` while the mock invented a different `separator` ended an apply run with `Provider produced inconsistent final plan` (verified), so override every attribute the plan fills in, or keep mocks and overrides in separate runs.
  • CI integration: `-filter=<file>` selects test files (two `-filter` flags ran two files in the verified run), `-var`/`-var-file` feed variables, `-verbose` prints each run's plan or state, and `-junit-xml=junit.xml` wrote `<testsuite ... tests="2" failures="1" errors="1">`, where the unknown-value problem counted as an error and the failed assertion as a failure (verified). `-parallelism=n` and parallel run annotations arrived in 1.12 (per its changelog; not exercised here).

Examples

A module with a validation and a count, tested with a plan run, an apply run and `expect_failures`.

bash
mkdir -p tests
cat > main.tf <<'HCL'
terraform {
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "~> 3.6"
    }
  }
}

variable "env" {
  type = string

  validation {
    condition     = contains(["dev", "prod"], var.env)
    error_message = "env must be dev or prod."
  }
}

variable "replicas" {
  type    = number
  default = 2
}

resource "random_pet" "app" {
  count  = var.replicas
  length = 2
  prefix = var.env
}

output "names" {
  value = random_pet.app[*].id
}
HCL

cat > tests/basic.tftest.hcl <<'HCL'
variables {
  env = "dev"
}

run "plan_count" {
  command = plan

  assert {
    condition     = length(random_pet.app) == 2
    error_message = "expected two pets"
  }
}

run "apply_prefix" {
  command = apply

  assert {
    condition     = alltrue([for n in output.names : startswith(n, "dev-")])
    error_message = "every name must start with the env"
  }
}

run "rejects_bad_env" {
  command = plan

  variables {
    env = "staging"
  }

  expect_failures = [var.env]
}
HCL

terraform init -input=false -no-color >/dev/null
terraform test -no-color

`mock_provider` with `mock_resource` defaults, and `override_resource` with `override_during = plan`.

bash
# Same main.tf as before; tests/mock.tftest.hcl needs no provider at all
mkdir -p tests
cat > tests/mock.tftest.hcl <<'HCL'
mock_provider "random" {
  mock_resource "random_pet" {
    defaults = {
      id = "dev-mocked-pet"
    }
  }
}

variables {
  env = "dev"
}

run "mocked_apply" {
  command = apply

  assert {
    condition     = output.names == ["dev-mocked-pet", "dev-mocked-pet"]
    error_message = "names"
  }
}
HCL

cat > tests/override.tftest.hcl <<'HCL'
mock_provider "random" {}

variables {
  env = "dev"
}

override_resource {
  target          = random_pet.app[0]
  override_during = plan
  values = {
    id        = "dev-fixed-name"
    separator = "-"
  }
}

run "plan_sees_override" {
  command = plan

  assert {
    condition     = random_pet.app[0].id == "dev-fixed-name"
    error_message = "override not applied"
  }
}
HCL

terraform test -no-color -filter=tests/mock.tftest.hcl -filter=tests/override.tftest.hcl

Failing output for a wrong assertion and for an unknown value in a plan run, plus the JUnit report.

bash
# Same main.tf as before
mkdir -p tests
cat > tests/fail.tftest.hcl <<'HCL'
variables {
  env = "dev"
}

run "wrong_count" {
  command = plan

  variables {
    replicas = 3
  }

  assert {
    condition     = length(random_pet.app) == 2
    error_message = "expected two pets"
  }
}

run "reads_unknown_at_plan" {
  command = plan

  assert {
    condition     = startswith(random_pet.app[0].id, "dev-")
    error_message = "id prefix"
  }
}
HCL

terraform test -no-color -filter=tests/fail.tftest.hcl -junit-xml=junit.xml; echo "exit=$?"
grep -o '<testsuite [^>]*>' junit.xml

In practice

  • terraform test — The built-in runner for `.tftest.hcl` files, with plan and apply runs, assertions, mocks and JUnit output.
  • Terratest — Gruntwork's Go library that applies real infrastructure and asserts from Go code, used when tests must call real endpoints (UNVERIFIED here).
  • LocalStack — Local cloud API emulator some teams point provider endpoints at to run apply tests without a real account (UNVERIFIED here).
  • OpenTofu — `tofu test` runs `*.tftest.hcl` files and also `*.tofutest.hcl` (a `.tofutest.hcl` file wins over a `.tftest.hcl` of the same base name) and supports `mock_provider` (per the OpenTofu docs; not run here).
  • Checkov — Complements tests: tests check behaviour, a scanner checks configuration against rules (see the policy entry).

Trade-offs

Pros

  • Plan runs and mocked applies finish in seconds and need no credentials
  • `expect_failures` tests that validations and preconditions really reject bad input
  • Assertions live in HCL next to the module, with a JUnit report for CI
  • Real apply runs destroy what they create, so the same file can also be an integration test

Cons

  • Mocks return invented values, so a passing mocked test says nothing about the real provider's behaviour
  • Plan runs cannot assert on values unknown until apply
  • Real apply runs cost money and can leave orphaned resources if teardown fails
  • Override and mock details change between 1.x releases, so pin the Terraform version the tests were written for

How `required_version`, provider constraints, `~>` and `.terraform.lock.hcl` pin versions, what `init -upgrade` changes, and why a newer state format locks older CLIs out.

The problem

A laptop upgrades to a new Terraform, runs one apply and the shared state is rewritten in a way the CI image's older Terraform refuses to read. Elsewhere the lock file was never committed, so a `~> 3.0` provider constraint picks a new minor release on the next fresh `init` and the plan suddenly shows replacements nobody asked for. Terraform has three independent version axes (the CLI, the providers, the modules) plus the state format, and each needs its own pin and its own upgrade routine.

How it works

  • Three constraints pin three things. `terraform { required_version }` constrains the CLI, `required_providers { version }` constrains each provider, and a module block's `version` constrains registry modules. The operators are `=` (or none), `!=`, `>`, `>=`, `<`, `<=` and `~>`. `~> 3.6` allows `>= 3.6.0, < 4.0.0`, while `~> 3.6.1` allows `>= 3.6.1, < 3.7.0` (the rightmost component may increase).
  • `required_version` is enforced on every command. With `~> 1.99` on Terraform 1.16.5, `terraform validate` failed with `Error: Unsupported Terraform Core version` and "This configuration does not support Terraform version 1.16.5" (verified, and the same for `>= 1.6, < 1.10`). Use a range such as `>= 1.6, < 2.0` for modules, and a tighter one such as `~> 1.16.0` for root configurations that your pipeline controls.
  • `.terraform.lock.hcl` records the exact provider version chosen, the constraint it satisfied and its hashes (`h1:` and several `zh:` entries). After `init` with `= 3.6.0` the file held `version = "3.6.0"` and `constraints = "3.6.0"`. Relaxing the constraint to `~> 3.6` and running `init` printed `Reusing previous version of hashicorp/random from the dependency lock file` and kept 3.6.0; only `terraform init -upgrade` installed 3.9.1 and rewrote the entry (all verified). Commit the file.
  • When the lock and the constraint disagree, plain `init` refuses to move: a lock at 3.9.1 and a new `~> 4.0` constraint gave `Error: Failed to query available provider packages ... locked provider ... 3.9.1 does not match configured version constraint ~> 4.0; must use terraform init -upgrade to allow selection of new versions` (verified). An upgrade is therefore always a deliberate `init -upgrade`, reviewed as a diff of the lock file.
  • Hashes are per platform. `terraform providers lock -platform=linux_amd64 -platform=darwin_arm64` added a second `h1:` entry for darwin_arm64 and left the linux one in place (verified: two `h1:` lines afterwards). Record every platform your developers and CI use, otherwise an `init` on an unrecorded platform typically has to add hashes itself or, under `-lockfile=readonly`, fail (the read-only failure for a missing lock was verified; the per-platform case is UNVERIFIED).
  • The lock file also detects tampering. With a lock entry edited to `3.6.0` but the 3.9.1 hashes kept, `init` failed with `Error: Failed to install provider` and "the current package ... 3.6.0 doesn't match any of the checksums previously recorded in the dependency lock file" (verified). In CI, `terraform init -lockfile=readonly` fails with `Error: Provider dependency changes detected` when the lock does not cover the configured requirements (verified with a missing lock file), instead of quietly choosing versions.
  • The state format has its own compatibility rule. A state file carries `version` (4), `terraform_version`, `serial` and `lineage`. Setting `version` to 5 made 1.16.5 stop with `Unsupported state file format: The state file uses format version 5, which is not supported by Terraform 1.16.5` (verified). Setting only `terraform_version` to 1.99.0 or 2.0.0 was accepted and rewritten to 1.16.5 on the next apply (verified); older releases are reported to refuse such a state with "state snapshot was created by Terraform vX, which is newer than current vY" (from public issue reports, UNVERIFIED for current releases).
  • Upgrade in steps. Upgrade the CLI everywhere together (laptops, CI image, any remote runs) because a newer CLI or provider may upgrade state in a way an older one cannot read; this one-way step is typical and UNVERIFIED here. For a provider, read its changelog and upgrade guide, change the constraint on a branch, run `terraform init -upgrade`, then `terraform plan -detailed-exitcode` on every state. A clean minor upgrade (3.6.0 to 3.9.1) exited 0 (verified); a major one typically brings `Unsupported argument` errors or in-place updates and replacements to resolve with edits, `moved` blocks or `import` before the first apply.

Examples

Pin to 3.6.0, loosen to `~> 3.6` (lock wins), `init -upgrade`, lock hashes for two platforms, and the `required_version` error.

bash
cat > main.tf <<'HCL'
terraform {
  required_version = ">= 1.6, < 2.0"
  required_providers {
    random = {
      source  = "hashicorp/random"
      version = "= 3.6.0"
    }
  }
}

resource "random_pet" "app" {
  length = 2
}
HCL

terraform init -no-color | grep -E 'Installing|Installed'
terraform apply -auto-approve -no-color | tail -1
grep -E '^\s+(version|constraints)' .terraform.lock.hcl

# Loosen the constraint: the lock file still wins, nothing moves
sed -i 's/= 3.6.0/~> 3.6/' main.tf
terraform init -no-color | grep -E 'Reusing|Using'

# Only -upgrade re-resolves within the constraint and rewrites the lock entry
terraform init -upgrade -no-color | grep -E 'Installing|Installed'
grep -E '^\s+(version|constraints)' .terraform.lock.hcl
terraform plan -detailed-exitcode -no-color > /dev/null; echo "plan exit=$?"

# Make the lock file valid for the platforms your team and CI really use
terraform providers lock -platform=linux_amd64 -platform=darwin_arm64 -no-color | grep Obtained
grep -c '"h1:' .terraform.lock.hcl

# A pinned Terraform floor/ceiling is enforced on every command
sed -i 's/>= 1.6, < 2.0/~> 1.99/' main.tf
terraform validate -no-color 2>&1 | grep -A1 'Error:'

A `terraform_version` stamp from the future is tolerated by 1.16.5, a newer state format `version` is refused.

bash
cat > main.tf <<'HCL'
resource "terraform_data" "demo" {
  input = "x"
}
HCL
terraform init -no-color >/dev/null
terraform apply -auto-approve -no-color | tail -1
jq '{version, terraform_version, serial}' terraform.tfstate

# A newer terraform_version stamp alone is accepted by 1.16.5 and overwritten on the next write
cp terraform.tfstate state.bak
sed -i 's/"terraform_version": "[0-9.]*"/"terraform_version": "1.99.0"/' terraform.tfstate
terraform plan -detailed-exitcode -no-color >/dev/null; echo "plan exit=$?"
terraform apply -auto-approve -no-color | tail -1
jq -r .terraform_version terraform.tfstate

# A newer state FORMAT version is refused outright
sed -i 's/"version": 4/"version": 5/' terraform.tfstate
terraform plan -no-color 2>&1 | grep -E 'Error|Unsupported state'
cp state.bak terraform.tfstate

In practice

  • tfenv, tenv — Version managers that select the CLI per project, commonly from a `.terraform-version` file (UNVERIFIED here).
  • Renovate, Dependabot — Open pull requests that bump provider and module constraints; lock-file refresh support varies, check each tool's documentation (UNVERIFIED here).
  • Terraform Registry — Source of provider and module versions; `init` resolves constraints against it.
  • HCP Terraform — Per-workspace Terraform version setting, so remote runs use the version you selected (UNVERIFIED here).
  • OpenTofu — Reads the same constraint syntax and `.terraform.lock.hcl` format but resolves providers from its own registry (UNVERIFIED: not installed here).

Trade-offs

Pros

  • A committed lock file makes `init` reproducible and detects altered provider packages
  • `required_version` stops an unintended CLI before it touches state
  • Upgrades become reviewable diffs: constraint change plus lock file change plus plan output
  • `~>` ranges accept patch and minor fixes without opening the door to a new major version

Cons

  • Pins need maintenance: stale locks and ceilings collect security fixes slowly if nobody bumps them
  • Every platform in use needs its hashes recorded, or `init` differs between laptop and CI
  • State upgrades are effectively one-way, so rolling back the CLI after an apply may not work
  • Tight `required_version` ceilings in shared modules force every consumer to upgrade in lockstep

Troubleshooting

plan or apply fails with `Error acquiring the state lock`: read the Lock Info block, decide whether the holder is alive, wait with `-lock-timeout`, and only then `terraform force-unlock` the exact ID.

The problem

A pipeline job or a laptop run ends with `Error: Error acquiring the state lock` and a `Lock Info:` block, and every later run fails the same way. The lock exists to stop two writers from corrupting one state, so the first question is never how to remove it but whether its holder is still working: a 20 minute apply that is simply slow looks identical to a runner that was killed. Unlocking under a live apply recreates exactly the lost update the lock prevents, while waiting on a dead holder blocks the whole team.

How it works

  • Symptom: `plan`, `apply` and `destroy` take the state lock before they read state, so even a read-only `plan` fails while another run holds it (verified: a `plan` during a running `apply` exited 1 after 0.1 s). Commands that write state, such as `import` or the `state` subcommands that modify it, typically lock as well. The error is not a Terraform fault; it reports that someone else holds the lock.
  • Read the `Lock Info:` block first. It has seven fields (verified): `ID` (a UUID, the only thing `force-unlock` needs), `Path`, `Operation` (for example `OperationTypeApply`), `Who` (user@host of the holder, such as a runner name), `Version` (the Terraform version, `1.16.5` here), `Created` (UTC start time) and `Info` (usually empty). `Who` and `Created` answer the real question: which job is this and how long has it been running. The local backend adds `Error message: resource temporarily unavailable`; the http backend printed `HTTP remote state already locked:` instead.
  • The default is to fail at once: `-lock-timeout` defaults to `0s`. With `-lock-timeout=5s` the run printed `Acquiring state lock. This may take a few moments...`, waited 5 s and printed the same error (verified). With `-lock-timeout=15s` against an apply that needed 8 more seconds, the plan waited 8 s and then ran normally (verified). In CI, a timeout of a few minutes turns most collisions into a short queue; it cannot fix a lock whose holder is dead.
  • A dead local run leaves no stale lock. After `kill -9` of an apply the hint file `.terraform.tfstate.lock.info` stayed, but the next `plan` ran normally and the file was gone afterwards, because the local backend uses an operating-system file lock that the kernel drops when the process dies (verified). `terraform force-unlock -force <ID>` on a local backend answers `Failed to unlock state: LocalState not locked` (verified). A normal Ctrl-C is the polite path: the apply printed `Interrupt received.` and `Stopping operation...` and released the lock (verified).
  • A dead remote run can leave a stale lock. Remote backends store the lock as a record, not an OS lock, and nothing removes it when the client vanishes. With the `http` backend as a stand-in, `kill -9` of an apply left the lock in place and every later `plan` failed with the dead holder's ID (verified). S3, GCS and the other remote backends typically behave the same way (not run here). That is the one case `force-unlock` exists for; CI runners that are cancelled hard (container killed, spot instance reclaimed) are the usual source.
  • Confirm the holder is gone before unlocking: check the job or pipeline named by `Who` and `Created`, look for a `terraform` process on that host, and compare the time with your longest known apply. Then run `terraform force-unlock <ID>` with the ID from the error. It asks `Do you really want to force-unlock?` and only `yes` is accepted; without a terminal it fails with `Error asking for confirmation: EOF`, and `-force` skips the question (all verified). Success prints `Terraform state has been successfully unlocked!`. It removes only the lock record: no infrastructure and no state content changes.
  • Passing the wrong ID: the http stand-in accepted any ID, so do not rely on a typo being rejected. Real remote backends typically compare the ID with the stored lock and refuse a mismatch (not verified here), which is why you copy the ID from the current error instead of reusing an old one. Treat `force-unlock` as a manual override and write down who ran it and why.
  • After the unlock, assume the dead run was cut in the middle: run `terraform plan` and read it for resources that were created but not recorded, `tainted` resources, or changes that look half applied; `-refresh-only` shows drift without changing anything (see drift-and-refresh). Prevent the next one: one pipeline job per state (a CI `concurrency` group), `-lock-timeout=5m` on every CI command, graceful cancellation (SIGINT, a grace period) instead of `kill -9`, and never `-lock=false` on `apply`. In the verified run `-lock=false` let a `plan` read while an apply held the lock, which can show half-written data.

Examples

Reproduce contention with a real second process: the error, the Lock Info fields, `-lock-timeout`, and the hint file.

bash
cat > main.tf <<'EOF'
resource "terraform_data" "slow" {
  provisioner "local-exec" {
    command = "sleep 20"
  }
}
EOF
terraform init -input=false > /dev/null
terraform apply -auto-approve > apply.log 2>&1 &
sleep 3

terraform plan; echo "rc=$?"
# Error: Error acquiring the state lock
#
# Error message: resource temporarily unavailable
# Lock Info:
#   ID:        184d893d-5fca-2a24-ce8b-b89af283391a
#   Path:      terraform.tfstate
#   Operation: OperationTypeApply
#   Who:       root@vm
#   Version:   1.16.5
#   Created:   2026-10-10 03:45:28.991253174 +0000 UTC
#   Info:
# rc=1

terraform plan -lock-timeout=5s; echo "rc=$?"
# Acquiring state lock. This may take a few moments...
# ... the same error after 5 s, rc=1

cat .terraform.tfstate.lock.info
# {"ID":"184d...","Operation":"OperationTypeApply","Info":"","Who":"root@vm",...}
wait

The unlock procedure for a stale remote lock (captured on the http backend as a stand-in): ID from the error, confirmation, `-force`, then wait on the next run.

bash
# 1. Read the ID from the error (a remote backend; checked with the http backend)
terraform plan
# Error: Error acquiring the state lock
# Error message: HTTP remote state already locked:
# Lock Info:
#   ID:        11111111-2222-3333-4444-555555555555
#   Who:       ci@runner-7
#   Created:   2026-10-10 01:00:00 +0000 UTC

# 2. Prove the holder is gone (CI job finished or cancelled, no terraform process
#    for that Who), then unlock. Without -force it asks, and only 'yes' passes.
terraform force-unlock 11111111-2222-3333-4444-555555555555
# Do you really want to force-unlock?
#   Terraform will remove the lock on the remote state.
#   Only 'yes' will be accepted to confirm.

terraform force-unlock -force 11111111-2222-3333-4444-555555555555
# Terraform state has been successfully unlocked!

# 3. Make later runs wait instead of failing
terraform plan -lock-timeout=60s

A killed local apply leaves the hint file but no lock; `force-unlock` has nothing to do on the local backend.

bash
terraform apply -auto-approve -replace=terraform_data.slow > apply.log 2>&1 &
PID=$!
sleep 3
kill -9 $PID
sleep 1
ls -a | grep lock.info
# .terraform.tfstate.lock.info      <- hint file survives the kill

terraform plan | grep -E '^(Plan|No changes)'
# Plan: 1 to add, 0 to change, 0 to destroy.   <- but the plan ran: no lock was held
ls -a | grep lock.info
# (nothing: removed by the next lock release)

terraform force-unlock -force 00000000-0000-0000-0000-000000000000
# Failed to unlock state: LocalState not locked

In practice

  • Terraform local backend — Takes an operating-system lock on `terraform.tfstate` and writes `.terraform.tfstate.lock.info` with `ID`, `Operation`, `Who`, `Version`, `Created` and `Path`; a killed process loses the lock with the process, and a second run fails at once with `resource temporarily unavailable`.
  • Terraform http backend — Locks with `LOCK` and `UNLOCK` requests to endpoints you host; the lock is just the JSON the server stores, so a client killed between the two leaves it behind until `force-unlock` sends the unlock.
  • AWS S3 backend — Documented to lock either with a lock object next to the state (`use_lockfile`, generally available in Terraform 1.11) or with the older DynamoDB table, which is deprecated; a lock written by a runner that died is typically removed with `terraform force-unlock` (not run here, check the backend docs for your version).
  • GitLab managed Terraform state — Exposes an http-style state API with lock and unlock endpoints, so a cancelled CI job can leave a lock that shows up in the Lock Info `Who` field (not run here).
  • HCP Terraform — Runs are queued per workspace and the workspace is locked by the platform instead of by a client lease, so the stuck-lock case turns into a stuck run to cancel (one-line mention, not run here).

Trade-offs

Pros

  • Locking makes concurrent writes fail loudly instead of silently losing one run's changes, and the Lock Info block tells you who and when.
  • `-lock-timeout` converts most collisions in CI into a short wait without any new infrastructure.
  • `force-unlock` touches only the lock record, so using it wrongly is recoverable in a way that editing state by hand is not.

Cons

  • On remote backends a crashed client keeps the lock until a human intervenes, so one cancelled job can block every later run on that state.
  • The error cannot tell a slow holder from a dead one; you have to judge from `Who`, `Created` and your CI system.
  • `-lock=false` and a premature `force-unlock` are easy to reach for under pressure and are the two ways to corrupt a state the lock was protecting.

A plan shows `-/+ must be replaced` for something you barely touched: find the `# forces replacement` attribute, trace unknown upstream values, and use `plan -out` plus `show -json` to prove the cause before apply destroys it.

The problem

A routine change, a provider upgrade or a rename produces a plan with `-/+ destroy and then create replacement` for a database, a file or a server that you expected to be updated in place. Replacement is a normal outcome when an attribute cannot be changed on a live object, but in a long plan it hides among dozens of `~` lines, and the same symbol is used for a real change, an unknown value flowing downstream, a tainted resource and a value that changes on every run. Applying without finding the cause is how data gets deleted.

How it works

  • Read the symbols first. A replacement has a header `# ADDR must be replaced` and the line `-/+ resource ...`; the attribute that cannot change in place ends with `# forces replacement`, and the summary counts both sides (`Plan: 2 to add, 0 to change, 2 to destroy.`). Other headers mean other causes: `will be replaced, as requested` (`-replace`), `is tainted, so must be replaced`, and `will be replaced due to changes in replace_triggered_by` (all verified). With `create_before_destroy` the order flips and the symbol is `+/-`.
  • Cause 1, a real change to a ForceNew attribute: changing `prefix` on `random_pet` from `web` to `api` printed `~ prefix = "web" -> "api" # forces replacement` (verified). Providers mark such attributes in their schema (ForceNew in the SDK, `RequiresReplace` in the plugin framework, typically); the list is per resource type and is not visible in your configuration, so read the plan rather than guessing. A plain `input` change on `terraform_data` is an in-place update, while `triggers_replace` forces replacement.
  • Cause 2, an unknown value flowing downstream: `local_file.motd` was never edited, yet its plan said `must be replaced` with `filename = "./motd-web-infinite-pug.txt" -> (known after apply) # forces replacement`, because `filename` interpolates `random_pet.server.id`, and that id is unknown once `random_pet.server` itself is replaced (verified). The reported attribute is the symptom; the root cause is the first replacement in the dependency chain, here `random_pet.server`. `(known after apply)` on a ForceNew attribute is the signature of this cascade.
  • Prove the cause without applying: `terraform plan -out=tfplan` then `terraform show -json tfplan`. Each `resource_changes[]` entry has `change.actions` (`["delete","create"]` for a replacement), `action_reason` and `change.replace_paths` (verified). The reason values seen were `replace_because_cannot_update` (with `replace_paths` such as `[["prefix"]]`), `replace_because_tainted`, `replace_by_request` and `replace_by_triggers`. A `jq` filter on this output also makes a good CI guard against surprise deletes (see terraform-in-ci).
  • Cause 3, a value that changes on every run: `keepers = { stamp = timestamp() }` on `random_pet` showed `~ "stamp" = "2026-10-10T03:46:22Z" -> (known after apply)` under `keepers = { # forces replacement }`, and `plan -detailed-exitcode` returned 2 on every run (verified). `uuid()` in `terraform_data.input` produced an in-place update on every plan instead (verified). Remove the volatile function from the config, move it behind a stable input, or exclude the attribute with `ignore_changes`.
  • Cause 4, a different instance address: renaming a `for_each` key from `b` to `c` plans `will be destroyed` (`because key ["b"] is not in for_each map`) plus `will be created`, with no `forces replacement` marker at all (verified), and the same happens when a resource moves into a module or a `count` index shifts. The fix is a `moved` block, not a lifecycle setting (see moved-and-removed and count-vs-for-each). A provider upgrade can also turn a new default or a newly tracked attribute into a ForceNew diff; the changelog and a plan on a copy of the state are the only reliable check (not reproduced here).
  • Intentional replacement has three tools: `terraform apply -replace=ADDR` (the flag was added in Terraform 0.15.2; it replaced `taint` in the usual workflow), `terraform taint ADDR` (marks state; verified `is tainted, so must be replaced`, and the downstream `local_file.motd` followed) with `terraform untaint ADDR` to undo (verified), and `replace_triggered_by` (Terraform 1.2 and later; verified: changing `terraform_data.note` replaced `follower`). A provisioner that fails also leaves its resource tainted (see plan-apply-lifecycle).
  • Guard and fix: if the change is wanted but the replacement is not, revert the attribute or add it to `lifecycle { ignore_changes = [prefix] }`, which made `-var prefix=api` plan `No changes` (verified; note that state keeps the old value and the config no longer governs that attribute). `prevent_destroy = true` turns an accidental replacement into `Error: Instance cannot be destroyed`, and it applies to `-replace` too (verified); `create_before_destroy` reduces the gap but cannot save data held inside the old object. Always read the `-/+` lines of a plan on stateful resources before approving (see lifecycle-rules).

Examples

Change one ForceNew attribute, watch the replacement cascade to a dependent, and read the cause from the saved plan.

bash
cat > main.tf <<'EOF'
variable "prefix" {
  type    = string
  default = "web"
}

resource "random_pet" "server" {
  prefix = var.prefix
}

resource "local_file" "motd" {
  filename = "${path.module}/motd-${random_pet.server.id}.txt"
  content  = "hello"
}
EOF
terraform init -input=false > /dev/null && terraform apply -auto-approve > /dev/null
terraform plan -var prefix=api -out=tfplan | grep -E 'must be replaced|forces replacement|^Plan'
#   # local_file.motd must be replaced
#       ~ filename = "./motd-web-infinite-pug.txt" -> (known after apply) # forces replacement
#   # random_pet.server must be replaced
#       ~ prefix   = "web" -> "api" # forces replacement
# Plan: 2 to add, 0 to change, 2 to destroy.

terraform show -json tfplan | jq -r '.resource_changes[]
  | select(.change.actions != ["no-op"])
  | [.address, (.change.actions | join("+")), .action_reason, (.change.replace_paths | tostring)]
  | @tsv'
# local_file.motd    delete+create  replace_because_cannot_update  [["filename"]]
# random_pet.server  delete+create  replace_because_cannot_update  [["prefix"]]

Deliberate replacement with `-replace` and `taint`, then `ignore_changes` to stop an unwanted one.

bash
terraform plan -replace=local_file.motd | grep -E 'will be replaced|^Plan'
terraform taint random_pet.server
terraform plan | grep -E 'tainted|must be replaced|^Plan'
terraform untaint random_pet.server
# local_file.motd will be replaced, as requested        (first plan)
# random_pet.server is tainted, so must be replaced     (second plan; local_file.motd follows)

cat > main.tf <<'EOF'
variable "prefix" {
  type    = string
  default = "web"
}

resource "random_pet" "server" {
  prefix = var.prefix

  lifecycle {
    ignore_changes = [prefix]
  }
}

resource "local_file" "motd" {
  filename = "${path.module}/motd-${random_pet.server.id}.txt"
  content  = "hello"
}
EOF
terraform plan -var prefix=api | grep -E 'No changes|^Plan'
# No changes. Your infrastructure matches the configuration.

A volatile value in a ForceNew attribute replaces the resource on every run.

bash
cat > main.tf <<'EOF'
resource "random_pet" "stamped" {
  keepers = {
    stamp = timestamp()
  }
}
EOF
terraform init -input=false > /dev/null && terraform apply -auto-approve > /dev/null
terraform plan -detailed-exitcode | grep -E 'must be replaced|forces replacement|stamp|^Plan'
echo "exit=${PIPESTATUS[0]}"
#   # random_pet.stamped must be replaced
#       ~ keepers   = { # forces replacement
#           ~ "stamp" = "2026-10-10T03:46:22Z" -> (known after apply)
# Plan: 1 to add, 0 to change, 1 to destroy.
# exit=2   (and again on every run)

In practice

  • Terraform CLI — Renders `-/+`, `+/-`, `# forces replacement` and `(known after apply)` from the provider's plan, and exposes the same facts as `action_reason` and `replace_paths` in `terraform show -json`.
  • terraform-provider-aws — Marks attributes such as a database identifier or a subnet's availability zone as requiring replacement in its resource schemas, so the same edit is in-place on one attribute and destructive on another (documented per resource; not run here).
  • hashicorp/random — `keepers` is the textbook trigger: any change to the map replaces the random value, which is useful on purpose and a trap when the map holds a timestamp.
  • Open Policy Agent with Conftest — A common way to turn `resource_changes[].change.actions` into a policy that fails a pull request when a protected resource type is deleted or replaced (a pattern; not run here).

Trade-offs

Pros

  • The plan states the replacement and its cause in text and in JSON before anything is destroyed, so the check costs one `plan -out`.
  • `-replace`, `taint` and `replace_triggered_by` make intentional rebuilds explicit and reviewable.
  • `prevent_destroy` and a JSON guard in CI turn most accidental replacements into a failed check instead of an outage.

Cons

  • The ForceNew list lives in the provider and changes with provider versions, so a harmless-looking upgrade can add a replacement.
  • Cascades report the downstream symptom first; finding the upstream root cause means walking the dependency chain by hand.
  • `ignore_changes` hides the attribute from the configuration permanently, which trades a replacement for silent drift.

`Error: Cycle:` lists the addresses that wait for each other: find the ring with `terraform graph -draw-cycles`, then cut one edge by using a static value, a third linking resource, or a smaller module boundary.

The problem

`terraform validate` or `plan` stops with `Error: Cycle:` and a short list of addresses, with no file name and no line number. Terraform orders every operation by the dependency graph, and two objects that each need the other's result have no valid order, so it refuses before calling any provider. The list names the members of the ring but not which reference to remove, and in a configuration with modules the entries are internal graph nodes such as `module.a.var.peer (expand)` that never appear in your code.

How it works

  • Symptom (verified on 1.16.5): `terraform validate` and `terraform plan` both fail with `Error: Cycle:` followed by indented addresses, for example `terraform_data.db` and `terraform_data.app`, and exit 1 before any provider call. The order of the listed addresses is not the order of the dependencies.
  • Cause: every reference such as `terraform_data.db.id` adds an edge to the graph, and `depends_on` adds one without reading an attribute. A ring appears when edges lead back to the start. In the verified minimal case `app.input = terraform_data.db.id` and `db.input = terraform_data.app.output`. Reading a value that is known statically from a variable or a `locals` entry instead of from the other resource removes the edge.
  • Mixed rings are the hard ones. With `app depends_on cache`, `db` reading `app.id` and `cache depends_on db`, the error listed all three (`terraform_data.db`, `terraform_data.app`, `terraform_data.cache`), and nothing in the message says that two of the edges came from `depends_on` (verified). Search for `depends_on` first when the attribute references look innocent: it is the edge people forget they wrote.
  • Modules add graph nodes. Two module calls that feed each other (`module.a` takes `module.b.id`, `module.b` takes `module.a.id`) failed with six entries: `module.a.var.peer (expand)`, `module.b.output.id (expand)`, `module.b.terraform_data.this`, `module.b.var.peer (expand)`, `module.a.output.id (expand)` and `module.a.terraform_data.this` (verified). Variables and outputs of a module are nodes, so the ring can pass through them; read the `module.X` prefixes to find the two calls to untangle.
  • See the ring: `terraform graph -type=plan -draw-cycles` prints the DOT graph even when a cycle exists and marks the cycle edges with `color = "red", penwidth = "2.0"` (verified). `terraform graph -type=plan -draw-cycles | grep red` lists exactly the ring's edges, and piping the full output to Graphviz `dot -Tsvg` (not run here) draws them in red. Plain `terraform graph` just prints `Error: Cycle:` again, without the flag.
  • A resource that refers to itself is reported differently: `Error: Self-referential block` with `Configuration for terraform_data.self may not refer to itself.` and a file and line number. In the verified run `terraform validate` accepted it and `terraform plan` raised the error, so a clean `validate` does not prove the graph is acyclic for self references.
  • Cut one edge, keeping the direction that carries real data. Options: replace an attribute reference with an input you already know (a variable, a local, a naming rule); move the shared bit into a third resource that references both sides, so both sides depend on nothing that depends on them (verified: `link` referencing `app.id` and `db.id` applied cleanly and the graph listed `link -> app` and `link -> db`); delete a `depends_on` that duplicates an attribute reference; or split a module in two so data flows one way. The classic cloud example is two security groups that allow each other, usually solved by moving the rules into separate rule resources (documented pattern; not run here).
  • Prevent it: run `terraform validate` in CI so a cycle fails the pull request, not the apply; keep data flowing one way between modules (network, then data, then app) and pass only the ids a callee needs; prefer attribute references to `depends_on`, because an attribute edge is narrower and `depends_on` on a whole module call makes everything inside it wait (see module-boundaries and composition-over-monoliths).

Examples

Build the smallest cycle, read the error, and list the ring's edges from the graph.

bash
cat > main.tf <<'EOF'
resource "terraform_data" "app" {
  input = terraform_data.db.id
}

resource "terraform_data" "db" {
  input = terraform_data.app.output
}

resource "terraform_data" "unrelated" {
  input = "fine"
}
EOF
terraform init -input=false > /dev/null
terraform validate; echo "rc=$?"
# Error: Cycle:
#   terraform_data.db
#   terraform_data.app
# rc=1
terraform graph -type=plan -draw-cycles | grep 'color = "red"'
#   "terraform_data.app (expand)" -> "terraform_data.db (expand)" [color = "red", ...]
#   "terraform_data.db (expand)" -> "terraform_data.app (expand)" [color = "red", ...]

A cycle through two module calls, then a cycle that exists only through `depends_on`.

bash
mkdir -p modules/node
cat > modules/node/main.tf <<'EOF'
variable "peer" {
  type = string
}

resource "terraform_data" "this" {
  input = var.peer
}

output "id" {
  value = terraform_data.this.id
}
EOF
cat > main.tf <<'EOF'
module "a" {
  source = "./modules/node"
  peer   = module.b.id
}

module "b" {
  source = "./modules/node"
  peer   = module.a.id
}
EOF
terraform init -input=false > /dev/null
terraform validate; echo "rc=$?"
# Error: Cycle:
#   module.a.var.peer (expand)
#   module.b.output.id (expand)
#   module.b.terraform_data.this
#   module.b.var.peer (expand)
#   module.a.output.id (expand)
#   module.a.terraform_data.this

cat > main.tf <<'EOF'
resource "terraform_data" "app" {
  input      = "app"
  depends_on = [terraform_data.cache]
}

resource "terraform_data" "db" {
  input = terraform_data.app.id
}

resource "terraform_data" "cache" {
  input      = "cache"
  depends_on = [terraform_data.db]
}
EOF
terraform validate; echo "rc=$?"
# Error: Cycle:
#   terraform_data.db
#   terraform_data.app
#   terraform_data.cache

The fix: a third resource that references both sides, so neither side references the other.

bash
cat > main.tf <<'EOF'
resource "terraform_data" "app" {
  input = "app"
}

resource "terraform_data" "db" {
  input = "db"
}

resource "terraform_data" "link" {
  input = {
    app = terraform_data.app.id
    db  = terraform_data.db.id
  }
}
EOF
terraform validate
# Success! The configuration is valid.
terraform apply -auto-approve | tail -1
# Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
terraform graph | grep -- '->' | grep link
#   "terraform_data.link" -> "terraform_data.app";
#   "terraform_data.link" -> "terraform_data.db";

In practice

  • Terraform core — Builds one directed acyclic graph per walk (plan, apply, destroy) from references and `depends_on` and rejects a cycle during graph validation with `Error: Cycle:`, before any provider is called.
  • Graphviz — `terraform graph` emits DOT text; `dot -Tsvg` renders it, and with `-draw-cycles` the ring comes out in red (the DOT output was verified; rendering was not run here).
  • Blast Radius — An open-source tool that turns `terraform graph` output into an interactive diagram, handy once a root module has hundreds of nodes (a third-party tool; not run here).
  • Rover — An open-source Terraform visualizer that reads a plan and draws resources and their dependencies in the browser (a third-party tool; not run here).

Trade-offs

Pros

  • Cycles are caught at validate time, before any API call, so they cost nothing but the fix.
  • `-draw-cycles` and the address list point at the ring's members even when the reference is several modules away.
  • Breaking a cycle with a linking resource or a smaller module usually leaves a clearer design than the one that had the cycle.

Cons

  • The message lists members, not the offending reference, and gives no file or line number.
  • `depends_on` edges are invisible in the listing, so a mixed ring looks like one made only of attributes.
  • Module variables and outputs appear as nodes (`(expand)` entries), which makes the output harder to map back to your code.

`Invalid for_each argument` means the instance keys depend on values only known after apply: derive the keys from static input and look the unknown values up by key, or apply the inputs first with `-target` as a one-off.

The problem

A new `for_each = toset(some_resource[*].id)` plans fine in a second run and fails on a clean state, or fails in CI where there is no state at all. The error says the set includes values that cannot be determined until apply. The loop's keys become part of the resource addresses, so Terraform must know them while planning, before it has created the objects that would supply them. Teams hit this the first time a module is applied to an empty environment, and the common advice to apply twice with `-target` leaves a pipeline that cannot converge in one run.

How it works

  • Symptom (Terraform 1.16.5, verified): the plan lists the planned actions, then `Terraform planned the following actions, but then encountered a problem:`, then `Error: Invalid for_each argument` with the line `for_each = toset(random_pet.zone[*].id)` and the note `random_pet.zone is tuple with 2 elements`, and exits 1. The text reads `The "for_each" set includes values derived from resource attributes that cannot be determined until apply`. Older releases word it as `depends on resource attributes that cannot be determined until apply` (not run here), so search for `Invalid for_each argument` rather than the full sentence.
  • Why it is an error: an instance key is part of the address (`terraform_data.record["a"]`) and the address is the identity in state, so the full key set must be known to build the graph and to show a plan. Values are allowed to be unknown, keys are not. The same rule produced `The "for_each" map includes keys derived from resource attributes that cannot be determined until apply` with `local.m will be known only after apply` for a map built as `{ (random_pet.p.id) = 1 }` (verified), and `count` fails the same way with `Invalid count argument` (see count-vs-for-each).
  • Where unknown keys come from: attributes created in the same apply (`random_pet.zone[*].id`, any cloud id or name that the provider assigns); module outputs that depend on those; and data sources whose read is deferred. A data source with only known arguments is read during plan and its result is a fine `for_each` input; adding `depends_on = [terraform_data.gate]` on a resource that does not exist yet deferred the read and failed with `data.local_file.names.content is a string, known only after apply`, and removing the `depends_on` fixed it (all verified).
  • Preferred fix: make the keys static and keep the unknown part in the value. Build the set from a variable or local (`locals { zones = toset(["a", "b"]) }`), then `for_each = local.zones` with `random_pet.zone[each.key].id` inside; the plan was `Plan: 4 to add` on an empty state and the resources came out as `random_pet.zone["a"]` and `terraform_data.record["a"]` (verified). Iterating over another resource that itself uses `for_each` (`for_each = random_pet.zone`) is also accepted, because its keys are static (verified, `Plan: 4 to add`).
  • One-off workaround, the one the message names: `terraform apply -target=random_pet.zone` creates the inputs, after which a normal `plan` knows the keys (verified: `Apply complete! Resources: 2 added`, then `Plan: 2 to add`). Treat it as a bootstrap step: `-target` prints `Warning: Resource targeting is in effect` and on apply `Warning: Applied changes may be incomplete`, it is not meant for routine use, and any pipeline that needs it is two applies, not one.
  • Keys taken from created ids are also unstable. If `random_pet.zone[0]` is ever replaced, its id changes, so the `record` instance with the old id is destroyed and one with the new id is created, in addition to whatever the replacement already caused. Static keys such as `a`, `b` or a subnet's logical name survive replacement of the objects behind them, which is the second reason to prefer them.
  • Neighbouring errors worth knowing: a list instead of a set or map fails with `Invalid for_each argument ... you have provided a value of type tuple` (wrap it in `toset()` or build a map), and a sensitive value cannot be used as a key (`var.db_password has a sensitive value`); both were verified in the count-vs-for-each entry. Keys that come from a `for` expression over a variable are static as long as the variable has a value at plan time.
  • Verify and prevent: run `terraform plan` against an empty state (a new directory or a fresh workspace) in CI, because a configuration that only plans after a first apply is the signature of this bug. If a plan on a clean state succeeds with `Plan: N to add` and no `Invalid for_each argument`, the keys are static. When a second environment would otherwise need two applies, redesign so that keys come from input, not from objects.

Examples

Reproduce the error with ids as keys, then use the `-target` bootstrap the message suggests.

bash
cat > main.tf <<'EOF'
resource "random_pet" "zone" {
  count = 2
}

resource "terraform_data" "record" {
  for_each = toset(random_pet.zone[*].id)
  input    = each.key
}
EOF
terraform init -input=false > /dev/null
terraform plan > plan.txt 2>&1; echo "rc=$?"
grep -A3 '^Error' plan.txt
grep -E 'includes values|Alternatively' plan.txt
# rc=1
# Error: Invalid for_each argument
#   on main.tf line 6, in resource "terraform_data" "record":
#    6:   for_each = toset(random_pet.zone[*].id)
# The "for_each" set includes values derived from resource attributes that
# Alternatively, you could use the -target planning option to first apply only

# the workaround the message names: converge the inputs first
terraform apply -target=random_pet.zone -auto-approve | tail -1
# Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
terraform plan | grep '^Plan'
# Plan: 2 to add, 0 to change, 0 to destroy.

The preferred fix: static keys, with the apply-time value looked up by key; it plans on an empty state.

bash
cat > main.tf <<'EOF'
locals {
  zones = toset(["a", "b"])
}

resource "random_pet" "zone" {
  for_each = local.zones
}

resource "terraform_data" "record" {
  for_each = local.zones
  input    = random_pet.zone[each.key].id
}
EOF
terraform init -input=false > /dev/null
terraform plan | grep "^Plan"
# Plan: 4 to add, 0 to change, 0 to destroy.
terraform apply -auto-approve | tail -1
terraform state list
# random_pet.zone["a"]
# random_pet.zone["b"]
# terraform_data.record["a"]
# terraform_data.record["b"]

A deferred data source makes `for_each` unknown; removing the needless `depends_on` lets it be read during plan.

bash
cat > main.tf <<'EOF'
resource "terraform_data" "gate" {
  input = "x"
}

data "local_file" "names" {
  filename   = "${path.module}/names.txt"
  depends_on = [terraform_data.gate]
}

resource "terraform_data" "per_line" {
  for_each = toset(split("\n", trimspace(data.local_file.names.content)))
  input    = each.key
}
EOF
printf 'x\ny\n' > names.txt
terraform init -input=false > /dev/null
terraform plan 2>&1 | grep -E '^Error|known only after apply'
#   Error: Invalid for_each argument
#   │ data.local_file.names.content is a string, known only after apply
sed -i '/depends_on/d' main.tf
terraform plan | grep -E '^Plan|Reading'
# data.local_file.names: Reading...
# Plan: 3 to add, 0 to change, 0 to destroy.

In practice

  • Terraform core — Evaluates `for_each` and `count` while building the plan graph, so it needs the full key set and reports `Invalid for_each argument` or `Invalid count argument` when a key is unknown.
  • hashicorp/random — Its ids are generated during apply, which makes `random_pet` and `random_id` the cleanest way to reproduce the error without any cloud account.
  • hashicorp/local — `data.local_file` is read during plan when its arguments are known and deferred to apply when it has a `depends_on` on a resource that is not created yet, which turns its content into an unknown `for_each` input.
  • terraform-aws-modules (community modules) — Typically iterate over input maps and lists supplied by the caller rather than over ids of objects they create, so the same module plans on an empty account (a design pattern; not run here).

Trade-offs

Pros

  • Static keys make the plan show every instance up front, so reviewers see what will exist before anything is applied.
  • Looking values up by key survives replacement of the underlying objects, because the address does not change.
  • Planning on an empty state in CI catches the problem before the first real apply of a new environment.

Cons

  • The key list must be maintained as input; you cannot just iterate over whatever a resource produced.
  • The `-target` bootstrap works once but leaves a pipeline that needs two applies and a warning-laden plan.
  • Deferred data sources are easy to create by accident with `depends_on`, and the error blames the `for_each` line, not the dependency.

A plan that takes minutes: time it, read `TF_LOG=TRACE` per-node times, separate graph width from critical path, then trim refresh, data sources and state size, and treat `-parallelism` and `-target` as tools with limits.

The problem

`terraform plan` takes eight minutes on a root module that nobody remembers being slow, CI jobs time out, and the team starts using `-target` or `-refresh=false` to get anything done. Time is spent in four different places: reading every resource during refresh, reading every data source, walking a graph that is either too narrow for the parallelism or too wide for the provider's API, and loading and diffing a very large state. Each has a different fix, and raising `-parallelism` helps only one of them.

How it works

  • Measure before changing anything: `time terraform plan` for the total and the timestamped `Refreshing state...` and `Reading...` lines for the shape. Zero API latency is still not zero cost: in this sandbox 2000 `random_id` resources (an 885 KB state, no network) took about 10.7 s to plan, 8.8 s with `-refresh=false`, 4.3 s with `-target` on one instance, and 49 s to apply (all measured once; absolute numbers vary by machine, the trend is linear in resource count).
  • Graph width versus parallelism: Terraform walks the graph with `-parallelism=10` operations at a time by default (see workflow-and-graph). Thirty independent `time_sleep` resources of 2 s took 6.7 s at the default, 2.5 s at `-parallelism=30` and 20.7 s at `-parallelism=3` (verified). A chain of five 2 s sleeps took 10.6 s at the default and 10.9 s at `-parallelism=50`: a long dependency chain is the critical path, and no setting shortens it. Only restructuring the dependencies does.
  • Find the slow nodes: `TF_LOG=TRACE TF_LOG_PATH=trace.log terraform plan` writes the whole walk. `TF_LOG=DEBUG` is lighter and shows provider start-up, `TRACE` adds `vertex "ADDR": starting visit` and `vertex "ADDR": visit complete` lines with timestamps (verified), which a small awk script turns into a ranking of the slowest nodes. `TF_LOG_PROVIDER` sets the level for provider logs separately, per the documentation. The files are large (about 50 KB for a tiny plan) and may contain values, so do not attach them to tickets unreviewed.
  • A caveat when reading those times: the visit includes waiting for a free parallelism slot. With 20 one-second data sources and the default 10 slots, ten nodes showed about 1.0 s and ten showed about 2.0 s; with `-parallelism=20` all showed about 1.1 s (verified). A node near the top of the ranking may be queueing, not slow. A genuinely slow node stays slow when you raise the limit.
  • `-refresh=false` skips reading managed resources, which was about 2 s of 10.7 s for 2000 resources here, but it does not skip data sources: all 20 `Reading...` lines still appeared (verified). The cost is blindness: drift is not detected and the plan may be wrong about reality, so use it for a quick local edit loop, never for the plan that is applied. Use `-refresh-only` on a schedule to catch drift separately (see drift-and-refresh).
  • `-target` prunes the graph, and a plan with one target on the 2000-resource state took 4.3 s instead of 10.7 s (measured), but it always prints `Warning: Resource targeting is in effect` and on apply `Warning: Applied changes may be incomplete` (verified). It is for recovery and bootstrap, not a performance setting: skipped resources go unchecked, and it hides the real problem, which is that one state holds too much.
  • Data sources are re-read on every plan. 20 `external` lookups that sleep one second cost about 2.8 s at the default parallelism and 1.7 s at `-parallelism=20` when planned alone (measured). Slow lookups (an `external` script, an HTTP call, a cloud API listing) are paid by every pull-request plan, including ones that change nothing nearby. Reduce them: fewer, narrower lookups, pass values in as variables, or read another stack's outputs instead of re-querying the platform (see data-sources).
  • Large states and rate limits. Refresh and diff time grow with the number of resources in one state, so split by change frequency and ownership (network, data, app) and connect the pieces through variables and outputs (see composition-over-monoliths and state-commands for moving resources between states). Provider API throttling is the other cap: typical symptoms are retry messages in `TF_LOG`, `429` or `Throttling` style errors and a plan that gets slower when `-parallelism` is raised; the fix is usually a lower `-parallelism`, fewer refreshed objects and the provider's retry settings (UNVERIFIED: cloud providers were not available here).

Examples

Measure wall time against `-parallelism` for 30 independent 2 s resources (graph width).

bash
cat > main.tf <<'EOF'
resource "time_sleep" "wide" {
  count           = 30
  create_duration = "2s"
}
EOF
terraform init -input=false > /dev/null
for p in 10 30 3; do
  start=$(date +%s.%N)
  terraform apply -auto-approve -parallelism=$p > /dev/null
  end=$(date +%s.%N)
  printf 'parallelism=%s wall=%.1fs\n' "$p" "$(echo "$end - $start" | bc)"
  terraform destroy -auto-approve > /dev/null
done
# parallelism=10 wall=6.7s   (30 x 2s / 10 slots = 6s, plus startup)
# parallelism=30 wall=2.5s   (one wave)
# parallelism=3  wall=20.7s  (30 x 2s / 3 = 20s)

Rank the slowest nodes from a `TF_LOG=TRACE` log, and see queueing for a parallelism slot.

bash
cat > main.tf <<'EOF'
data "external" "lookup" {
  count   = 20
  program = ["sh", "-c", "sleep 1; echo '{\"n\":\"${count.index}\"}'"]
}
EOF
cat > slowest.sh <<'EOF'
#!/bin/sh
# Print the slowest graph nodes from a TF_LOG=TRACE log.
awk '
function ts(s, a) { split(substr(s, 12, 12), a, /[:.]/); return a[1]*3600 + a[2]*60 + a[3] + a[4]/1000 }
/vertex ".*": starting visit/ { v = $0; sub(/.*vertex "/, "", v); sub(/": starting.*/, "", v); st[v] = ts($1) }
/vertex ".*": visit complete/ { v = $0; sub(/.*vertex "/, "", v); sub(/": visit.*/, "", v); if (v in st) printf "%.2f %s\n", ts($1) - st[v], v }
' "$1" | sort -rn | head -5
EOF
terraform init -input=false > /dev/null
TF_LOG=TRACE TF_LOG_PATH=trace.log terraform plan > /dev/null
sh slowest.sh trace.log
# 2.06 data.external.lookup[10]
# 2.06 data.external.lookup (expand)
# 2.05 data.external.lookup[17]
TF_LOG=TRACE TF_LOG_PATH=trace20.log terraform plan -parallelism=20 > /dev/null
sh slowest.sh trace20.log | sed -n 2,3p
# 1.08 data.external.lookup[9]

What `-refresh=false` and `-target` do and do not skip, and applying a saved plan without a second plan.

bash
cat > main.tf <<'EOF'
data "external" "lookup" {
  count   = 20
  program = ["sh", "-c", "sleep 1; echo '{\"n\":\"${count.index}\"}'"]
}

resource "random_id" "item" {
  count       = 300
  byte_length = 4
}
EOF
terraform init -input=false > /dev/null
terraform apply -auto-approve > /dev/null

# -refresh=false skips resource refresh, but data sources are still read
terraform plan -refresh=false | grep -c 'Reading\.\.\.'
# 20

# -target prunes the graph, and says so
terraform plan -target=random_id.item | grep -E 'Warning|^No changes'
# Warning: Resource targeting is in effect

# a saved plan is applied without a second plan
terraform plan -out=tfplan > /dev/null
terraform apply tfplan

In practice

  • Terraform core — Walks the graph with a semaphore of `-parallelism` (default 10) slots and logs `vertex "ADDR": starting visit` and `visit complete` at TRACE level; a vertex's time includes waiting for a slot.
  • hashicorp/time — `time_sleep` is a credential-free way to make a resource take a known time, which is how the graph-width numbers here were produced.
  • hashicorp/external — Each `data "external"` runs its program during every plan, so a slow script is paid on every pull request; it is the simplest way to simulate a slow data source.
  • terraform-provider-aws — Typically retries throttled API calls with backoff, which shows up as a plan that slows down rather than fails; retry limits are provider settings (documented behaviour; not run here).

Trade-offs

Pros

  • Timing, `TF_LOG=TRACE` and a ranking script need no changes to the configuration and work on any provider.
  • Splitting state shortens plans and also narrows blast radius and lock contention.
  • `-refresh=false` and `-target` give a fast edit loop when used locally and knowingly.

Cons

  • Higher `-parallelism` only helps wide graphs and can trigger provider throttling that makes things slower.
  • `-refresh=false` and `-target` produce plans that skip real checks, so they must not feed the apply that CI runs.
  • Splitting a large state is a migration with its own risk, and cross-stack outputs create ordering between stacks.