Backends and State Locking
A backend decides where state lives and whether it is locked during writes: the local backend uses an OS file lock, remote backends add their own locks, and `-lock=false`, `-lock-timeout`, `-backend-config` and `init -migrate-state` control the edges.
The problem
Two engineers run `terraform apply` against the same state on the same afternoon, or a CI job and a laptop do. Without a lock, both read serial 12, both write their own result, and one change silently disappears from state while the real objects exist. With a lock the second run stops with `Error acquiring the state lock`, but then someone's laptop dies mid-apply and nobody knows whether it is safe to force the lock open. Knowing what a backend can and cannot guarantee decides how a team stores state.
How it works
- A backend is chosen in a `terraform { backend "<type>" { ... } }` block; at most one is allowed, and without it Terraform uses the `local` backend and `terraform.tfstate`. The block cannot use variables or other named values: `path = var.p` fails at `init` with `Error: Variables not allowed` (verified), which is why partial configuration exists.
- The local backend locks with an operating-system file lock on the state file and writes a hint file `.terraform.tfstate.lock.info` holding the holder's `ID`, `Operation`, `Who`, `Version`, `Created` and `Path` (verified). The hint file is removed when the lock is released. Contention fails immediately: a `terraform plan` during a running apply exited 1 with `Error: Error acquiring the state lock`, `Error message: resource temporarily unavailable` and a `Lock Info:` block listing `ID`, `Path`, `Operation: OperationTypeApply`, `Who`, `Version` and `Created` (verified).
- `-lock-timeout=30s` makes a command print `Acquiring state lock. This may take a few moments...` and keep retrying until the holder releases or the timeout passes; the verified run waited out a 15 s apply and then planned normally. The default is `0s`, so no waiting. `-lock=false` skips locking completely: in the verified run it let a `plan` read state while an apply held the lock, which can show half-written data and must never be used for `apply`.
- A killed process does not leave a stale local lock. After `kill -9` of an apply the `.lock.info` file remained, but the OS lock was gone, so the next `plan` ran and the file disappeared (verified). `terraform force-unlock -force <ID>` on a local backend only answers `Failed to unlock state: LocalState not locked` (verified).
- Remote backends hold locks as remote records (a lease, an object, a table item) and these can outlive a dead client; that is the case `terraform force-unlock <LOCK_ID>` exists for, using the ID printed in the error. Confirm the holder is really gone first: unlocking under a live apply invites exactly the lost update the lock prevents.
- Partial configuration keeps environment-specific values out of the `.tf` files: write `backend "local" {}` and pass `-backend-config=dev.local.tfbackend` (a file of `key = "value"` lines) or `-backend-config="path=state/dev.tfstate"` at `init`. The resolved settings are cached in plain text in `.terraform/terraform.tfstate`, so do not pass credentials this way if that directory is shared (verified path and contents).
- Changing a backend is an explicit act. A plain `init` asks `Do you want to copy existing state to the new backend?` and with no terminal fails with `Error asking for confirmation: EOF`. `-migrate-state` (add `-force-copy` to skip the question) copies state to the new place; `-reconfigure` switches without copying, so `terraform state list` afterwards printed `No state file was found!` (all verified). Changing the settings of an already initialized backend without either flag gives `Error: Backend configuration changed`.
- Remote backends from the documentation (not run here, so verify against your version): S3 takes `bucket`, `key`, `region` and locks either through an object `<key>.tflock` with `use_lockfile = true` (S3-native locking was added in 1.10; 1.11 deprecated the DynamoDB arguments in its favour) or through the deprecated `dynamodb_table`; GCS stores one object per workspace as `<prefix>/<name>.tfstate`; `azurerm` stores a blob in a storage account container with native locking; the `cloud` block uses HCP Terraform. Docs for S3 and GCS recommend enabling object versioning.
Examples
A second command hits the local lock: the error, the hint file, -lock=false and -lock-timeout.
mkdir lock-demo && cd lock-demo
cat > main.tf <<'HCL'
resource "terraform_data" "slow" {
provisioner "local-exec" {
command = "sleep 15"
}
}
HCL
terraform init > /dev/null
# terminal 1 (simulated): a long apply holds the lock
terraform apply -auto-approve > apply.log 2>&1 &
sleep 3
# terminal 2: any command that locks fails at once with exit code 1
terraform plan 2>&1 | grep -E 'Error acquiring|Error message|Operation:|Path:'
echo "exit=${PIPESTATUS[0]}"
# the local lock is an OS file lock plus a hint file with the holder
cat .terraform.tfstate.lock.info; echo
# -lock=false skips locking: the plan runs, and can read state while it is being written
terraform plan -lock=false | grep -E '^Plan:'
# wait for the lock instead of failing (retries until the timeout)
terraform plan -lock-timeout=30s | grep -E 'Acquiring|^No changes'
wait
ls -a | grep lock.info || echo "lock info file removed after the apply finished"Partial backend configuration, -migrate-state, and -reconfigure leaving the state behind.
mkdir backend-demo && cd backend-demo
cat > main.tf <<'HCL'
resource "terraform_data" "x" {
input = "demo"
}
HCL
# 1. start with the implicit local backend
terraform init > /dev/null
terraform apply -auto-approve > /dev/null
ls
# 2. declare a backend with no arguments: values arrive at init time
cat > backend.tf <<'HCL'
terraform {
backend "local" {}
}
HCL
echo 'path = "state/dev.tfstate"' > dev.local.tfbackend
# a plain init would ask "Do you want to copy existing state to the new backend?"; with no terminal it fails
terraform init -backend-config=dev.local.tfbackend 2>&1 | grep -E '^Error'
# 3. -migrate-state copies the existing state into the new location
terraform init -backend-config=dev.local.tfbackend -migrate-state -force-copy > /dev/null
ls state
terraform plan -detailed-exitcode > /dev/null; echo "plan exit=$?"
# the chosen settings are cached here, not in your .tf files
grep -A3 '"config"' .terraform/terraform.tfstate
# -reconfigure points at a different location WITHOUT copying state
terraform init -backend-config="path=state/other.tfstate" -reconfigure > /dev/null
terraform state list 2>&1 | head -1 | sed 's/\x1b\[[0-9;]*m//g'In practice
- Amazon S3 — The most common remote backend; docs describe lock objects named `<key>.tflock` via `use_lockfile` and recommend bucket versioning (not run here).
- Google Cloud Storage — Stores one `<prefix>/<workspace>.tfstate` object per workspace and supports customer-supplied or Cloud KMS keys per the docs (not run here).
- Azure Blob Storage — The `azurerm` backend stores state in a container blob with the platform's native locking per the docs (not run here).
- HCP Terraform — The `cloud` block stores and locks state in a workspace and can run plan and apply remotely.
- OpenTofu — Typically uses the same backend block syntax; its S3 backend documents `use_lockfile` and still supports `dynamodb_table` (no deprecation planned per its docs). Details of newer features may differ (not run here).
Trade-offs
Pros
- A lock turns concurrent applies into a clear error instead of a lost update
- `-lock-timeout` lets CI queue behind a running job without custom scripting
- Partial configuration keeps one codebase usable across environments
- Remote backends with versioning give recovery points for state
Cons
- Remote locks can go stale after a crash and need a human decision to force-unlock
- `-lock=false` and `force-unlock` remove the only protection against lost updates
- Backend settings cannot use variables, so initialisation scripts carry `-backend-config` arguments
- Backend-specific features (lock mechanism, encryption) differ and are only partly covered by the common docs