Remote State

Terraform allows you to store your state file in a remote location, which can provide better collaboration and sharing capabilities across multiple users and teams. This is especially useful when working on a large infrastructure project, where multiple people need to make changes to the same resources.

In this section, we will explore how to use remote state backends in Terraform!

What is a Remote State Backend?

A remote state backend is a storage location where Terraform can store its state file. The state file contains information about the current state of your infrastructure, such as the resources that have been created, their attributes, and their dependencies. By storing this file in a remote location, multiple people can access and modify the state file, which makes it easier to collaborate and share resources.

Common remote backend providers include:

• Amazon S3 (with optional DynamoDB for locking)
• HashiCorp Cloud Platform (fka Terraform Cloud/Enterprise)
• HashiCorp Consul
• Azure Blob Storage
• Google Cloud Storage

Using a Remote Backend

Using a remote state backend in Terraform is straightforward. First, you’ll need to configure your backend.

Here’s an example of how to configure an S3 backend:

terraform {
  backend "s3" {
    bucket = "my-terraform-state-bucket"
    key    = "terraform.tfstate"
    region = "us-east-1"
  }
}

In this example, we configure an S3 backend and specify the name of the bucket, the name of the state file, and the AWS region.

Next, you’ll need to initialize your backend. This creates the bucket if it doesn’t already exist and sets up the backend configuration. To initialize your backend, run the following command:

terraform init

After initializing the backend, you can use Terraform as you normally would, by creating resources and modifying your infrastructure. When you apply your changes, Terraform will store the state file in the remote location.

Here’s an example of how to use a remote state backend in a Terraform module:

module "web" {
  source = "./modules/web"

  backend "s3" {
    bucket = "my-terraform-state-bucket"
    key    = "modules/web/terraform.tfstate"
    region = "us-east-1"
  }
}

In this example, we specify the backend configuration inside the module. This allows us to store the state file for the module separately from the main state file.

Using different remote backends for different modules can help you manage complex infrastructure more effectively by providing isolation, security, and performance benefits. It is particularly useful in large-scale applications, multi-environment setups, or when different teams are managing different parts of the infrastructure.

Remote State Locking

State locking is a mechanism in Terraform that prevents multiple users or processes from making concurrent changes to the same Terraform state. Without locking, multiple operations could run at the same time, potentially leading to state corruption, race conditions, or inconsistent infrastructure deployment.

When remote state locking is enabled, Terraform ensures that only one Terraform process can modify the state at any given time. This is especially important in environments where multiple people or automation systems are managing the same infrastructure.

How Remote State Locking Works

When you run a command like terraform plan or terraform apply, Terraform attempts to lock the state before performing any operations. If a lock is successfully acquired, Terraform proceeds with the operations. After the operations complete, Terraform releases the lock.

If another user or process tries to acquire the lock while it’s in use, that operation will be blocked until the lock is released. This prevents any concurrent changes that could result in conflicting updates to the infrastructure.

Handling State Lock Conflicts

If someone else is already running an operation, Terraform will block your command and inform you that the state is locked. You can choose to wait until the lock is released.

However, if a lock is accidentally left in place (e.g., due to a process being killed), you might need to manually force-unlock the state:

terraform force-unlock LOCK_ID

Backend Providers that Support State Locking

Not all backend providers support remote state locking. However, many of the most commonly used ones do.

Here are a few examples:

Amazon S3 with DynamoDB: When using the S3 backend, you can configure a DynamoDB table to serve as a lock database. This setup prevents concurrent state modifications.

Terraform Cloud/Enterprise: State locking is natively supported in Terraform Cloud and Enterprise without any extra configuration. It automatically locks the state file when you run a Terraform operation.

HashiCorp Consul: Consul as a backend provider supports state locking natively through its key-value store.

Google Cloud Storage (GCS): State locking is supported using Google Cloud Storage with native locking mechanisms.

Azure Blob Storage: Azure Blob Storage backend supports state locking as part of its native functionality.

This command forcibly removes the state lock. The LOCK_ID will be provided by Terraform when it encounters a locked state.

Benefits of Remote State Locking

• Prevents Conflicts: Locking ensures that only one user or process can apply changes to the infrastructure at a time, avoiding conflicts and potential state corruption.
• Safe Collaboration: In teams or multi-environment setups, locking enables safe collaboration across team members, preventing two users from running Terraform at the same time.
• Automated Workflows: State locking is crucial in CI/CD pipelines where Terraform may be run by automation tools. Locking ensures that pipeline runs are isolated and do not interfere with each other.

Conclusion

Using a remote state backend in Terraform can provide better collaboration and sharing capabilities across multiple users and teams. By storing the state file in a remote location, you can ensure that everyone is working with the same version of the infrastructure and can easily see what changes have been made - along with adding additional security benefits, such as not losing the current state of your entire application!