Create Multiple Resources at Once With Terraform for each

Creating dynamic infrastructures with Terraform used to be a challenge. Start using the for_each-meta-argument to safely and predictably create your infrastructure while limiting code duplication.

This post gives you a real-world example of how to effectively use the for_each meta-argument of Terraform 0.12.

As an example, I will take the GCP storage bucket module I talked about in my previous post about the object type.
The module will be refactored so I can create multiple buckets by providing it with all of the bucket settings at once. We will see how using the for_each meta-argument will make it easier to add, and remove buckets with Terraform.

The previous approach using count

Previously you would use the count meta-argument to create multiple resources with one definition. If you did, you’ll also know the problems of this workaround as it doesn’t allow you to override the variables for each resource that easily, and it can give unwanted results if you change the count value and thus removing a resource from the list.
In the example below, I will show you how you may have used this meta-argument:

resource "google_storage_bucket" "count" {
  count      = 4

  name               = format("bucket_%s", count.index)
  location           = "europe-west4"
  storage_class      = "REGIONAL"
  bucket_policy_only = true
}

When you use count as in the example above, you are limited in the variables passed to the resource definition. As a solution, you might have used a lookup variable like this:

locals {
  bucket_settings = [
    { name = "bucket_1", location = "europe-west4", bucket_policy_only = false },
    { name = "secrets", location = "europe-west1", bucket_policy_only = true },
    { name = "documents", location = "europe-west1", bucket_policy_only = true },
    { name = "public", location = "europe-west4", bucket_policy_only = false },
  ]
}

resource "google_storage_bucket" "list" {
  count      = 4

  name               = local.bucket_settings[count.index].name
  location           = local.bucket_settings[count.index].location
  storage_class      = "REGIONAL"
  bucket_policy_only = local.bucket_settings[count.index].bucket_policy_only
}

While this does give you more flexibility, it still gives you some problems when you want to remove or change the order of our resources.
Terraform will create the resources, each with their own state key, which is using the count index number:

google_storage_bucket.list[0]
google_storage_bucket.list[1]
google_storage_bucket.list[2]
google_storage_bucket.list[3]

When removing a bucket, which is not the last one in the list, all buckets after that will shift 1 position. This results in Terraform wanting to delete them and recreate them with a new state key.

The Terraform for_each Meta-argument

As of Terraform 0.12.6, we can use the for_each function in the creation of resources.
I’ve updated the previous example with this new for_each function.

locals {
  bucket_settings = {
    "bucket_1"  = { location = "europe-west4", bucket_policy_only = false },
    "secrets"   = { location = "europe-west1", bucket_policy_only = true },
    "documents" = { location = "europe-west1", bucket_policy_only = true },
    "public"    = { location = "europe-west4", bucket_policy_only = false }
  }
}

resource "google_storage_bucket" "map" {
  for_each      = local.bucket_settings

  name               = each.key
  location           = each.value.location
  storage_class      = "REGIONAL"
  bucket_policy_only = each.value.bucket_policy_only
}

I have changed the local variable from a list into a map and replaced the list-lookup with each.value.<attribute>. for_each can work either with a set of string or a map value. In my case, I’ve chosen a map, so I can pass separate settings for each bucket while using the keys for generating my bucket names.
If you run this code, you will see Terraform will use the map keys as an index for the state keys of your resources:

google_storage_bucket.map["bucket_1"]
google_storage_bucket.map["documents"]
google_storage_bucket.map["public"]
google_storage_bucket.map["secrets"]

This will allow you to add or remove buckets to your map without worrying that Terraform will touch the other resources created.

Conclusion

You now know how to utilize this awesome for_each functionality that has been added in Terraform 0.12.
If you want to see this fully in action, you can check out the GCP bucket module I’ve created. In this module, I’ve combined the use of the terraform object type with the for_each meta-argument having a flexible module as a result.