Terraform has two looping mechanisms for creating multiple resources, count
and for_each
. The count
meta-argument has been around for a long time, but for_each
is a relative newcomer (introduced in version 0.12). Each meta-argument allows you to create more than one resource or module with a single configuration block.
A common question is when to use count
versus for_each
. I would make the argument that for_each
is almost always preferred, and in this post I hope to show you why.
Before I explain why for_each
is generally superior, it would be useful to understand what is actually happening when you add a looping meta-argument to a Terraform configuration. Let’s start with a simple example.
resource "local_file" "count_int_loop" {
count = 3
content = "This is file number ${count.index}"
filename = "${path.module}/int-${count.index}.count"
}
Running terraform apply
will generate three files: int-1.count, int-2.count, int-3.count. If we take a look at the state data:
$> terraform state list
local_file.count_int_loop[0]
local_file.count_int_loop[1]
local_file.count_int_loop[2]
We have three resources created by the count
meta-argument with an integer based index. The object local_file.count_int_loop
is an ordered list of local_file
resource objects.
If we want four files instead of three, all we have to do is increase the count value by one and Terraform will create a fourth file. This is what count
was meant for, undifferentiated resource creation based on an integer.
But what if instead of an integer, we were dealing with a list of items?
Before the introduction of for_each
, the count
argument was all we had. (And we liked it.) You could use a list to create mulitple resources by finding the number of elements in the list and using that for the count
value. The length()
function does an admirable job of accomplishing this.
Consider the following configuration with a list of toppings defined as a local value.
locals {
toppings = ["lettuce","tomatoes","jalapenos"]
}
resource "local_file" "count_loop" {
count = length(local.toppings)
content = "${local.toppings[count.index]}"
filename = "${path.module}/${local.toppings[count.index]}.count"
}
Running terraform apply
will generate three files: lettuce.count, tomatoes.count, and jalapenos.count. If we take a look at the state:
$> terraform state list
local_file.count_loop[0]
local_file.count_loop[1]
local_file.count_loop[2]
Once again, we have three resources created by the count
meta-argument with a number based index. Just like before, the object local_file.count_loop
is an ordered list of local_file
resources.
So far, so good, right? What’s the point of a for_each
arguemnt if we can simply use the count
argument with a length
function? Let’s try the same thing with for_each
instead:
resource "local_file" "for_each_loop" {
for_each = toset(local.toppings)
content = "${each.value}"
filename = "${path.module}/${each.value}.foreach"
}
Looking at our state now:
$> terraform state list
local_file.for_each_loop["jalapenos"]
local_file.for_each_loop["lettuce"]
local_file.for_each_loop["tomatoes"]
We have three resources created by the for_each
meta-argument with a key based reference. The object local_file.for_each_loop
is a map (aka hashtable). The keys will be the strings in the set, or if you submit a map it will be the keys of the map. The map values are the local_file
resources.
From a practical standpoint, we have essential generated three files with the same content. Either argument seems to do the trick, so why would you prefer one over the other? Two reasons: consistency and referencing.
Something that’s not immediately obvious is how much the order of the items used by count
matters. Let’s say we want to add a topping to our list. How about some onions?
locals {
toppings = ["lettuce","tomatoes","onions","jalapenos"]
}
What do you think will happen when we run terraform plan
against the count
example? Notice the order of the toppings. Onions is now in index 2 and jalapenos is in index 3.
$> terraform plan
Terraform will perform the following actions:
# local_file.count_loop[2] must be replaced
-/+ resource "local_file" "count_loop" {
~ content = "jalapenos" -> "onions" # forces replacement
~ filename = "./jalapenos.count" -> "./onions.count" # forces replacement
~ id = "626451b23e9097d6a2c081959703df63424602bf" -> (known after apply)
# (2 unchanged attributes hidden)
}
# local_file.count_loop[3] will be created
+ resource "local_file" "count_loop" {
+ content = "jalapenos"
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "./jalapenos.count"
+ id = (known after apply)
}
# local_file.for_each_loop["onions"] will be created
+ resource "local_file" "for_each_loop" {
+ content = "onions"
+ directory_permission = "0777"
+ file_permission = "0777"
+ filename = "./onions.foreach"
+ id = (known after apply)
}
Plan: 3 to add, 0 to change, 1 to destroy.
Terraform is going to destroy our jalapenos! And that is because when Terraform runs through the count
loop, it sees the value onions in index 2 and that value used to be jalapenos. Terraform has to destroy the original local_file.count_loop[2]
resource and replace it with the new value. Then it will create a new resource called local_file.count_loop[3]
using the jalapenos value.
The for_each
loop doesn’t have this problem. Since it is using a key based reference, it doesn’t care about order. In fact, the for_each
argument requires a set or map as input, neither of which are ordered. Terraform simply sees the key onion without a corresponding entry in local_file.for_each_loop
and decides to create one. The jalapenos file is not touched. Thank goodness! I like my jalapenos.🌶️
Count is going to unnecessarily destroy and recreate the jalapenos file, which might not be a problem for a text file. But imagine that’s a Kubernetes cluster running 100+ applications, and you just destroyed it because you added a new cluster in the wrong order. That’s… bad. Possibly a resume generating event.
Of course, that would never happen becuase you run terraform plan
first, right? Right???
As pointed out by the previous section, using count
results in an ordered list and for_each
results in a map. When you need to reference the resources somewhere else in your configuration, you might find that being able to refer to a resource by key instead of index is much easier. Let’s look at a slightly more advanced example where we are trying to create users and groups in Terraform Cloud.
Users are created with the resource type tfe_organization_membership
. I could create users with a count
like this:
resource "tfe_organization_membership" "org_members" {
count = length(local.users)
organization = local.organization_name
email = local.users[count.index]
}
Or with a for_each
like this:
resource "tfe_organization_membership" "org_members" {
for_each = toset(local.users)
organization = local.organization_name
email = each.value
}
To add a user to a team on Terraform Cloud, the resource type tfe_team_organization_member
is used. The two arguments team_id
and organization_membership_id
both require a value that is an attribute of the previously generated team or user. It is a value we must look up using a reference. If we’re using the for_each
loop to create users, the reference for organization_membership_id
looks like this:
organization_membership_id = tfe_organization_membership.org_members[each.value["member_name"]].id
We are using the key member_name
to find the correct instance of tfe_organization_membership
and returning the id
attribute of that instance. While the code looks a little confusing at first, trust me it works. The full block is shown below.
resource "tfe_team_organization_member" "team_members" {
for_each = { for member in local.team_members : "${member.team_name}_${member.member_name}" => member }
team_id = tfe_team.teams[each.value["team_name"]].id
organization_membership_id = tfe_organization_membership.org_members[each.value["member_name"]].id
}
If we had used a count
argument to create the users, we would need to use a for
expression with a filter to look up the membership id value. Something along the lines of this:
organization_membership_id = ([for user in tfe_organization_membership.org_members : user.id if user.email == each.value["member_name"]])[0]
Would it work? Sure. Is it efficient? Nope. The for
expression has to loop through all the tfe_organization_membership
resources to find the one that matches. No big deal if we have three users. Pretty big deal if we have three thousand. Reference by key is one of the best things about maps/hashtables/dictionaries, whatever you want to call them.
You can use count
if you don’t care about uniqueness or references in your configuration. If every item created by a loop is ephemeral and functionally identical, then there’s probably no benefit to using for_each
. If you don’t need to refer to anything by the key, then using count
could be fine. On the other hand, it’s about the same amount of work to use either, and for_each
has some serious benefits.
One use that still seems relevant is using count with a zero value to make the creation of a resource optional. What do I mean? Consider this:
resource "local_file" "count_optional" {
count = local.create_file ? 1 : 0
content = "Hello!"
filename = "${path.module}/count-create.txt"
}
The creation of a new organization will happen if the variable create_new_organization
is set to true
and not if its set to false
. You’re only ever creating one or zero of an item. Is there a way to replace this with for_each
? If so, is there any benefit?
To answer the first question. Yes, you can do it.
resource "local_file" "for_each_optional" {
for_each = local.create_file ? toset(["any_value"]) : toset([])
content = "Hello!"
filename = "${path.module}/for-each-create.txt"
}
Is there any benefit? Not that I can think of. The count
version feels more intuitive, but I don’t think it would be any more effective than the for_each
loop.
I started out the looping overview by using an integer with count
. This is the one time when count
has a clear advantage. The count argument takes a number and counts up to that number. You can access the current iteration using the count.index
expression. Count
makes more sense if I am creating resources based off a number, instead of set, list, map, or other object.
Could you replace it with a for_each
argument? Would there be any benefit?
To answer the first question, yes you can do it.
for_each = toset(range(3))
The range
function creates a list of integers starting with 0 and going to the max, non-inclusive. The toset()
function turns that list of integers into a set. The each.value
value will be the integer of the current iteration, so it’s basically the same as index.count
.
Is there any benefit? I’d have to say no. The syntax is clunky and requires the execution of two functions to get it working. Contrasted to the count
argument, there is no tangible benefit of using for_each
in this situation.
To sum up, here’s the general advice. If you are creating multiple resources based off an integer, and the resources are undifferentiated, then count
works just fine. Any time you are using an input that is more complex than an integer, the proper answer is going to be for_each
. Conditional resource creation is a bit of a toss up, so just do what feels intuitive to you or aligns with your team’s conventions.
Deploying Azure Landing Zones with Terraform
November 12, 2024
October 18, 2024
What's New in the AzureRM Provider Version 4?
August 27, 2024
Debugging the AzureRM Provider with VSCode
August 20, 2024