Terraform For Expressions
Brendan Thompson • 14 October 2022 • 11 min read
The for
expression in the Terraform configuration language allows us to construct complex objects
out of pieces of information from multiple sources! The result of a for
expression can either be
a map
or a list
as well as any combination of those two things. These expressions can be very
simple or insanely complex (and yes, I mean insanely). Today we will try and cover a few
common scenarios when dealing with the for
expression. I will also cover the for_each
meta-argument in this post as these two concepts are often used together.
for
Expression#
I am going to split this section into a few sub-sections to make it easier to digest the information, those will be:
But first, let's have a look at the syntax for a for
expression. Before I begin that we need
something to work with, in this instance, I am getting a list of 5 random pet names.
resource "random_pet" "this" {
count = 5
}
In this first example, we are using the result of random_pet.this
(which is a list
) splitting
each item on the -
and then taking the first index. This example does nothing other than taking a
list
splitting the values and returning a list
with the split values.
locals {
first = [
for i in random_pet.this :
split("-", i.id)[0]
]
}
In this next example, we are taking the same result of random_pet.this
and creating a map
. We will
split each value the same as before except this time the first element will be the key and the second
the element will be the value for our map
.
locals {
both = {
for i in random_pet.this :
split("-", i.id)[0] => split("-", i.id)[1]
}
}
In summary, we have a few components:
- The result type, which is shown by the use of
[]
for alist
and{}
for amap
- The
for
keyword - Either an index (
i
) or key-value (k, v
) temporary symbol, depending on your input type - The
in
keyword - The input value itself, which can be a;
list
,set
,tuple
,map
orobject
- The
:
symbol marks where thefor
expression finishes and the assignment begins - Assignment:
- No special symbols are required for the
list
result type - The
=>
assignment symbol is used when the result type is amap
with the left-hand side being thekey
and the right-hand side thevalue
- Finally the optional
if
keyword for filtering things out or validation
for
expressions, and we will look into this later.Filtering#
Now we have the basics of the for
expression down let's see how we can filter some inputs and
return only what is relevant to us. Firstly let's set up some test data:
locals {
projects = {
customer_api = {
region = "australiasoutheast"
environments = ["dev", "uat", "prd"]
}
partner_api = {
region = "australiasoutheast"
environments = ["dev", "prd"]
}
internal_api = {
region = "australiaeast"
environments = ["prd"]
}
}
}
In the test data map
called projects we have various API project names, what region they
are in as well as their environments. If we wanted to filter out everything that isn't in
australiasoutheast
we would do that by using the following for
expression:
locals {
australiasoutheast_region = [
for k, v in local.projects :
k
if v.region == "australiasoutheast"
]
}
On the highlighted line you can see we are using the if
keyword to ensure that values that will
go into our returned list
have in the region australiasoutheast
. We could of course do the
opposite:
locals {
australiasoutheast_region = [
for k, v in local.projects :
k
if v.region != "australiaeast"
]
}
Now we are doing a negative check to say when the region is not australiaeast
. Either of these
will work and will return to us only the piece of information we require. But wait, there's more! We
have multiple conditions on our if
using the ||
(or) and &&
(and) symbols, and these can
further be grouped using ()
. Let's see an example where we only want to return projects in
australiasoutheast
with a uat
environment.
locals {
australiasoutheast_region = [
for k, v in local.projects :
k
if v.region == "australiasoutheast" && contains(v.environments, "uat")
]
}
As you can see the filtering available to us is pretty powerful.
Grouping#
What happens when you have a data structure that has different projects, users, or maybe subscriptions,
and you want to do something on the point of commonality between them this is where grouping comes
into play. An example of this is you might have an object containing users or service accounts and
with these the roles or groups they need to be in, we could use grouping to pass the users for a
particular group into a resource or module that assigns them that access. Another example might be
operating on services or projects in a given common region or performing tasks on things within a
common Azure subscription. For our example, we will use an extended version of the projects
local
variable above.
locals {
projects = {
customer_api = {
region = "australiasoutheast"
environments = ["dev", "uat", "prd"]
}
partner_api = {
region = "australiasoutheast"
environments = ["dev", "prd"]
}
internal_api = {
region = "australiaeast"
environments = ["prd"]
}
payments_api = {
region = "norwayeast"
environments = ["dev", "tst", "uat", "sit", "stg", "prd"]
}
returns_api = {
region = "norwayeast"
environments = ["dev", "uat", "prd"]
}
refunds_api = {
region = "norwayeast"
environments = ["dev", "uat", "prd"]
}
}
}
As you can see we have added three new APIs in the Norway East region, now what we want to do is
create an object that gives us a map where the key
is the region
and the values are a list
of
projects in that region.
locals {
projects_grouped_by_region = {
for k, v in local.projects :
v.region => k...
}
}
The important thing above is the ...
symbol after our key value symbol k
, this activates
grouping mode in Terraform. Think of it like a variadic function. The application of this is
fairly simple but this functionality can be really powerful, especially when it comes to user
permissions or network rules in my experience.
Mutating#
In this section, we are going to look at a scenario where we want to mutate the structure of some
data with a for
expression. The most common use case for this is where we need to combine two values
where one is common to create a unique key.
For the sake of consistency, we will continue to use the above API data structure. We want to be able
to run a for_each
over every environment
in every project
, to do that we will need to create
a unique key.
locals {
prepared_projects = {
for i in flatten([
for proj, data in local.projects : [
for env in data.environments : [{
format("%s-%s", proj, env) = {
project = proj
environment = env
region = data.region
}
}]
]
]) : keys(i)[0] => values(i)[0]
}
}
For this, we will work from the inside out. Right at the center, we are creating a new object that has
the project
, environment
and region
properties from our original data structure and the key
is a project
and environment
concatenated together with a -
. The for
expression itself
is looping through all the environments of the current data
value map, which corresponds to a
single project. The middle for
expression is where we loop through all of our projects
. The
first for
expression is looping over all the indexes on the flattened result of the inner for
expressions.
prepared_projects
will be a list
rather than a map
.Without the flatten
the result would look like this:
prepared_projects = [
[
[
{
"customer_api-dev" = {
"environment" = "dev"
"project" = "customer_api"
"region" = "australiasoutheast"
}
},
],
[
{
"customer_api-uat" = {
"environment" = "uat"
"project" = "customer_api"
"region" = "australiasoutheast"
}
},
],
[
{
"customer_api-prd" = {
"environment" = "prd"
"project" = "customer_api"
"region" = "australiasoutheast"
}
},
],
],
]
Once we have flattened the structure we will get the following:
prepared_projects = [
[
"customer_api-dev",
],
[
"customer_api-uat",
],
[
"customer_api-prd",
],
]
The above is what we get when we return a flattened list
, and grab the keys using keys(i)
. The i
is out temporary index symbol for the first for
expression. If we were to grab keys(i)[0]
instead
we would get:
prepared_projects = [
"customer_api-dev",
"customer_api-uat",
"customer_api-prd",
]
This is much closer to what we want these are the keys that will be used when we return the full object.
locals {
prepared_projects = {
for i in flatten([
...
]) : keys(i)[0] => values(i)[0]
}
}
So, from our flattened object using the above-highlighted line we are going to retrieve the i
index
on the list
and then take the 0
index on that returned list
. This gives us the data we are
looking for!
Below is a snippet of the projects
local variable, this is the object we are going to mutate.
locals {
projects = {
customer_api = {
region = "australiasoutheast"
environments = ["dev", "uat", "prd"]
}
}
}
Below is the resultant object of the mutated projects
variable.
prepared_projects = {
"customer_api-dev" = {
"environment" = "dev"
"project" = "customer_api"
"region" = "australiasoutheast"
}
"customer_api-prd" = {
"environment" = "prd"
"project" = "customer_api"
"region" = "australiasoutheast"
}
"customer_api-uat" = {
"environment" = "uat"
"project" = "customer_api"
"region" = "australiasoutheast"
}
}
This link here inspired a simplification of the above implementation that does away with
the requirement for us to grab the keys(i)[0]
and values(i)[0]
, its actually rather elegant.
locals {
prepared_projects = {
for i in flatten([
for proj, data in local.projects : [
for env in data.environments : {
project = proj
environment = env
region = data.region
}
]
]) : format("%s-%s", i.project, i.environment) => i
}
}
In this implementation, we are simply accessing the properties of the indexed item.
As you can see mutating data with for
is absurdly powerful, but also absurdly complex and they
can become far more complex than the above example. However, this example has all the key components
you would need to build out increasingly complex mutating for
expressions.
for_each
Meta-argument#
One common scenario when using the for_each
meta-argument is to use it as a type of feature flag,
this allows us to enable (or disable) one or more Terraform resources, data sources or even modules.
In this first instance, we will look at how we can enable a single resource to be switched on or off with an input variable. Think of this like a feature flag, this is extremely useful when you have portions of a module that you might want to turn on or off.
variable "is_enabled" {
type = bool
default = true
}
resource "scratch_string" "this" {
for_each = var.is_enabled ? { enabled = true } : {}
in = each.value
}
Above we have declared the is_enabled
input variable, when this is true
- which it is by default -
we will have an instance of the scratch_string
. This instance will have the key of enabled
, the
below is how we would reference the resource once it is created.
output "scratch_string_id" {
value = scratch_string.this["enabled"].id
}
All of the above examples of for
expressions can also be used with the for_each
meta-argument,
this helps them more powerful than just reading in a static map
or set
!!
Closing out#
In this post, we have gone through the components of a for
expression, and how we can use them for
filtering, grouping and mutating. We also looked a little at the for_each
meta-argument which is
generally used with the for
expression, as well as a great little gem for using it as a feature flag.
Hopefully, this post has given you some guidance or a solid starting point for for
expressions and
the for_each
meta-argument. Please feel free to reach out if you have any questions or if you'd like
further explanation on anything covered.