Terraform For Expressions

Brendan Thompson • 14 October 2022 • 11 min read

The for expression in the Terraform configuration language allows us to construct complex objects out of pieces of information from multiple sources! The result of a for expression can either be a map or a list as well as any combination of those two things. These expressions can be very simple or insanely complex (and yes, I mean insanely). Today we will try and cover a few common scenarios when dealing with the for expression. I will also cover the for_each meta-argument in this post as these two concepts are often used together.

for Expression#

I am going to split this section into a few sub-sections to make it easier to digest the information, those will be:

But first, let's have a look at the syntax for a for expression. Before I begin that we need something to work with, in this instance, I am getting a list of 5 random pet names.

resource "random_pet" "this" {
  count = 5
}

In this first example, we are using the result of random_pet.this (which is a list) splitting each item on the - and then taking the first index. This example does nothing other than taking a list splitting the values and returning a list with the split values.

locals {
  first = [
    for i in random_pet.this :
    split("-", i.id)[0]
  ]
}

In this next example, we are taking the same result of random_pet.this and creating a map. We will split each value the same as before except this time the first element will be the key and the second the element will be the value for our map.

locals {
  both = {
    for i in random_pet.this :
    split("-", i.id)[0] => split("-", i.id)[1]
  }
}

In summary, we have a few components:

  1. The result type, which is shown by the use of [] for a list and {} for a map
  2. The for keyword
  3. Either an index (i) or key-value (k, v) temporary symbol, depending on your input type
  4. The in keyword
  5. The input value itself, which can be a; list, set, tuple, map or object
  6. The : symbol marks where the for expression finishes and the assignment begins
  7. Assignment:
  1. Finally the optional if keyword for filtering things out or validation

Filtering#

Now we have the basics of the for expression down let's see how we can filter some inputs and return only what is relevant to us. Firstly let's set up some test data:

locals {
  projects = {
    customer_api = {
      region       = "australiasoutheast"
      environments = ["dev", "uat", "prd"]
    }
    partner_api = {
      region       = "australiasoutheast"
      environments = ["dev", "prd"]
    }
    internal_api = {
      region       = "australiaeast"
      environments = ["prd"]
    }
  }
}

In the test data map called projects we have various API project names, what region they are in as well as their environments. If we wanted to filter out everything that isn't in australiasoutheast we would do that by using the following for expression:

locals {
  australiasoutheast_region = [
    for k, v in local.projects :
    k
if v.region == "australiasoutheast"
] }

On the highlighted line you can see we are using the if keyword to ensure that values that will go into our returned list have in the region australiasoutheast. We could of course do the opposite:

locals {
  australiasoutheast_region = [
    for k, v in local.projects :
    k
if v.region != "australiaeast"
] }

Now we are doing a negative check to say when the region is not australiaeast. Either of these will work and will return to us only the piece of information we require. But wait, there's more! We have multiple conditions on our if using the || (or) and && (and) symbols, and these can further be grouped using (). Let's see an example where we only want to return projects in australiasoutheast with a uat environment.

locals {
  australiasoutheast_region = [
    for k, v in local.projects :
    k
    if v.region == "australiasoutheast" && contains(v.environments, "uat")
  ]
}

As you can see the filtering available to us is pretty powerful.

Grouping#

What happens when you have a data structure that has different projects, users, or maybe subscriptions, and you want to do something on the point of commonality between them this is where grouping comes into play. An example of this is you might have an object containing users or service accounts and with these the roles or groups they need to be in, we could use grouping to pass the users for a particular group into a resource or module that assigns them that access. Another example might be operating on services or projects in a given common region or performing tasks on things within a common Azure subscription. For our example, we will use an extended version of the projects local variable above.

locals {
  projects = {
    customer_api = {
      region       = "australiasoutheast"
      environments = ["dev", "uat", "prd"]
    }
    partner_api = {
      region       = "australiasoutheast"
      environments = ["dev", "prd"]
    }
    internal_api = {
      region       = "australiaeast"
      environments = ["prd"]
    }
    payments_api = {
      region       = "norwayeast"
      environments = ["dev", "tst", "uat", "sit", "stg", "prd"]
    }
    returns_api = {
      region       = "norwayeast"
      environments = ["dev", "uat", "prd"]
    }
    refunds_api = {
      region       = "norwayeast"
      environments = ["dev", "uat", "prd"]
    }
  }
}

As you can see we have added three new APIs in the Norway East region, now what we want to do is create an object that gives us a map where the key is the region and the values are a list of projects in that region.

locals {
  projects_grouped_by_region = {
    for k, v in local.projects :
    v.region => k...
  }
}

The important thing above is the ... symbol after our key value symbol k, this activates grouping mode in Terraform. Think of it like a variadic function. The application of this is fairly simple but this functionality can be really powerful, especially when it comes to user permissions or network rules in my experience.

Mutating#

In this section, we are going to look at a scenario where we want to mutate the structure of some data with a for expression. The most common use case for this is where we need to combine two values where one is common to create a unique key.

For the sake of consistency, we will continue to use the above API data structure. We want to be able to run a for_each over every environment in every project, to do that we will need to create a unique key.

locals {
  prepared_projects = {
    for i in flatten([
      for proj, data in local.projects : [
        for env in data.environments : [{
          format("%s-%s", proj, env) = {
            project     = proj
            environment = env
            region      = data.region
          }
        }]
      ]
    ]) : keys(i)[0] => values(i)[0]
  }
}

For this, we will work from the inside out. Right at the center, we are creating a new object that has the project, environment and region properties from our original data structure and the key is a project and environment concatenated together with a -. The for expression itself is looping through all the environments of the current data value map, which corresponds to a single project. The middle for expression is where we loop through all of our projects. The first for expression is looping over all the indexes on the flattened result of the inner for expressions.

Without the flatten the result would look like this:

prepared_projects = [
  [
    [
      {
        "customer_api-dev" = {
          "environment" = "dev"
          "project" = "customer_api"
          "region" = "australiasoutheast"
        }
      },
    ],
    [
      {
        "customer_api-uat" = {
          "environment" = "uat"
          "project" = "customer_api"
          "region" = "australiasoutheast"
        }
      },
    ],
    [
      {
        "customer_api-prd" = {
          "environment" = "prd"
          "project" = "customer_api"
          "region" = "australiasoutheast"
        }
      },
    ],
  ],
]

Once we have flattened the structure we will get the following:

prepared_projects = [
  [
    "customer_api-dev",
  ],
  [
    "customer_api-uat",
  ],
  [
    "customer_api-prd",
  ],
]

The above is what we get when we return a flattened list, and grab the keys using keys(i). The i is out temporary index symbol for the first for expression. If we were to grab keys(i)[0] instead we would get:

prepared_projects = [
  "customer_api-dev",
  "customer_api-uat",
  "customer_api-prd",
]

This is much closer to what we want these are the keys that will be used when we return the full object.

locals {
  prepared_projects = {
    for i in flatten([
      ...
]) : keys(i)[0] => values(i)[0]
} }

So, from our flattened object using the above-highlighted line we are going to retrieve the i index on the list and then take the 0 index on that returned list. This gives us the data we are looking for!

Below is a snippet of the projects local variable, this is the object we are going to mutate.

locals {
  projects = {
    customer_api = {
      region       = "australiasoutheast"
      environments = ["dev", "uat", "prd"]
    }
  }
}

Below is the resultant object of the mutated projects variable.

prepared_projects = {
  "customer_api-dev" = {
    "environment" = "dev"
    "project" = "customer_api"
    "region" = "australiasoutheast"
  }
  "customer_api-prd" = {
    "environment" = "prd"
    "project" = "customer_api"
    "region" = "australiasoutheast"
  }
  "customer_api-uat" = {
    "environment" = "uat"
    "project" = "customer_api"
    "region" = "australiasoutheast"
  }
}

This link here inspired a simplification of the above implementation that does away with the requirement for us to grab the keys(i)[0] and values(i)[0], its actually rather elegant.

locals {
  prepared_projects = {
    for i in flatten([
      for proj, data in local.projects : [
        for env in data.environments : {
          project     = proj
          environment = env
          region      = data.region
        }

      ]
    ]) : format("%s-%s", i.project, i.environment) => i
  }
}

In this implementation, we are simply accessing the properties of the indexed item.

As you can see mutating data with for is absurdly powerful, but also absurdly complex and they can become far more complex than the above example. However, this example has all the key components you would need to build out increasingly complex mutating for expressions.

for_each Meta-argument#

One common scenario when using the for_each meta-argument is to use it as a type of feature flag, this allows us to enable (or disable) one or more Terraform resources, data sources or even modules.

In this first instance, we will look at how we can enable a single resource to be switched on or off with an input variable. Think of this like a feature flag, this is extremely useful when you have portions of a module that you might want to turn on or off.

variable "is_enabled" {
  type    = bool
  default = true
}

resource "scratch_string" "this" {
  for_each = var.is_enabled ? { enabled = true } : {}

  in = each.value
}

Above we have declared the is_enabled input variable, when this is true - which it is by default - we will have an instance of the scratch_string. This instance will have the key of enabled, the below is how we would reference the resource once it is created.

output "scratch_string_id" {
  value = scratch_string.this["enabled"].id
}

All of the above examples of for expressions can also be used with the for_each meta-argument, this helps them more powerful than just reading in a static map or set!!

Closing out#

In this post, we have gone through the components of a for expression, and how we can use them for filtering, grouping and mutating. We also looked a little at the for_each meta-argument which is generally used with the for expression, as well as a great little gem for using it as a feature flag.

Hopefully, this post has given you some guidance or a solid starting point for for expressions and the for_each meta-argument. Please feel free to reach out if you have any questions or if you'd like further explanation on anything covered.


Brendan Thompson

Principal Cloud Engineer

Discuss on Twitter