Today I Learned: HCL TFE Variable resources don't work the way you expect them to

If like many people you are using the TFE Provider to manage your TFE/C workspaces the it will come as no surprise to you that you can manage your terraform variables and environment variables through the use of tfe_variable. One argument this gives you is the ability to mark this variable as hcl, meaning it will be evaluated as a hcl string. An example of this is as follows:

resource "tfe_variable" "this" {
    key          = "my_variable"
    value        = local.value
    hcl          = true
    category     = "terraform"
    workspace_id = ""
}

Lets say that you're constructing the value of this dynamically within your code -this has been highlighted- and your resource definition will consume this as a local variable. Below is an example of what that local variable might look like.

locals {
    value = {
        a = 1
        b = "meow"
    }
}

One would assume this would work, right? I mean, we have marked our variable as hcl, and we are passing in a hcl object as our value. That assumption would be incorrect, you will get an error like the below:

There are a few ways you can get this to work, and those are namely by converting them into json, yaml, or regex inappropriate characters.

1. Lets add an output to our terraform code so that we can see the results in real time.

output "tfe_variable" {
    value = tfe_variable.this.value
}

2. We are going to alter our resource definition, and wrap the right hand side of the value attribute in jsonencode()

resource "tfe_variable" "this" {
    key          = "my_variable"
    value        = jsonencode(local.value)
    hcl          = true
    category     = "terraform"
    workspace_id = ""
}

By running an apply we will get the following result as our output value:

tfe_variable = "{\"a\":\"1\",\"b\":\"zoo\",\"c\":\"42\"}"

3. Lets do the same thing except this time with yamlencode()

resource "tfe_variable" "this" {
    key          = "my_variable"
    value        = yamlencode(local.value)
    hcl          = true
    category     = "terraform"
    workspace_id = ""
}

This one will return us the following result:

tfe_variable = <<EOT
"a": "1"
"b": "zoo"
"c": "42"

EOT

4. Finally, there is an interesting solution posed osterman on GitHub, which goes as follows:

locals {
    hcl = replace(jsonencode(local.value), "/(\".*?\"):/", "$1 = ")
}

This gives us the following result:

tfe_variable = "{\"a\" = \"1\",\"b\" = \"zoo\",\"c\" = \"42\"}"

Now that we have all of that out of the way we get to the crux of this TIL. When you try to consume each of these, some work and some do not.

• jsonencode() - Will return an invalid character error
• yamlencode() - Will return an invalid character error
• magic_regex - Works beautifully

The thing I find interesting here is that there is no out of the box solution for actually working with hcl enabled input variables, which seems strange seeing as it is a valid option. In theory, you could get both the json and yaml versions working if you were to decode them inside your code however the whole point -in my opinion- of having hcl enabled input variables is being able to access the properties directly like you would with a locals{}.

There is a feature request on GitHub (Missing hcldecode/hclencode encoding functions #25584) that would provide new functions that might better deal with this. However, for now the little hack created by osterman works pretty bloody well!