Experiments with Terraform 0.12

A shared central folder consumed by multiple environment folders

A common pattern I’ve used in recent projects is to use a shared folder to contain the Terraform files which:

  • sets the provider (AzureRm)
  • sets any tooling requirements (terraform.exe of at least a certain version)
  • declares variables
  • sets default values for variables at a global level
  • uses the merge function for environment-specific variables
  • creates resources (in this example, Azure Resource Groups)

I combine this with separate folders per environment which contain:

  1. env-variables.tfvars - sets the variables for this environment
  2. env-variables-secure.tfvars - sets any secure variables, and is not checked in to version control
  3. env-backend-secure.tfvars - sets details of a backend for storing Terraform state remotely and securely (e.g. an Azure Storage Account or AWS S3 bucket)

For these examples, I’ve gone with the simplest example - just an env-variables.tfvars per environment. The Terraform:

  • pulls in the AzureRM provider in order to create Azure resources
  • specifies two default tags to be applied to each resource Terraform creates
  • merges in a stage tag (Dev, UAT)
  • creates three resource groups per environment (so that we can see the differences between 0.11 and 0.12 when deleting a resource from the beginning or middle of a list.)

Terraform 0.11

Folder structure 0.11

Under the root c:\work\projects\ I have a folder structure like this:


terraform0.12
├─ 0.11
│  ├─ environments
│  │  ├─ dev
│  │  │  └─ dev-variables.tfvars
│  │  └─ uat
│  │     └─ uat-variables.tfvars
│  └─ shared
│     ├─ provider.tf
│     ├─ resource_group.tf
│     └─ variables.tf

File contents 0.11

shared\variables.tf

  • the variable resource_groups is defined with no default, as its values will be set from a per-environment .tfvars file
  • I use the description property to remind me of the order of the elements I must supply

variable "resource_groups" {
  description = "A map of two values [resource_group_name,location]"
  type = "map"
}

variable "tags" {
  default = {
    creationmethod = "terraform"
    live = "no"
    project = "TF12Experiments"
  }
}

environments\dev\dev-variables.tfvars

  • sets the values in the correct order (resource_group_name, location)
  • for this worked example, I’m varying the location - one resource group in UK West, and two in UK South:

resource_groups = {
  "rg0" = ["RG_CONTAINERS_DEV","uksouth"]
  "rg1" = ["RG_SQL_DEV", "uksouth"]
  "rg2" = ["RG_STORAGE_DEV", "ukwest"]
}

shared\resource_group.tf

  • count means if ${var.resource_groups} is empty, no resources will be created.
  • the element at position 0 is passed in as the name
  • the element at position 1 is passed in as the location

resource "azurerm_resource_group" "rg" {
  count               = "${length(var.resource_groups)}"
  name                = "${element(var.resource_groups["${element(keys(var.resource_groups),count.index)}"],0)}"
  location            = "${element(var.resource_groups["${element(keys(var.resource_groups),count.index)}"],1)}"
  tags                = "${var.tags}"
}

Run the example - 0.11

  • Change into the shared directory: cd C:\work\projects\terraform0.12\0.11\shared
  • Initialise with: terraform init
  • Plan the dev environment by using the --var-file option:

terraform plan --var-file=C:\work\projects\terraform0.12\0.11\environments\dev\dev-variables.tfvars

The output of the plan command shows that Terraform will attempt to create 3 Resource Groups, with name and location pulled from the values set in \environments\dev\dev-variables.tfvars, and tagged with the values set in \shared\variables.tf:


Terraform will perform the following actions:

  + azurerm_resource_group.rg[0]
      id:                  <computed>
      location:            "uksouth"
      name:                "RG_CONTAINERS_DEV"
      tags.%:              "3"
      tags.creationmethod: "terraform"
      tags.live:           "no"
      tags.project:        "TF12Experiments"

  + azurerm_resource_group.rg[1]
      id:                  <computed>
      location:            "uksouth"
      name:                "RG_SQL_DEV"
      tags.%:              "3"
      tags.creationmethod: "terraform"
      tags.live:           "no"
      tags.project:        "TF12Experiments"

  + azurerm_resource_group.rg[2]
      id:                  <computed>
      location:            "ukwest"
      name:                "RG_STORAGE_DEV"
      tags.%:              "3"
      tags.creationmethod: "terraform"
      tags.live:           "no"
      tags.project:        "TF12Experiments"


Plan: 3 to add, 0 to change, 0 to destroy.

Create the resources with:


terraform apply --var-file=C:\work\projects\terraform0.12\0.11\environments\dev\dev-variables.tfvars

Note: I’ve chosen the Resource Groups for this example partly because they only require a name - even the location parameter is optional. Creating Resource Groups should not incur any Azure charges, but as with any Terraform action, do check the output of the plan command and the apply command.

Delete the non-final element

If we now find we no longer need the RG_CONTAINERS_DEV resource group, if we adjust \dev\dev-variables.tfvars

From:


resource_groups = {
  "rg0" = ["RG_CONTAINERS_DEV","uksouth"]
  "rg1" = ["RG_SQL_DEV", "uksouth"]
  "rg2" = ["RG_STORAGE_DEV", "ukwest"]
}

To:


resource_groups = {
  "rg1" = ["RG_SQL_DEV", "uksouth"]
  "rg2" = ["RG_STORAGE_DEV", "ukwest"]
}

We see:

Terraform 0.11 upgraded

Folder structure 0.11 upgraded - before fixes

I copied the 0.11 folder to 0.11-upgraded-before-fixes I deleted the .terraform folder from the \shared\ folder

Run the example - 0.11 upgraded - before fixes

Terraform 0.11.4 introduced the additional command 0.12checklist


cd C:\work\projects\terraform0.12\0.11-upgraded-before-fixes\shared
terraform init
terraform 0.12checklist

Gives us the result


Looks good! We did not detect any problems that ought to be
addressed before upgrading to Terraform v0.12.

This tool is not perfect though, so please check the v0.12 upgrade
guide for additional guidance, and for next steps:
    https://www.terraform.io/upgrade-guides/0-12.html

Then we swap the terraform.exe to a version above 0.12.6. Here I’m using 0.12.8

If we run a simple terraform plan we get an error:


Error: Unsupported Terraform Core version

  on provider.tf line 6, in terraform:
   6:   required_version = "~> 0.11.4"

This configuration does not support Terraform version 0.12.8. To proceed

Run terraform 0.12upgrade which should give:


Would you like to upgrade the module in the current directory?
  Only 'yes' will be accepted to confirm.

  Enter a value: yes

-----------------------------------------------------------------------------

Upgrade complete!

Folder structure 0.11 - 0.11 upgraded - before fixes - upgraded to 0.12

At this point, our folder looks like this:


├─ 0.11-upgraded-before-fixes
│  ├─ environments
│  │  ├─ dev
│  │  │  └─ dev-variables.tfvars
│  │  └─ uat
│  │     └─ uat-variables.tfvars
│  └─ shared
│     ├─ .terraform
│     │  └─ plugins
│     │     └─ windows_amd64
│     │        ├─ lock.json
│     │        └─ terraform-provider-azurerm_v1.33.1_x4.exe
│     ├─ provider.tf
│     ├─ resource_group.tf
│     ├─ variables.tf
│     └─ versions.tf

Plan: terraform plan --var-file=C:\work\projects\terraform0.12\0.11-upgraded-to-0.12\environments\dev\dev-variables.tfvars


Error: Unsupported Terraform Core version

  on versions.tf line 3, in terraform:
   3:   required_version = ">= 0.12"

We notice from the folder structure that we have a new file shared\versions.tf which has

Change shared\versions.tf to be


terraform {
  required_version = ">= 0.12"
}

But we also have the requirement in the the shared\provider.tf. Remove from this file.

Run the plan command again:


terraform plan --var-file=C:\work\projects\terraform0.12\0.11-upgraded-to-0.12\environments\dev\dev-variables.tfvars

Gives the output


Error: Invalid value for input variable

  on C:\work\projects\terraform0.12\0.11-upgraded-to-0.12\environments\dev\dev-variables.tfvars line 1:
   1: resource_groups = {
   2:   "rg0" = ["RG_CONTAINERS_DEV","uksouth"]
   3:   "rg1" = ["RG_SQL_DEV", "uksouth"]
   4:   "rg2" = ["RG_STORAGE_DEV", "ukwest"]
   5: }

The given value is not valid for variable "resource_groups": element "rg2":
string required.

Type constraints

https://www.terraform.io/docs/configuration/types.html

Locals

Adapted an example from Microsoft on Terraform 0.12 for Azure:

Defaults

Single object

Multiple objects

Upgrade

Follow the guide in https://www.terraform.io/upgrade-guides/0-12.html

References

Updated article: https://blog.gruntwork.io/terraform-tips-tricks-loops-if-statements-and-gotchas-f739bbae55f9 https://github.com/hashicorp/terraform-guides/tree/master/infrastructure-as-code/terraform-0.12-examples/rich-value-types https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each https://medium.com/oracledevs/lessons-learned-when-upgrading-to-terraform-0-12-6d894d3ab20e https://github.com/oracle-terraform-modules/terraform-oci-oke/blob/master/terraform.tfvars.example https://discuss.hashicorp.com/t/produce-maps-from-list-of-strings-of-a-map/2197/2 https://github.com/hashicorp/terraform/issues/17179 https://alexharv074.github.io/2019/06/02/adventures-in-the-terraform-dsl-part-iii-iteration-enhancements-in-terraform-0.12.html https://ilhicas.com/2019/08/20/For-each-resource-terraform.html https://www.hashicorp.com/blog/terraform-0-12-rich-value-types

https://www.hashicorp.com/blog/terraform-0-12-rich-value-types with type = map(object({ https://www.terraform.io/docs/configuration/types.html

Flatten - https://discuss.hashicorp.com/t/help-with-for-each-in-resource/2435 SetProduct - https://github.com/hashicorp/terraform/issues/17179 toset - https://www.reddit.com/r/Terraform/comments/ckdtou/terraform_0126_is_out_mapped_for_each_for/

https://cloudblogs.microsoft.com/opensource/2019/06/25/how-to-migrate-to-hashicorp-terraform-0-12-microsoft-azure/

Works like locals:


# Terraform 0.12 Configuration. Some sections omitted for clarity.
locals {
 app_services = [
   {
     kind = "Linux"
     sku = {
       tier = "Standard"
       size = "S1"
     }
   },
   {
     kind = "Windows"
     sku = {
       tier = "Basic"
       size = "B1"
     }
   }
 ]
}


# Terraform 0.12 Configuration. Some sections omitted for clarity.
resource "azurerm_app_service" "example" {
 count               = length(local.app_services)
 name                = "${lower(local.app_services[count.index].kind)}-appservice"
 location            = azurerm_resource_group.example.location
 resource_group_name = azurerm_resource_group.example.name
 app_service_plan_id = azurerm_app_service_plan.example[count.index].id

 site_config {
    # omitted for clarity
 }
}

Locals with flatten

https://github.com/hashicorp/terraform/issues/20893


locals {
  group_permissions = flatten([
    for group in var.groups: [
      for permission in group.permissions: {
        group_id = group.group_id
        permission = permission
      }
    ]
  ])
}

resource "google_organization_iam_member" "role" {
   count = length(local.group_permissions)
  
   role   = local.group_permissions[count.index].permission
   member = local.group_permissions[count.index].group_id
}

Dynamic

Dynamic blocks look very interesting:


See also