Provider Iteration in Terraform: A Dream Come True or a Dilemma of Design?
OpenTofu recently delivered on their promise of bringing new ideas to the table. This time, they implemented an age-old dream: the ability to iterate over providers.
In Terraform, provider blocks authenticate with control planes like AWS, Azure, GCP, Kubernetes, or even a Minecraft server. However, provider blocks have historically been sacred; you couldn’t iterate over them. This meant you were constrained by the provider’s design choices.
Since each Terraform provider is an independent piece of software, if the authors imposed constraints on what a single instance could do, you were stuck — unless you created multiple provider instances with aliases.
Why Can’t We Iterate Over Providers in Terraform?
One of the most prominent examples of provider constraints is AWS. The AWS provider follows a common pattern where a single instance is scoped to a single region. If you wanted a multi-region AWS deployment, you had to explicitly declare multiple provider blocks and use aliases.
Defining Input Variables for Regions
You’d probably start out by defining input variables for your primary and secondary regions.
variable "primary_region" {
type = string
default = "us-east-1"
}
variable "secondary_region" {
type = string
default = "us-west-1"
}
Declaring Provider Blocks for Each Region
Then you would declare a provider block for each. Explicitly — a single provider block for both primary and secondary.
# Define your provider blocks
provider "aws" {
alias = "primary"
region = var.primary_region
}
provider "aws" {
alias = "secondary"
region = var.secondary_region
}
Explicitly Defining Modules for Each Region
Then you would declare a module block for each. Again, explicitly — a single module block for both primary and secondary.
# Define your Regional Stamp Modules
module "primary_stamp" {
source = "./modules/regional-stamp"
providers = {
aws = aws.primary
}
region = var.primary_region
}
module "secondary_stamp" {
source = "./modules/regional-stamp"
providers = {
aws = aws.secondary
}
region = var.secondary_region
}
This means that, while Terraform allows iteration over module, resource, and data blocks, provider blocks are exempt. If we could iterate over them, we might be able to simplify this setup significantly:
variable "regions" {
type = set(string)
default = ["us-east-1", "us-west-1" ]
}
provider "aws" { }
module "secondary_stamp" {
source = "./modules/regional-stamp"
for_each = var.regions
region = each.key
}
Alas, this is not possible in Terraform. Why? Well, technically, Terraform doesn’t support iteration over providers. But is that the real reason? Why? Because Terraform doesn’t support iteration over providers? Well, I suppose so… “from a certain point of view”.
How Other Providers Handle Scope Differently
However, I argue that the reason why this isn’t possible is because of a conscious design decision by the good folks responsible for building and maintaining the aws provider decided to scope it to a single region.
SHOCKER: The above code is perfectly valid using either the azurerm provider or the googlecloud provider.
Now before you go torching me over “CloudWars” or something. My point is not to say that Azure or GCP’s Terraform providers are somehow superior. I am just pointing out that a conscious design decision has been made in each of these providers. I’ve talked about this subject at great length when it comes to my beloved provider, azurerm, and how I wish the scope for it were expanded to allow multi-subscription deployments — a pattern that is possible on the googlecloud provider — thanks to a conscious design decision.
Should We Remove These Constraints?
So now the question is, should we be unburdened by what has been and start adding things like provider iteration? Or maybe there was a good reason why that limitation was there in the first place?
In the age old wisdom, as seen in one of my favorite movies “Goonies”, one of the band of would-be treasure-hunting youngsters beriddled by boobie trap laden caverns of a band of pirates from yesteryear exclaims “God put that rock there for a reason”.
Maybe the reason wasn’t to make our lives, as Terraform or Infrastructure-as-Code developers, hard, maybe it was to separate the responsibility from the Platform and the Application. In this case, the Terraform CLI itself is that Platform and the Terraform Providers act like Applications. Each Application is essentially the master of its own destiny. It can decide what boundaries to place on its end users. It can change those boundaries if it sees fit.
The job of the Platform is to maintain a stable and reliable place where all Applications can live and be merry. In order for the Platform to do that we probably shouldn’t make that Platform more and more complex unnecessarily. Sure we can add new features but if the capability already exists, why add more complexity to do it?
Just Because We Can, Should We?
Software design is an art form. Adding new features just because we can isn’t always the best idea. If the functionality already exists through aliasing and conscious design decisions within the providers, do we really need provider iteration? Or does adding it unnecessarily complicate Terraform’s core design?
Maybe one day, there will be an Olympic Games of Software Design — or even a Grammy for Best Infrastructure-as-Code. But until then, we should ask ourselves:
Just because we can, does that mean we should?