Terraform Cloud (TFC) is a pretty cool service with a bunch of excellent features. But what if you decide it’s just not working out? Sorry TFC. It’s not you. It’s me. How do you migrate off of Terraform Cloud and onto another platform? In particular, how do you migrate your state data off TFC to another backend? That is what I’ll address in this post.
I want to start with a few Terraform basics, since this is going to factor heavily into how state data is managed and migrated. For starters, state data is the mapping between what exists in your Terraform config and what is deployed in the target environment. I’ve written about it extensively in my post of what happens when changes occur outside of Terraform, so I won’t rehash it here.
State data needs somewhere to live. The default location is the local backend, creating state files in the same directory as your configuration. There are two different scenarios to consider when using the local backend: working with the default workspace only and working with multiple workspaces. Terraform OSS workspaces allow you to use the same configuration to support multiple target environments. Each workspace has its own state data, which means Terraform needs to create multiple state data files when using the local backend.
When you first spin up a Terraform configuration using the local backend, you have a single default workspace. You cannot remove the default workspace, but you don’t have to deploy anything to it. Simply create a new workspace and then run your terraform apply. A few key questions come to mind:
Starting with the first question, assuming you’re using the local backend, when you create the first non-default workspace, Terraform creates the directory terraform.tfstate.d. Each non-default workspace gets a subdirectory inside the terraform.tfstate.d directory. For instance, if I have the following workspaces: default, development, and production, then I will have the following file tree.
> tree terraform.tfstate.d
terraform.tfstate.d/
├── development
└── production
The state data for each workspace will be stored in a terraform.tfstate file inside each workspace directory. You might notice the default workspace doesn’t have a directory. It will store state data in the file terraform.tfstate in the configuration directory.
When you run terraform workspace list, Terraform looks at the subdirectories inside terraform.tfstate.d and compiles a list from there, adding the default workspace since it won’t have a directory.
Terraform knows which workspace is currently active by writing it to the file .terraform/environment. Terraform will create this file when you create your first non-default workspace. The file will have a single entry, the name of the currently active workspace. Running terraform workspace select simply changes the entry in this file.
The way that Terraform organizes and writes state data for workspaces differs based on the backend being used. The information above is specific to the local backend. The azurerm backend we’ll use later takes a different approach, adding env:workspace_name to the end of each state data blob and storing all the blobs in the same storage account container.
One thing all the backends have in common is the local .terraform/environment file that tells the Terraform CLI which workspace is currently selected.
Yes, I know this is all about migrating off Terraform Cloud, but first we have to get our data on Terraform Cloud to begin with. If you want to follow along, you can check this directory on my terraform-tuesdays GitHub repo. The configuration I’m going to deploy has the following terraform configuration block:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "~> 2.0"
}
}
#backend "azurerm" {
# key = "webapp"
#}
cloud {
organization = "ned-in-the-cloud"
workspaces {
name = "tfc-migration-test"
}
}
}
You might notice the azurerm backend config commented out. We’ll eventually try and get our config migrated to an Azure Storage Account, but first we’ll look into how to migrate to a local backend.
After we’ve run a terraform init and terraform apply our local directory does not have a terraform.tfstate file or terraform.tfstate.d directory. That’s because we’re using the cloud backend. However, in the .terraform directory we have an environment file and a terraform.tfstate file. What’s in those files?
The environment file serves the same function as before, it has a single entry identifying the currently selected workspace. The terraform.tfstate file holds information about the cloud backend:
{
"version": 3,
"serial": 1,
"lineage": "8e0c46e8-ef07-a6f3-1558-08d5bba7d574",
"backend": {
"type": "cloud",
"config": {
"hostname": null,
"organization": "ned-in-the-cloud",
"token": null,
"workspaces": {
"name": "tfc-migration-test",
"tags": null
}
},
"hash": 4214871454
},
"modules": [
{
"path": ["root"],
"outputs": {},
"resources": {},
"depends_on": []
}
]
}
The actual state data is securely stored in the Terraform Cloud workspace. We can grab that state data by going to the UI or by running terraform state pull.
If you were going to migrate your state between any other two backends, the process would generally be:
terraform init to update the backend and migrate state dataBut, as of right now, Terraform Cloud doesn’t work that way. If we comment out the cloud block from our configuration and run terraform init we will get the following error:
> terraform init
Initializing the backend...
Migrating from Terraform Cloud to local state.
╷
│ Error: Migrating state from Terraform Cloud to another backend is not yet implemented.
│
│ Please use the API to do this: https://www.terraform.io/docs/cloud/api/state-versions.html
│
│
╵
Ouch. And BTW, that link doesn’t really help you migrate via the API.
That’s okay! We can figure this out with all the knowledge we’ve already gained. So what do we need in place locally to support our tfc-migration-test workspace with the local backend?
terraform.tfstate.d directory with a tfc-migration-test subdirectoryterraform.tfstate file in the tfc-migration-test directoryterraform.tfstate file in the .terraform directory that points at the cloud configcloud blockterraform init to prepare local filesThat should do it!
> mkdir -p terraform.tfstate.d/tfc-migration-test
> terraform state pull > terraform.tfstate.d/tfc-migration-test/terraform.tfstate
> mv .terraform/terraform.tfstate .terraform/terraform.tfstate.old
# Remove the cloud block in the config
> terraform init
Sure enough, if we make a change to the configuration, a plan will run successfully. This works well enough for a single workspace. If you have multiple workspaces, you’ll need to create a directory for each workspace and pull the state data for that workspace.
Will the migration process be any easier if we’re moving to another remote backend instead of the local backend?
> terraform init -backend-config="backend.txt"
Initializing the backend...
Migrating from Terraform Cloud to backend "azurerm".
╷
│ Error: Migrating state from Terraform Cloud to another backend is not yet implemented.
│
│ Please use the API to do this: https://www.terraform.io/docs/cloud/api/state-versions.html
│
│
╵
No. No, it will not.
We should be able to follow the same basic process, only this time we need to create the necessary files in the target storage account. Here’s a reminder of what is in the azurerm config block:
backend "azurerm" {
key = "webapp"
}
That’s just the partial configuration, following HashiCorp’s guidance. The rest of the configuration comes from environment variables and a backend.txt file with this content:
storage_account_name="tfc40300"
resource_group_name="tfc-40300"
container_name="terraform-state"
The container being used in Azure is terraform-state. The key value is webapp. The azurerm backend will add env: and the workspace name to the end of the key value. So our state file will be webappenv:tfc-migration-test. The migration process will be similar to the local migration:
webappenv:tfc-migration-test in the storage account containerterraform.tfstate file in the .terraform directory that points at the cloud configcloud block and add the azurerm blockterraform init to download files and validate configWe can grab the state data with the same terraform state pull command we used before. And then use the Azure CLI to copy it to our storage account.
> terraform state pull > statedata
> az storage copy -s statedata --destination-account-name tfc40300 --destination-container terraform-state --destination-blob "webappenv:tfc-migration-test"
Now we’ll update the backend configuration, rename the terraform.tfstate file, and run a terraform init.
> mv .terraform/terraform.tfstate .terraform/terraform.tfstate.old
# Remove the cloud block in the config
> terraform init -backend-config="backend.txt"
For authentication and access to the storage account, I have Azure Service Principal information stored in environment variables.
After running terraform init, we can update the config and run a plan or apply to show that our migration was successful.
In this post we’ve seen how to migrate from Terraform Cloud to either the local or azurerm backend. The process for any other backend would be similar, except you’ll need to know how it handles workspaces and the actual state data.
As an alternative, you could perform a two stage migration from Terraform Cloud to local and then from local to your remote backend of choice. Either way, you’ll still need to pull the state data down to an intermediary location before uploading to the new remote backend.
In all likelihood, HashiCorp will update the cloud backend to support direct migration in the not too distant future, making this entire post moot. But until then, I hope this has helped you migrate off Terraform Cloud or at least learn a bit more about how state data and workspaces are managed by Terraform.
Creating Your First Terraform Repository
March 4, 2024
Comparing OpenTofu and Terraform
January 22, 2024
January 17, 2024
