Moving the Remote Terraform State Items

Last update: March 28, 2020

Due to recent refactoring, I figured out that I need to move some Terraform state items from one S3 path to another. And then to merge configurations with other stuff at the destination directory. Terraform can move state items around, but this feature doesn't work with remote states. Here is one way of doing it.

Example Use Case

First, let's consider the following situation, this is configuration directory tree output:

.
├── db
│   └── test
│       ├── main.tf (s3 key: aws/db/test/terraform.tfstate)
│       └── rds.tf
├── test
│   ├── main.tf (s3 key: aws/test/terraform.tfstate)
│   ├── sqs.tf

You want to merge db/test state items into aws/test/terraform.tfstate state and move Terraform files together to match the following directory structure:

.
├── test
│   ├── main.tf (s3 key: aws/test/terraform.tfstate)
│   ├── sqs.tf
│   ├── rds.tf

NOTE: If you are using multiple workspaces, make sure they are the same when going through directories.

First, make local backups of both remote state files with the following command:

terraform state pull > terraform.tfstate

Then go to the directory which state items you want to migrate, in this case, db/test. Run the following commands:

# List all available items
terraform state list

# Move item from one state to another
terraform state mv -state-out=../test/terraform.tfstate aws_rds_cluster.test aws_rds_cluster.test

Each time you run terraform state mv command, Terraform automatically creates a backup of state file as well. Keep in mind that all this is happening locally, regardless of how you configured the backend. Go to the consolidated directory test and push the state file to the remote:

terraform state push terraform.tfstate

Then, if you do terraform state list in this directory, you should see moved state items. You can copy the remaining .tf files in this directory and make changes as needed.

If you want to move all state items with one command you could use simple bash one-liner:

for i in $(terraform state list); do terraform state mv -state-out=../test/terraform.tfstate $i $i; done

After you check that all is good, you can delete local state files and all auto-created backups with rm terraform.tfstate*.

Summary

Maybe there is a better way to do this, but I didn't want to spend too much time researching. I hope this article will help someone with a similar issue.