What will you learn?
In this guide our data engineer Edgar Ochoa will help you to set up a Terraform to work with a GCP Project using Cloud Storage as a backend.
What do you need before starting?
To follow this guide you must be familiar with the G Cloud Console and have some basic understanding of GCP services such as IAM and Cloud Storage. You need to have cloud storage creation permissions and an IAM role.
Let’s get started
1. Set up your G Cloud Configuration
Set up your G Cloud Configuration to use the project that you will be working with.
gcloud config set project PROJECT_ID
The next step is to set your own user credentials for Terraform in order to access the APIs:
gcloud auth application-default login
2. Create a service account for your project
We recommend you to use the following naming convention: sa-{short_project_name}-tf-{Environment}.
Tip: The short name can be something related to the project name you are using.
Example:
The name of my service account is sa-demo-tf-sbx
- Demo: my project is called demo-playground
- Sbx: the environment I’m using is called sandbox
gcloud iam service-accounts create sa-demo-tf-sbx \
–description=”Terraform Service account Demo Sandbox Environment”
\
–display-name=”Terraform Service Account”
3. Provide your freshly created service account with the necessary roles and permissions
We will now provide the service account with the necessary roles and permissions. We will be granting the project Editor permission for a full list of possible roles you that can find here.
gcloud projects add-iam-policy-binding PROJECT_ID \
–member=”serviceAccount:sa-demo-tf-
sbx@PROJECT_ID.iam.gserviceaccount.com” \
–role=”roles/editor”
We will be impersonating this service account to make all our changes. In order to do this, we need to grant ourselves the necessary permissions. You can do it like this:
3.1. First: get the policies for the service account and save it in policy.json
gcloud iam service-accounts get-iam-policy sa-demo-tf-
sbx@PROJECT_ID.iam.gserviceaccount.com \
–format=json > policy.json
3.2. Modify the policy.json to add yourself as member to the role (roles/iam.serviceAccountTokenCreator). Remember to append the rest of policies that already exist:
{
“bindings”: [
{
“members”: [
“user:user_name@domain.com”
],
“role”: “roles/iam.serviceAccountTokenCreator”
}
],
…
}
3.3. Update the policies with the policy.json file
gcloud iam service-accounts set-iam-policy sa-demo-tf-
sbx@PROJECT_ID.gserviceaccount.com \
policy.json
4. Create a bucket that will hold your Terraform Stat
Now choose the name of the bucket. You can use this naming convention: {short_project_name}-{Environment}-tf-state.
Example: in this case it will be demo – sbx – tf- state
This bucket will help you to keep the Terraform state in a location that is shared across all developers. It is a good practice to keep the states of Terraform with versioning.
gsutil mb -l gs://demo-sbx-tf-state
gsutil versioning set on gs://demo-sbx-tf-state
5. Write the Terraform Main
Now, let’s write the Terraform Main. In order to do this we will need:
- A Terraform Main
- A Tfvars
- A couple of files
Are you unsure how to structure the Terraform code? Then use this as a starting point.
For now we will focus on the main.tf, gcp-demo-sbx.tfvars, demo-, sbx.backend and variables.tf.
tf-code
│ main.tf
│ gcp-demo-sbx.tfvars
│ gcp-demo-sbx.backend
│ variables.tf
│ version.tf
│
└───module1
│ │ main.tf
│ │ outputs.tf
│ │ varialbes.tf
│
└───module2
│ │ main.tf
│ │ outputs.tf
│ │ varialbes.tf
│
…
5.1. Main.tf
This is the main of our Terraform code. We will be using the Google Provider. Use the following code as starting point:
provider “google” {
project = var.project_id
region = var.region
zone = var.zone
impersonate_service_account = var.tf_service_account
}
5.2. Gcp-demo-sbx.tfvars
Fill this with the variables that can be used for each environment.
project_id = “demo-sbx-tf-state”
region = “europe-west1”
zone = “europe-west1-b”
tf_service_account = “sa-demo-tf-
sbx@PROJECT_ID.iam.gserviceaccount.com”
5.3. Gcp-demo-sbx.backend
This file contains the definition of the backend, the bucket name, the prefix to use to save the state and the service account to impersonate.
bucket = “demo-sbx-tf-state”
prefix = “static.tfstate.d”
impersonate_service_account = “sa-demo-tf-
sbx@PROJECT_ID.iam.gserviceaccount.com”
5.4. Version.tf
This will enable you to keep track of exactly which version of Terraform you are using and each provider that is required.
#### Configure backend ####
terraform {
required_version = “~>0.14.0”
backend “gcs” {}
required_providers {
google = {
source = “hashicorp/google”
version = “~>3.64.0”
}
}
}
6. Initialise the Terraform code
The next step is to initialise the Terraform code using the following command:
terraform init -backend-config=gcp-demo-sbx.backend
7. Create a workspace
Now you can create a workspace. Workspaces should be created for each environment.
terraform workspace new gcp-demo-sbx
8. Plan and apply
Now you can plan and apply the solution.
terraform plan –out tf.plan –var-file=gcp-demo-sbx.tfvars
Next steps
- It is possible to leverage the IAM module from the Google Cloud Provider to manage all your permissions.
- Migrate your current setup using bulk export to Terraform
Conclusion
Congratulations! You can now set up Terraform and set up its backend in the GCP Cloud Storage. Do you want to learn more or get help from our experts? Get in touch here.
