Automating Terraform Deployments with GitLab CI/CD: GCP

Introduction:

Introduce the concept of automating infrastructure deployments using GitLab CI/CD and Terraform.
Highlight the benefits of using CI/CD pipelines for managing infrastructure as code.

Prerequisites

Before diving into the GitLab CI/CD YAML configuration, ensure that you have the following in place:

A GitLab repository containing your Terraform code.
A service account JSON key file (service_account.json) with the necessary permissions for your infrastructure. To get service account from the console you can check the Creating a GCP Service Account and Key
Add the JSON Key to the gitlab variable. In GitLab, enhance security by adding your GCP service account JSON file under Settings -> CI/CD -> Variables. Click on ‘Add Variable,’ select Type as File, set the Key as ‘tf_sa,’ and paste the JSON file content under Value. This ensures secure and centralized management of your Terraform deployments’ service account credentials

You can find the source code and the complete GitLab repository at gcp_terraform_demo.

YAML Configuration for GitLab CI/CD with Terraform

image:
  name: hashicorp/terraform:light
  entrypoint:
    - "/usr/bin/env"
    - "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

stages:
  - fmt
  - validate
  - test
  - plan
  - apply

before_script:
  - mkdir -p ~/creds
  - cat $tf_sa > ~/creds/service_account.json
  - export TF_VAR_GOOGLE_CREDENTIALS=~/creds/service_account.json
  - export GOOGLE_APPLICATION_CREDENTIALS=~/creds/service_account.json
  - rm -rf .terraform
  - terraform --version
  - TF_LOG=trace terraform init

TF_Format:
  stage: fmt
  script: 
    - terraform fmt

TF_Validation:
  stage: validate
  script: 
    - terraform validate
  dependencies:
    - TF_Format

TF_TFSec:
  stage: test
  script: 
    - wget -O /usr/local/bin/tfsec https://github.com/tfsec/tfsec/releases/download/v1.28.4/tfsec-linux-amd64
    - chmod +x /usr/local/bin/tfsec
    - tfsec . -s
  dependencies:
    - TF_Format

TF_Plan:
  stage: plan
  script: 
    - terraform plan -out "planfile"
  dependencies:
    - TF_TFSec
  artifacts:
    paths:
      - planfile

TF_Apply:
  stage: apply
  script:
    - terraform apply -auto-approve -input=false "planfile"
  dependencies:
    - TF_Plan
  only:
    - main
  when: manual

Understanding the Configuration

The provided GitLab CI/CD YAML configuration automates the deployment of Terraform infrastructure. Here’s a breakdown of the different stages and jobs within the configuration:

fmt: Formats the Terraform code using terraform fmt.
validate: Validates the Terraform code using terraform validate.
test: Uses tfsec to perform security scanning on the Terraform code, ensuring adherence to best practices.
plan: Generates a Terraform execution plan using terraform plan.
apply: Applies the Terraform changes using terraform apply. This job is set to manual triggering and is restricted to the main branch.

Pipeline Execution Flow

When a new commit is made to the repository, the GitLab CI/CD pipeline executes the defined stages and jobs in the following order:

fmt stage: Formats the Terraform code.
validate stage: Validates the Terraform code.
test stage: Performs security scanning on the Terraform code.
plan stage: Generates a Terraform execution plan and creates an artifact file called planfile.
apply stage: Applies the Terraform changes manually, triggered only on the main branch.

Conclusion

By leveraging GitLab CI/CD with Terraform, you can automate the deployment of your infrastructure, ensure code quality and security, and enable efficient collaboration within your team. The provided YAML configuration serves as a starting point for setting up your pipeline and can be customized to fit your specific requirements.

Remember to adjust the configuration based on your infrastructure and project needs. Regularly review and enhance your pipeline to incorporate additional stages, tests, or security measures as your project evolves.

DevToolHub