Deploying Azure Resources with GitLab CI/CD and Terraform

Introduction

This document provides a comprehensive overview of setting up a GitLab CI/CD pipeline to deploy resources in Azure using Terraform. The pipeline automates the infrastructure provisioning process, ensuring consistency, reliability, and efficiency in your Azure environment.

Prerequisites

Before you begin, make sure you have the following prerequisites in place:

A GitLab repository for your Terraform code (e.g., GitLab Azure Terraform Demo).
An Azure Service Principal for authentication. Follow the step-by-step guide here to create one.

GitLab CI/CD Pipeline Configuration

Managing Terraform Variables with GitLab CI/CD Secrets

To enhance security and avoid exposing sensitive information in your GitLab CI/CD pipeline, it’s recommended to store Terraform variables as GitLab CI/CD secrets. This allows you to securely manage and pass sensitive data to your pipeline jobs.

Follow these steps to set up GitLab CI/CD secrets for your Terraform variables:

Navigate to your GitLab project.
In the left sidebar, click on Settings and then go to CI / CD.
Scroll down to the Variables section.
Click on Add Variable.
Add the following Terraform variables, corresponding to the ones used in your .gitlab-ci.yml file:
- ARM_CLIENT_ID: Azure Service Principal client ID.
- ARM_CLIENT_SECRET: Azure Service Principal client secret.
- ARM_SUBSCRIPTION_ID: Azure subscription ID.
- ARM_TENANT_ID: Azure Active Directory tenant ID. Set the values for each variable according to your Azure environment.
Click on Add Variable to save the secrets.

Your GitLab CI/CD pipeline configuration is defined in a .gitlab-ci.yml file. Let’s break down the key components:

image:
  name: hashicorp/terraform:1.5.7
  entrypoint:
    - ""

This specifies the Docker image to use for running the Terraform commands.

stages:
  - fmt
  - validate
  - test
  - plan
  - apply
  - destroy

Defines the stages of the CI/CD pipeline, from formatting and validation to planning, applying, and destroying the infrastructure.

before_script:
  - rm -rf .terraform
  - terraform --version
  - TF_LOG=trace terraform init
  - export TF_VAR_ARM_CLIENT_ID=${ARM_CLIENT_ID}
  - export TF_VAR_ARM_CLIENT_SECRET=${ARM_CLIENT_SECRET}
  - export TF_VAR_ARM_SUBSCRIPTION_ID=${ARM_SUBSCRIPTION_ID}
  - export TF_VAR_ARM_TENANT_ID=${ARM_TENANT_ID}

The before_script section sets up the environment before running Terraform commands, including initializing Terraform and setting Azure Service Principal details.

Stages:

1. Format Terraform Code

TF_Format:
  stage: fmt
  script: 
    - terraform fmt

Runs terraform fmt to format your Terraform code. This ensures a consistent code style across your project.

2. Validate Terraform Code

TF_Validation:
  stage: validate
  script: 
    - terraform validate
  dependencies:
    - TF_Format

Validates your Terraform code using terraform validate. This ensures that your code is syntactically correct and follows Terraform best practices.

3. Run TFSec (Terraform Security Checks)

TF_TFSec:
  stage: test
  script: 
    - wget -O /usr/local/bin/tfsec https://github.com/tfsec/tfsec/releases/download/v1.28.4/tfsec-linux-amd64
    - chmod +x /usr/local/bin/tfsec
    - tfsec . -s
  dependencies:
    - TF_Validation

Integrates security checks using TFSec to identify potential security issues in your Terraform code.

4. Plan Infrastructure Changes

TF_Plan:
  stage: plan
  script: 
    - terraform plan -out "planfile"
  dependencies:
    - TF_TFSec
  artifacts:
    paths:
      - planfile

Generates a Terraform execution plan and saves it to a file named “planfile”. This plan outlines the changes that will be applied to your infrastructure.

5. Apply Infrastructure Changes (Manual Step)

TF_Apply:
  stage: apply
  script:
    - terraform apply -auto-approve -input=false "planfile"
  dependencies:
    - TF_Plan
  only:
    - main
  when: manual

Manually triggers the execution of the Terraform plan to apply changes to the Azure infrastructure. This step ensures that potentially destructive changes are only applied with explicit approval.

Conclusion

By implementing this GitLab CI/CD pipeline, you can automate the deployment of your Azure resources, improving efficiency and reducing the risk of errors. Regularly check the pipeline output and logs for any issues, and make adjustments to your Terraform code as needed.

For more details and customization options, refer to the official GitLab CI/CD documentation.

DevToolHub