Deploying AWS Resources with GitLab CI/CD and Terraform

Introduction

This document provides a comprehensive overview of setting up a GitLab CI/CD pipeline to deploy resources in AWS using Terraform. The pipeline automates the infrastructure provisioning process, ensuring consistency, reliability, and efficiency in your AWS environment.

Prerequisites

Before you begin, make sure you have the following prerequisites in place:

A GitLab repository for your Terraform code (e.g., GitLab AWS Terraform Demo).
An AWS Access Key , Secret Access Key for authentication. Creating a new user with Access Key and Secret key.

GitLab CI/CD Pipeline Configuration

Managing Terraform Variables with GitLab CI/CD Secrets

To enhance security and avoid exposing sensitive information in your GitLab CI/CD pipeline, it’s recommended to store Terraform variables as GitLab CI/CD secrets. This allows you to securely manage and pass sensitive data to your pipeline jobs.

Follow these steps to set up GitLab CI/CD secrets for your Terraform variables:

Navigate to your GitLab project.
In the left sidebar, click on Settings and then go to CI / CD.
Scroll down to the Variables section.
Click on Add Variable.
Add the following Terraform variables, corresponding to the ones used in your .gitlab-ci.yml file:
- AWS_ACCESS_KEY_ID: AWS Access Key ID.
- AWS_SECRET_ACCESS_KEY: AWS Secret Access Key.
- AWS_DEFAULT_REGION: AWS Region.
Click on Add Variable to save the secrets.

Your GitLab CI/CD pipeline configuration is defined in a .gitlab-ci.yml file. Let’s break down the key components:

image:
  name: hashicorp/terraform:1.5.7
  entrypoint:
    - ""

This specifies the Docker image to use for running the Terraform commands.

stages:
  - fmt
  - validate
  - test
  - plan
  - apply
  - destroy

Defines the stages of the CI/CD pipeline, from formatting and validation to planning, applying, and destroying the infrastructure.

before_script:
  - rm -rf .terraform
  - terraform --version
  - TF_LOG=trace terraform init
  - export TF_VAR_AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
  - export TF_VAR_AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
  - export TF_VAR_AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}

The before_script section sets up the environment before running Terraform commands, including initializing Terraform and setting AWS Access Key , Secret Access Key.

Stages:

1. Format Terraform Code

TF_Format:
  stage: fmt
  script: 
    - terraform fmt

Runs terraform fmt to format your Terraform code. This ensures a consistent code style across your project.

2. Validate Terraform Code

TF_Validation:
  stage: validate
  script: 
    - terraform validate
  dependencies:
    - TF_Format

Validates your Terraform code using terraform validate. This ensures that your code is syntactically correct and follows Terraform best practices.

3. Run TFSec (Terraform Security Checks)

TF_TFSec:
  stage: test
  script: 
    - wget -O /usr/local/bin/tfsec https://github.com/tfsec/tfsec/releases/download/v1.28.4/tfsec-linux-amd64
    - chmod +x /usr/local/bin/tfsec
    - tfsec . -s
  dependencies:
    - TF_Validation

Integrates security checks using TFSec to identify potential security issues in your Terraform code.

4. Plan Infrastructure Changes

TF_Plan:
  stage: plan
  script: 
    - terraform plan -out "planfile"
  dependencies:
    - TF_TFSec
  artifacts:
    paths:
      - planfile

Generates a Terraform execution plan and saves it to a file named “planfile”. This plan outlines the changes that will be applied to your infrastructure.

5. Apply Infrastructure Changes (Manual Step)

TF_Apply:
  stage: apply
  script:
    - terraform apply -auto-approve -input=false "planfile"
  dependencies:
    - TF_Plan
  only:
    - main
  when: manual

Manually triggers the execution of the Terraform plan to apply changes to the AWS infrastructure. This step ensures that potentially destructive changes are only applied with explicit approval.

Conclusion

By implementing this GitLab CI/CD pipeline, you can automate the deployment of your AWS resources, improving efficiency and reducing the risk of errors. Regularly check the pipeline output and logs for any issues, and make adjustments to your Terraform code as needed.

For more details and customization options, refer to the official GitLab CI/CD documentation.

DevToolHub