← Back to Blog

Automating Terraform State Drift Detection and Remediation

How I built a reliable WhatsApp AI shopping assistant for Clickmothercare that survives hallucinated products, silent save failures, and multi-agent handoff bugs.

Anas Rhimi
Anas Rhimi August 2026 • 8 min read

The Silent Threat of Infrastructure Drift

Infrastructure as Code (IaC) is only as reliable as its alignment with reality. When team members make manual changes via the AWS Console or scripts, Terraform state diverges from the actual infrastructure. This drift can lead to catastrophic overwrites during the next deployment. Let's automate the detection and remediation of this drift using GitHub Actions.

Building the Drift Detection Workflow

We need a scheduled GitHub Action that runs terraform plan and evaluates the exit code. Terraform provides a specific flag, -detailed-exitcode, which returns 0 for no changes, 1 for errors, and 2 if changes are present (drift).

name: Terraform Drift Detection
on:
  schedule:
    - cron: '0 * * * *' # Run hourly
jobs:
  detect-drift:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_wrapper: false
          
      - name: Terraform Init
        run: terraform init
        
      - name: Terraform Plan (Detect Drift)
        id: plan
        # The '|| exit 0' prevents the step from failing immediately on code 2, 
        # allowing us to handle the logic in the next step.
        run: |
          terraform plan -detailed-exitcode -out=tfplan || export exitcode=$?
          echo "exitcode=$exitcode" >> $GITHUB_OUTPUT
          if [ $exitcode -eq 1 ]; then
            echo "Terraform plan failed with an error."
            exit 1
          fi

      - name: Alert on Drift
        if: steps.plan.outputs.exitcode == '2'
        run: |
          echo "Drift detected! Changes are required to match the state."
          # Here we trigger a Slack notification or create a Jira ticket
          curl -X POST -H 'Content-type: application/json' \
          --data '{"text":"🚨 Terraform drift detected in production! Please review immediately."}' \
          ${{ secrets.SLACK_WEBHOOK_URL }}

Automated Remediation: Proceed with Caution

While detecting drift is crucial, automatically remediating it (by forcing a terraform apply) is highly aggressive and can cause outages if the manual change was an emergency hotfix. Instead, a safer remediation pattern is to automatically open a Pull Request that updates the Terraform configuration to match the drifted state, using tools like terraformer or by analyzing the plan JSON output.

Is your AI agent's infrastructure secure and reliable?

Book a Free 15-Min Technical Audit
Hire Me