Automating Terraform State Drift Detection and Remediation

Infrastructure Drift

Infrastructure as Code operates effectively only when registered state reflects live reality. When engineers introduce manual modifications via the AWS Console or execute ad-hoc scripts, the Terraform state file diverges from actual cloud resources. During the subsequent terraform apply, Terraform may unintentionally overwrite those out-of-band updates or cause production service disruptions.

You can proactively prevent drift issues by executing automated drift detection workflows using scheduled GitHub Actions.

Building the Drift Detection Workflow

To identify drift automatically, schedule a GitHub Action that executes terraform plan incorporating the -detailed-exitcode flag:

• 0: No changes identified (live infrastructure aligns with state).
• 1: Error encountered during execution.
• 2: Resource discrepancies detected (drift confirmed).

Below is a configuration that runs hourly and notifies your operations team upon detecting state drift:

name: Terraform Drift Detection
on:
 schedule:
 - cron: '0    ' # Run hourly
jobs:
 detect-drift:
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4

name: Setup Terraform uses: hashicorp/setup-terraform@v3 with: terraform_wrapper: false


name: Terraform Init run: terraform init


name: Terraform Plan (Detect Drift) id: plan # Catch exitcode so step doesn't fail on code 2 run: | terraform plan -detailed-exitcode -out=tfplan || export exitcode=$? echo "exitcode=$exitcode" >> $GITHUB_OUTPUT if [ $exitcode -eq 1 ]; then echo "Terraform plan failed with an error." exit 1 fi


name: Alert on Drift if: steps.plan.outputs.exitcode == '2' run: | echo "Drift detected! Changes are required to match the state." curl -X POST -H 'Content-type: application/json' \ --data '{"text":" Terraform drift detected in production! Please review immediately."}' \ ${{ secrets.SLACK_WEBHOOK_URL }}

Automated Remediation Strategies

Automatically executing terraform apply to overwrite detected drift involves operational risk. If an engineer manually updated a production resource during an incident as a temporary remediation, triggering an unreviewed apply could revert the critical fix and re-introduce downtime.

A more robust pattern involves sending immediate notifications or generating an automated Pull Request that updates the HCL codebase to match the drifted state utilizing tools such as terraformer or evaluating plan JSON output. This guarantees peer review before infrastructure changes are applied.