Infrastructure Drift Detection and Remediation Across IaC Tools

Infrastructure drift is the gap between what your IaC declares and what is actually running in your cloud account, and it appears in 90% of large-scale deployments that use IaC, per Firefly AI's analysis. Drift is the default condition of running infrastructure at scale, and any serious conversation about Terraform, CloudFormation, or Pulumi has to start there.

Why infrastructure drift is nearly universal on IaC teams

Infrastructure as Code was supposed to kill the "snowflake server" problem, the one-off machine configured by hand that nobody could reproduce. What IaC didn't solve is the fact that cloud infrastructure keeps changing after the code is written and the pipeline runs. Firefly looked at cloud infrastructure drift and found that even teams with disciplined IaC hygiene and strict GitOps processes can't eliminate it entirely, because the causes are structural.

Four causes recur constantly across teams, regardless of how careful the engineers are. The fourth cause is that cloud providers quietly change default settings on resources that already exist.

The managed-service case deserves a closer look, because it shows drift isn't always a mistake. A contractor who made a one-time change, an incident responder who widened a rule to stop the bleeding, a cloud provider patching a default: none of these are IaC failures. They are the normal operating conditions of a living system, and that's why drift management has to be built as an ongoing practice, not a one-time cleanup project.

The three types of drift that matter operationally

Treating every instance of drift as the same problem is what makes drift management feel impossible to keep up with. The useful framework sorts drift by what actually changed, because the risk profile is different in each case.

Configuration drift is the most familiar type: a resource still exists and is still tracked in state, but one or more of its properties has changed since IaC last declared it.

State drift is a different problem. A resource exists in the cloud but has no record in the state file, or it exists in the state file but has already been deleted outside of IaC. The infrastructure and the code no longer agree on what exists at all, which is a harder problem than disagreeing on settings.

Shadow resources are infrastructure that IaC never touched in the first place: a test database a developer spun up and forgot to tear down, a one-off resource created directly in the console that quietly became load-bearing for a production workflow.

The compliance and security exposure concentrates in the first and third categories. A shadow resource holding protected health information can trigger a HIPAA violation the moment it exists, whether or not anyone notices it. These are incidents that haven't been discovered yet.

AI and GPU workloads add a fourth category: behavioral drift, where a workload's actual resource consumption diverges from what the declared configuration would predict. GPU utilization running far below provisioned capacity, or inference traffic that has outgrown the autoscaling bounds defined in code, both qualify. This category gets its own full treatment later, once the cost implications are on the table.

Native drift detection capabilities and limits by IaC tool

Every major IaC tool ships with some capacity to detect drift. Each one also has a specific blind spot, and knowing where that blind spot sits determines what a team still needs to build or buy.

Terraform and OpenTofu lean on terraform plan as the baseline detection mechanism. It compares the current state against the declared configuration and surfaces every difference as a speculative execution plan; an exit code of 2 signals that drift is present. The limitation is timing: terraform plan only catches drift at the moment someone runs it. Between runs, drift can accumulate invisibly.

AWS CloudFormation takes a different approach, native to the AWS ecosystem and integrated with higher-level authoring tools like AWS CDK and SAM. The constraint is portability: CloudFormation templates are AWS-specific by design, so its drift detection only applies inside the AWS account boundary and offers nothing for a multi-cloud footprint.

A practical bridge exists for teams whose only detection tool is a command someone has to remember to run. Scheduling terraform plan -detailed-exitcode on a cron, through GitHub Actions, turns a point-in-time check into something close to continuous detection. A common starting cadence is to run the check every six hours for production and daily for staging. It costs nothing beyond setup time and converts "detection whenever someone remembers" into "detection on a schedule."

What none of these native tools offer is a unified view across accounts, environments, or cloud providers. Each one detects drift only within its own scope, so cross-environment reconciliation is left to the team running it.

Dedicated drift detection platforms and their added value

Dedicated drift detection platforms exist to solve a different problem than the one native IaC commands solve. They give a team continuous visibility, cross-account mapping, and policy enforcement, so it doesn't have to build custom automation on top of terraform plan output.

Porter takes a structural approach to the problem. For teams that want drift kept in check without building a detection pipeline from scratch, Porter deploys production environments directly into a customer's own AWS, GCP, or Azure account and handles cluster management, CVE patching, and autoscaling automatically. That reduces the surface area where unmanaged drift can accumulate in the first place, because fewer manual intervention points exist for drift to enter through. This fits best if a team is a startup or growth-stage team that needs IaC-level control over its infrastructure without staffing a dedicated DevOps function to maintain detection tooling.

Firefly works from the opposite direction: it maps every live cloud resource back to the IaC code, Terraform, Pulumi, OpenTofu, CloudFormation, that manages it, then detects configuration drift such as manually added security group rules, changed instance types, and resources that exist in AWS but have no IaC definition. Firefly enforces policy no matter which pipeline or engineer made the underlying change, and it processes nearly 320,000 drifts per month across the cloud accounts it scans. Its position in the stack is governance and recovery, sitting above whatever IaC tool a team already uses.

driftctl is an open-source option that scans live AWS, GCP, Azure, and GitHub resources and compares them against Terraform state. You can self-host it for free, so it's a reasonable starting point if you're evaluating drift detection without going through procurement. It's currently in maintenance mode, and the project can't promise timely review of contributions, so it's better treated as a way to get started than a long-term foundation.

StackGen positions itself as an autonomous operations platform, with AI agents called Aiden that act rather than recommend, detecting drift, generating Terraform to fix it, and remediating through self-validating pipelines. Its MCP Server extends this into developer IDEs, letting engineers check for drift from inside Claude, Cursor, or other MCP-compatible environments without switching context.

How remediation should be routed

Detecting drift is only half the job. Acting on it carelessly can cause more damage than the drift itself. Auto-remediation without a defined scope is its own source of incidents, because not every drift event is a mistake that needs reverting.

The question to answer before configuring any auto-remediation is whether the change was deliberate. If an engineer widens a security group rule during an incident, that may be exactly the right call under pressure. A system that reverts both of those automatically, without distinguishing intent, causes more harm than having no remediation.

One pattern that works in practice is the reconciliation window: instead of reverting drift the moment it's detected, collect drift events over a defined period and route them to a human for review before any action is taken.

Teams managing drift at scale route it by severity, and that's the operational standard now. Critical resources, security groups, IAM roles, IAM policies, KMS keys, route to PagerDuty and a dedicated high-priority Slack channel for immediate human review. Low-impact resources, like CloudWatch log groups and SNS topics, just aggregate into a weekly email digest.

This routing exists because blast radius varies enormously by resource type. A drifted KMS key binding is a potential data exposure event. A drifted CloudWatch log group retention setting is noise that doesn't need anyone paged. If both are routed the same way, engineers learn to ignore every alert, which defeats the purpose of detection.

Auto-remediation is safest when scoped to low-blast-radius resources, where the remediation action is unambiguous and rollback is trivial. Above that threshold, the right default is alert-and-queue-for-human-review. The drift events that most urgently need that human review tend to sit on exactly the resource classes regulated industries care about most.

Why drift on regulated resources is a compliance incident

For teams handling regulated data, certain classes of drift are compliance violations the moment they occur, not debt to clean up later. HIPAA compliance in the cloud is a continuous state. Infrastructure changes daily, permissions widen under pressure, protected health information spreads across services as new integrations get built, and a configuration that passed last quarter's audit can drift out of compliance before the next one arrives.

The resource classes where drift itself constitutes the incident include security group rules on any workload touching PHI, IAM policy bindings, and KMS key configurations. A manually widened security group rule on an RDS instance holding patient data carries a potential breach notification obligation, not just a flag for the next code review.

AWS, GCP, and Azure all offer BAA-eligible managed services, including AWS HealthLake, Azure Health Data Services, and Google Cloud Healthcare API. BAA coverage isn't automatic. It has to be explicitly requested and signed, and IaC state needs to reflect that the BAA-covered service is the one actually in use, rather than a non-covered equivalent someone provisioned by hand during a rush.

Discovering a compliance gap in production, instead of catching it through a drift alert, is dramatically more expensive and far slower to fix than catching it early. You need legal review, audit documentation, and engineering time pulled away from everything else on the roadmap to remediate after the fact. That cost asymmetry is the argument for isolating compliance-sensitive resource classes in detection policy, routing them separately from general infrastructure drift, and tying them to the on-call system rather than a weekly digest, applying the same severity-based routing covered above with the compliance stakes made explicit.

Unmanaged drift and cloud cost waste for AI and GPU workloads

Drift that goes undetected doesn't stop at compliance exposure. It accumulates as idle or oversized resources that inflate the cloud bill without delivering any value back to the business.

AI workloads make this problem sharper. Drift detection that maps actual resource state against declared intent is the mechanism that surfaces this gap; without that comparison, an oversized GPU fleet just looks like a line item on the bill.

IaC-enforced cost governance closes the loop between detection and control.

If a team runs inference and training workloads on infrastructure that Porter manages, they get resource-based, transparent pricing while the platform handles automatic cluster optimization. Keeping the infrastructure lifecycle under managed control, instead of leaving it to ad-hoc console changes, shrinks the surface area where cost-generating drift can take hold.

Building a drift detection workflow that does not become its own maintenance burden

A common failure in drift management programs is designing them for the threat model of a large platform team and then imposing that design on a five-person engineering team, producing alert fatigue and abandonment of the whole system within a few months.

Detection frequency should match the team's actual ability to act on what it finds, not a default setting left over from a vendor's documentation. If a team runs scheduled drift detection, it can catch manual console changes within 6 hours instead of weeks, so hourly detection fits production resources where drift is a compliance event, daily detection fits staging, and weekly aggregation is enough for low-blast-radius resources. Running detection more often than the team can review generates noise.

As a team grows past a certain number of engineers and managed resources, manual review stops scaling. So you need real-time detection, automated policy enforcement, and restricted console access; they stop being optional. This is the point where dedicated platforms, Firefly, env0, StackGen, or a managed platform like Porter that shrinks the unmanaged surface area directly, earn their place in the stack.

Multi-cloud environments add a layer of complexity that point tools can't resolve on their own. Inconsistent resource models across AWS, GCP, and Azure, differing provider APIs, and fragmented state backends mean unified drift observability needs an abstraction layer sitting above all three, either a governance platform or a managed PaaS that enforces consistency at the deployment layer itself.

Prevention still costs less than detection. Restricting console access, requiring every change to flow through an IaC pipeline, and using dynamic short-lived credentials issued through a federated identity mechanism, as some infrastructure management platforms support, reduces the rate at which drift enters the system instead of relying entirely on catching it after the fact. A workflow built with both levers, prevention and detection working together, holds up far better than one that leans on detection alone.

Which approach fits depends almost entirely on where a team sits today. A team whose infrastructure has outgrown that model, running multiple clouds, regulated workloads, or GPU fleets with real spend behind them, has a natural next step in evaluating a managed platform built for that scale. Porter's approach to startups and growth-stage teams, handling the infrastructure lifecycle so drift has fewer places to start, is one version of that next step worth putting on the list.