Forecasting

Why Your Cloud Forecast Is Wrong (And How to Fix It)

· Finopscraft
Abstract concept: cloud cost forecast lines diverging, illustrating why cost forecasts fail

Every month, someone on the team opens the AWS Cost Explorer dashboard, looks at last month's spend, adds 5%, and calls that the forecast. Finance accepts the number. Engineering moves on. Then the bill lands and it's 23% over the "forecast" and there's a retrospective with slide decks nobody wanted to make.

We've talked to enough engineering leads to know this is the dominant approach to cloud cost forecasting at growing companies. It's not because these teams are careless. It's because the naive model is fast and the tools that exist for cloud cost data are mostly built for looking backward, not forward.

This article explains the three structural reasons why last-month extrapolation fails, and the data inputs you need to build a forecast that actually holds.

Reason 1: Your spend is a lagging signal from a leading pipeline

Cloud spend in any given month reflects decisions made 2-6 weeks earlier: infrastructure provisioned for a feature release, a new data pipeline that started ingesting, a caching layer someone stood up for an A/B test that then got left running. By the time the bill arrives, those decisions are already history.

The naive extrapolation model treats last month's spend as representative of next month's baseline. It's almost never right, because cloud consumption follows your deployment pipeline with a lag. If your engineering team pushed two significant features in April and has three more queued for May, May's bill will not look like April's. The slope of your spend curve is determined by your release calendar, not by your cost history.

What you need instead: a signal from the deployment side. At minimum, this means pulling your active workload count and comparing it week-over-week against your spend growth rate. If workloads are growing at 12% per month and your spend is growing at 8%, something is being provisioned efficiently. If workloads are flat and spend is growing at 15%, something is accumulating that isn't tied to features.

Finopscraft pulls both signals and computes a workload-adjusted baseline. The gap between spend trajectory and workload trajectory is one of the most informative numbers in cloud cost management, and almost nobody tracks it explicitly.

Reason 2: Reserved instance coverage and savings plans create hidden discontinuities

Consider a scenario we saw with a team running about $90K/month in AWS: they renewed their EC2 reserved instances in February, which dropped their on-demand compute spend by roughly $14K in March. Their March bill was unusually low. Then in April, a few instances came off the RI coverage window before the next renewal batch, and on-demand spend jumped back up. Finance looked at February, March, April in sequence and saw: flat, then a dip, then a spike. They forecasted May as another spike and prepared budget accordingly. In reality, May renewed a fresh RI batch and came in $11K under the "forecast."

Reserved instances and savings plans are purchasing decisions, not consumption signals. When you extrapolate from a month where RI coverage was unusually high or low, you're extrapolating from a pricing artifact, not from real demand. Your actual compute consumption may be completely stable even while the on-demand vs. reserved split is shifting.

A correct forecast model needs to separate the commitment layer from the consumption layer. You need to track: what's covered by active reservations, what's covered by savings plans, what's genuinely on-demand, and when each commitment expires. That's not a spreadsheet calculation unless you have a lot of time and very few accounts.

Reason 3: Service-level composition changes faster than total spend

Your total AWS bill might grow by only $3K from one month to the next. But inside that stable total, you might have: RDS spend up $8K (new production database for a just-launched product), EC2 spend down $6K (an old cluster you finally decommissioned), S3 spend up $4K (a new data archival pipeline), CloudFront spend down $3K (reduced traffic from a retired marketing campaign). A flat total hides four significant changes.

When you forecast at the account or total level, these service-level shifts are invisible until after the fact. The problem is that service-level changes are where budget accountability breaks down. The team that owns the new RDS instance doesn't have visibility into whether their provisioning fits within their allocated budget when the only number available is a company-level total.

Forecasting needs to happen at the service and team level, not just the account level. This requires consistent tagging (covered in a separate post), but even with imperfect tags, per-service trending is more actionable than per-account totals.

The inputs a real forecast model needs

To build a forecast that performs better than naive extrapolation, you need at minimum:

  • Service-level spend history, not just totals. At least 90 days, ideally 6 months, broken down by AWS service (EC2, RDS, S3, data transfer, etc.).
  • Reserved instance and savings plan expiration schedule. This is available from AWS Cost Explorer's coverage reports and the Reservations console, but it needs to be pulled proactively, not discovered after the fact.
  • Anomaly detection on per-service lines. A spend line that shifts more than 2 standard deviations from its trailing 8-week average is either a new workload, a removed workload, or something worth investigating.
  • Deployment pipeline signals. At minimum: a rough count of active production services and their EC2 family distribution. Better: infrastructure change events from your CI/CD system (a new Terraform apply that provisions RDS is a direct cost event).
  • Known future commitments. Planned feature launches, known data ingestion growth, upcoming RI renewals. These are calendar events, not historical signals, but they belong in the forecast.

Where AWS Cost Explorer falls short

We're not saying Cost Explorer is useless. For looking at last month's spend by service, by region, by tag, it's genuinely good and the API is solid to build against. The gap is specifically in forward projection. Cost Explorer's built-in forecast is a straight-line extrapolation from recent history. It's the naive model, just with a better UI. It doesn't account for RI expiration schedules, doesn't decompose by workload, and doesn't surface anomalies that haven't yet grown large enough to appear in the trailing average.

If your spend is flat and your architecture is stable, Cost Explorer's forecast is probably fine within a 10-15% margin. If you're actively building, if you have reservations cycling, or if you have multiple teams with separate cost centers, the built-in forecast will miss.

The model we use at Finopscraft

Our forecasting engine starts with per-service spend decomposition over a trailing 13 weeks. We fit a trend component and a seasonality component separately, because some services (data transfer, often) have weekly patterns while others (RDS) are mostly flat with step changes. We then overlay the RI/savings plan commitment schedule to adjust the pricing layer independently of the consumption layer.

The output isn't a single number. It's a per-service forecast with a confidence interval, plus a list of the line items driving the most variance. That last piece is the one that matters most for engineering teams: knowing which specific service lines are uncertain helps engineers confirm or deny whether a planned infrastructure change is already reflected in the forecast.

We also run anomaly detection on every service line on a rolling weekly basis. A line that jumps outside its normal range triggers a flag before the month is over, not after the bill arrives. That's the difference between managing cost proactively and doing retrospective apologetics.

What to do this week

If you're relying on naive extrapolation now, here's a concrete starting point that doesn't require new tooling:

  1. Export your AWS Cost Explorer data by service, monthly, for the last 6 months.
  2. Identify the 5 services that account for 80%+ of your spend.
  3. For each of those 5 services, check your RI coverage report. Note any expirations in the next 90 days.
  4. Talk to whoever owns each of those 5 services. Ask: "Do you have anything provisioning or deprovisioning in the next 30 days that would affect this line?" That conversation alone will surface more forecast signal than any historical extrapolation.

The forecast problem in cloud isn't fundamentally statistical. It's an information-routing problem. The people who know what's going to run are not the same people who see the bill. The job of cloud cost forecasting tooling is to close that gap before the invoice arrives, not after.