The Databricks bill doubled and nothing changed

Guide, October 2026. Applies to Databricks on Azure, AWS or GCP with Unity Catalog, which is where the system tables live.

First, find out where the money went

Databricks keeps a billing ledger in a system table. Before changing anything, read it. This query gives you a month of spend by product and by the cluster or warehouse that spent it.

SELECT
  usage_date,
  sku_name,
  coalesce(usage_metadata.cluster_id, usage_metadata.warehouse_id, usage_metadata.job_id) AS spender,
  sum(usage_quantity) AS dbus
FROM system.billing.usage
WHERE usage_date >= current_date() - INTERVAL 30 DAYS
GROUP BY 1, 2, 3
ORDER BY dbus DESC;

Two patterns explain most doubled bills. A small number of spenders dominate, usually all-purpose clusters with a person's name on them, or a SQL warehouse. Or a steady daily line steps up on one date, which is when somebody changed a setting. Join to system.compute.clusters for the cluster names and owners, and to system.billing.list_prices if you want the number in currency rather than DBUs.

The five settings, in the order they usually matter

1. All-purpose clusters running scheduled work

An all-purpose cluster is for people: notebooks, exploration, a quick look. A job cluster is created for one scheduled run and terminated when it ends. The all-purpose rate per DBU is roughly double the jobs rate, and an all-purpose cluster left on through the working day runs for hours while the job it hosts needed minutes. Moving a scheduled job from an all-purpose cluster to a job cluster is the single largest saving on most bills, often a large multiple rather than a percentage, and it is a change to the job definition, not to the code.

2. Auto-termination set too long, or off

Every all-purpose cluster should terminate after a short idle period. Ten to twenty minutes is right for most teams; the default is longer, and "never" is common on clusters created in a hurry. SQL warehouses have the same setting, called auto stop. Serverless compute sidesteps the problem by billing only while work runs, which is why it is the right default for SQL warehouses and for many notebooks.

3. No cluster policy

Without a policy anyone can create a cluster of any size with any settings. A policy fixes the node types, caps the size, enforces auto-termination and tags the cluster with a cost centre. It takes an afternoon to write and it stops the next accidental 64-node cluster before it exists.

4. Oversized or unspot clusters

Jobs are often sized once, generously, and never revisited. Check the job's Spark UI for the actual CPU and memory use of the last few runs and shrink to fit, or enable autoscaling with a sensible minimum. On Azure, spot instances for worker nodes cut the virtual machine cost substantially for batch work that can tolerate an occasional retry; keep the driver on demand. Photon speeds up SQL-heavy workloads enough to pay for its higher rate on many jobs, but measure it on yours rather than assuming.

5. Pipelines doing full reloads

A pipeline that reprocesses the whole history every run costs the whole history every run. Streaming reads with Auto Loader, incremental processing in Lakeflow pipelines, and MERGE on changed rows mean the daily run touches a day's worth of data. If run time grows with the size of the table rather than with the size of the day's input, this is the problem.

Then make it stay fixed

• A budget alert that reaches a person. Databricks account budgets, or a scheduled query over the billing table posting to a channel when the daily spend exceeds a threshold. Cost surprises should arrive within a day, not at month end.
• Tags on everything. Cluster policies can require them. Spend per team or per project becomes a query instead of an argument.
• A monthly five-minute review. Run the query above, look at the top five spenders, ask whether each one is earning its keep.

What this looks like in practice

In the lakehouse walkthrough, the daily pipeline runs on a job cluster for about twenty minutes. The same pipeline hosted on an all-purpose cluster left running from nine to six would cost, at the higher rate and the longer hours, on the order of fifty times as much for the same result. The code is identical. The difference is entirely in the two settings above, which is why they come first.

When to ask for help

If the billing query is hard to read because nothing is tagged and dozens of clusters have names like "test" and "new cluster 2", a short engagement to put policies, tags, job clusters and a budget alert in place usually pays for itself inside the first month, and we put that number in writing before starting.

Related: What does a lakehouse cost to run? and job cluster in the glossary.