"Explain the Databricks architecture" is one of the most common data engineering questions. Here is the answer as a picture, built up layer by layer.
1. It starts with your cloud account
Databricks runs on Azure, AWS or Google Cloud, and your cloud account is the foundation everything else sits on.
2. The control plane
Databricks runs as two halves. The control plane is hosted and managed by Databricks. It is where the web application, the notebooks, the job scheduler and the cluster manager live.
3. The compute plane
The compute plane (you will also hear "data plane") is where the actual compute runs. In the classic setup it sits inside your own cloud account, in your own virtual network. With serverless compute, it runs in the Databricks account instead.
4. Your data lake
At the bottom is your data lake: ADLS Gen2 on Azure, S3 on AWS. Your raw files live here, and they never leave your account.
5. Delta Lake
On top of the lake sits Delta Lake, a storage layer that adds ACID transactions, schema enforcement and time travel to plain Parquet files. This is what turns a data lake into a lakehouse.
6. Bronze, silver, gold
Inside Delta, data is usually organised in three zones:
- Bronze: raw ingested data
- Silver: cleaned and joined data
- Gold: business-level aggregates, ready for reporting
7. Unity Catalog
Across all of it sits Unity Catalog, the governance layer: permissions, lineage, and a single catalog across every workspace.
8. What happens when you run a notebook
The control plane sends the instruction. A cluster spins up in the compute plane, reads from Delta and writes results back. Only results and metadata go back to the control plane. The data stays put.
9. The consumers
SQL warehouses for BI, machine learning workloads and streaming pipelines all read from the same gold layer.
Watch it drawn step by step
This is episode 1 of a free series that builds Databricks up one layer at a time. Watch it in order: https://www.youtube.com/watch?v=B5iHmoYgnqY&list=PLDB5WDkDOYF4
