Topic 02 · DP-700 · DP-600
Spark Notebooks and Delta Tables
Build from Spark compute and DataFrames to persistent, queryable Delta tables in a Fabric Lakehouse.
Compute
Spark pools run distributed work.
Transform
DataFrames and Spark SQL shape data.
Persist
Delta tables provide reliable storage.
The Spark-to-Delta mental model
A Spark notebook uses distributed compute to read and transform data. A DataFrame represents the data while it is being processed. When the result must be reused by other Fabric workloads, write it as a persistent Delta table rather than leaving it as a session-only view.
| Concept | What it is | Exam decision |
|---|---|---|
| Spark pool | Compute cluster that runs the driver and executor processes | Choose capacity and autoscale settings for workload size |
| DataFrame | Distributed tabular structure for loading and transforming data | Use select, filter, groupBy, and write operations |
| Temporary view | Session-scoped SQL name created from a DataFrame | Useful for combining Python or PySpark with Spark SQL |
| Delta table | Persistent table format with transactions and version history | Use for reliable Lakehouse tables and time travel |
Exam checklist
- • Use explicit schemas when stable types matter more than automatic inference.
- • Partition by columns that are commonly filtered, while avoiding excessive small partitions.
- • Use temporary views for session-only SQL and Delta tables for persistent Lakehouse data.
- • Remember that Parquet is a file format, while Delta adds transaction and table-version management.