Garbage Collection
Reclaim storage by expiring snapshots, removing orphan files, and vacuuming stale data safely.
Garbage collection reclaims storage that is no longer referenced. Done carelessly, it is the fastest way to lose data in a lakehouse — so Crunch runs it with cross-catalog validation and a configurable recovery window. Crunch runs a two-stage deletion process to maximize resource utilization and minimize S3 throttling. Our internal benchmarks show that Crunch is 50x+ more efficient than open source implementations.
What Crunch collects
- Expired snapshots — Iceberg snapshots older than the retention window, along with the manifests that only they reference.
- Orphan files — objects in the table's storage location that no live snapshot or log entry references, typically left behind by failed writes.
- Vacuum — both full and partial (Lite) vacuum of your Delta tables.
Safety model
Crunch is non-destructive by default. Before any object is deleted:
- Retention check — the object must be older than the retention window configured in the policy.
- Reference check — the object must not be referenced by any live snapshot, manifest, or transaction log entry.
- Cross-catalog shared-file detection — Crunch checks every connected catalog for references to the same file. This is what prevents data loss during Hive-to-Iceberg or Hive-to-Delta migrations, where two catalogs describe overlapping sets of files and format-specific tools see only their own.
- In-flight write check — objects written by a job that is still running are never eligible for collection.
Set the retention window to be longer than your longest-running query or batch job. A reader that opened a snapshot before collection began can fail if the files behind that snapshot are removed while it is still running.
Per-format behavior
| Format | Operations |
|---|---|
| Apache Iceberg | Snapshot expiration, orphan file deletion, metadata and manifest compaction |
| Delta Lake | VACUUM, transaction log cleanup, checkpoint pruning |
| Hive / Parquet | Orphan file deletion against catalog partition metadata |
Configure garbage collection
Garbage collection is enabled per table through a policy in the Granica Console or the Granica API. The policy sets the retention window, the collection schedule, and the resource pool used for the scan. Collection jobs are scheduled like any other Crunch job, so they respect SLA windows and resource limits rather than competing with production workloads.
See also
Lifecycle Management
Apply retention, tiering, and deletion policies to partitions and rows across your lakehouse tables.
Query Acceleration
Optimize data layout — sort order, clustering, file and row group sizing, stats, and metadata — against real query patterns.