Apache Iceberg exposes metadata as queryable tables

Apache Iceberg exposes its internal metadata through queryable virtual tables, letting users inspect table internals with standard SQL [devto]. These include $snapshots, $history, $files, $manifests, and $partitions—each revealing critical operational details.

The $snapshots table lists all historical snapshots with snapshot_id, committed_at, operation, and summary data, enabling time travel queries and change tracking [devto]. The $files table shows every data file in the current snapshot, including file_path, file_size_in_bytes, record_count, and partition, which helps identify small file issues or hot partitions. The $partitions table provides per-partition statistics like record_count and file_count, supporting partitioning strategy reviews.

These tables let teams debug schema changes via $history, audit writes, or optimize compaction cycles. Because they’re SQL-accessible, they integrate directly into monitoring dashboards or alerting systems—no custom tooling required.

Time travel queries rely on $snapshots to reconstruct table state at prior points, useful for reproducing reports or rolling back erroneous writes. Incremental processing pipelines use $files to track new or deleted files between snapshots, reducing full-scan overhead.

The $manifests table reveals which file manifests belong to each snapshot, helping assess scan efficiency and plan rewrites. Together, these tables turn Iceberg’s internals into an observable system, reducing reliance on external metadata scanners or guesswork.

This isn’t just observability—it’s operational leverage. Teams can now base compaction, expiration, or schema evolution decisions on live metadata, not heuristics.