Hey Tanvir, I can confirm the retention change went through, all your databases are showing 0 days now. But I want to flag that it won't fully fix this on its own. The storage won't drop right away, since those bytes move into failsafe, which is a fixed 7-day window we can't configure. So it'll look flat for about a week before it falls off. And it won't settle as low as you'd want, because the storage is coming from lake_cache and the driver is your refresh loop. You're running CREATE OR REPLACE TABLE ... FROM iceberg_scan(...) about every 5 minutes across 10 tables, and each rewrite keeps a copy of the previous version, so as long as that cadence continues you'll keep a large multiple of your actual data.
What would actually close the gap is making it a transient database. Transient databases have no historical snapshots and a one-day failsafe, and since lake_cache is a cache you rebuild from Iceberg anyway,. However, transient can only be set when the database is created, so it would mean CREATE DATABASE lake_cache_v2 (TRANSIENT), repointing the pipeline, and dropping the old one. Would you be open to that?
More on how the lifecycle works here: motherduck.com/docs/concepts/storage-lifecycle
.png){kind=logo}
