.png){kind=logo}
Market Data Recovery Complete: Root Cause Identified and Writes Committing Cleanly Again
Hi team (Yana C., @Yahia, Olivia P.) — closing the loop from our side: market_data is fully recovered and writes are committing cleanly again. Huge thanks for the fast turnaround. 🙏 Confirming the resolution for the record:
Root cause(matches Yahia's diagnosis): one of our one-off migration scripts ranDROP SEQUENCE seq_ledger_idwhile a table column (position_ledger.id) still depended on it viaDEFAULT nextval(...). That left adangling catalog dependencywhich blocked checkpoint/commit for theentiredatabase — every write failing at COMMIT even though connections were clean,readonly=false, andCHECKPOINTitself succeeded. That was exactly the puzzling signature we saw (error 43ce3050 etc.).
Fix that worked: rollback to the pre-incident checkpoint + re-creating the sequence (we keep it now as an unused sequence; the column no longer references it) + Duckling reset.
Since then writes have been rock solid — we're currently running a large full-market rebuild (thousands of per-stock DELETE+INSERT transactions into market_data) with zero COMMIT failures. One bit of constructive feedback that might save others the same trap: it'd be great if DuckLake could refuse a DROP SEQUENCE while a column still depends on it (or surface a clear dependency error), rather than letting it succeed and silently wedging checkpoints — the failure gave no hint that a dropped-but-referenced sequence was the cause, which made it very hard to self-diagnose. Thanks again for getting us unblocked!