Adam E.

Build away! I look forward to seeing what you build.

Hey Patrick, The Flight handles the write concurrency provided the logic is written into the pipeline. If two flights run in parallel modifying the same database, you may run into a race condition the same as any other parallelized situation. Parallel flights actually happened to me earlier today:

  Called MotherDuck (ctrl+o to expand)

⏺ There's now a run 4 on version 4 — run 3 must have finished. Let me check what happened.
  
  Called MotherDuck (ctrl+o to expand)
  
⏺ Both run 3 AND run 4 are showing RUNNING simultaneously — something triggered a second
  concurrent backfill. Let me check run 4's logs to understand the situation.

  Called MotherDuck (ctrl+o to expand)
  
⏺ Good news — the two runs are cooperating. Run 4 correctly skipped everything run 3 already
  loaded (1990–2010) and picked up from 2011. It's now through 2018 with 1.87M rows. The
  idempotency logic is working exactly as designed.

The key here was that the pipeline had an upsert, so rows would not be duplicated.

Adding on to Neil's comment, the Python runner has the following specs: • 2 vCPU • 16 GB RAM • 150 GB scratch disk This is distinct from whichever Duckling instance you are running your query load on.

Hey Patrick, Good question. You are charged at $0.60/hr billed to the second for the Python work only while the script is running. Any other questions? Anything cool you're planning to build?

(I have pet ducks, so I'm actually MotherDuck's token Mother Duck)

I was thinking curdled milk. Believe it or not, eggs actually do fine on the counter for about a week if they weren't washed before getting shipped out.

I've been spoiled by getting left on the counter for a few days. What am I?

I'll float it internally as well

Smahht