Building a Self-Verifying Text-to-SQL Agent with DuckDB and Custom Eval Harness

I've been studying agents and getting into the weeds with evals. So finished a project to help walk myself through some challenges. text-to-sql is a solved demo where the hard part is teaching the agent to refuse. I built an agent over DuckDB that verifies its own SQL and declines when the schema can’t answer (so no inventing an email column), and a 64-question eval harness that scores “correctly declined” as its own metric, separate from accuracy. Used DuckDB’s native TPC-H to make the whole thing one-line reproducible. Repo + writeup below. FYI: the headline number is the harness validating itself, not a model just yet! repo: https://github.com/mehrabr/agentic-duckdb-analyst write up: https://mehrabr.com/engineering/2026/06/20/text-to-sql-is-a-solved-demo.html