Data Engineering's Durable Skills: What AI Can't Automate
The data engineer who survives the next five years isn't the one who codes the fastest — it's the one who understands context. Here's what that means in practice.
Blog
AI & Agents
5 min read
Your AI Risk Register Has Four Bullets. FINOS Has 46.
SR 26-2 punted generative AI out of model risk scope and told you to govern it anyway. The FINOS AI Governance Framework is a free catalogue of 46 risks and controls, with the regulatory mapping already done.
By Jeremy
Continue reading →
Technology
5 min read
An Alternative to Buying Capacity to Fix Dataflow Stockouts
A Dataflow job that can't get its machine type sends most teams straight to Compute Engine reservations. There's a free config change that went GA in April and might work better.
By Jeremy
Continue reading →
AI & Agents
5 min read
Thinking of an agent as a Colleague
I was challenged to work out what it would take to think of agents as colleagues. Taken seriously it becomes an operating model: identity, scoped access, governed onboarding context, supervision, evaluation, deprovisioning. Here's what worked when I built that, and the three places it broke.
By Jeremy
Continue reading →
Technology
4 min read
BigQuery Omni Reads Your S3 Data In Place. Just Don't Treat It Like a Pipeline.
I ran BigQuery Omni against a real Apache Iceberg lake in AWS S3 to see where cross-cloud querying shines and where it quietly breaks — and why it's a query window, not a data pipeline.
By Jeremy
Continue reading →
Career
4 min read
You Don't Really Understand It Until You've Taught It
Someone called me a good teacher and I wanted to argue. Here's what mentoring juniors, teaching my six-year-old drums, and handing off platform frameworks taught me about ego, understanding, and why teaching is the real work.
By Jeremy
Continue reading →
AI & Agents
6 min read
SR 26-2 Punted on AI. Here's What Actually Regulates Your Models.
The new model risk guidance excludes generative AI in a single footnote and defers the rest to a future rulemaking. But out of scope isn't unregulated - a stack of existing law still governs every AI decision you ship.
By Jeremy
Continue reading →
Technology
8 min read
I Made Snowflake Query GCS Data Without Copying a Single Byte
A GCS-resident Iceberg table, a workload identity trust handshake, and Snowflake reading it live -- no pipeline, no copy, no ETL. Here's exactly how zero-copy federation works, and where it falls apart.
By Jeremy
Continue reading →