An Alternative to Buying Capacity to Fix Dataflow Stockouts
A Dataflow job that can't get its machine type sends most teams straight to Compute Engine reservations. There's a free config change that went GA in April and might work better.
Blog
Filtered by: Technology × clear
Technology
4 min read
BigQuery Omni Reads Your S3 Data In Place. Just Don't Treat It Like a Pipeline.
I ran BigQuery Omni against a real Apache Iceberg lake in AWS S3 to see where cross-cloud querying shines and where it quietly breaks — and why it's a query window, not a data pipeline.
By Jeremy
Continue reading →
Technology
8 min read
I Made Snowflake Query GCS Data Without Copying a Single Byte
A GCS-resident Iceberg table, a workload identity trust handshake, and Snowflake reading it live -- no pipeline, no copy, no ETL. Here's exactly how zero-copy federation works, and where it falls apart.
By Jeremy
Continue reading →
Technology
5 min read
Apache Beam Streaming Windowing: Quiz
Ten questions on Beam's windowing methods, watermarks, triggers, and late data — with sourced explanations for every answer.
By Jeremy
Continue reading →
Technology
5 min read
Cloud Run Is Fine: When You Actually Need Kubernetes
An instructor at Next '26 shrugged and said Cloud Run is fine. He's mostly right. Here are the five things that actually justify reaching for Kubernetes instead, and why the two are closer to the same thing than the debate admits.
By Jeremy
Continue reading →
Technology
7 min read
Managing Data Products in Google Cloud's Knowledge Catalog
Dataplex's Knowledge Catalog turns data products into first-class, discoverable, requestable objects. A full field tour — finding, describing, sharing, and securing certified data products — plus the gotchas the docs skip.
By Jeremy
Continue reading →
Technology
7 min read
Finally Found a Way to Make a Monolithic Beam Pipeline Modular
For years I shipped Apache Beam pipelines as one giant file in a Dataflow Flex Template because custom containers never worked for me. The fix I had missed: a Flex Template is two images, a launcher and a worker, each with its own entrypoint.
By Jeremy
Continue reading →
Technology
3 min read
Stop Your GCP Bill Before It Stops You
AI has made it almost too easy to spin up GCP services without thinking through the cost implications. Here's how I built a hard cap that actually cuts off spending - using Google's own recommended pattern.
By Jeremy
Continue reading →