Posts
All the articles I've posted.
- 31 MIN READ•Sep 2, 2026
Moving Iceberg Tables Between Catalogs Without Rewriting Data
Why moving Iceberg tables between catalogs is a pointer copy, and the protocol that makes a cutover safe for one table or thousands.
Apache IcebergCatalogsMigration - 30 MIN READ•Sep 2, 2026
Logs, Traces, and Metrics as Tables: Building an OpenTelemetry Data Lake on Iceberg
Building an OpenTelemetry data lake on Iceberg: schemas for spans, logs, and metrics, ingestion, query patterns, and retention.
OpenTelemetryApache IcebergLogs - 31 MIN READ•Sep 2, 2026
Orchestration in 2026: Airflow 3 vs Dagster vs Prefect vs Event-Driven
Where Airflow 3, Dagster, Prefect, and event-driven triggering stand for lakehouse pipelines in 2026, after the Prefect acquisition of Dagster.
AirflowDagsterPrefect - 29 MIN READ•Sep 2, 2026
Postgres Meets the Lakehouse: pg_lake, pg_duckdb, and When Postgres Is Enough
What pg_lake, pg_duckdb, and pg_mooncake do at the Iceberg level, and honest thresholds for when Postgres is enough.
Postgrespg_lakepg_duckdb - 30 MIN READ•Sep 2, 2026
Schema Registries and Event Schemas: Avro, Protobuf, and JSON Schema on the Way Into the Lakehouse
How Avro, Protobuf, and JSON Schema evolve through a registry, and how that maps to the schema evolution rules of Iceberg.
Schema RegistryAvroProtobuf - 30 MIN READ•Sep 2, 2026
Storage-Partitioned Joins and the Bucket Transform
How the spec-defined bucket transform lets engines skip the shuffle in joins, and how to set it up and keep it engaged in Spark.
Apache IcebergBucketingQuery Planning - 28 MIN READ•Sep 2, 2026
Synthetic Data in the Lakehouse: Generation, Governance, and Testing
What synthetic data in a lakehouse is for, the generation methods, how to preserve fidelity, and where synthetic tables belong.
Synthetic DataTestingLakehouse - 24 MIN READ•Sep 2, 2026
Will AI Replace Data Engineers?
What the evidence shows about whether AI replaces data engineers, which parts of the job compress, and which parts do not.
AIData EngineeringCareers - 31 MIN READ•Aug 25, 2026
Agent-Driven Storage Tiering for Apache Iceberg: Moving Cold Data Without Breaking Queries
A background agent can move cold Iceberg partitions to cheaper tiers without breaking live queries. Heatmaps, path-safe moves, and restore paths.
Apache Icebergstorage tieringcost optimization - 31 MIN READ•Aug 25, 2026
Securing the Agentic Lakehouse Gateway: Preventing Prompt Injection and Data Exfiltration
Agentic lakehouse gateways face prompt injection and exfiltration through query results. A threat model and defenses for the layer in front of data.
AI agentssecurityMCP - 31 MIN READ•Aug 25, 2026
Apache Ossie and Apache Polaris: Putting Semantic Models in the Open Catalog
Apache Ossie and Polaris put metric definitions in the open catalog. What the spec covers, what Polaris stores, and what is still unfinished.
Apache OssieApache Polarissemantic layer - 31 MIN READ•Aug 25, 2026
Arrow Flight SQL and ADBC: Why the Database Driver Is the Slowest Part of Your Query
JDBC and ODBC often dominate large-result time. Flight SQL and ADBC keep data columnar from server to client, with Python, Go, and Rust examples.
Apache ArrowFlight SQLADBC