Posts
All the articles I've posted.
- 8 MIN READ•Apr 29, 2026
Writing to an Apache Iceberg Table: How Commits and ACID Actually Work
Here is exactly how an engine writes to an Iceberg table, step by step, from data files through the atomic commit that makes ACID guarantees possible.
writing to Apache IcebergIceberg ACID transactionsatomic commit - 8 MIN READ•Apr 29, 2026
Volcano, Vectorized, Compiled: How Engines Execute Your Query
The Volcano model processes one row at a time. Vectorized execution processes batches with SIMD. Code generation fuses operators into compiled code. Here is how each works.
query execution modelsvectorized executionvolcano iterator model - 8 MIN READ•Apr 29, 2026
Hidden Partitioning: How Iceberg Eliminates Accidental Full Table Scans
Iceberg's hidden partitioning separates physical layout from user queries using transform functions. Here is how it works and why it eliminates accidental full scans.
Iceberg hidden partitioningpartition transformsaccidental full table scan - 8 MIN READ•Apr 29, 2026
Inside the Query Optimizer: How Engines Pick a Plan
Query optimizers transform SQL into execution plans using rule-based rewrites, cost-based search, and adaptive runtime adjustments. Here is how each approach works.
query optimizercost-based optimizationquery planning - 8 MIN READ•Apr 29, 2026
Partition Evolution: Change Your Partitioning Without Rewriting Data
Iceberg lets you change partition schemes without rewriting data. Here is how partition evolution works internally and why Hive-style partitioning could not do this.
Iceberg partition evolutionpartition spechidden partitioning - 8 MIN READ•Apr 29, 2026
B-Trees, LSM Trees, and the Indexing Tradeoff Spectrum
B-trees balance reads and writes for OLTP. LSM trees maximize write throughput. Bitmap indexes accelerate OLAP filtering. Here is when to use each.
database indexing strategiesB-tree vs LSM treebitmap index - 8 MIN READ•Apr 29, 2026
Performance and Apache Iceberg's Metadata
Iceberg's three-layer metadata tree eliminates directory listing and enables multi-level data skipping. Here is how scan planning actually works.
Apache Iceberg metadata performanceIceberg scan planningmanifest pruning - 8 MIN READ•Apr 29, 2026
How Databases Organize Data on Disk: Pages, Blocks, and File Formats
Databases structure data on disk as heap files, sorted files, or LSM trees, then wrap it in formats like Parquet with metadata that lets engines skip irrelevant blocks.
data file formatsParquet file formatdatabase storage internals - 8 MIN READ•Apr 29, 2026
The Metadata Structure of Modern Table Formats
Iceberg uses a metadata tree, Delta Lake uses a transaction log, Hudi uses a timeline. Here is exactly how each format organizes metadata and why it matters.
table format metadataApache Iceberg metadataDelta Lake transaction log - 8 MIN READ•Apr 29, 2026
Row vs. Column: How Storage Layout Shapes Everything
Row stores keep records together for fast transactions. Column stores keep field values together for fast analytics. Here is how each layout works and when to use it.
columnar vs row storagecolumn-oriented databaserow store performance - 9 MIN READ•Apr 29, 2026
What Are Table Formats and Why Were They Needed?
Table formats like Apache Iceberg solved the ACID, schema, and performance problems that turned data lakes into data swamps. Here is how each one works.
data lake table formatsApache IcebergDelta Lake - 9 MIN READ•Apr 29, 2026
How Query Engines Think: The Tradeoffs Behind Every Data System
Every database is a collection of engineering tradeoffs. Learn the 9 design decisions that shape how query engines store, index, and process your data.
query engine designdatabase tradeoffsquery optimization