I have read quite a bit this week, I’m also preparing a summary of the RDD paper.
After reading the Snowflake paper, I got curious about how similar engines work. Also, as I mentioned in that article, I like knowing how the data sausage is made. So, here I will summarise the Delta Lake paper by Databricks.
I have dropped the Weekly from the title. It was about time.
In the midst of chaos, there is also opportunity.
Sun Tzu
The one where Airflow messes with you.
I didn’t know much about Snowflake, so I decided to have a look at its SIGMOD (ACM Special Interest Group on Management of Data) paper and investigate a bit more what special capabilities they offer, and how they compare to others.