Below you will find pages that utilize the taxonomy term “Databricks”
Managing logging in Spark ain’t easy, and is even harder in managed clouds like Databricks or EMR.
This is the next instalment on my quest to read and help understand interesting papers in the data space.
After reading the Snowflake paper, I got curious about how similar engines work. Also, as I mentioned in that article, I like knowing how the data sausage is made. So, here I will summarise the Delta Lake paper by Databricks.
The one where Airflow messes with you.