This article outlines how Delta Lake tables managed in Databricks and those managed in Snowflake can be shared in both directions without duplicating data. This is achieved through the use of shared object storage (ADLS) and open table formats. The article discusses the benefits, tradeoffs, business and cost advantages, the approach to implementation, and main technical aspects to consider.
Overview
Interoperability enables one platform to access data that is owned by another platform, without moving the data.Both Databricks and Snowflake store a single copy of the underlying Parquet files in ADLS, and they share only table metadata through catalog APIs. Databricks stores data in Delta (or Delta Uniform), which Snowflake can read via catalog integration.On the other hand, Snowflake-managed Iceberg tables can be accessed by Databricks through catalog federation.
East Texas is gaining a new state park at Lake Palestine that will include amenities like hike-and-bike trails, campsites, and lakefront access. The Texas Parks and…
Back in May we spotted a Linux patch adding "Panther Lake R" as a rugged version of Panther Lake intended for "use in harsh environments". Since Linux 7.2 we have seen…
Discussion (0)
No comments yet. Start the conversation!