Migration from Hive Metastore to Unity Catalog - 2023.08.30
HTML-код
- Опубликовано: 5 сен 2024
- Are you excited to leverage serverless, unity catalog model registry, #lakehouse monitoring, fine-grain governance, and data lineage? This session will walk through migrating from Hive Metastore to #UnityCatalog
slides drive.google.c...
scripts drive.google.c...
#databricks #hivemetastore
Excellent session and material. Thanks a lot!
Happy its helpful!
Awesome, glad to find so useful content before migration
Happy its helpful!
Great resource for UC migration. Thanks for sharing.
Glad it was helpful!
Hi Stephanie, thanks for the video. i am currently using DLT with apply changes and write output to hive metastore, which has AWS glue connect with it. The output is a streaming table, however, it is actually a view build from __apply_changes_storage_xxx table. Any idea how this could be migrate from hive to UC? Also, when i change the same DLT pipeline target to a UC schema, it seems AWS glue is not able to get the table meta. Is there any documentation i can follow for DLT build table migrate from hive to UC? Thanks
Hey Stephanie, I migrated everything from hive_metastore to unity just now, but when I'm executing my pipelines it's throwing class and library errors. I had the same libraries installed which were in the old clusters. In fact I edited the old cluster and changed the mode to "shared" in order to make it unity. The same libraries work fine in the old cluster. Do you happen to know what I'm missing here.
Can you reach out to your account team? I don't know whats going on.
@@stephanieamrivera Thanks for replying. The issue was resolved actually. In shared mode, it does not support some of the APIs and spark context according to the documentation. So we used single-user, multi-node cluster and it's all working fine. Thanks.