Видео

Thank you 👍🙏

Hi Donald, I don’t have any script for that, but I can create and share it over the weekend.

Thank you for the feedback :) I am working on improving the quality of videos. Future videos will be better for sure :)

Hi Joshua, I hope you enjoyed the video and got to learn something new. Here is the link to the code. gist.github.com/Databracket9/f6507607048697fc403e0753d64e1bf4 Thanks for your support :)

Hey sunny That’s a good question. Azure synapse is new to market and have dependency on azure cloud A more generalised and intelligent solution to handle and manage data at scale is databricks. We can use synapse to get the job done, it is more azure centric and not extendible with majority of data and AI solutions.

Happy to hear that you found the content useful. Thank you for your support 🙂

Thank you very much 🙂

Thank you very much 🙂

Thank you very much :)

Hey, For snowpark trainee, in my opinion you need to familiarize yourself with following topics. python basics snowflake and spark architecture pyspark and snowflake basics underlying cloud essentials (AWS, GCP or Azure) and mindset to not get stressed/scared of tasks - You are in a learning phase, everything will appear and feel alien. stick with your routine, learn and implement without giving up.

@@data_bracket thank you

Hi atharva Here is the link to the API. storage.googleapis.com/generall-shared-data/startups_demo.json Thanks.

Sounds good. I will try to curate and publish series on Snowflake and Snowpark soon. :)

@@data_bracket waiting here!

Glad you liked it and found it useful. Thanks for the comment :)

Thank you for the comment. Hope this was useful!

Noted on this. My apologies. I will not repeat that in upcoming videos. thanks for the feedback!

Unfortunately yes😔 I regret it I will try to generate better quality content going forward. Thanks for the feedback 🙂

Sorry about that. Upcoming videos are going to be better. Thanks for the feedback!

Yes, My bad. I’ll improve the quality of my upcoming content. Thanks for the feedback 🙂

Understood. I will create a video on Git integration with Databricks soon. Thanks for your comments.

thank you for your feedback. I will work towards improving the audio quality.

Sure.

Have you thought of considering step functions? You can use initiate start job run sync functionality which will sit in a wait status until completed. If the glue trigger is happening through boto3, pull the job run ID from the response of the start job run. Create a while not true loop with sleep, then call get job run and check the JobRunState, if completed initiate the next job run. for parallel run, check this docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-map-state.html

😂 Thank you

Yes, My bad. My initial videos have bad audio quality. Working towards improving them. Thanks for the feedback.

Thank you for the feedback. I'll try to produce better quality audio and visuals going forward. Really appreciate your inputs🙏❤️

Hi I dont have any specific dataset for this demo. You can use virtually any dataset. Just truncating the column names from csv file will give you a file with no header. I manually removed the column names from the file and uploaded the sample to S3 for this demo.

Thank you very much.

Glad you liked the video! ☺️

sarcasm bro @@data_bracket

Hi Prabhat Here is the reference to the dummy just used in the video. opensource.adobe.com/Spry/samples/data_region/JSONDataSetSample.html If you have any specific use case in mind. please drop a comment . I have create a video out of it. Thanks for watching the Demo.

Hello Abdul Jaweed. Thanks for your comment and support. I will surely try to create a video according your request very soon.

Hi @rayees_thurkki, Its great to know that you want to advance your skill in data field. you can use databricks community edition for free to learn all the features. community.cloud.databricks.com/

Hi @techproductowner Thank you for your comments, glad you liked the video. The dataframe write is happening on databricks file system(dbfs), which is not a data warehouse. if you want to learn how to write to data warehouse like snowflake, check this demo -> ruclips.net/video/KHsxlN9XKww/видео.htmlsi=qCvkrg8wJJCpCRLe FYI - Star schema can be defined by the maintainer.

@@data_bracket Thank you for your reply , can you pls help me related question : If i go to databricks (premium version ) -> compute -> create sql warehouse -> provision it . I dont' see it anywhere in azure portal under azure resource ? where are the sql warehouse contents stored when we create the sql warehouse from within the Databricks -> compute section

Hi @vemedia5850, can you please help with the error stack to understand why the filesystem call failed? and please refer this notebook to check if you have any typo's or missing permissions. databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/1356463967729483/4352720773097014/5548585219097941/latest.html

You are welcome 😀 Hope it was helpful.

You're very welcome! Glad it was helpful :)

Thank you very much :)

Glad it was helpful! Thank you for the support :)

Hi @sreeharis3989 If you have a list of libraries to be installed you can run pip freeze > requirements.txt in your local to capture the dependency list and move it into docker containers workdir Post that, it's just a simple call to run pip install requirements.txt instead of manually installing one dependency after other.

FYR - dev.to/behainguyen/python-docker-image-build-install-required-packages-via-requirementstxt-vs-editable-install-572j

Additional : You can try to replace the pip3 install streamlit with pip3 install -r requirements.txt, But make sure to copy the file to image first by adding command before the run pip3 install with COPY requirements.txt <destination_path_in_instance_image>

Through docker, we can expose ports. But if you want to expose IP the container needs to be in a network, you need orchestration platforms or cloud offertins such as EKS or Kubernetes where you can expose cluster_ip within a network of pods to communicate with other pods.

Glad it was helpful 😀

Thank you. Glad it was helpful. More videos are on the way...

HI @furry2fun I am afraid we don't have any code for this demo. But if you have any specific use case in mind, kindly drop a comment and I'll put up a video demonstrating your use case. Thanks.

Really glad it was useful for your use-case.

Noted, Thank you for the feedback. 🙇 For upcoming videos, I'll try to maintain professional audio.

Thank you for expressing your feedback :) much appreciated.

Glad to know the demo was helpful. Appreciate your feedback. Thank you for your support :)