10 frequently asked questions on spark | Spark FAQ | 10 things to know about Spark

This video talks about the most frequently asked question on spark. The terms that should be known by each developer and Architect who works on Spark
Spark architecture in 3 mins
• Spark Architecture in ...
Spark Memory Management
• Spark Memory Management
Spark performance optimization Part1
• Spark performance opti...
Spark performance optimization Part2
• Spark performance opti...
Spark Date Timestamp datatypes
• Spark Date Timestamp d...

[@vaibhavjoshi6853]

Getting confidence in spark because of you only. Thanks so so much!

[@BigDataThoughts]

Thanks

[@nikhilgupta110]

I'm impressed , no copy paste , Covered whole architecture and done full justice to the title. Great work ,you are heading in the right direction, keep going :).

[@BigDataThoughts]

Thanks Nikhil

[@iwonazwierzynska4056]

The next great video

[@angelangeli2089]

good explanation and easy!! :)

[@sachinragde]

Thank you ma'am u explain it very nicely due to u I understand it
And ur voice is very beautiful
Thank you so much

[@rajeshd3940]

quite formative for beginners and Appreciate the efforts made

[@BigDataThoughts]

Thanks rajesh

[@dhavaldalasaniya]

Very nice !! Simple & very knowledgeable 👍🏻

[@BigDataThoughts]

Thanks dhaval

[@harshmohan8419]

Mam you are awesome.. teaching talent with example is like god gifted us.....
Mam, can u briefly tell me if we use yarn, then what would be the role of the cluster manager and its application master in spark.. because driver directly communicating to executor

[@sathwikemmadi4428]

Great content shreya ,I really like all the videos and wanted to learn more about datalakes and best practices designing datalakes.If you have some time and bandwidth please keep a post on the above.
Thanks for all your hardwork.

[@BigDataThoughts]

Thanks sathwik.
Will surely do. Have posted one on what are data lakes ruclips.net/video/wHG0ljN3plg/видео.html
Will post on best practices too.

[@BigDataThoughts]

This week have posted on data lake best practices

[@samirharris1]

Thankyou 😊

[@ramkumarananthapalli7151]

Thanks for making this video. Kindly make more videos on big data engineering.

[@BigDataThoughts]

Thanks Ramkumar surely

[@khajaabdulmujeebbasha6327]

mam there is shuffle in coalese even it is considering as narrow transformation why?

[@SagarSingh-ie8tx]

Nice

[@BigDataThoughts]

Thanks sagar

[@geetakadam6728]

Hello Mam I had gone through your video the way you have delivered is very nice.
I have one dought though silly but I wants to know in local system that means on our laptop when we execute pyspark code who is executor and who is worker and on cloud please explain me executor and worker?
Thanks in advance

[@guptaashok121]

Is it driver which reads all data and divide in partition and sends to executor?

[@BigDataThoughts]

Yes executors job is to just execute.

[@guptaashok121]

@@BigDataThoughts will it not create bottleneck..

[@BigDataThoughts]

@@guptaashok121 no it won't be cos actual data resides on executors, driver is just the coordinator. while assigning task principal of data locality is kept in mind.

[@guptaashok121]

@@BigDataThoughts that's what my doubt is, if driver is not reading the data then how it sends command to executor on which data to work on

[@winfred4193]

Love it!! Become an online boss - Promo-SM !!

[@mithunnambiar1433]

This is what you call WOMEN IN TECH.. absolutely fantastic!!! 💩

[@BigDataThoughts]

Thanks Mithun

[@ranvijaymehta]

Thanks mam ... Today I have subscribed BigData Thoughts

[@BigDataThoughts]

Thanks

[@techmahesh9560]

Nice

[@BigDataThoughts]

Thanks