Spark performance optimization Part1 | How to do performance optimization in spark

[@sky-i8d]

Absolutely amazing, the way content is delivered!!

[@darnasibujji]

Earlier I watched some videos regarding this topic ,no one can explained in this way ,I am glad to see this video,now clearly understood spark optimization techniques

[@BigDataThoughts]

Thanks

[@durairaj4296]

I am searching this kind of topics from the web in long time.. Thank you very much.. And i appreciate your stuff..

[@BigDataThoughts]

@@durairaj4296 thanks

[@harshmohan8419]

best video on internet for spark performace...

[@BigDataThoughts]

Thanks harsh

[@sunnyd9878]

This is excellent and valuable knowledge sharing... Easily one can make out these trainings are coming out of personal deep hands-on experience and not the mere theory ..Great work

[@BigDataThoughts]

thanks

[@funnysatisfying426]

Very informative 👏👍
Keep such videos coming

[@husnabanu4370]

what a wonderfull explanation to the point... thank you

[@BigDataThoughts]

Thanks

[@gancan1654]

perfectly went into my brain, what a clean explanation.
can you please do videos on Pyspark from scratch.

[@hemanthkumar9757]

Very good explanation keep create more videos in spark

[@BigDataThoughts]

Thanks

[@SagarSingh-ie8tx]

One of the best explanation on RUclips 😊

[@thedarkknight579]

I only have one word for this video
"Awesome!!"

[@BigDataThoughts]

Thanks

[@namratashinde9157]

It's very very easy to understand whatever you explained, thank you so much

[@BigDataThoughts]

Thanks namrata

[@sahilmittal7426]

Very Good vedio, awesome work. To the point and one can understand easily

[@BigDataThoughts]

Thanks sahil

[@rajnimehta5156]

Great expectations Mam ... eagerly waiting for your upcoming video

[@BigDataThoughts]

Thanks Rajni

[@quadribrothers4396]

Many thanks for making such informative video

[@samk_jg]

All yours tutorials are too good!

[@theanatomyofreliability2168]

Thanks for sharing. Very informative .

[@BigDataThoughts]

Thanks karan

[@ChethanKarur]

This is excellent maam. Looking forward to watching more videos from you

[@BigDataThoughts]

Thanks Chethan

[@kiranmudradi26]

Another much needed video on Spark optimizations. point to point. Thank you very much for the video.

[@BigDataThoughts]

Thanks Kiran

[@gunasekaranr8029]

Neat and clean explanation. Looking forward to the videos on Spark Optimization.

[@BigDataThoughts]

Thanks Gunasekaran

[@vibhad-cv4sf]

Very well explained. Loving your videos!❤

[@BigDataThoughts]

Thanks

[@satviknaren9681]

REALLY helped me get better at my work

[@BigDataThoughts]

Thanks

[@terrificmenace]

Nice video it was really good 👍🏻 Thank you

[@BigDataThoughts]

Thanks

[@ramkumarananthapalli7151]

Thanks a lot mam for making these videos. These are extremely useful. One of the best videos I have come across.

[@BigDataThoughts]

Thanks Ram

[@ayushgour3984]

Amazing Job Shreya... Keep it Up..

[@BigDataThoughts]

Thanks Ayush

[@arjunkharat121]

Thanks for video ma'am., you made it very simple to understand.. waiting for more video's on this topic and spark

[@BigDataThoughts]

Thanks Arjun

[@BigDataThoughts]

ruclips.net/video/snPYj3TqM1g/видео.html you can chk this too. There are others videos on spark that i have posted

[@abhisekhmishra4029]

So nicely explained Shreya..

[@BigDataThoughts]

Thanks Abhishek

[@sandeepchoudhary4900]

Awesome explanation of the optimisation techniques. If possible please create a video to cover the realtime challenges which you faced in your project and the solution you provided. That will be really helpful.

[@khaderbasha7592]

Awesome Shreya, if possible could you please upload realtime challenges which we faced in realtime environment

[@BigDataThoughts]

Thanks Khader

[@chessforevery1]

where do I get practical session on this optimization technique of spark..?

[@sathisha1702]

If count is not adviced, how can we count the number of rows in data frame?

[@SpiritOfIndiaaa]

thanks a lot , i have case where someother modules write parquet file , i need to process in my module by reading it, so how should i apply bucketing on that day ...can it be possible without writing ???

[@Technology_of_world5]

Good explanation 😊

[@BigDataThoughts]

Thanks

[@puneetnaik8719]

Great explaination !!

[@BigDataThoughts]

Thanks Puneet

[@Sharath_NK98]

Tnks Amigo
It's very helpful

[@BigDataThoughts]

thanks

[@harshalpatel555]

very well explained. but you are telling everything i know to exclude. ?? We need count, group,agg

[@karthikvenkataram4790]

Ultimate 👏👏👏

[@BigDataThoughts]

Thanks

[@anirbanrc1]

Great explanation

[@BigDataThoughts]

Thanks

[@foodietraveller4591]

Nice video mam

[@mohitupadhayay1439]

Coalesce doesn't do Shuffle and that's why it's less expensive than repartition. I believe.

[@BigDataThoughts]

It does but not as much as repartition. Repartition does entire data shuffle as it can reduce or increase no of partitions.

[@mohitupadhayay1439]

@@BigDataThoughts thanks! Can you build an end to end project or some mini project where one can see how and where these properties arte getting implemented? Just watching these in silos only give half knowledge. Thanks.

[@DASHTeknik]

Excellent.

[@npl4295]

good optimization tips

[@BigDataThoughts]

Thanks Neethu

[@tanushreenagar3116]

so nice it helped a lot

[@BigDataThoughts]

Thanks tanushree

[@shaileshc4994]

What are configurations file in spark ?
Plz anyone can ans me please

[@mdatasoft1525]

❤

[@tolasebrisco6565]

Keep the good work #Prinetechs.
I can clearly see all the good reviews about you man…I never believed my account can fixed after 7 months hahaha

[@ahmedaly6999]

how i join small table with big table but i want to fetch all the data in small table like
the small table is 100k record and large table is 1 milion record
df = smalldf.join(largedf, smalldf.id==largedf.id , how = 'left_outerjoin')
it makes out of memory and i cant do broadcast the small df idont know why what is best case here pls help

[@tolasebrisco6565]

Keep the good work #Prinetechs.
I can clearly see all the good reviews about you man…I never believed my account can fixed after 7 months hahaha

[@tolasebrisco6565]

Keep the good work #Prinetechs.
I can clearly see all the good reviews about you man…I never believed my account can fixed after 7 months hahaha