Spark Interview Questions | Spark Context Vs Spark Session

#Spark #Context vs #Session
Please join as a member in my channel to get additional benefits like materials in BigData , Data Science, live streaming for Members and many more
Click here to subscribe : / @techwithviresh
About us:
We are a technology consulting and training providers, specializes in the technology areas like : Machine Learning,AI,Spark,Big Data,Nosql, graph DB,Cassandra and Hadoop ecosystem.
Mastering Spark : • Spark Scenario Based I...
Mastering Hive : • Mastering Hive Tutoria...
Spark Interview Questions : • Cache vs Persist | Spa...
Mastering Hadoop : • Hadoop Tutorial | Map ...
Visit us :
Email: [e-mail]
Facebook : / tech-greens
Twitter :
Thanks for watching
Please Subscribe!!! Like, share and comment!!!!

[@sndselecta]

By far the best explanation after researching for a few days. I think main take away is config is moved from sc to each session, creating independently isolated sessions via one overall sc for the cluster.

[@shr2000]

Thank you, explained very well. better than anything i read/ referred before

[@TechWithViresh]

Your Welcome:)

[@manubhatnagar8519]

Very nice, keep them coming

[@rudrasingh21]

All the lectures really helping me to understand deeper. Thanks

[@TechWithViresh]

Thanks :)

[@arindamnath1233]

The best explanation I ever get about Spark Session and Spark Context.

[@TechWithViresh]

Thanks :)

[@krishnakishorenamburi9761]

Viresh, Please mention the version of spark in the Slide on the top @4:25 and @5:45 as an improvement to slides.

[@vinaykumardudi154]

Very nice explanation. Crisp and clear.

[@TechWithViresh]

Thanks :)

[@BibekMishra84]

Great explanation. Quick question: What Code Editor/IDE are you using at 7:06 ? If its specific to Scala, what is best code editor/IDE for pySpark ?

[@TechWithViresh]

I am using Databricks Notebooks, for PySpark - PyCharm is a good IDE.

[@vijaybigdata752]

I have a doubt, in this scenario if we have 2 spark sessions for a single spark context, when spark context goes down will all 2 spark sessions killed? Please confirm.

[@unknown_traveller12]

Correct

[@vinujackify]

Had a query....Suppose I have a spark-submit job, does this mean I can create multiple spark session within .scala code of this spark-submit job

[@TechWithViresh]

one spark-submit job can have only one spark session

[@mohdrayyankhan6623]

@@TechWithViresh so my doubt is one spark-submit will have only one spark session along with spark context ? What is the use of having both.
2- what is the meaning of multiple user/ multiple sessions.
I am not clear

[@mahammadshoyab9717]

Viresh, can you make video on spark architectire ,because i had gone through so many videos of spark architecture but i m not satisfying, so can you make deeper spark architecture video.

[@TechWithViresh]

check these videos : ruclips.net/video/JSvzGJC8vjQ/видео.html
ruclips.net/video/LfsL4ot6iSU/видео.html

[@danteatwork1255]

is possible to get sparksession id from python/pyspark?

[@TechWithViresh]

Yes

[@sarjfud]

In case of multiple spark sessions can the data/dataframe of each session interact with each other? Can we take output from both spark session and do a union operation?

[@MrVivekc]

I tried this scenario and it works..we can perform union or two DFs from diff spark session can interact.

[@sarjfud]

@@MrVivekc can you please let me know some use case scenarios where we can use multiple spark sessions?