spark overview | lec-1
HTML-код
- Опубликовано: 4 окт 2024
- From this video, we are going to start talking about Apache spark in great depth.
Directly connect with me on:- topmate.io/man...
For more queries reach out to me on my below social media handle.
My Second Channel -- / @competitivegyan1
Interview series Playlist:- • Interview Questions an...
Follow me on LinkedIn:- / manish-kumar-373b86176
Follow Me On Instagram:- / competitive_gyan1
Follow me on Facebook:- / manish12340
My Gear:-
Rode Mic:-- amzn.to/3RekC7a
Boya M1 Mic-- amzn.to/3uW0nnn
Wireless Mic:-- amzn.to/3TqLRhE
Tripod1 -- amzn.to/4avjyF4
Tripod2:-- amzn.to/46Y3QPu
camera1:-- amzn.to/3GIQlsE
camera2:-- amzn.to/46X190P
Pentab (Medium size):-- amzn.to/3RgMszQ (Recommended)
Pentab (Small size):-- amzn.to/3RpmIS0
Mobile:-- amzn.to/47Y8oa4 ( Aapko ye bilkul nahi lena hai)
Laptop -- amzn.to/3Ns5Okj
Mouse+keyboard combo -- amzn.to/3Ro6GYl
21 inch Monitor-- amzn.to/3TvCE7E
27 inch Monitor-- amzn.to/47QzXlA
iPad Pencil:-- amzn.to/4aiJxiG
iPad 9th Generation:-- amzn.to/470I11X
Boom Arm/Swing Arm:-- amzn.to/48eH2we
My PC Components:-
intel i7 Processor:-- amzn.to/47Svdfe
G.Skill RAM:-- amzn.to/47VFffI
Samsung SSD:-- amzn.to/3uVSE8W
WD blue HDD:-- amzn.to/47Y91QY
RTX 3060Ti Graphic card:- amzn.to/3tdLDjn
Gigabyte Motherboard:-- amzn.to/3RFUTGl
O11 Dynamic Cabinet:-- amzn.to/4avkgSK
Liquid cooler:-- amzn.to/472S8mS
Antec Prizm FAN:-- amzn.to/48ey4Pj
![KSI - Thick Of It (feat. Trippie Redd) [Official Music Video]](http://i.ytimg.com/vi/At8v_Yc044Y/mqdefault.jpg)
I have cleared 'Databricks Certified Associate Developer for Apache Spark' certificate yesterday but will still follow this playlist because I like your explanation. You're doing great work🙏🙏.
The fact that you answer to all the "why" and "what" of a simple definition makes learning so much in depth and enjoyable! Great teaching!
I have seen so many english tutorial for spark but the way you are explaining is just amazing. Content are more clear and easy to understand. Keep up the good teaching work. Hats off 👏
sir, aap ne hum sab ke liye phir se video banaya hai...hum sab ke liye bahut he achhi baat ho gai...
Sir , genuine review … You are the best teacher 💯
Hey Manish. Really appreciate your efforts. These are gem lectures man.
nice explanation as well as apka sikhate samay bahut hi sahajeta se samjhana achha laga .pl. continue this simplicity hame encourage karti hai. thanaks Sir..🙏🙏🙏gr8 work...
Loved the "baap & 4-beta" analogy... gr8 content !!!
Bro I am from Patna working as a web scraper in b2b company . I can assured u have millions subscriber one day …… keep going best of luck
nicely explained,no videos from almost 3 weeks, waiting for next
heads off to your honesty Manish for each video lecture you deliver.
Attendance Marked
keep making vdos brother ......ur every vdo outclass other so called data techs
Hi Manish Sir
Good Evening 🙂
First I want to say thank you so much for this playlist.
I'm watching this playlist from 1st video and making my notes parallelly from this video.
Once I've watched this playlist, I will watch another playlist with my notes.
I scheduled a call with you to discuss more about data engineering through the Topmat portal.
Any suggestion for me or my notes which I'm making based on this playlist?
Thanks Again
Bahut badiya!! Kya baat hain. You are so good Man!!
Spark doesn’t have its own storage but in the computer cluster architecture explanation it’s mentioned that each slave has certain RAM, CPU and Storage. What does storage meant here?
examples ache hai smjhne ke liye
Very nicely explained. Thank you Bhaiya.
How it's possible to comment before
10days when video is released today.
I have decided to follow your channel, funny thing is i am poor in hindi so, i don't know if I'll learn DE. Most probably I'll end up learning Hindi. Anyway looking forward for more videos on how to crack DE .
bhai ye videos continue rkhna.
Bilkul
Excited for upcoming next video s❤
Will upload soon
Hello bhai apne jo example Iiya usmein total slave RAM 64 GB hai to kya master ko bhi 64GB RAM ki need hai ya 16GB RAM is enough to distribute 64GB data?
baap waala example in parallel processing is amazing
very good explanatiom
Nice to see with spark...sir
Excellent
❤ your content
Hello, when you will be creating playlist for Hadoop. Please make
what is configuration
bro why hdfs used please add this too
❤
Hello sir, I had learnt sql, power bi & python for Data analyst role. but as of now, there are very less job for freshers in this field, what about data enginner I dont have any idea in this field is there are jobs for fresher in DE....
Idhar to Data analyst se v Kam openings hote hai. But few freshers get chance to work as DE directly. Openings are bit less compared to Data analyst
bhaiya is both of your series enough for interviews?
waiting for ur next vidio..😊
It will come today
very nice
Awesome explanation bro.... Must continue the series
Yes I will upload video soon
Thank you waiting for more
Coming soon
how to learn both spark overview playlist and spark practical playlist. Pahle kaun sa padhe ? ya agar sath me padhna h to kya flow rahega .
After 5-6 videos of fundamentals you can start practical in parallel
Great video !!
Thankyou so much bhaiya 😭
Thank you. This means that the storage or disc is irrelevant in spark cluster
Aisa nhi hai. Aapko storage to lagega data ko permanently store karne ke liye. Aapke paas option bahut hai ki aap ise kaha rakhna chahte ho like hdfs, cloud or your on premise server
@@manish_kumar_1 g thek hai. Means ky spark itself ka storage sy koi laina daina nae hai but hum spark ky sath different storage services use kr sakty hain. Thank you
Great work bhiya
Thank you so much for great content and explanation. can we get pdf notes of all these theories?
You are very great person bro 🙏
Thank you so much 😀
Thanks a lot for this!, this is really helpful :)
done
please start soon, waiting
Already started sir
Directly connect with me on:- topmate.io/manish_kumar25
Thanks bro, sharing info
spark data store nhi karta. Lekin shuffling me jo disk I/O hota hai, us data ke sath kya hota hai..post execution?
Clean kar diya jata hai
super!!
achha bhaiya agar hamare cluster ka size kam baa aur agar usase jyada size ka data aa gaya tab ye kaise handel karta hai...aur data streaming hai..
Streaming ka bilkul idea nahi hai mere ko
very nice class ...looking for full series
Will upload soon
@@manish_kumar_1bhai can u plz make how to read spark ui whenever spark job is slow , kase identify karenge , not able to get proper video
Optimization playlist bna skte ho mostly asked in interview
Bhai ek resume ka video banado 2 years dusre stream se data engineer k liye
Ok
is your videos in English version, can you please advise
No currently only in hindi
@@manish_kumar_1 , Thanks for your response
Storage nahi hota but architecture me aap bol rahe storage 5TB, yeh confusion please clear kare
Spark ka khud ka nhi Hota hai. Agar storage nhi hoga to Data persists kaise hoga.
@@manish_kumar_1 data persistence RAM me hona chahiye, Jaisa aapne bola yahi toh diffrence hai Hadoop and Spark me, kya aap example bata sakte jab data HDD/SDD me store hoga instead of RAM.
@@Binge_View you need to work a lot on understanding fundamentals of computer science. RAM short lived memory hai.
@@manish_kumar_1 Sir, I have done B.E in computer science, I understand RAM is temporary memory, HDD we can use as heap memory or for data view swapping, even I know how can we see 4GB file with 1GB RAM and some more in depth details about it, what iam saying is
Hadoop Map Reduce used to work like, whenever data transformation come to it, it will perform some transformation and store the result to HDFS and then perform another transformation and as required fetch the previous data from HDFS, as HDFS is slow in reading and writing spark introduced.
What I am not understanding is, if spark also stores data in HDD then in which case it's happening and how it's different from Hadoop.
Please don't take anything in a negative manner, I really like your way of explaining and clearing the concept, you are the best teacher I found for spark concepts.
I hope you understand.
Sir, this playlist would have totally how many lectures, approx?
Around 20-25 for experienced and 12-15 will be enough for Beginner
@@manish_kumar_1 Thank you Sir
Namaste Manish bhaiya. Please see this series through to the end. Hope to learn a lot from you. 🙏
Yes sure
aap course start kb kroge data engineer ka
Yahi hai course bro - and this is lec 1 of series
thanks manish!!
Stay tuned for the series 🙂
@@manish_kumar_1 I have experience in support domain. but now looking to switch into data domain. hope your series helps me.
@@Somnath-je9nd Definitely it will
Bhai syllabus spark ka refer krdo ,proper mil nhi rha h
Spark internals
Spark submit
Spark process life cycle
Spark optimization
Data frame API coding
Bro , how much coding knowledge is required for data engineer
python
Array and string ke easy and medium level ka questions kijiye from leetcode. Itna me kaam ho jayega
@@manish_kumar_1 and python pura aana chahiye ? Ya basic concept sufficient hai
@@amansoniamanat syntax aana chahiye like loops, variables,list,dictionary,function and basics of Oops concept
@@manish_kumar_1 okay, Thank you bro 🙏
Has anyone made notes.
9 months and still reached till 13th video.
its really difficult to be consistent.
Hope i will complete the series soon😑
Main leta hi hua tha 😂
😂😂