👋 Guys everyday we upload high quality in depth tutorial on your requested topic/technology so kindly SUBSCRIBE to our channel👉( bit.ly/Intellipaat ) & also share with your connections on social media to help them grow in their career.🙂
Following topics are covered in this video: 01:07: what is big data 03:22 - issues with big data 04:26 - hadoop vs spark 05:40 - what is apache spark 05:58 - features of apache spark 06:47 - industry usage of spark 12:12 - what are spark Rdds 13:15 - how to create rdds (demo) 23:45 - spark components 25:27 - spark sql 29:10 - how to create dataframe in spark 33:20 - spark mllib 38:40 - spark & scala with json 01:06:32 - how to deal with xml data 01:27:33 - spark sql with hive 01:46:15 - performance tuning 02:02:40 - what is the need of kafka 02:10:35 - what is kafka 02:21:10 - architecture of kafka cluster 02:38:05 - apache zookeeper 02:47:09 - how to configure single node single broker cluster 03:20:55 - what is batch processing 03:21:36 - what is stream processing 05:19:32 - apache flume
For beginners, it would be a very good tutorial. Please prepare next vedio with more examples with real-time cases with huge datasets and huge processing and show distinction that how spark would be faster comparatively with other technologies and performance metrices with how can we specify number of cores and executors and what factors influence to choose them kind of things etc .. All the best @intellipaat
Sure, will forward the topics to the respective team and will upload the video soon. Please subscribe to Intellipaat channel & press the bell icon so that you don't miss that video from us.
Dear Sir/Madam, thank you for this valuable tutorial. Can you also kindly suggest some books to read on this tutorial? It will be greatly appreciated. Thank you.
Nice video.. In the spark streaming example, the data stream is processed in the form of word count as we coded. just wanted to know where the data stream will be sent back after processing in real time or in production.
Thanks Gopi. Sure will forward the topic to the respective team and will upload the video soon. Please subscribe to Intellipaat channel & press the bell icon so that you don't miss that video from us. :)
Hi Shruti, sure will forward the topic to the respective team and will upload the video soon. Please subscribe to Intellipaat channel & press the bell icon so that you don't miss that video from us. :)
i have installed the updated ver of c++ and also change the port 80 to 8080 but in second file i am not able to chg 443 port in ssl ,confg i find nathing there please help
Hi Gowthami, thanks for your comment. We are working on this video. Give us some time please and we will upload it soon. :) Please subscribe to Intellipaat channel & press the bell icon so that you don't miss any videos from us.
Hi Abid, sorry the slides are a part of course artifacts which can't be shared on here. If you want it please enroll to Intellipaat Apache Spark course. Here's the course link: intellipaat.com/apache-spark-scala-training/
HI Subhana Nazar, Click on the links mention for Hypothesis & Anova Link for Hypothesis: ruclips.net/video/jde9Ws9WPoQ/видео.html Link for Anova: ruclips.net/video/ABvkRlENB_c/видео.html
Glad you liked it Nat Shaw. Keep learning with Intellipaat. :) Also subscribe to Intellipaat channel and press the bell icon so that you never miss any video from us.
Hi Yukti, that is a course artifacts so that can't be provided with the video. If you are interested to learn Apache Spark, please enrol for Intellipaat online course. Here's the course link: intellipaat.com/apache-spark-scala-training/
Yes. There is Cassandra connectors, we can supply credentials and load processed data to Cassandra, note that Cassandra will have tombstones for deleted items
For a short and crisp understanding of spark architecture you can also refer to this post: optimizedlive.wordpress.com/2020/06/11/understand-spark-architecture/
You should have mentioned as scala in the tile, Many are coming from python bg and they want to use pyspark api for the spark.Thought the instruction is about spark using pythong..Disappointing.
👋 Guys everyday we upload high quality in depth tutorial on your requested topic/technology so kindly SUBSCRIBE to our channel👉( bit.ly/Intellipaat ) & also share with your connections on social media to help them grow in their career.🙂
Following topics are covered in this video:
01:07: what is big data
03:22 - issues with big data
04:26 - hadoop vs spark
05:40 - what is apache spark
05:58 - features of apache spark
06:47 - industry usage of spark
12:12 - what are spark Rdds
13:15 - how to create rdds (demo)
23:45 - spark components
25:27 - spark sql
29:10 - how to create dataframe in spark
33:20 - spark mllib
38:40 - spark & scala with json
01:06:32 - how to deal with xml data
01:27:33 - spark sql with hive
01:46:15 - performance tuning
02:02:40 - what is the need of kafka
02:10:35 - what is kafka
02:21:10 - architecture of kafka cluster
02:38:05 - apache zookeeper
02:47:09 - how to configure single node single broker cluster
03:20:55 - what is batch processing
03:21:36 - what is stream processing
05:19:32 - apache flume
Best video watched on youtube for spark. Very well explained session.
You are best teacher .... terrific teaching style ...god bless you 🙏
Thanks a lot
where ever we go this man is there for explaining this
Do like, share & subscribe to our channel to get updates on upcoming videos.
@@Intellipaat 👍
One of best video on spark.. Thanks alot for this!
Glad you liked it. Keep learning with Intellipaat. :)
yes i agree with you
Best free tutorial video I've ever watched
good narration and presentation
Thank you kindly! @Abdu Chadili Do like, share to get updates on upcoming videos.
For beginners, it would be a very good tutorial. Please prepare next vedio with more examples with real-time cases with huge datasets and huge processing and show distinction that how spark would be faster comparatively with other technologies and performance metrices with how can we specify number of cores and executors and what factors influence to choose them kind of things etc
.. All the best @intellipaat
Sure, will forward the topics to the respective team and will upload the video soon. Please subscribe to Intellipaat channel & press the bell icon so that you don't miss that video from us.
@@Intellipaat great quick response...
A good hands on training with details .
Yes we have covered that in this tutorial.
13:15 - how to create rdds (demo),where we have to write this command, i meant ki on which prompt??
Dear Sir/Madam, thank you for this valuable tutorial. Can you also kindly suggest some books to read on this tutorial? It will be greatly appreciated.
Thank you.
@Grandmaster GyoroGyoro If you wish to get Personal support Kindly call our course advisors on IND: +91-7022374614 US: 1-800-216-8930 (Toll-Free).
Good one.
Thanks for watching😊 Do like, share & subscribe to our channel to get updates on upcoming videos.
I learned a lot about Spark. Was there something about Scala? -:/) Still a Great video...
@Bryan Chance Do like, share & subscribe to our channel to get updates on upcoming videos.
Nice video.. In the spark streaming example, the data stream is processed in the form of word count as we coded. just wanted to know where the data stream will be sent back after processing in real time or in production.
Great work
Glad you liked it. Keep learning with Intellipaat. :)
Wonderful explaination with good examples.
Can you please also do a vedio about powershell
Thanks Gopi. Sure will forward the topic to the respective team and will upload the video soon. Please subscribe to Intellipaat channel & press the bell icon so that you don't miss that video from us. :)
What's the max retention period of kafka message that can be configured? Can it be indefinite?
Yes it can be configured
Sir which one should I learn Spark or Hadoop? I am confused about this
Please upload apache solr tutorials video
Hi Shruti, sure will forward the topic to the respective team and will upload the video soon. Please subscribe to Intellipaat channel & press the bell icon so that you don't miss that video from us. :)
Good Job Guys..
Thanks. Keep learning with Intellipaat. :)
i have installed the updated ver of c++ and also change the port 80 to 8080 but in second file i am not able to chg 443 port in ssl ,confg i find nathing there please help
Nice tutorial Sir. Please tell me from where we can get the Slide for the same.
Best tutorial. Can you please post a detailed end to end handson tutorial on Flink?
Hi Gowthami, thanks for your comment.
We are working on this video. Give us some time please and we will upload it soon. :)
Please subscribe to Intellipaat channel & press the bell icon so that you don't miss any videos from us.
Can we get job in spark as fresher.
It would really help if you provide case studly document or file as you show in tutorial
Sure. Kindly Share your mail ID.
@@Intellipaat please share the doc with me alsoo. will be great help.
great
Thanks Bernard. Keep learning with Intellipaat. :)
thanks
You're welcome!
Hello! good work. can you please share these slides ?
Hi Abid, sorry the slides are a part of course artifacts which can't be shared on here. If you want it please enroll to Intellipaat Apache Spark course. Here's the course link: intellipaat.com/apache-spark-scala-training/
Do u have Apache spark with python
Hi Abdullah, here's the video: ruclips.net/video/dK_F1tuK9C4/видео.html
Good session, but should have been start with Introduction of Environment. How to install & setup spark.
We will upload separate another video for that soon.
@@Intellipaat Any chance on a video on how to integrate Mongodb with Spark?
please how to save a spark job from spak-shell ? please i need it
Can you make a video on hypothesis and anova!!
HI Subhana Nazar, Click on the links mention for Hypothesis & Anova
Link for Hypothesis: ruclips.net/video/jde9Ws9WPoQ/видео.html
Link for Anova: ruclips.net/video/ABvkRlENB_c/видео.html
love this video ❤️
Glad you liked it Nat Shaw. Keep learning with Intellipaat. :) Also subscribe to Intellipaat channel and press the bell icon so that you never miss any video from us.
@intellipaat can you make crash course on hive for big data
Hi Makarand, sure, will forward this topic to the respective team and will upload this video soon. Stay tuned!
can you make 10 hour videos please.
Sure . Noted. Will upload soon. :)
When we open spark shell why it is opened as Scala> ? And what is the Scala?
SCALA is a machine language, if u want to work up on spark u need to learn scala also. hope u understand buddy.
Very informative but "roughness" in voice made it difficult to hold on for long 5 hours. Thanks for the video
Sorry to hear that
Can anybody tell me that are there any pre-requisites of learning Spark ??
There is no pre requisite to learn spark.
@@Intellipaat thanks
need help i installed xamp server while starting apache i getting the following error 0x00007b
Thanks for your trainings, could please help to share apache nifi training.?
Hi Vijaya, here's Apache Nifi link: ruclips.net/video/kwOqGhnvMfY/видео.html
please provide the video for using it with csv files
Hi Yukti, that is a course artifacts so that can't be provided with the video. If you are interested to learn Apache Spark, please enrol for Intellipaat online course. Here's the course link: intellipaat.com/apache-spark-scala-training/
Is it helpful for hadoop admin
Of course Madhuri, it is in fact important for a hadoop admin to learn apache spark for implementation.
Thank you for the great job!! I have a question: Is it possible to export that json file in Cassandra database?
Yes. There is Cassandra connectors, we can supply credentials and load processed data to Cassandra, note that Cassandra will have tombstones for deleted items
where can i find code and dataset
Can you enable subtitles
Sure will do that.
Please try to lose the extra roughness or base in voice . Its not helping
Edit : Btw thanks for the good information !
Thanks for the feedback. :)
it is advised to get a good microphone.
Thanks for the feedback. :)
It is good session, but should have been started with spark Introduction.
Hi, we have covered from scratch that's why.
For a short and crisp understanding of spark architecture you can also refer to this post:
optimizedlive.wordpress.com/2020/06/11/understand-spark-architecture/
You should have mentioned as scala in the tile, Many are coming from python bg and they want to use pyspark api for the spark.Thought the instruction is about spark using pythong..Disappointing.
I need 🙄🙄it
It is not for beginners
Please change title.... its not for beginners....
Hi, please suggest a better title and we will change that. :)
:)
love this video ❤️
Thanks. Kep learning with Intellipaat. :)