Thanks for the detailed setup of a spark application in Intellij and SBT integration . Please do make some more videos on real world use cases in both python and scala .
Thanks for a great video. i know it may be a longshot to seek your help, I am at step to verify spark install and it is not working, yet. Q - without setting spark env variable, how did your spark-shell work ? As you only showed, HADOOP-HOME env variable only Microsoft Windows [Version 10.0.19043.1151] (c) Microsoft Corporation. All rights reserved. C:\Users\kiran>echo %JAVA_HOME% C:\Java\jdk1.8.0_291 C:\Users\kiran>java --version java 16.0.2 2021-07-20 Java(TM) SE Runtime Environment (build 16.0.2+7-67) Java HotSpot(TM) 64-Bit Server VM (build 16.0.2+7-67, mixed mode, sharing) C:\Users\kiran>echo %HADOOP_HOME% C:\hadoop\hadoop-3.2.1 C:\Users\kiran>echo %SPARK_HOME% C:\spark\spark-3.1.2-bin-hadoop3.2 C:\Users\kiran>spark-shell 'spark-shell' is not recognized as an internal or external command, operable program or batch file.
Please do include it in your PATH. 05:13 In this video am executing the command from the spark bin folder itself. If you want to access from any directory check my next comment.
I had already done it in this ruclips.net/video/3LTSSzBZvXE/видео.html Please check this portion from the timestamps in the above video. Setting Spark_Home for accessing Pyspark from all directories.
Finally I found the complete guide. All previous I've seen are too old and magically got this thing work
Glad it helped 👍
Thanks a lot, quite a detailed coverage of all spark stuff.
NOTE:
Please use a headset to get the optimal sound as RUclips has decreased the volume considerably 😆!
If you enjoyed this video, then consider subscribing today!
ruclips.net/channel/UCjO8Jq2sdpuI134axhMp0Fg
Thanks for the detailed setup of a spark application in Intellij and SBT integration . Please do make some more videos on real world use cases in both python and scala .
Thank you!!! Keep watching.
sir Very nice and detailed video
Thank you.
@@stream2learn sir , is it possible to connect you through Skype or any other ways
bro i don't have word for you🥰🥰
Always Welcome!
Good. Buy I need Pyspark using Pycharm, can you do it.
Let me try to put one short video for it, As we have most of the steps covered here.
As promised, here is my video covering pyspark and pycharm.
ruclips.net/video/3LTSSzBZvXE/видео.html
@@stream2learn Thanks a lot mate, Appreciate it.
Thanks for a great video. i know it may be a longshot to seek your help, I am at step to verify spark install and it is not working, yet.
Q - without setting spark env variable, how did your spark-shell work ? As you only showed, HADOOP-HOME env variable only
Microsoft Windows [Version 10.0.19043.1151]
(c) Microsoft Corporation. All rights reserved.
C:\Users\kiran>echo %JAVA_HOME%
C:\Java\jdk1.8.0_291
C:\Users\kiran>java --version
java 16.0.2 2021-07-20
Java(TM) SE Runtime Environment (build 16.0.2+7-67)
Java HotSpot(TM) 64-Bit Server VM (build 16.0.2+7-67, mixed mode, sharing)
C:\Users\kiran>echo %HADOOP_HOME%
C:\hadoop\hadoop-3.2.1
C:\Users\kiran>echo %SPARK_HOME%
C:\spark\spark-3.1.2-bin-hadoop3.2
C:\Users\kiran>spark-shell
'spark-shell' is not recognized as an internal or external command,
operable program or batch file.
Please do include it in your PATH.
05:13
In this video am executing the command from the spark bin folder itself. If you want to access from any directory check my next comment.
I had already done it in this
ruclips.net/video/3LTSSzBZvXE/видео.html
Please check this portion from the timestamps in the above video.
Setting Spark_Home for accessing Pyspark from all directories.