Install HADOOP on MacOS (Macbook M1, M2 / Apple Silicon) | HDFS
HTML-код
- Опубликовано: 4 авг 2024
- In this video i have shown how you can install HADOOP on you MacOS specific on Macbook M1, and M2 which are apple silicon chips based macbooks. I have also shown how to set up java jdk 8 for hadoop and set up all daemons (namenode, datanode, job tracker etc.) and also how to format namenode.
Medium Link : / install-hadoop-on-maco...
Remove Java JDK : • Remove Java JDK from M...
PayPal : paypal.me/codewitharjun
Patreon : / codewitharjun
➡️ Subscribe : / codewitharjun
Timelapses :
00:00 - Introduction to Hadoop
00:27 - Install Java JDK 8 for Hadoop
02:15 - Set up Java HOME for Hadoop
04:00 - Creating a security key for SSH Hadoop
05:59 - Download Hadoop
07:03 - Configure Environment variable for Hadoop
09:42 - Make changes to coresite.xml
10:48 - Make changes to hdfs-site.xml
11:37 - Make changes to mapred-site.xml
12:10 - Make changes to yarn-site.xml
12:35 - Format Namenode Hadoop
13:25 - Start all daemons of hadoop
14:22 - Make directory on Hadoop using terminal
15:25 - Add file to hadoop
16:05 - Outro
Stay Connected :
➡️ LinkedIn : / arjungautam1
➡️ Github : github.com/arjungautam1
Join Community :
➡️Discord : / discord
Follow me on :
➡️ Medium: / codewitharjun
➡️ Dev.to : dev.to/codewitharjun
#hadoop #CodeWithArjun #Programming #hadooptutorial #hadooptraining #hadoopinstallation 
Наука
This is one of the best hadoop installation video.
Includes all the solutions, GREAT VIDEO!
Happy to help. Be sure to subscribe.
really great tutorial, thanks!
Thank you so much, helped a lot!!
Amazing ! Many thanks
Thank u so much. I installed without error
Happy to help! Be sure to subscribe.
can you also make video to further install hive and spark scala ?
in there any differences in installation between apple silicon and intel core i5?
Thankss You are the Best!!
thank you man
this is very helpful!
Did it work fine for you ?? Without any issue ??
@@CodeWithArjun yeah, it works fine👍
Hi, How do you connect your terminal to VsCode? I'm currently doing a school project and struggling to get to that part. Thanks!
Thanks dude you are a god-send!
In Configure Environment variable for Hadoop when i write Source ~/.zprofile it is giving me an error(/.zprofile:4: command not found: JAVA_HOME) can you tell me what to do
I’m not getting nodemanger and resourcemanager on entering command jps
how to add yarn ui and ambari and hive to this?
I could install hadoop on Mac as well as Ubunutu but can't compile a java program due to missing FileDefNot found . I feel something is wrong with Eclipse/JDK set up.
can you/someone help ?
Thank you!
You're welcome!
@codewitharjun this is very helpful
can also please please make a video on __Installing Hadoop with HIVE on macOS__ without Homebrew
Hello i get Operation not permitted when i run hdfs namenode -format
and installation of R also
Hi, namenode is not coming up on m2
I have a 2015 Macbook Pro with Intel Core i7 - will this tutorial work for that or are there extra configurations I need?
is it working?
Once path is set for java home then need to restart the terminal and then it worked
I am getting this ,can anyone suggest me solution
Starting resourcemanager
ERROR: Cannot set priority of resourcemanager process 7904
Starting nodemanagers
localhost: ERROR: Cannot set priority of nodemanager process 8004
Hi, I faced this error when trying to put a file into Hadoop : could only be written to 0 of the 1 minReplication nodes. There are 0 datanode(s) running and 0 node(s) are excluded in this operation. Does anyone has any idea how to solve this?
FYI, when I typed :JPS, it showed only these :
48528 ResourceManager
2339 QuorumPeerMain
48627 NodeManager
48084 NameNode
48327 SecondaryNameNode
52697 Jps
but not the DataNode
Did u able to solve this
Hi, i get the below error despite trying all possible solutions I Googled, Any help would be appreciated sudo hdfs --daemon start namenode
Password:
ERROR: Cannot set priority of namenode process 23949
o/p of jps
MacBook-Pro ~ % jps
23200 DataNode
23955 Jps
23338 SecondaryNameNode
While browsing the file system, it shows: Failed to retrieve data from /webhdfs/v1/?op=LISTSTATUS: Server Error
Did ayone get a message stating "This key is not known by any other names. Are you sure you want to continue connecting?" What should I do? It also states that the authenticity of the localhost cannot be established.
I did :( Not quite sure what it means tho and what to do with it.
Make a video on Windows installation on Mac M1
yaa please
there is a error saying (code .zprofile not found) any help?
use nano. I guess you haven't configured vscode to run from cmd
instead of code, give open instead... it will open the file in your text editor
bro when i type jps i only get "jps,resource mangaer , node mangaer" . i dont get " name node ,datanode and secondary namenode" also localhost 9870 not working
Stop all and format the namenode and again start all.
@@CodeWithArjun still not working , please help how to do , only jps and resource manager are visible in jps
same for me, only jps and resource manager are visible in jps
@@ayushikhullar8818 Did u get any solution from another side.
@@ayushikhullar8818 did you find the solution? I am also facing same issue
what is the terminal you're using with autocomplete? 😳
I have got seperate video for that check out on my channel.
oh-my-zsh with history plugin.. There are a ton of walkthroughs around.
im at format namenode part and i get this error "zsh: command not found: hadoop "
and if i use "/Users/ogunfoan/hadoop-3.3.1/" Instead of hadoop it says "zsh: permission denied: /Users/ogunfoan/hadoop-3.3.1/"
@@muratkara2012 Hi i found same error too did find the solution for it?
@@ragapranathi3748 no, i just quit
got the same error, type this into cmd "brew install hadoop". Then check version "hadoop version". and then just follow the steps from the namenode format onwards
code .zprofile command not found error can anyone please help me this
I got the same error do you got any solution?
@@johnsnow-yu5sz i guess that's because you don't have visual studio, download it then you can use the code .command
@@johnsnow-yu5sz 5 months late so i assume you figured it out somehow, but you want to do cmd+shift+p in VScode, type "shell command" and choose to "Uninstall" first. Then, choose to "Install" by typing shell command again. After this is installed, you can use any "code ." commands on shell.
If the command is not found, then you haven't enabled the VS Code cli extension. Just search for it in the VS Code command palette or over the web. Secondly, it depends on which directory you're running it from. The .zprofile file is located in your home directory. So run code ~/.zprofile instead
hello sir i had followed each and ever step, im getting an error after opening the browse the file system on the local host, the error is:
same here
Did u get any solution for that ??
@@harshpatel9954 hey I have got the same error, did you get the solution?
Edit yarn-site.xml
when running later
ERROR: Invalid HADOOP_YARN_HOME
An error occurs.
In this case, how should I solve it?
Can you please try to install the entire Hadoop ecosystem such as Hive, Sqoop, Spark, Hbase, Pig, airflow etc.,
Much thanks in advance.
Sure I will publish soon.
Do you uploaded the playlist on full hadoop ecosystem ?@@CodeWithArjun
@@CodeWithArjun when will you make this video?
Failed to retrieve data from /webhdfs/v1?op=LISTSTATUS: Server Error
getting error at hadoop_yarn_home can anyone help me with that??
Bro code .zprofile is not working it's massage like command not found
@codewitharjun this is very helpful _can also please please make a video on _Installing Hadoop with HIVE on macOS_ without Homebrew
when will you make a _cuz i have lab on Monday before that 8th oct