Это видео недоступно.
Сожалеем об этом.
Hadoop Installation Guide Step by Step | How to install Hadoop on ubuntu | Installing Hadoop local
HTML-код
- Опубликовано: 13 янв 2021
- Hadoop Installation Guide Step by Step | How to install Hadoop on ubuntu | Installing Hadoop local
#HadoopInstallationStepByStep #InstallingHadoopInUbuntu #UnfoldDataScience
Hello ,
My name is Aman and I am a Data Scientist.
About this video:
In this video, I demonstrate step by step process on installing hadoop in your local machine. I explain each and every step in detail about how to install single node hadoop in your local machine. Below topics are explained well in this video:
1. Hadoop Installation Guide Step by Step
2. How to install Hadoop on ubuntu
3. How to install hadoop single node cluster in ubuntu
4. How to install hadoop in vmware workstation
5. How to install hadoop on ubuntu virtualbox
6. How to install hadoop on windows 10 using vmware
please find the installation guide file here : drive.google.com/drive/folder...
If you need Data Science training from scratch with me, Please fill this form (Please Note: Training is chargeable)
docs.google.com/forms/d/1Acua...
If you need one on mentorship from me(Career Guidance/Resume Tips/Mock interview/data science training etc), Please fill this form and I will get back to you (Please Note: One on One mentorship/training is chargeable as it will be a live engagement)
docs.google.com/forms/d/1Acua...
Book recommendation for Data Science:
Category 1 - Must Read For Every Data Scientist:
The Elements of Statistical Learning by Trevor Hastie - amzn.to/37wMo9H
Python Data Science Handbook - amzn.to/31UCScm
Business Statistics By Ken Black - amzn.to/2LObAA5
Hands-On Machine Learning with Scikit Learn, Keras, and TensorFlow by Aurelien Geron - amzn.to/3gV8sO9
Ctaegory 2 - Overall Data Science:
The Art of Data Science By Roger D. Peng - amzn.to/2KD75aD
Predictive Analytics By By Eric Siegel - amzn.to/3nsQftV
Data Science for Business By Foster Provost - amzn.to/3ajN8QZ
Category 3 - Statistics and Mathematics:
Naked Statistics By Charles Wheelan - amzn.to/3gXLdmp
Practical Statistics for Data Scientist By Peter Bruce - amzn.to/37wL9Y5
Category 4 - Machine Learning:
Introduction to machine learning by Andreas C Muller - amzn.to/3oZ3X7T
The Hundred Page Machine Learning Book by Andriy Burkov - amzn.to/3pdqCxJ
Category 5 - Programming:
The Pragmatic Programmer by David Thomas - amzn.to/2WqWXVj
Clean Code by Robert C. Martin - amzn.to/3oYOdlt
My Studio Setup:
My Camera : amzn.to/3mwXI9I
My Mic : amzn.to/34phfD0
My Tripod : amzn.to/3r4HeJA
My Ring Light : amzn.to/3gZz00F
Join Facebook group :
groups/41022...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/profile/Aman-Ku...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Learn Ensemble Model, Bagging and Boosting here:
• Introduction to Ensemb...
Build Career in Data Science Playlist:
• Channel updates - Unfo...
Artificial Neural Network and Deep Learning Playlist:
• Intuition behind neura...
Natural langugae Processing playlist:
• Natural Language Proce...
Understanding and building recommendation system:
• Recommendation System ...
Access all my codes here:
drive.google.com/drive/folder...
Have a different question for me? Ask me here : docs.google.com/forms/d/1ccgl...
My Music: www.bensound.com/royalty-free...
Access English, Hindi courses here- www.unfolddatascience.com/s/store
Don't forget to register on the website guys ☺
bro @Unfold Data Science asap very urgent
plz help during installation of this step ie sudo nano .bashrc it says below
sudo nano .bashrc
[sudo] password for hdoop1:
hdoop1 is not in the sudoers file.
Thanks a lot, this finally helped after so many searching here and there found one perfect video for Hadoop Installation
Thanks for watching Abhijit.
Made it look so simple. Thanks a lot. Finally installed after so many tries. Good Work...!!!
Glad it helped Kashish.
Brother, I was struggling with the installation for the last month till coming across your video. It was a god send. I am glad that I could complete following your steps. The best part is how to tackle the errors during installation. Looking forward to more such stuff from you
did you install successfully ?
I literally installed it 3 times in my college lab system but the instructions were so unclear, I couldn't install it properly at all. I watched this video and installed it on my Virtual Box Ubuntu and I FINALLY FINALLY FINALLY GOT IT 😭😭😭😭😭🎉🎉🎉 Thank you so so so much for this video!!!
Worked like charm
THANK YOU SOO MUCH
Your video stopped me from smashing my laptop ;)
Thank you very very much......this was really helpful. Hoping to see more from you.
Thanks Aniket.
Loved it bro, was struggling to install from other sites, nicely explained. Keep it up....:-)
Glad it helped. Thanks for watching.
i spent 1/2month to install hadoop but every time i failed, but allah showed me your channel and after i watched your video now i installed it thanks.
Glad to hear that Anwar. Happy Learning!
guys, there is a slight change while editing .bashrc file @9:22 of the video where the last line is
export HADOOP_OPTS"-Djava.library.path=$HADOOP_HOME/lib/nativ"
please change it to
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
later when you type
source .bashrc
it won't throw any errors
thank you.
Thats what heroes do.
@@juhisharma6873 same error, can you fix it
Hope u are able to fix it, if not please read other comments or just google it, It may be a very minor issue
Thanjks
Thanks
BRO I JUST WANTED TO SAY, THANK YOU SO MUCH !!!
Thanks a lot, definitely the video to watch if you want to install hadoop ! keep posting this kind of quality videos !
Thanks a lot.
Thank you so much, finally installed hadoop, god bless you!
You're welcome!
COME ON MAN! Thank you so much. This video really helpful for me as a beginner
Thanks Taufiq.
@@juhisharma6873 I have same issue.. can u pls help?
Great Video Bro!!!!!!!!!!!!!!!! Excellent explanation and the file provided saves a lot of time. Thanks a lot bro!!!!!!!!!!!!!
Welcome Satyam.
highly recommend :- with the help of this video i installed Hadoop successfully, but make sure u follow all the steps properly.
Thanku.
Cheers Harsh. Your comments mean a lot 😃
Thanks a lot . This process is really working.
Welcome Riya.
Thank you for making this video, it really helped me! I'm in a Big Data training right now, and I've been trying to install Hadoop properly for weeks at this point. The only error I got this time was "port 22: Connection Refused" when using the ssh localhost command. I followed a suggestion from Stack Overflow and did "sudo service ssh restart". Then ssh localhost worked fine for me, and I was able to continue with the rest of your instructions. Thank you!!
Welcome Morgan.
thanks a lot, that works
Thanks very much a million times, your tutorial was very helpful. Thanks again and stay bless..
You're welcome!
Thanks a lot for this tutorial!
Thanks Manoj for watching.
thanks, it installed properly and working without any errors.
🍻
best video for downloading hadoop thank you sir g
Thanks for u r efforts.. i am able to install hadoop :)
Helpful Video :) Thanks
If I’d like to install one name node and two data nodes, should I install 3 Ubuntu systems on virtual box and link then?
Could you do a video on creating 1 master and 2 salve nodes?
is there is any problem using current super
user to install hadoop or we need to adduser for the purpose to install hadoop. i studied hadoop in a years ago so i forgot everything in hadoop
hello sir,
I want to create a cluster were one master and two slave
so, I will have to follow this steps for all VMs right??
thank u so much i learned alot and it worked for me
Glad it helped
Really good one. thank you
Welcome.
Thank You soooo much for such video
It took 6 hrs to finally do everything successfully after resolving many hurdles.
Anyone face any issue then may be I can help. Please reply your problem in comment
Awesome. Love your comment. Let's keep helping each other.
where to get that localhost gui
in the end when we are putting the local file to hadoop i am getting a error please can you help me resolve it.
This is the error:
hdfs dfs -put home/abdeali/demo.txt /
2022-10-02 12:00:36,644 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
put: `home/abdeali/demo.txt': No such file or directory
Thank you in advance
I have the same problem that @Abdeali Hazari has. Any help?
@@dhilipkarthik1807 after starting your Hadoop nodes
- go to port 9870 in browser to see it running locally
@20:00 when you try to enter the folder it should be 3.2.3 not 3.2.1 if you are having problems. Thank you again!
U're a life saver tysm
While running ~/hadoop-2.3.1/sbin/
It is showing an error that no such file/directory and before that while modifying the yarn files, it was asking for rewrite y/n? I gave yes to it. Please help with this
Thanks a lot...!!🙏❣️
Welcome.
nandri thala... thanks
Welcome.
Superb !!!!
Thank you very much
Is this Hadoop installation done in pseudo distributed mode?
Very Nice Explanation. Please mention in which Linux and Version you are working Thank You
First of all, let me thank you for explaining so beautifully. It was really helpful. Now coming to my concern - I am facing issue in running Hadoop. I followed all the steps in this video, but when am running the command "ssh localhost" or trying to start services, I am getting connection refused message. Would really appreciate your help here.
This is a very common issue, below link will help:
stackoverflow.com/questions/17335728/connect-to-host-localhost-port-22-connection-refused/29868328
@@UnfoldDataScience
Im getting error in 2nd file sudo nano $HADOOP_HOME/etc/Hadoop/Hadoop -env.sh command
@@deekshanayak895 did the error ressolved for u?
@@deekshanayak895 same for me
Hi Sir,
1st of all, thank you for the proper-made video, it is really nice. However, I have a question:
If I download the hadoop package via web browser and extract it in Download file, do I need to modify any directory? Or should I just follow your instruction as suppose?
You can install from downloads folder as well, just give the correct path
thank you sir best video
Thanks Abhishek
Thanks a lot!
Welcome.
thank you very much
Hi sir, Thanks for the video.
I have one question please clarify it.
In the core-site.xml and in the other .XML files you gave path like
home/hadoop/tmpdata
home/hadoop/ dfsdata/namenode
home/hadoop/ dfsdata/datanode
And all...
Now my question is will system automatically create this directories at the back end ?
Or we have to manually create and give the mentioned path that are given in the document?
Thanks
System will pick from these places, i mean its already there if i am getting question right.
Hey sir. Lovely video. It worked flawlessly. Thanks.
I have few questions.
1) where exactly is this data getting stored ?? India itself?? Where??
2) How can we store data for free?? Why isn't Hadoop paid???
1. If you are doing local installation, in your local machine, if u do in a cluster, wherever cluster machines are located physically.
2. You will need to pay for hardware - not for using hadoop
Hello,
When i’m trying to unzip Hadoop file using tar xzf command, it is throwing “no space left on device” error. Can you please help me on how to resolve this.
cool thank u😀
Welcome.
Good one.
Thanks a ton.
will it run spark-shell --master yarn commands?
Hello sir, can u plz tell where to get that localhost gui
@UnfoldDataScience i was facing some problems regarding the ssh localhost and i tried the commands from stack overflow too but it is not working .. even after serve ssh restart command it is asking for password and after entering the password it is showing that "hdoop is not in the sudoers file.Thiis incident should be reported" this is being shown
Thank you so much for this video it really worked! I have only one question How to install hadoop in multiple nodes in ubuntu?
Thanks, in multinode, we need to do some setting change in different files, process wise 80% its same
Brother while downloading hadoop its showing that link as 404 error what to do
great💯💯
Thanks a lot.
its getting error when i am adding txt file for the first time what is the problem please help me out . error :put: File /data.txt._COPYING_ could only be written to 0 of the 1 minReplication nodes. There are 0 datanode(s) running and 0 node(s) are excluded in this operation.
Awesome.
Thanks Vignesh.
bro did you download hadoop 3.2.1 only?
great job bro
Thanks alot
Great video
Thanks Adithi
Thanks a lott! sir can you start making tutorials on hadoop, spark, hive and other big data related subjects, with their complete executions in virtual box, There is no other resources to learn.
Hi Aritra, go to my playlist and look for Big data playlist. It's already there.
Or else you can search on RUclips, "unfold data science big data and unix playlist"
You will find it. If not let me know
@@UnfoldDataScience Okay sir.
I am unable to put file in user(aman) to hdoop. i.e. i am unable to use last commands. i am getting the error "No such file or directory". later i came to know hdoop cant access files in user(aman). what should i do.
Is creating a new user for hadoop is necessary ?
Thank you Bro
@@juhisharma6873 can you tell me in which minute ?
@@juhisharma6873 create a file in the same directory and call it hadoop-env.sh and add java home in it
You can create a file using command "put go to the same directory first"
touch hadoop-env.sh
Then edit it like the vedio
@@juhisharma6873 you are welcome
Thanks Abdul and Juhi
Is there any video of yours for single node cluster or multiple?
Please check hadoop playlist.
bro while i gonna put this sudo nano .bashrc command it showing like "this hadoop user is not in sudoers file this incident will be reported it" ,will u help me to come out from this
This is a very common issue, if you google you will easily find solution, you need to add user in sudo group.
Sir i don't know why but put command didn't worked it said no such file or directory
Thanks for the video, question: When I run ssh localhost it gives me this: ssh: connect to host localhost port 22: Connection refused. Any idea why and how to solve it?
Please read previous comments, it's a common issue
hello sir, I tried following your installation but hadoop 3.2.1 is no longer available for download, and the .bashrc is compleytely different in hadoop 3.2.2, will this be a problem as I am missing some important alteration in that part.
Try with the version available, I dont think its a problem.
@@UnfoldDataScience Sir the .bashrc file is completely different
No such file or directory for cat command what should I do
It is showing error when I execute the $hdfs namenode -format
hdfs not found error
Hello !!
Thanks for this video.
Everything is working fine but Datanode is not showing after running jps.
Kindly stop all and start all once. If it does not work try starting components separately using shell scripts in same folde.
I followed your steps, but in the first step, sudo apt update it's showing error e: sub error occurred. Please help sir
sir i am not able to start namenode on local host ,it is starting on my virtual machine and at the end it is showing permission denied
Goods Tutorial😍
Glad you like it pls share with friends also.
is it compulsory to install jdk 8, can't we install latest version of java like jdk 17
Not compulsory
sir please make video on multinode hadoop cluster installation
Hello thank you so much for your easy to understand tutorial, when i run hdfs namenode -format i get an error message "Cannot execute /usr/local/Cellar/hadoop/3.3.0/libexec/libexec/hdfs-config.sh." pls kindly help
HADDOP_HOME is not set properly.
Below link will help
stackoverflow.com/questions/49063060/hadoop-installation-error-error-cannot-execute-hdfs-config-sh
Plaise when i run this cmd sudo nano . Bashrc
This message show wht i can do
(Hadoop not in the folder sudoers. Incident will signal)???
Check previous comments, people have answered it in comment.
you can check this link as well:
www.tecmint.com/fix-user-is-not-in-the-sudoers-file-the-incident-will-be-reported-ubuntu/
after command jps secondarynamenode is not showing.. what should i do now...
Restart hadoop
Sir you installed hadoop on ubunto in VM......i have pc with ubuntu as a system, can i use the same installation of hadoop on ubuntu ?
Yes same way u can do, no need of VM.
After ssc localhost
It was showing host key verification failed what I need to do
Hope u are able to fix it, if not please read other comments or just google it, It may be a very minor issue.
somehow secondaryname node is not popping up when i run jps. However i was able to execute put statement and placed the file in hadoop root directory. Is it an issue if secondarynamenode is not running and still others are working ??
stop all and start all once.
@@UnfoldDataScience
Somehow started working when I login in as root. Thanks
I dont know how to geta java runtime that supports apt. I downloaded a new java jdk yet it doesn't work
This is a java issue. U can follow different solution on various websites one may work
I had a problem with the HADOOP_OPTS, was that supposed to be an "=" and not a dash??
Yes
getting the following error:
WARNING: HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX.
ERROR: Invalid HADOOP_YARN_HOME
I have seen some posts in stackoverflow and modfied HADOOP_PREFIX to HADOOP_HOME and still unable to resolve the error. can you pleae help to resolve this.
See if below link helps:
dbversity.com/warning-hadoop_home-is-deprecated/
Help me plz...it's showing me this error :(
Command 'hdfs' not found, did you mean:
command 'hdfls' from deb hdf4-tools (4.2.14-1ubuntu1)
command 'hfs' from deb hfsutils-tcltk (3.2.6-14)
Try: sudo apt install
may be you are not running from correct directory.
see it once:
stackoverflow.com/questions/46896502/hdfs-command-not-found-when-trying-to-execute-command-hdfs-namenode-format/46912585
[ Directory ' / home/hdoop/hadoop-3.2.1/etc/hadoop ' does not exist ]
^ this is the error i am facing when I try to get into the env.sh of hadoop , also i have 3.2.2 version of hadoop installed, I am in desperate need of help.
check manually if files and folders exist at right location, if not delete and re download
Great work, but i am getting an issue after completion of hadoop installation... I could not send my local files to hdoop user, where hdfs exists. I have approached the same procedure... no idea why it is not recognising
hey did you get any answer to your issue as i am getting the same error
hdfs dfs -put home/abdeali/demo.txt /
2022-10-02 12:00:36,644 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
put: `home/abdeali/demo.txt': No such file or directory
can you please help me regarding this. Thank you
@@abdealihazari4484 I am getting the same error. help me please.
Thanks , great stuff, Facing one issue at the last both the namenode and secondary name node starting in the same virtual box.
after doing ./start-dfs.sh
Output :
hdoop@sayantan-VirtualBox:~/hadoop-3.2.1/sbin$ ./start-dfs.sh
Starting namenodes on [sayantan-VirtualBox]
Starting datanodes
Starting secondary namenodes [sayantan-VirtualBox]
2021-05-24 17:54:10,907 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Find library on web and put it
i am getting command hdfs not found error at the command hdfs namenode -format can u help me with that
installation did not happen or u r not in right directory
unable to retrive (put) file from /home/Desktop to hdfs. it says put: `/home/qsadmin/test.txt': No such file or directory. can you help? couldnt find any right explanation in google
File is not existing in source
Hi sir. I followed the same as mentioned in video. But finally when I type jps , datanode is not showing. I tried restarting 2 commands start-dfs.sh and start-yarn.sh but still not getting datanode in jps. please help me out. I tried many things present in internet but nothing worked.
i have the same problem too....
Does stop-all.sh and start-all.sh help?
Go through the comments on this video. I think people faced similar issue and resolved
I fixed the problem. It was in the first file and specifically this line: "Djava.library.path=$HADOOP_HOME/lib/nativ". It's supposed to be native and not nativ. When I fixed that then my program started working perfectly.
@@simery9066 Nope still not working.
how to check the web interface after the installation???
When I paste wget link to download hadoop it's showing HTTP request sent awaiting response... 404 not found please help me with this soon
Yes,same issue, please help if anyone has tackle this error
Google the link, it may have changed
Google it, may be link has changed
It shows hdoop not in sudoers file. When I run sudo nano .bashrc command what to do?
please read previous comments, its an easy issue to fix.
Sir , while starting hdfs format , it is showing error.
same here.
Where is the document to copy paste the commands ?
48 th line is not opening in my terminal y
BRO after "cat ~/.ssh/id " its showing no file or directory found
Can we get localhost web UI after this
yes
For the 2 file in giving hadoop the java info.. Directory not found shows to me
Anyone can help??
No sure what issue you are facing. Please post your error.