Hive Installation Demo On Ubuntu | Hive Installation On Hadoop | Hive Installation on Ubuntu 20.04
HTML-код
- Опубликовано: 8 сен 2024
- Hive Installation Demo On Ubuntu | Hive Installation On Hadoop | Hive Installation on Ubuntu 20.04
#HIveInstallationOnUbuntu #UnfoldDataScience
Hello ,
My name is Aman and I am a Data Scientist.
About this video:
In this video, I demonstrate step wise process of hive installation. I explain different steps involved in hive installation on ubuntu. I also explain what all issue can one expect in hive installation on ubuntu. Below topics are explained well in this video:
1.Hive Installation Demo On Ubuntu
2. Hive Installation On Hadoop
3.Hive Installation on Ubuntu 20.04
4.How to install hive on Hadoop
5. How to install hive on ubuntu
Reference doc is placed here : drive.google.c...
About Unfold Data science: This channel is to help people understand basics of data science through simple examples in easy way. Anybody without having prior knowledge of computer programming or statistics or machine learning and artificial intelligence can get an understanding of data science at high level through this channel. The videos uploaded will not be very technical in nature and hence it can be easily grasped by viewers from different background as well.
If you need Data Science training from scratch . Please fill this form (Please Note: Training is chargeable)
docs.google.co...
Book recommendation for Data Science:
Category 1 - Must Read For Every Data Scientist:
The Elements of Statistical Learning by Trevor Hastie - amzn.to/37wMo9H
Python Data Science Handbook - amzn.to/31UCScm
Business Statistics By Ken Black - amzn.to/2LObAA5
Hands-On Machine Learning with Scikit Learn, Keras, and TensorFlow by Aurelien Geron - amzn.to/3gV8sO9
Ctaegory 2 - Overall Data Science:
The Art of Data Science By Roger D. Peng - amzn.to/2KD75aD
Predictive Analytics By By Eric Siegel - amzn.to/3nsQftV
Data Science for Business By Foster Provost - amzn.to/3ajN8QZ
Category 3 - Statistics and Mathematics:
Naked Statistics By Charles Wheelan - amzn.to/3gXLdmp
Practical Statistics for Data Scientist By Peter Bruce - amzn.to/37wL9Y5
Category 4 - Machine Learning:
Introduction to machine learning by Andreas C Muller - amzn.to/3oZ3X7T
The Hundred Page Machine Learning Book by Andriy Burkov - amzn.to/3pdqCxJ
Category 5 - Programming:
The Pragmatic Programmer by David Thomas - amzn.to/2WqWXVj
Clean Code by Robert C. Martin - amzn.to/3oYOdlt
My Studio Setup:
My Camera : amzn.to/3mwXI9I
My Mic : amzn.to/34phfD0
My Tripod : amzn.to/3r4HeJA
My Ring Light : amzn.to/3gZz00F
Join Facebook group :
www.facebook.c...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Learn Ensemble Model, Bagging and Boosting here:
• Introduction to Ensemb...
Build Career in Data Science Playlist:
• Channel updates - Unfo...
Artificial Neural Network and Deep Learning Playlist:
• Intuition behind neura...
Natural langugae Processing playlist:
• Natural Language Proce...
Understanding and building recommendation system:
• Recommendation System ...
Access all my codes here:
drive.google.c...
Have a different question for me? Ask me here : docs.google.co...
My Music: www.bensound.c...
![ARGENTINA vs. CHILE [3-0] | RESUMEN | ELIMINATORIAS SUDAMERICANAS | FECHA 7](http://i.ytimg.com/vi/De3AWmXzhuA/mqdefault.jpg)
Thank you bro ! I was also getting guava error. Now resolved. Great video
Welcome Shubhangi.
Thank you again ! Just one correction to document .
export HADOOP_HOME=/home/hdoop/hadoop-3.2.1 ; this needs to be updated with 3.2.3 in the document .
Otherwise- we will get the error as "Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path".
Thanks buddy
if you have found the solution of hadoop home/prefix, share it with us
Nice!! Thanks for pointing out the multiple bindings!
Thanks! Magnificent tutorial, very well explained and done! Congratulations
Glad you found it helpful. Continue learning and watching
Thank you Sir for this tutorial
Welcome Annu.
Great
Thanks a lot :)
I was facing the issue with the installation thanks
Thank you
GREAT THANKS for this helpful tutorial !
tks bro 👌
such a nice tutorial video... thanks bro
You are most welcome
Thank you for the tutorial and for the resources.
do you have any tutorial for installing Hadoop as a cluster?
Hi sir, in real time.
Is it recommended to use derby database? Is it safe?
I saw people use MySQL to store metadata coz it's safe and fast.
Plz let me know is it safe to use derby to store huge metadata?
Thanks 😊
Throwing me this error plz help me out !!
Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path
It can be one of few error, see it:
stackoverflow.com/questions/36044498/issues-with-installing-hive-on-ubuntu-hadoop-home-or-hadoop-prefix-must-be-se
in my case, i solved editing hive-config.sh change hadoop-3.2.1 (as it say in the video) for hadoop-3.3.1 (the last version i downloaded today)
Exception in thread "main" java.lang.RuntimeException: The dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x
and when i check there is not folder as hive in tmp
but my hive installation was successful
Error is coming schema tool is not found while running this command : schematool -initSchema -dbType derby
what is prerequisite to use any RDBMS in hive? do we have to install in our machine first ?
Thank you sir
Welcome Akash.
@@UnfoldDataScience Best Wishes to you sir and thank you ones again🙂❤️💐
I have an error...i put show database....it showing...failed hiveexception java.lang.runtimeexception.unable to instantiate org apache.hadoop.hive.ql.metadata.sessionhiveMetaStoreClient....wht should I do..... please let me know.....sir
Thanks, please help me to understand, hive can be connected with a remote Hadoop? I mean a Hadoop which is not in the same hive machine?
sir i do installation of hadoop by using ur lecture it succesfully running but now iam trying to install hbase but its is giving same errors which we faced in this video so,pls make a video on hbase installation also on standalone mode
Hi Rohit,probably some version or library issue might be coming for you, what is the exact error for you?
i am getting error
Exception in thread "main" java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
Also, dont you need to run hiveserver or metastore as service.
Sir kindly reply,
I am getting 2 errors
When I am opening the sudo nano $hive_home/bin/hive-config.sh
It's a empty file but your file I saw is not empty in the video
And
Second one is when I am creating TMP directory
It's giving error unable to load native Hadoop library for your platform and connection refused
@Safa Khan Check if your hive_home is set correctly, for first question.
@@divyanshim1how can we check
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
very common error - please google
Hello man how are you, I installed both hadoop 3.2.2 and hive 3.1.2, at the end of hive installation ($HIVE_HOME/bin/schematool -initSchema -dbType derby), I will faced with this error (HiveSchemaTool:Parsing failed.), how can i solve it??? thanks for your cooperation.
Same did you found to tackle with
hi, i followed every single step but when i tried to launch it threw an error : log4j:ERROR Could not instantiate class [org.apache.hadoop.hive.shims.HiveEventCounter].
Can you pls help.. Thanks
Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path
rm: cannot remove '/lib/guava-19.0.jar': No such file or directory
how to solve it?
While running hive query "Exception in thread "main" java.lang.RuntimeException: java.net.ConnectException:" I got this error,Any solution for this?
Try restarting.
@@UnfoldDataScience OK I will try
@@shubhangidargude getting same error . how did it resolve ?
Hey Aman, I'm getting this error, can you plz help.
hdoop@dipak:~/apache-hive-3.1.2-bin/bin$ schematool -initSchema -dbType derby
Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path
Hey even i got the same error did you found the solution
It works.
Thanks a lot!
hadoop path should also given or simply hive path is enough?
Great video.
Thanks a lot.
getting an error schematool: command not found
Hey have you found out the solution for the error?
Just too good!
Thanks a lot.
sir please make video on hbase installation
How do I solve this issue?
Exception in thread "main" java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader
YOu need to find the ,missing class from web and put it.
@MSCBARATHS any luck on this?
@@UnfoldDataScience have u covered this in some video because we are unable to resolve that. Suggestions try to cover the errors also that a user might face then it would be helpful as it might worked for some and not for all.
While making dir user/hive/warehouse ...it is showing mkdir: Call from ........ to localhost:9000 failed on connection
Highly unlikely case.
I'm also getting same, how did you Sol it?
Follwed the same procedure but while starting hive I am getting the following error: Hive Session ID = 354ea424-88e5-46b6-bc75-7381026b1e8f
Exception in thread "main" java.lang.ClassCastException: class jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.hadoop.hive.ql.session.SessionState.(SessionState.java:413)
at org.apache.hadoop.hive.ql.session.SessionState.(SessionState.java:389)
at org.apache.hadoop.hive.cli.CliSessionState.(CliSessionState.java:60)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:683)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Kindly help me to resolve this issue
Can you done some videos on streaming technologies - Kafka, Storm etc
Sure :)
Sir I can not access the hdoop folder which is located in home.
check permissions
sir , if i don't have guava-27.0-jre.jar in hdfs lib what i have to do ?
Hi Uttam, downlaod from web and put in lib folder.
Hello Aman, thanks for this installation video! hey can you please tell me that I am not able to delete guava-19.0.jar file as it is write protected file. How could I do?
Change permission and try.
Thanks man...✌
Welcome Sandesh.
how much ram and memory have you allocated for the virtual box , please tell , its urgent
when i am running hive queries, my map reducer is getting out of memory
schematool: command not found ¸??
while i try to eun "schematool" command i am getting error like exception in thread main like that help me
Please restart derby.
rm $HIVE_HOME/lib/guava-19.0.jar
run this command inside the apache-hive-3.1.2-bin/bin folder
and then run the schema tool
the creator of this video does not know how to solve this and is giving you random advice. He did not even tell you how to restart derby. Pathetic
Missing Hive execution Jar
why in my case after installation of hadoop, hdfs command is not working.
Start hadoop by "start-all.sh". This file will be in sbin folder of hadoop directory.
I also faced this issue but then realised in spelling is different as i.e. hdoop in video and I used hadoop
My issue was related to Hadoop version, latter I realised I installed Hadoop v1 and was trying hdfs commands.
How to stop it
While running any query in Hive, getting this error: "FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient", Any solution for this?
delete metadata_db folder and again initialize derby
@@harishb7085 Thank you it worked :-)
Thanks Harish,
@@harishb7085 Where is metadata_db folder....I got same error but I am unable to fix it..
@@harishb7085 thanks
my config.sh file is been created new and due to this it was empty
Did you get the solution for that bro
My config.sh file was totally new i.e nothing was written in it.
same here bro u find any solution about it?
@@jaiminrana2754 try opening the config.sh file manually.. it would be there..
I had the same issue and i resolved it by going to the directory and opening the file manually
@@rijothomas8577 how to open it manually bro
bhai hbase v install karwa do
-bash: export: `HADOOP_OPTS-Djava.library.path=/home/hdoop/hadoop-3.2.1/lib/nativ': not a valid identifier
-bash: export: `/home/hdoop/apache-hive-3.1.2-bin': not a valid identifier
Can anyone help?
Thanks in advance.
every gets the error not a valid identifier. It doesn't effect.
there is one extra space before /home remove that
replace it with:
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
Should work :)
Here are the errors that I received: (in hdoop user)
1) Command - rm $HIVE_HOME/lib/guava-19.0.jar
error - rm: cannot remove '/lib/guava-19.0.jar': No such file or directory
2) Command - cp $HADOOP_HOME/share/hadoop/hdfs/lib/guava-27.0-jre.jar $HIVE_HOME/lib/
error - cp: cannot create regular file '/lib/guava-27.0-jre.jar': Permission denied
3) Command - schematool -initSchema -dbType derby
error - schematool: command not found
And when I run hive it says:
hive
Command 'hive' not found
Kindly help
These are common errors. please google , if unable to fix, mail me.
ashutosh@ashutosh:~/apache-hive-3.1.2-bin/bin$ schematool -initSchema -dbType derby
Cannot find hadoop installation: $HADOOP_HOME or $HADOOP_PREFIX must be set or hadoop must be in the path
Sir please help asap.
It is showing this error after schematool command
thank you sir
I am getting a Java error while running hive shell
check java installation
@@UnfoldDataScience my java version is 11 that s y it showing error... Thanks
@@varun6322 does hive require java8 only? because I too have java11 in ubuntu