True! in every video of the Big Data playlist there will be a moment where you feel like you just learnt something that will blow away your senior's mind!!!1
Bro ur videos are genuinely so helpful.. they way you give actual examples are really nice.. easy to understand.. thanks to you.. keep up the good work we would watch the inbetween youtube ads 👍
what if the bucketed column is a string how will it perform the hash partition on it using Modulus division ...will it take the ASCII value of string into consideration?
excellent work and explanation but i have a question. what if the remainder is bigger than the number of buckets ? for example if we have 3 buckets and hash % 3 is 6 . in which bucket that record will be ? thanks.
If i get it correctly you are assuming what happen any hash value hash (x) % (mod a) > (mod a) and then whil bucket will it go? The simple answer is no this is not possible in current math . What you are assuming is the qoutient and this is separte than remainder . Lets take an example of hash value 95 and mod value 3 . Here qoutient would be 31 that is (95 - (31*3)) = 2. If you have 10001 also in hash value still you will get qoutient 3333 that is (10001-(3333*3)) = 2. Remainder will always be
While scanning the buckets for the records , does it scan all the buckets one by one until the record is found and will break out once the record is found ?
Thank you very much Gowtham for making Big Data concepts so easy and understandable. The contents are great and help millions of people like me who are moving toward Big Data. Keep on good work :)
I am working in internal audits ( 2 years exp..) - role is to be checking the business transactions with large data and making the observations... Companies can consider the internal audit Experience for data analyst jobs ???
When you have taken 2 buckets and if you insert values in same order 1. 9876 2.1234 3.5678 4.0014 Then, if we are searching for 0014 Now it come under Full Scan, So that is the reason, you have taken 1&4 in 1st bucket and the remaining in 2nd bucket. Hahaha....good cover drive
Ha ha it's not like that , and the records will not store in the order the way we insert in the bucket , the hash partition algorithm will decide the buckets destinations for the records , 0014 can be in 1st bucket too . Using bucket we are not completely avoiding full scannn... 🙂
![How to Decide [Bucket Count] in Hive #hive #apachehive](http://i.ytimg.com/vi/pHvyhfWK43s/mqdefault.jpg)
![How to Decide [Bucket Count] in Hive #hive #apachehive](/img/tr.png)
![Hive Partition [ Static Vs Dynamic]](http://i.ytimg.com/vi/RFS4JJFlzGc/mqdefault.jpg)
You nailed the game, I do not need expensive courses to learn when I can have a brother like you.
Thank you so much bro 🙏
True! in every video of the Big Data playlist there will be a moment where you feel like you just learnt something that will blow away your senior's mind!!!1
great explanation. I was having hard time understanding why bucketing concept is required if partitioning is there in hive. Now i got the idea
Love the way you explain. Completed whole hive series. Keep posting more.
Bro ur videos are genuinely so helpful.. they way you give actual examples are really nice.. easy to understand.. thanks to you.. keep up the good work we would watch the inbetween youtube ads 👍
Thanks for the series, gives deep understanding and clear picture on topics. Waiting for Pyspark videos.
This explaination on hive bucketing was really useful👍it has really helped me to clear technical interview
Keep updating this playlist. Very useful Thanks!
Outstanding explanation..Thanks much..-Mohamed
what if the bucketed column is a string how will it perform the hash partition on it using Modulus division ...will it take the ASCII value of string into consideration?
excellent work and explanation
but i have a question. what if the remainder is bigger than the number of buckets ?
for example if we have 3 buckets and hash % 3 is 6 .
in which bucket that record will be ?
thanks.
Thanks
So you mean hash is 3 and number of bucket is also 3
Therefore 3 mod 3 right ?
If i get it correctly you are assuming what happen any hash value hash (x) % (mod a) > (mod a) and then whil bucket will it go? The simple answer is no this is not possible in current math . What you are assuming is the qoutient and this is separte than remainder . Lets take an example of hash value 95 and mod value 3 . Here qoutient would be 31 that is (95 - (31*3)) = 2. If you have 10001 also in hash value still you will get qoutient 3333 that is (10001-(3333*3)) = 2. Remainder will always be
It is property of natural numbers that any number n divided by x will have remainder
While scanning the buckets for the records , does it scan all the buckets one by one until the record is found and will break out once the record is found ?
Thank you very much Gowtham for making Big Data concepts so easy and understandable. The contents are great and help millions of people like me who are moving toward Big Data. Keep on good work :)
Bro, your interview questions vd was very useful. Can you do the same for each topic separately pls?
It was a great session.thank you for your efforts.
Easily understandable explaination❤
Good explanation. thanks for the video
Very nicely explain ...keep posting new videos pls
what is that 's' at the end of the command SELECT avg(amt) FROM bk_test_data TABLESAMPLE(BUCKET 1 OUT OF 3 ON rand()) s;
I am working in internal audits ( 2 years exp..) - role is to be checking the business transactions with large data and making the observations... Companies can consider the internal audit Experience for data analyst jobs ???
Found a gem in you tube!💎💎
Good content as always, thank you.
How bucket works on top of partition?
Nice explanation brother.
hi can you post video on how to prepare and crack faang interviews?
for the begginer hard to find the order, to follow, would appreciate if you rename it with part-1,part-2...
Great bro thanks
Nice explanation 😁
Thank you brother
Thankyou
When you have taken 2 buckets and if you insert values in same order
1. 9876
2.1234
3.5678
4.0014
Then, if we are searching for 0014
Now it come under Full Scan,
So that is the reason, you have taken 1&4 in 1st bucket and the remaining in 2nd bucket.
Hahaha....good cover drive
Ha ha it's not like that , and the records will not store in the order the way we insert in the bucket , the hash partition algorithm will decide the buckets destinations for the records , 0014 can be in 1st bucket too . Using bucket we are not completely avoiding full scannn... 🙂
First view 😀