sorry to be offtopic but does anyone know a trick to log back into an Instagram account? I was stupid forgot my account password. I would love any tricks you can give me.
Good Presentation, however I do have question. Since the file is split to other nodes, doesn't replication also take places so that if a node does go down, then you can retrieve. Node 3 went down, but I would have figured I could get it from another node. Does master save a copy of all files as well? I didn't see how that works on the video.
I'm just now learning about the methods used in distributed file systems. I'm an application developer and it's a bit difficult to wrap my head around the lower level storage systems like HDFS. This video explained replication in an easy to understand way. Now only if I could have one other BIG question answered. What kind of file system would we use if we want an application to use a virtualized file system stored over many nodes? For example, we want each node to add additional storage capacity. To the application layer, this would look like a single big storage drive but to the lower level facilities this would use network coordination to serve the files to the application.
nice illustrative way of teaching HDFS. Would have been wonderful if some more information was given about fallback mechanism of Name node or coordinator
Question: when we have to read from let's say the red file. Would all three nodes be processing simultaneously different data (chunks) and give us an combined output or would only one node process the complete the data processing alone?
if a data node malfunctions, then name node instructs remaining data nodes to create replicas of files that malfunctioned data node held. why create another replica when there are two others already? I mean they are there for the backup, right? Do correct if I am missing something. Thanks.
Hi, While setting up pseudo or full cluster, do we need to format data node with HDFS file system or we only have to format namenode. I am asking this because, I have read this on many blogs that, HDFS stores the data in sequential order on the hard disk and it is an abstract layer which stores data on big blocks rather than default block size storage provided host file system. If we are not formatting datanode than the powerful feature of HDFS will come to toss.
Hello, nice job, I have a question, as a file system how hadoop can manage a database file(like mssql or mysql) file? how can it devide an .MDF file to other separeted files to store them on different machines? Because such files, have meta data and overhead and they are not like a basic txt file. Does hadoop have special system to treat each file type differently? Regards.
why s3 service streaming with embedded link is slow (get stock every 2-3 second) when embedded on wp site? after understanding what you saying how can i improve the speed? i read something about the hdfs but i don't understand how it's related to s3 if at all. thanks in advance
Great explanation! I assume that another benefit of the HDFS is that reading large files will be quicker, because you would be able to effectively "BitTorrent" from your cluster. Does HDFS do that?
I don't understand. If each set of blocks is "one" file (red, yellow, blue) why he says that blocks are replicated? He should say "distributed", not replicated! Replicating involve data resiliency, erasure code and so on. Is it? Then he put the case that one of the cluster's node crash. So in this case replications come on play. And, of course, he should not call the set of blocks "one" file. But it should say there's one file composed of "n" chunks that are replicated among nodes of cluster.
The way I understood it, = . I suppose (if my understanding is correct of course) the confusion then comes from the fact that at the beginning of the video he said that a = . He really should have started with just one row of lego bricks for each file, and just explained that a data node sends a copy of the block it just wrote to one its data peers as directed by the control node.
Why do Americans call Lego bricks Legos? Lego is a company name and small building bricks is what they make, they don't make Legos! You can't implicitly type Lego bricks as Legos it sounds silly.
cafecapes Every nation has it's own way of speaking. There's a very elaborate section on the Stack Exchange website that goes into great depth on the difference in languages, pronunciation, and word usage. If this is a serious inquiry, I'd suggest you go there. It's a very informative place to learn anything you want - and if the question hasn't already been asked, you can ask it yourself. One example is that there is a topic on how Americans pronounce the word solder as "sodder" while other countries pronounce it as "sold-er" and where this distinction came from. You might be surprised to know that the language variations have a lot to do with heritage dating way way back. Every language and dialect, regardless of what it is has become bastardized - and it's just a part of how languages evolve. For what it's worth, I used to pronounce it as "sold-er" until I got tired of being corrected - and I have no indication of where I learned this from. I now colloquially pronounce it as "sodder" just because of tradition and geographical linguistics. Also, pronunciations and accent vary widely in the United States from coast to coast also. Apologize for the long comment.
Nice explanation man, if I could, I would buy you an air conditioner.
You deserve it.
may the force be with you
sorry to be offtopic but does anyone know a trick to log back into an Instagram account?
I was stupid forgot my account password. I would love any tricks you can give me.
@Billy Dariel it worked and I actually got access to my account again. Im so happy!
Thank you so much you really help me out!
@Miguel Chris Happy to help :D
@@billydariel9140 nice ad
This video was uploaded 5 years ago and it still one of the best explanations I've ever seen.
the first explanation at all that could tell me what Hadoop has to do with MapReduce
This is a best explanation I have seen yet! It's a lot more engaging and informative than the traditional PowerPoint! Thank You!!
Nice job Jesse. Very informative and creative. Thank you.
TeraTom
11:06 R.I.P. Data Node 3
He was a good node, admired by his node friends.
I'm still crying, can't get over it, such a good node dude rippp
He is happy in the Node Heaven and is saving all the replicated data it ever wanted to save.
Excellent video. I actually understand Hadoop somewhat after watching this video.
You really have a " hands-on" approach of teaching this. : )
Good Presentation, however I do have question. Since the file is split to other nodes, doesn't replication also take places so that if a node does go down, then you can retrieve. Node 3 went down, but I would have figured I could get it from another node. Does master save a copy of all files as well? I didn't see how that works on the video.
I love learning with legos, even watching it at 1.5x, I was able to follow along easily. Well Done.
I'm just now learning about the methods used in distributed file systems. I'm an application developer and it's a bit difficult to wrap my head around the lower level storage systems like HDFS.
This video explained replication in an easy to understand way. Now only if I could have one other BIG question answered.
What kind of file system would we use if we want an application to use a virtualized file system stored over many nodes? For example, we want each node to add additional storage capacity.
To the application layer, this would look like a single big storage drive but to the lower level facilities this would use network coordination to serve the files to the application.
ĺm ĺ
Wow man, you explain it in a creative way.
Excellent teaching. Conceptually things are very clear now. Thank you.
The explanation is so clear I understood everything
This video just made my day! Thank you New Circle Training! And thank you Doug Cutting for sharing this video on Twitter.
Excellent, thank you. A serious question: What would happen if two of the four nodes crash?
Really nice and innovative way to teach hdfs concept. loved it and understood it very clearly. Thank you.
nice illustrative way of teaching HDFS. Would have been wonderful if some more information was given about fallback mechanism of Name node or coordinator
Amazing explanation !! one of the best videos I've seen about HDFS
Very nice video! Only one question: If node 3 is down, how can the data from node 3 be moved to the other nodes?
@@brianbitchballs3902 thanks for the great explanation BrianBitchBalls
Question: when we have to read from let's say the red file. Would all three nodes be processing simultaneously different data (chunks) and give us an combined output or would only one node process the complete the data processing alone?
Superb, I had it running at 1.5x and it was still easy to follow! Thanks :)
Can you explain every CS concept with Lego? - that would be amazing
thanks dude lol
Jesse! Great explanation as always.
Did we find who broke node 3 yet?
What if hulk smashes the naming node?
If a node goes down, why does it need 3 replications to pull the data, could it not just read from the existing two?
if a data node malfunctions, then name node instructs remaining data nodes to create replicas of files that malfunctioned data node held. why create another replica when there are two others already? I mean they are there for the backup, right? Do correct if I am missing something. Thanks.
Hi,
While setting up pseudo or full cluster, do we need to format data node with HDFS file system or we only have to format namenode.
I am asking this because, I have read this on many blogs that, HDFS stores the data in sequential order on the hard disk and it is an abstract layer which stores data on big blocks rather than default block size storage provided host file system.
If we are not formatting datanode than the powerful feature of HDFS will come to toss.
Hello, nice job,
I have a question, as a file system how hadoop can manage a database file(like mssql or mysql) file? how can it devide an .MDF file to other separeted files to store them on different machines? Because such files, have meta data and overhead and they are not like a basic txt file. Does hadoop have special system to treat each file type differently? Regards.
Awesome, Thank you for the explanation, Could you please make videos of Spark with Yarn and how the communication is handled?
For the red file, are EACH of the replicas still 64 mb in size?
very nice teaching methodology jesse, thanks for sharing
Thank you for such a good explanation!
Nice work. Really help me to understand how HDFS works.
why s3 service streaming with embedded link is slow (get stock every 2-3 second) when embedded on wp site? after understanding what you saying how can i improve the speed? i read something about the hdfs but i don't understand how it's related to s3 if at all. thanks in advance
Great introduction to HDFS!
Incredible illustration! Would love to see more on this topic in this form :-)
what does happen when you add a new and empty node?
Thanks for your efforts! Smart display!
What will happen once Data Node 3 is alive again? will the Name Node asks other nodes to copy some data to Data Node 3?
Great explanation! Voted!
The fact that hbase write on a node as you say is the reason why it corrupts the HDFS filesystem so easily?
Nice clear explanation
This was great, thank you! Should have more views.
Nice job, it is helping getting familiar with HDFS. I'm new to Data Engineering and so on.. Could you please explain to me what is a cluster.
Thanks;
gorgeous presentation, thanks
Very good work for begginers - THANKS A LOT!
Very innovative presentation thanks a lot!
Great explanation! I assume that another benefit of the HDFS is that reading large files will be quicker, because you would be able to effectively "BitTorrent" from your cluster. Does HDFS do that?
Great Job Jesse! Nicely done! :-)
Excellent video, thanks!
did you get a AC by now ?
This was great, thanks.
very informative, thank you
thanks for the explanation
feel like buying Legos already!
Great explanation!
Amazing !!! great Job
Interesting, it's very similar to how Elasticsearch works
yoou haveabsolutely no need to be nervous! Doing a great job here
Nice Job. Brazil thanks you!
Perfect! Thanks so much
I don't understand. If each set of blocks is "one" file (red, yellow, blue) why he says that blocks are replicated? He should say "distributed", not replicated! Replicating involve data resiliency, erasure code and so on. Is it?
Then he put the case that one of the cluster's node crash. So in this case replications come on play. And, of course, he should not call the set of blocks "one" file. But it should say there's one file composed of "n" chunks that are replicated among nodes of cluster.
The way I understood it, = .
I suppose (if my understanding is correct of course) the confusion then comes from the fact that at the beginning of the video he said that a = .
He really should have started with just one row of lego bricks for each file, and just explained that a data node sends a copy of the block it just wrote to one its data peers as directed by the control node.
Brilliant!
Thank You
Thanks a lot
Very creative, indeed.
tnx
thanx!
Cool
Now I like Legos :)
狡兔三窟说的就是这个意思
play in 1.25x
英语不太好，但是觉得很棒
its lego, not legos... but nice vid.
It started off pretty well but then it got very confusing. He didn't seem so sure of what he was saying. It didn't work for me. Thank you, anyway.
SUPER informative, but also i hear "Hadoop" and just think "Hadooken" just me? yeah okay hahaha
Why is he so sweaty?
Yeah, it was the lighting. We tried moving the lights around, but I didn't have any makeup on which mitigates the lights.
You should not answer this moron. u doing a good job for free
Looks like you are a moron!
He replied so kindly and genuinely, you are calling him moron.
Pretty sure he was referring to the initial commenter, not Jesse. Rude remark nevertheless.
Nican
It's been quite a challenge to hear this video. Headphones are in but it doesn't seem to help.
audio volume is fine to me, must be an issue on your machine
Freddy yes, that’s what it was. Guess my Mac needed a reboot. Worked fine afterwards.
Why do Americans call Lego bricks Legos? Lego is a company name and small building bricks is what they make, they don't make Legos! You can't implicitly type Lego bricks as Legos it sounds silly.
we do the same in Europe. Where are you from?
I've been thinking deeply about this and decided I'm the worst person to be dictating English.
Briton mate.
cafecapes Every nation has it's own way of speaking. There's a very elaborate section on the Stack Exchange website that goes into great depth on the difference in languages, pronunciation, and word usage. If this is a serious inquiry, I'd suggest you go there. It's a very informative place to learn anything you want - and if the question hasn't already been asked, you can ask it yourself.
One example is that there is a topic on how Americans pronounce the word solder as "sodder" while other countries pronounce it as "sold-er" and where this distinction came from. You might be surprised to know that the language variations have a lot to do with heritage dating way way back. Every language and dialect, regardless of what it is has become bastardized - and it's just a part of how languages evolve.
For what it's worth, I used to pronounce it as "sold-er" until I got tired of being corrected - and I have no indication of where I learned this from. I now colloquially pronounce it as "sodder" just because of tradition and geographical linguistics. Also, pronunciations and accent vary widely in the United States from coast to coast also.
Apologize for the long comment.
Thanks 😊