1 more thing to add; sorted set, internally is implemented with skip lists and hashtables in Redis. Thats why adding value to sorted set is O(logN) on average and searching is O(logN + M). If you bring this up, congratulations you passed your interview
Very informative and helpful video as always. I would appreciate if the visualization of the internals could be a bit more detailed and in sync with what you describe, rather than just a couple of boxes. Regardless, this is a refreshing take to step away from the usual system design videos to dive into technologies that are useful in both interviews and real life. Hope to see more of this in the future.
Hi i just want to say your work is truly fantastic and you are an excellent educator! I started watching your videos to prepare for system design interviews and your explanations and examples are top notch. I'm motivated to continue watching not just to prep but also to expand my knowledge and improve my craft. Eagerly awaiting the great work you'll do next!
Wait .... I thought I clicked on a Dua Lipa music video. 😀. Just kidding. Stefan I love this - clear, concise, articulate and I learned something. Keep 'em coming.
Great content. This is a great resource for learning before i apply Redis to my own projects. One thing I was wondering was if your content would be able to be added into Spotify so I could listen along on the go?
I really enjoy your content on the website, the videos as well as posts on reddit. Can you make another video on behavior/leadership principles interview next? I would like to learn how to tell a convincing story about my leadership potential.
The problem of hot key is clear but solution of hashtag is very difficult to understand. do we have to handle that all in the client? like figure out what are the hot keys?? and what will adding hashtag do really?
It's a great deep dive. I'd like to give a suggestion, Can you guys add chapters to the video, so that, if I want to rewatch again about certain portion of the video then that would be easy thing to do.
You should have chapters on this one! If you look at the description "Key Moments" will take you to the relevant sections. LMK if we're not doing something else we should be doing!
@@hello_interview I see that, you missed to add "00:00" time stamp in your "Key moments", that's causing YT to not auto chaptering your video. I have checked couple of other videos in the channel, videos which are having 00:00 in their key moments, respective chapters are added.
Loved it! Want to see more on how communication between service on global scale like using kafka with async or consistent connection around this please, if you get the idea, Sorry i am bad at explaining
Hi, you talked about append a random number to solve the hotkey issue. But when you want to retrieve the data, you won't know the key because you append a random number.
At 22:55 how does hashing tweet IDs help if we're looking for the most liked tweets overall? Or were you referring to getting top liked tweets for specific keywords when splitting that key across multiple nodes?
In practice? Substantially faster and cheaper if you can accept some of the (many) compromises. In an interview setting it can be helpful to have a few multifaceted tools vs having to know all about Kafka (video on that soon). If your business was entirely built around append-only logs I probably wouldn't take Redis as an acceptable substitute, though it works in a pinch.
Can i book sys design coaching sessions on hello interview or is it only for mock interviews?I'm a senior engineer and I've just started preparing do you recommend to jump straight into mocks?
@@hello_interview sorry, what's the approach? I think somewhere else in the video talks about it, but I also somehow did not understand that. Is it roughly the following? Generate X random suffixes. Maintain a map on the (Redis') client side of . Whenever we want to do some sort of "write", we get a random suffix from our list and append it to our key in the Redis command. Whenever we want to do some sort of "read", we get _all_ random suffixes (X of them) from our set, make X reads from Redis, and aggregrate on the client side. ---- Or maybe it's sometimes the opposite way. For example, in a cache, I'm guessing we'd want to write our value to each and every one of the key+randomsuffix. Then for a read, we can (randomly) pick one of the suffixes. ---- Maybe the missing link (for me and others similarly confused) is how/where the random suffixes are managed. There's also the question of how we (automatically?) identify a hotkey, or develop an algorithm that generates (or increases) the number of suffixes depending on some sort of access rate.
You can use Kafka as pub/sub! Differences in durability (Redis in memory, Kafka persists to disk), which then has consequences on other things like ease of disconnection, etc. But they can both be used.
Redis pub/sub is really dumb, at-most-once delivery of messages. You can think of it as a hashmap from a topic (string) to a socket connection to a subscribing server with all the associated problems (messages might get lost?) and benefits (incredibly lightweight, low-latency etc.) Kafka is more full-featured. Kafka has message persistence, you can make guarantees on delivery, etc.
@@hello_interview Geohashing algorithm is too complex. I just cant take in all this diagrams. I also dont understand how proximity systems can search something inside the circle using square blocks of geohashes
@@maxvettel7337 Gotcha! I'll earmark this for follow-up. The nice thing is you don't necessarily need to know about the geohashing internals to use something like Redis' proximity search in practice!
14:40 the nice thing about this stream is if we have got something we need to make sure all of the items of the stream are processed, the redis give us a bunch of premitives to work with. What does this mean?
I've been waiting for videos on deep dives. More deep dives please!! These rock
🫡
1 more thing to add;
sorted set, internally is implemented with skip lists and hashtables in Redis. Thats why adding value to sorted set is O(logN) on average and searching is O(logN + M). If you bring this up, congratulations you passed your interview
Can't thank you enough. You made system design so much easier to understand.
Very informative and helpful video as always. I would appreciate if the visualization of the internals could be a bit more detailed and in sync with what you describe, rather than just a couple of boxes. Regardless, this is a refreshing take to step away from the usual system design videos to dive into technologies that are useful in both interviews and real life. Hope to see more of this in the future.
Good feedback, thank you! Will try to make the visuals more engaging in the future.
Hi i just want to say your work is truly fantastic and you are an excellent educator! I started watching your videos to prepare for system design interviews and your explanations and examples are top notch.
I'm motivated to continue watching not just to prep but also to expand my knowledge and improve my craft. Eagerly awaiting the great work you'll do next!
Couldn't focus, too handsome
The GOAT has arrived and has something important to say!
@@hello_interview Gigachad onlyfans allocator (of) toes, that's very kind of you
Hey Jordan looks like you are getting your life back.
@@coledenesik unclear at best
Loving the consistency of uploads! :)
would love to see a deep dive on Kafka as well
Kafka is in the works! Written resource will be posted to the website this week or early next!
Echoing the existing sentiment in that this content is incredibly informative and well put together. Really appreciate your work
Glad it's been helpful! More soon.
Absolutely loved it !!!
Can we have one deep dive on SQL vs NoSQL, a topic always asked in system design interview.
We actually have an opinion on that very topic, check this out: www.hellointerview.com/learn/system-design/in-a-hurry/key-technologies#core-database
You are best channel for system design interview. Such a indepth technical understanding, I really appreciate you making these videos.
Wait .... I thought I clicked on a Dua Lipa music video. 😀. Just kidding. Stefan I love this - clear, concise, articulate and I learned something. Keep 'em coming.
Lol. The intro is a banger, no :)?
Great content. This is a great resource for learning before i apply Redis to my own projects. One thing I was wondering was if your content would be able to be added into Spotify so I could listen along on the go?
Great content. Really liking it. Can you please do a deep dive on TiDB, Rocks DB and Cassandra too
Cassandra is soon!
Really cool deep dive, updated with time as well. Thanks a lot for making this. Looking forward to more!
Just love the way you add practical aspects! Thankyou for the thoughtful, concise and articulate deep dive! Was definitely a high ROI on time :)
So glad it was useful!
Great watch! A similar deep dive video for Kafka would be amazing 🙏
Kafka is in the works! Written resource will be posted to the website this week or early next!
@@hello_interview amazing!
I really enjoy your content on the website, the videos as well as posts on reddit. Can you make another video on behavior/leadership principles interview next? I would like to learn how to tell a convincing story about my leadership potential.
Enjoyed both your videos on Kafka and Redis. Can you please make a video on CQRS?
this video is awesome! exactly what I expected!
So glad to hear it!
Excellent video. Please also make video on mock interviews
Mock interviews are what we do! What is it you want to see?
The problem of hot key is clear but solution of hashtag is very difficult to understand. do we have to handle that all in the client? like figure out what are the hot keys?? and what will adding hashtag do really?
It's a great deep dive. I'd like to give a suggestion, Can you guys add chapters to the video, so that, if I want to rewatch again about certain portion of the video then that would be easy thing to do.
You should have chapters on this one! If you look at the description "Key Moments" will take you to the relevant sections. LMK if we're not doing something else we should be doing!
@@hello_interview I see that, you missed to add "00:00" time stamp in your "Key moments", that's causing YT to not auto chaptering your video. I have checked couple of other videos in the channel, videos which are having 00:00 in their key moments, respective chapters are added.
Re-upload, sorry! Stefan is a RUclips noob.
Facts.
🤣
Can you please do deep dives on Kafka and Spark too ? I love your videos
Written deep dive on Kafka coming in a couple days!!
Loved it! Want to see more on how communication between service on global scale like using kafka with async or consistent connection around this please, if you get the idea, Sorry i am bad at explaining
Hi, you talked about append a random number to solve the hotkey issue. But when you want to retrieve the data, you won't know the key because you append a random number.
At 22:55 how does hashing tweet IDs help if we're looking for the most liked tweets overall? Or were you referring to getting top liked tweets for specific keywords when splitting that key across multiple nodes?
The latter!
One can imagine the implementation of the geo-spacial index using 2d indexing data structures - quad tree, R-tree and others.
Why would I want to use Redis stream over something like Kinesis or Kafka streams
In practice? Substantially faster and cheaper if you can accept some of the (many) compromises. In an interview setting it can be helpful to have a few multifaceted tools vs having to know all about Kafka (video on that soon).
If your business was entirely built around append-only logs I probably wouldn't take Redis as an acceptable substitute, though it works in a pinch.
Can i book sys design coaching sessions on hello interview or is it only for mock interviews?I'm a senior engineer and I've just started preparing do you recommend to jump straight into mocks?
Send us an email at support@hellointerview.com and we might be able to make something work. Have some products planned here for people in your shoes.
@@hello_interview done
10:00 didn't quite catch how adding random suffix would solve hot node problem
Splits the hot shard into multiple, so you can distribute that load across nodes.
@@hello_interview sorry, what's the approach? I think somewhere else in the video talks about it, but I also somehow did not understand that.
Is it roughly the following?
Generate X random suffixes. Maintain a map on the (Redis') client side of .
Whenever we want to do some sort of "write", we get a random suffix from our list and append it to our key in the Redis command.
Whenever we want to do some sort of "read", we get _all_ random suffixes (X of them) from our set, make X reads from Redis, and aggregrate on the client side.
----
Or maybe it's sometimes the opposite way. For example, in a cache, I'm guessing we'd want to write our value to each and every one of the key+randomsuffix.
Then for a read, we can (randomly) pick one of the suffixes.
----
Maybe the missing link (for me and others similarly confused) is how/where the random suffixes are managed.
There's also the question of how we (automatically?) identify a hotkey, or develop an algorithm that generates (or increases) the number of suffixes depending on some sort of access rate.
@@hello_interview but how will you query the key then, as at read time you don't know what random suffix got added
@@hello_interview What if the new key conflicts with any keys added later?
@@Xiao-p9m How will it conflit, we append the certain value every single time so there wont be a conflict
Can we have a deep dive on Apache Flink please ?
We'll put it on the list!
What is the difference between Redis Pub/Sub and Kafka? From the other video, what I understand is Kafka is queue and event driven as well.
You can use Kafka as pub/sub! Differences in durability (Redis in memory, Kafka persists to disk), which then has consequences on other things like ease of disconnection, etc. But they can both be used.
Redis pub/sub is really dumb, at-most-once delivery of messages. You can think of it as a hashmap from a topic (string) to a socket connection to a subscribing server with all the associated problems (messages might get lost?) and benefits (incredibly lightweight, low-latency etc.)
Kafka is more full-featured. Kafka has message persistence, you can make guarantees on delivery, etc.
Thank you.
Thanks so much for the video. I would be really interested in deep dive on elastisearch/opensearch
That one's next!
Amazing Content.. Can't thank you enough!! Any Plans for DSA Mock Interviews 👀
What are you looking to see?
@@hello_interview DSA Mock Interviews with FAANG Candidates..
But how will you handle the at most one delivery of redis in the case messaging?
Check out our Whatsapp guide for one example of this: www.hellointerview.com/learn/system-design/answer-keys/whatsapp
Nice! Can you also do Deep Dive on Graph Databases(eg, neo4j) as well. please!
Will add it to the list! Typically, graph dbs are not as commonly used at scale.
Please make a video on Elastic Search
It would be nice to see a Deep Dive video about location database. Pretty hard to understand for me
Which part is most confusing? Geohashing? Quad trees? Where to use them?
@@hello_interview Geohashing algorithm is too complex. I just cant take in all this diagrams. I also dont understand how proximity systems can search something inside the circle using square blocks of geohashes
@@maxvettel7337 Gotcha! I'll earmark this for follow-up. The nice thing is you don't necessarily need to know about the geohashing internals to use something like Redis' proximity search in practice!
this video is too high level..need more simpler deep dive on redis if possible
Which part did you miss? Maybe we can point you in the right direction.
Why not talking about cache invalidation when using as a cache?
Finding it difficult to understand stream and consumer group
I have never clicked so fast!
Kafka next, please!
This week!
I see , I like.
Could Even cover Redis again?
Nah, this explanation is killer. Couldn’t do it better if I tried!
-Evan
@@hello_interviewPlease
@@hello_interview at least we can highlight the knowledge point . Have some summary on the diagram
14:40 the nice thing about this stream is if we have got something we need to make sure all of the items of the stream are processed, the redis give us a bunch of premitives to work with. What does this mean?
I agree with you . This is the first out of ten video I felt I learned nothing after watching in this channel
I missed the co founder now
do a deepdive on ethereum latest version, l2 rollups
Format reminds me of acloudguru