Видео

I am a senior/lead engineer in a niche field/networking swe. I want to transition to AI networking area. Not sure how will I do it. My decade of experience has nothing to do with AI.

for the second approach why dont we use the same consistent hashing algorithm in the load balancer so that when user hits the lb, he/she will be connected to the correct server (granted in order to know which one is the correct server/ the most up to date server map we prob need to rely on a consensus tool such as zookeeper/etcd anyways)

How is mapping from id to title stored ? When a query comes for lazy, we should return : 'lazy dog', 'lazy days are best days' not IDs (12,53) ID: 12 lazy dog Id: 53 lazy days are best days. Index lazy: [12,53] dog: [12,53] days: [53] are: [53] best: [53] Is the mapping from ID to title stored in every shard or some specific shard. If it is stored in every shard, it would cause a lot of duplication of data. If it is stored in some central shard, when we iterate over Ids(12,53) to retrieve title we have to make a network call to retrieve title. That can introduce latency in order of milliseconds per request. If Id list goes to 1000 then we are already close to 1s mark. For complex queries, performance will degrade quickly. Also can you give us verbs(eg _search, _update, _doc, _pit) that are important. Eg: i was unable to search _doc related info online.

Super useful!! Thanks!

Insanely good explanations !

Nice video! Wouldn't the Redis cache be an issue even to store the swipes? As we will still have 10M * 100 * 32Bytes *365 = 12TB of space?

Great overview, thanks! One "gotcha" that I've run into before managing a system like this in production is that DynamoDB has a maximum item size of 400KB. This will implicitly limit the file sizes you can support if you store chunks like that as a list on the DynamoDB item. Let's say each chunk metadata is 50B, you could support ~8000 chunks per file before you run into that 400KB limit, so with 5MB per chunk, you can actually only support 40GB files. Of course, if you used something like 10MB chunks, you could double this limit to 80GB and fulfill the requirements. But I think this would actually push me towards a SQL db like Postgres instead, or you have to have some more fancy logic to store/associate the chunk metadata.

Great video. Making use of TTLs in DynamoDB would be a way to reduce some of the work the Cleanup Svc has to do.

I liked the previous method of going through and identifying the core entities and subsequent APIs before diving into high level design. Is there a reason you chose to omit that in this video?

Why have you chosen DynamoDB for storing ride, driver entities? I was expecting this to be relational DB, because there are a lot of updates to Ride entity and the relational itself is pretty straightforward

♥

Hey guys, first things first, love this channel! The content is amazingly explained and I can't recall the amount of new concepts I have learned, really got me exploring things on my own too! As a recommendation for a future video, would you guys consider a system design on google maps? Not sure if it's asked in interviews but it would be nice to see the inner workings. Thanks again for all the content!

GEM

very nice way to explain! I would just avoid the clicking tongue sounds ;)

Whoa, since I'm applying to Engineering Manager roles I'm also preparing for System Design interviews (never had it before in my career). I've read both ByteByteGo books, watched all of the youtube content from them, also read "Designing Data Intentive Applications" bible. Don't get me wrong, these are great resources. And I had 2 System Design interviews that I pretty much failed. Now I feel that I should start from a simple and clean videos like yours that don't start from the definition of Consistent Hashing but first learn how you should play with basic blocks. Great content, thx!

Bro just made the entire system design course industry free. Golden content! Free content FTW !

Hi Evan. Humble request to please make a video on eCommerce system like Amazon and a service like doordash

Hi Evan. Humble request to please make a video on eCommerce system like Amazon and a service like doordash

Humble request to please make a video on eCommerce system like Amazon and a service like doordash

Hi Evan. Humble request to please make a video on eCommerce system like Amazon and a service like doordash

Humble request to please make a video on eCommerce system like Amazon and a service like doordash

Thanks for making this video. Your delivery is calm and confident, making the content very easy to digest.

FYI - DOCX files (and XLSX, PPTX, etc.) are zip files. You can unzip them to see what they hold.

Thanks, when I first strated preparing for the system desing interviews, I had no idea if I could even try to get started. I love how you made things systematic in such a simple but effective framework. I had a short time to prepare my system design interviews, got one passed, one failed. I don't have any sd interviews lined up, but i don't want to lose what I studied, so i'll keep grinding from here. Really love your contents and actually am enjoying learning these things! Thanks for the great contents.

Hi! Thanks for the great video with detailed explanation. One question regarding dispatching a user with "videoId" to the realtime-comment server(the server that propagates comments to end users). Can't we just use consistent hashing based on the videoId, when connecting the end-user to the server? In this way, the scaling would be easier and there is one less service to worry about(the zookeeper aka the dispatcher)

Great video but a quick comment: when you classify the HTTP verb as needing to be one thing or another, you're inadvertently leading people to believe that it actually matters. It matters with GET vs POST but PUT or PATCH? Seriously? Just make it a POST and lets stop confusing the children as to what those verbs really do.

Top notch content as always, can't thank you enough for this channel! Out of topic, but is there any chance you guys might do a quick post or video about designing an i18n system, please? I saw two separate posts on Leetcode Discuss that it got asked at Meta product architecture interview within the last 6 months. Can find my way around in most problems after intensive prep on Hello Interview but totally thrown off at i18n design :( Can't find much content anywhere either , must be a really niche question... Can't even think of core entities, APIs etc, just blanking out and panicking... Feels like it's just getting the user's lang and throwing some CDNs around but I must be missing something cause that's nowhere enough content to be talking about for 35-40 minutes...

nice! one suggestion: move your cursor out of the way when you are writing. :)

You still get logs from your CDN services, so you don't lose the analytics of which shortened urls are being requested. The main downside to using a CDN is the cost, although properly implemented, CDNs save you money on egress and content delivery. Obviously there is also the cost of complexity, but for high scale, globally distributed services, CDNs are a must-have. The 301 vs 302 redirect only applies to repeated lookups of a shortened url by a single client host device. All end-users (host devices) will still need to hit your service to look up the long url the first time they want to use it. So when a shortened url is shared, either way you'll still get a count of users (really, devices) that hit that url.

This is awesome content! Also found the new 10-min duration format to be very helpful. Maybe it's because I have a short attention span but these shorter videos allowed me to stay engaged the whole time whereas when watching longer ones I lost focus after 20-30 minutes.

Great guidance on system design. One nit to pick... I believe that "ensure uniqueness of short codes" is a functional requirement. Non-functional requirements are not specific to the application (qualities like security, performance, scalability, reliability, resiliency, legal compliance, etc.).

By far one of the best system designs and very clear explaining, cheers

Awesome !

how did i come across this video? this is by far the best system design tutorial i've seen

So Partition Tolerance refers only to the network connection between servers? It has nothing to do with data partitioning like sharding in mongoDB, right ?

It makes zero sense to place a ELB in front of API gateway MongoDB cannot handle twitter scale I/O Ops Systems design is luck sometimes

In redis pub sub, there is a topic for each receiving user. We publish to that topic, and the WS connection associated with that userid sends the message to the receiving user. We only have 1 WS connection per user. Why are we saying that all chat servers subscribed to a topic get message?

Can we cdc match data also into Elastic Search? and filter matched ones inside ES. If we do in the profile service, it may have pagination problem, as we filter out items in the app memory.

We can use web sockets to get the location of the driver after every few seconds, making an API call (polling) is unoptimised

Hi, can someone really explain me why videoId had a a colon : before it ? Because it’s an object being uploaded to a bucket?

More such videos please. Thank you so much

This was very interesting. But I would rather have a long form, full system design video for this

Thank you very much, the best channnel for prep to sys design on RUclips!! ❤

Looking forward to the next one!

Is the chunks s3 url different than the initial pre-signed url ?

So it is super similar to search systems, where more and more are using embedding retrieval and ML reranking now? Just the input to recommendations are user information while the input to search are user queries?

Is a cache a great solution for storing the stacks? That cache will then be nearly as large if not larger than the profiles DB, for every profile in that table there is a profile with a stack in the cache db. Would another stack db be a better potential solution?

Maybe a community similar to leetcode where there is a way for users to post their solutions to common design questions & other users can interact & provide feedback/learn from each other.

Amazing

Excellent explanation. Best part of your system design interview videos are that you teach step by step and along with this you also pin point the common mistakes. That's really helpful.