9:30 key insight from comparing options: the best Replication Log is one that is at least: 1) Deterministic (outcome pre-determined), 2) invariant of any environment changes.
It is worth noting that `Consistent prefix reads` problem arises only if all the writes are not ordered in time, which doesn't happen in case of a single-leader replication setup.
Enjoy your content. Only 1 concern you mention that Write-ahead log is not scalable in case db engine changes but I think u r referring to extensibility which means easy to replace in case of new implementation whereas scalable means able to handle 1mil requests similar to the way handles 1k requests for ex.
I have a question - what do you mean by a node? I am confused because we're talking about DB replication here. Is a DB follower/leader referred to as a node in this context? Then when you said at 3:31 that "All clients send writes to new leader" - DBs arent smart to actually do that right? Who makes this change?
When I say a node I mean a distinct computer. Could be a leader, follower, basically any two things that require a network call to communicate with one another. On a leader failover, we need to get a coordination service involved so that all replicas agree on who the new leader is. Clients can either ask an existing replica who the new leader is, or ask the coordination service itself.
Hey, great content! Can you share the links to all slides somewhere? The link you've posted below is pointing to only one video. Thanks for doing this. I'm learning a lot watching your videos.
Excellent content. Quick question though - I got the understanding that the replication log conveys the changes to the followers, so once the follower reads the changes, follower has no clue of that log anymore. So if this the case how can replication log helps with follower failure , does it store a copy of log too? Also WAL is the way of storing logs(by appending changes) and Copy over SQL statements and logical logs are ways of storing those changes in that WAL. Is my understanding correct?
Really good content!! Thanks for making them. One question about "Consistent Prefix Read" you mentioned "causal relationship between writes lost because preceding write takes longer to replicate (due to being on different partitions)". If client only writes to the leader node, how would writes be on different partitions?
Great Content. Thanks for making them. My Question: Regarding Causal relationship writes on different partitions. Does that mean that one database can be run on multiple partition on the same machine? And can those partitions be different physical disks or logical partitions?
@@jordanhasnolife5163 thanks a lot! Finally I found high quality system design video that teaches concepts so well. BTW someone recommended you on Reddit. I love your jokes, keep sending more!
Well at this point I'm pretty limited on time to study but the channels that I did like were probably martin kleppman, Gaurav sen, code karle, any random conference that talks about what I want to hear
Great point - I begin to deviate from his content around episode 15, but hopefully the suggested reading should allude to the fact that a lot of my knowledge comes from him :)
@@jordanhasnolife5163 i think, this is not true.. in certain scenarios leader should also be able to handle the read.. for example, we talked of this as a possible to resolve RYOW problem
![Seungmin "그렇게, 천천히, 우리(As we are)" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/kAzmhLHePqU/mqdefault.jpg)
bro no one is talking about this guy's humor, he is absolutely fucking hilarious LMAO
+1
Among lots of bullsh*ts in RUclips, I found your content most articulate and rich. Keep up the great work!
I appreciate that! That's why I made the channel :)
brooo, no one's talking about "my friends are giving me their loads constantly"
They can relate
The quality of your videos is exactly what I Want. most other channels overly simplify the topic. leaving me more confused.
I'd just like to say that you are doing brilliant work. Keep up the efforts. May you be blessed with millions of followers !!
Thanks Anupam!
Jordan has an awesome system design channel
9:30 key insight from comparing options: the best Replication Log is one that is at least: 1) Deterministic (outcome pre-determined), 2) invariant of any environment changes.
Well said thanks for summarizing!
Great Video Man ! Much Love from Egypt . keep it up
Wow! Glad to have you as part of the community, love the geographically diverse audience, and thanks for the feedback!
It is worth noting that `Consistent prefix reads` problem arises only if all the writes are not ordered in time, which doesn't happen in case of a single-leader replication setup.
Yep!
I have to admit I googled Kate Upton after (actually during) this video...
Man those chats are very innovative 🤣
I laugh out so loud at the consistent prefix read chats. "On my way!!"
Enjoy your content. Only 1 concern you mention that Write-ahead log is not scalable in case db engine changes but I think u r referring to extensibility which means easy to replace in case of new implementation whereas scalable means able to handle 1mil requests similar to the way handles 1k requests for ex.
Yes apologies, you're completely correct
I will never think about replication the same way again kkkkkk
I have a question - what do you mean by a node? I am confused because we're talking about DB replication here. Is a DB follower/leader referred to as a node in this context? Then when you said at 3:31 that "All clients send writes to new leader" - DBs arent smart to actually do that right? Who makes this change?
When I say a node I mean a distinct computer. Could be a leader, follower, basically any two things that require a network call to communicate with one another.
On a leader failover, we need to get a coordination service involved so that all replicas agree on who the new leader is. Clients can either ask an existing replica who the new leader is, or ask the coordination service itself.
Hey, great content! Can you share the links to all slides somewhere? The link you've posted below is pointing to only one video. Thanks for doing this. I'm learning a lot watching your videos.
Of course, and glad you're learning - here's the link: drive.google.com/drive/folders/1ChodcbMZ4KqS9WP9gin4sLVdCsgD3uoE?usp=sharing
@@jordanhasnolife5163 Thanks a lot bro!
@@jordanhasnolife5163 I was about to request you for slides. Thanks for sharing!
@@jordanhasnolife5163 about to ask you this in another video and randomly found the links here. Thanks 👍👍
2:02 That's what she said
teehee
Excellent content. Quick question though - I got the understanding that the replication log conveys the changes to the followers, so once the follower reads the changes, follower has no clue of that log anymore. So if this the case how can replication log helps with follower failure , does it store a copy of log too?
Also WAL is the way of storing logs(by appending changes) and Copy over SQL statements and logical logs are ways of storing those changes in that WAL. Is my understanding correct?
Yes the follower has its own copy of the log so in the event of failure changes can be replayed
@@jordanhasnolife5163 Thank you.
Can you share a pdf or so for all your videos so that we can have a quick revise before interviews. Just a request feel free to ignore it.
Both are out there: see my channel description
Hi Jordan, can you pls share your PPT link for this and other videos?
Check my channel description
Really good content!! Thanks for making them.
One question about "Consistent Prefix Read"
you mentioned "causal relationship between writes lost because preceding write takes longer to replicate (due to being on different partitions)".
If client only writes to the leader node, how would writes be on different partitions?
Single leader replication means that per partition, there is one leader. You can have partitioning and replication at the same time!
@@jordanhasnolife5163 Thanks! Okay this totally makes sense now.
thank you for this channel
Great Content. Thanks for making them.
My Question: Regarding Causal relationship writes on different partitions. Does that mean that one database can be run on multiple partition on the same machine? And can those partitions be different physical disks or logical partitions?
Yes!
where do I read in depth about the concept in video?
I'd start on Google and go until you can't
@@jordanhasnolife5163 thanks a lot! Finally I found high quality system design video that teaches concepts so well. BTW someone recommended you on Reddit.
I love your jokes, keep sending more!
hey since i like your content and humour , i am pretty sure i will like the youtubers you refer too can you drop the channels you refer to study ?
Well at this point I'm pretty limited on time to study but the channels that I did like were probably martin kleppman, Gaurav sen, code karle, any random conference that talks about what I want to hear
Thanks for creating this great content! I notice lots of the content are directly from ddia, maybe give some credit to Martin at the beginning😊
Great point - I begin to deviate from his content around episode 15, but hopefully the suggested reading should allude to the fact that a lot of my knowledge comes from him :)
By “partition” do you mean replica or is partition an orthogonal concept here?
Nope, partitioning/sharding is different, I cover this later in the series
I have some confuses. Is it true that only leader is allowed to operate write, and slave is just for reading.
That's correct - writes to leader, reads from any of the replicas
@@jordanhasnolife5163 i think, this is not true.. in certain scenarios leader should also be able to handle the read.. for example, we talked of this as a possible to resolve RYOW problem
Also to add read databases and write databases could be different and even read database could have different schema that write database schema.
This is true.
Hi Jordan,
Can you also share the ppt you used in this video
See channel description
Loved it!
Your slides would make great reference notes. Do you think you could share those too?
docs.google.com/presentation/d/1tkz2iksFao7_yaUXuKAvcvgKNlh_xYTRQJmKwYT0GHI/edit?usp=sharing
Pretty good bro, solid fundamental knowledge, good presentation
Thanks Dave!
I come here for system design(NO), awesome humour(Yes)
Thank you!
thanks
You have friends what!!!!!
They're imaginary
this is the type of nerd i mean when I say nerds are hot
^Conclusive proof that being a FAANG engineer makes you 10x hotter
I thought you are a software developer and has no friends.
I am a swe with some friends