This section is present in the book designing data intensive applications. I highly recommend everyone to read it. There is lots of other concepts there also like how db handles transactions and instant backups etc.
I have a doubt. @18:00 in this video, we are telling if Active File size exceeds 100 MB we will create chunks and compaction will happen on chunks. But if in compaction Active File does not participate, then its could happen reader will not get data until Chunks does not gets created out of Active file. Am I missing anything ?
Good video for basic understanding but still lot of things to discuss. In real world, we would have a distributed environment . That means there would be multiple clients writing to multiple db nodes. We need to handle concurrent updates in such scenarios. We would also need a conflict resolving mechanism like vector clocks. Also writing to database itself involves multiple step. We write to commit logs, then memtables(inside ram) and then flush it to sstables (append only logs on disk). Compaction is a big topic and there are multiple ways of compacting data. For deletes, most dbs use a construct called tombstone.
I think inspiration for this video is from this book -> Designing Data Intensive Applications by Martin Klepman. Amazing book, I am halfway through it.
@@praveenkurapati691 I don't have a hardcopy, I read from a pdf available online. The benefit of that pdf is that you can take notes on a doc from pdf.
Nice! So database system will take care of copying data to slaves, it is matter of configuration on database system level....what about routing read n writing request to the respective db node,would that b implemented on the application level via some gateway to filter n send to respective db i.e., read or write???eagerly curious :-))
one question - how is this in-memory cache index built, what if machine restarted ? Or it's database engine responsibility to build this in memory DS when machine starts (more precisely when data base services in that machine starts)
Hey rachit, i am a 2021 passout currently working as a Software tester. I want to switch to SDE. Considering it would take me around next 6 months to be good at DSA , that would sum up 1 YOE as a tester. Would that create a problem for me in getting interview call for SDE profile? Or any other suggestion from your side about what should i do now? Thanks
This section is present in the book designing data intensive applications. I highly recommend everyone to read it. There is lots of other concepts there also like how db handles transactions and instant backups etc.
I am acually Reading it and searched for the video
I have a doubt. @18:00 in this video, we are telling if Active File size exceeds 100 MB we will create chunks and compaction will happen on chunks. But if in compaction Active File does not participate, then its could happen reader will not get data until Chunks does not gets created out of Active file. Am I missing anything ?
9:50 when storing index in-memory, what if the index grows in size ? how can we handle this case ?
How did you (programmatically) get the disk offset for the cache at time 12:39?
Good video for basic understanding but still lot of things to discuss.
In real world, we would have a distributed environment . That means there would be multiple clients writing to multiple db nodes. We need to handle concurrent updates in such scenarios. We would also need a conflict resolving mechanism like vector clocks.
Also writing to database itself involves multiple step. We write to commit logs, then memtables(inside ram) and then flush it to sstables (append only logs on disk).
Compaction is a big topic and there are multiple ways of compacting data. For deletes, most dbs use a construct called tombstone.
I think inspiration for this video is from this book -> Designing Data Intensive Applications by Martin Klepman. Amazing book, I am halfway through it.
Is it boring without practicals?
@@praveenkurapati691 No, it's one of the most interesting book , I have ever read.
@@praveenkurapati691 It's useful only if you have worked in software industry atleast for 1 year.
@@umangmalhotra1222 thanks I'm thinking to buy hardcopy. Will test it. 😀
@@praveenkurapati691 I don't have a hardcopy, I read from a pdf available online. The benefit of that pdf is that you can take notes on a doc from pdf.
Thanks for the video!
Nice! So database system will take care of copying data to slaves, it is matter of configuration on database system level....what about routing read n writing request to the respective db node,would that b implemented on the application level via some gateway to filter n send to respective db i.e., read or write???eagerly curious :-))
one question - how is this in-memory cache index built, what if machine restarted ? Or it's database engine responsibility to build this in memory DS when machine starts (more precisely when data base services in that machine starts)
Which programming language do you use in Code Jam?
hi rachit bhaiya.. can you please comment on what should i prep for upcoming summer Microsoft intern?
Hey rachit, i am a 2021 passout currently working as a Software tester. I want to switch to SDE. Considering it would take me around next 6 months to be good at DSA , that would sum up 1 YOE as a tester. Would that create a problem for me in getting interview call for SDE profile? Or any other suggestion from your side about what should i do now?
Thanks
this video needs to be swapped with 5th video.. videos are out of order in this playlist.
plz make a video on Innovaccer Hackercamp challenge.. they will give jobs upto 15 lacs. very good opptny for students like me, i really want to win