00:04 Common sense application is key in engineering principles. 01:00 Understanding problem-solving in engineering through practical experience. 03:13 Understanding hash-based routing in distributed databases. 04:16 Understanding routing is essential before diving into core concepts. 06:21 Discussing data distribution strategies for handling large volumes of requests. 07:19 Understanding router functionality and efficient key storage. 09:00 Exploring alternatives to routers for database communication. 10:03 Alternative approaches in problem-solving are essential. 11:44 Understanding real-world analogies in engineering and computer science. 12:34 Explains hash functions in routing for efficient data management. 14:20 Discussing data movement challenges and system consistency. 15:43 Understanding minimal data movement in engineering processes. 17:56 Efficient data movement in programming with minimal changes. 18:54 Apply first principles to optimize data structures in engineering. 21:22 Exploration of data storage methods and their applications in engineering. 22:54 Understanding database connection management in programming. 25:52 Understanding linked lists and hashing in data structures. 26:54 Explaining value storage in linked lists and accessing nodes efficiently. 29:05 Linear search is simpler than it appears. 29:58 Understanding consistent hashing and its evolution in programming. 31:53 Implementing effective caching strategies in engineering solutions. 33:01 Efficient approaches to data ownership and database migration. 35:01 Optimizing queries requires understanding implementation through practical knowledge. 35:52 Discussing the cost and efficiency of running a linear search algorithm. 38:00 Effective strategies for scaling data ownership and management. 39:16 Understanding the producer-consumer model in data buffering. 41:10 Minimizing 'stop the world' issues in data processing. 42:45 Real engineering focuses on efficient problem-solving using practical analogies. 45:48 Swapping variables has evolved from complex to instantaneous operations. 46:59 Discussion on effective programming approaches in engineering. 48:56 Strategies for managing buffer data in engineering contexts. 49:56 Understanding immutability and single-threading in JavaScript. 52:49 Discussion on threading in programming, particularly JavaScript's single-threaded nature. 54:14 Understanding the fundamentals of web server coding from scratch. 56:48 Understanding TCP connections and event loops in server programming. 57:49 Understanding single socket connections in server-client communication. 59:56 Understanding JavaScript's single-threaded nature and I/O operations. 1:01:08 Understanding socket operations in real-time engineering. 1:02:53 Understanding client data retrieval in network programming. 1:04:16 Data handling process in networking and system calls. 1:06:15 Understanding effective blocking system calls in coding. 1:08:40 Understanding file descriptors and connection handling in web servers. 1:10:46 Understanding single-threaded server implementation and its significance. 1:11:44 Understanding the advantages and pitfalls of multithreading in engineering. 1:14:07 Importance of readiness and data management in engineering. 1:15:25 Survival in engineering requires curiosity and practical knowledge.
For the folk's who thinks this is overwhelming stick till the end and watch it saw alot of comments from such guys i am a newbie too still i understood few portion of it for sure combing back again after gaining more knowledge.Great content thanx kirat sir and arpit
Really don't care. One could just patch up bunch of projects from GitHub and make clones or even make better versions. India's tiktok clone Moj was bootstrapped in just 90 hours. Literally, probably nobody in their dev team start with first principles.
Timestamps: 4:30 Why Distributed DB reqd? + Monolith vs MicroService + 1st principle for system design patterns -> If single node cannot handle the service, then distribute. Distribution requires data splitting or sharding 07:00 Routing basics -> static routing + storing metadata - 08:00 Framework of opposites : don't wanna store metadata -> use a mathematical function named hash don't wanna use routers -> Either middleware knows databases .. or client knows databases 11:00 - CS design patterns are inspired by real-world basics 13:00 - Consistent Hashing
Love Arpit Sir's way of teaching, the way he simplified "Senior Engineer" concepts to basic principles / algorithms, truly shows his depth of knowledge in computer science topics. Please make more such videos, learnt a lot !!
It's really nice to see how things actually work behind the fancy words and on such fundamental concepts which everyone knows about, the only difference is taking a step further to dig deep on the implementation. Loved the complete session 🤍🤍
Most of these guys are FAKE as well - they are more of social media gimmicks and fakes, real coders and designers do not talk much. Their aim is very high, very very high, in fact.
amazing session about software engineering. the way he taught about thinking by first principle and breaking big problems into smaller ones and solving it is amazing. thanks Arpit sir for wonderful session🙏🏻
These concepts are really not that hard to think of (of course excluding the epoll kernel syscall, you either know about it or don't). It's kinda surprising people can't put that 2 and 2 together. This is why I'll always die on the hill that more people need to care and learn about what's going on under-the-hood. You're so much confident when theorising or building stuff when you know what's _actually_ happening (what/how many connections are made, what syscalls might be happening, what part is too computationally/IO heavy, what are the libraries doing, etc.). It also naturally develops this first-principles thinking. All of this came to me naturally as a consequence of knowing my stuff, I did not get stuck at any of his questions. I'm not special or very smart. Just dive deep.
My job is so much saturated that I keep doing the same things. After almost 6 months, I used my brain to the question Consistent Hashing implementation and got it right in the first attempt. Proud.
I don't know why this seems a big problem you basically need to rebalance only 1 node and do bisect ordering when adding a new node. Use the shard key as array values, and when a new shard is added, use bisect ordering to figure out new pos and refills. The Shard key should not be monotonically incrementing. I don't have CSE background.
4:55 for god sake, THANK YOU! someone finally said that! -- they think its fancy and scalable but project shuru hua nahi microservice thokna hota hai newbies ko, cuz its cool for them -- its a fucking overhead -- go to market is most important
name's super30 , folks are stupid. you guys have literally kirat and arpit like ppl around you... consider this as your golden time and learn without bitting around bush. you guys have no ideas how many of us wanted to be a part of this cohort 💔
I respect Arpit as an engineer, but this was not a good session . Maybe people who attended it offline might find it useful. The things seem to be bit rushed. Rather than sparking curiosity this will overwhelm a lot of students. PS:- No hate to him, I think he is a good engineer, but teaching things is different.
Exactly bro.. thank god I found your comment. I felt so overwhelmed so worthless in the first 10 mins. I stopped watching. Will re-watch it maybe after year.
@@priyanshrawat442 You can say that. But context matters. The students he is teaching are freshers and I don't think this is how you teach topics. If it would have been for people who have more experience and already know most of the concepts on a high level it would be ok. But still freshers or experienced, this is such a vague style of teaching. Everything seems to be all over the place.
It's normal to feel overwhelmed sometimes; not everyone has the same background or understanding. The key is to keep learning and pushing forward. One day, you'll look back and see how far you've come. Many concepts discussed here are part of CS syllabuses, though they're often not taught in ways that highlight their real-world value. Arpit's approach might feel fast-paced, especially if you're not familiar with some of the topics yet, but that's part of the learning process. It's a great reminder of the importance of these concepts.
Hi , I have one question , Do we keep this consistent topology [Array/LL] in-memory or in a persistent storage . Because if we only rely at in-memory service can die/boot-up at anytime .....?
this is system design thing. Just knowing this and some famous algorithm and system design principle would be the prerequisite. (this is my opinion, i dont know how much is absolutely correct)
Don't watch this if u are a newbie like me.Youll get overwhelmed. It's fine if you don't understand what he says rn we can always learn things step by step.Tbh I stopped watching after 10 mins into it. P.s - no hate to Arpit
Jo bhi ye skill issue likh rhe hai saalo tumne konsa facebook ya twitter ban diya kis liye hai overwhelming hai to hai Arpit sir 10+ experience hai unke liye kuch bhi nhi hai but jo newbie hai unke liye to ye nhi smjha aayga starting mein isliye gyan kam do aur video dekho
what I don't like about these sessions is that it is still not very hands on! real engineering is I write actual code for say a basic server in say C or Rust and show some tool like iptraf to see the sockets, use tcpdump/wireshark to capture packets sent & received, display threads etc on a unix system! can get more deep also! Write unit tests!!! He talks about epoll and io_uring but show some code in action for event loop! go deep and not talk 10 things, that real engineering for an hour tbh.
Why they hell are oyu sharing all this with folks? this is supposed to be learnt on the job. You guys have already screwed up the design interviews by sharing designs of various systems, even when they have not designed any system. Now, they know all these things in bits and pieces and behave as if they have built it, in the interview. It only makes our lives difficult as interviewers. We have learnt all the concepts you mentioned on the job and implemented them in low-latency systems in C.
![Blox Fruits Dragon Rework Update [Full Stream]](http://i.ytimg.com/vi/EqDAp8udhm0/mqdefault.jpg)
00:04 Common sense application is key in engineering principles.
01:00 Understanding problem-solving in engineering through practical experience.
03:13 Understanding hash-based routing in distributed databases.
04:16 Understanding routing is essential before diving into core concepts.
06:21 Discussing data distribution strategies for handling large volumes of requests.
07:19 Understanding router functionality and efficient key storage.
09:00 Exploring alternatives to routers for database communication.
10:03 Alternative approaches in problem-solving are essential.
11:44 Understanding real-world analogies in engineering and computer science.
12:34 Explains hash functions in routing for efficient data management.
14:20 Discussing data movement challenges and system consistency.
15:43 Understanding minimal data movement in engineering processes.
17:56 Efficient data movement in programming with minimal changes.
18:54 Apply first principles to optimize data structures in engineering.
21:22 Exploration of data storage methods and their applications in engineering.
22:54 Understanding database connection management in programming.
25:52 Understanding linked lists and hashing in data structures.
26:54 Explaining value storage in linked lists and accessing nodes efficiently.
29:05 Linear search is simpler than it appears.
29:58 Understanding consistent hashing and its evolution in programming.
31:53 Implementing effective caching strategies in engineering solutions.
33:01 Efficient approaches to data ownership and database migration.
35:01 Optimizing queries requires understanding implementation through practical knowledge.
35:52 Discussing the cost and efficiency of running a linear search algorithm.
38:00 Effective strategies for scaling data ownership and management.
39:16 Understanding the producer-consumer model in data buffering.
41:10 Minimizing 'stop the world' issues in data processing.
42:45 Real engineering focuses on efficient problem-solving using practical analogies.
45:48 Swapping variables has evolved from complex to instantaneous operations.
46:59 Discussion on effective programming approaches in engineering.
48:56 Strategies for managing buffer data in engineering contexts.
49:56 Understanding immutability and single-threading in JavaScript.
52:49 Discussion on threading in programming, particularly JavaScript's single-threaded nature.
54:14 Understanding the fundamentals of web server coding from scratch.
56:48 Understanding TCP connections and event loops in server programming.
57:49 Understanding single socket connections in server-client communication.
59:56 Understanding JavaScript's single-threaded nature and I/O operations.
1:01:08 Understanding socket operations in real-time engineering.
1:02:53 Understanding client data retrieval in network programming.
1:04:16 Data handling process in networking and system calls.
1:06:15 Understanding effective blocking system calls in coding.
1:08:40 Understanding file descriptors and connection handling in web servers.
1:10:46 Understanding single-threaded server implementation and its significance.
1:11:44 Understanding the advantages and pitfalls of multithreading in engineering.
1:14:07 Importance of readiness and data management in engineering.
1:15:25 Survival in engineering requires curiosity and practical knowledge.
really
I really appreciate this, thank you
For the folk's who thinks this is overwhelming stick till the end and watch it saw alot of comments from such guys i am a newbie too still i understood few portion of it for sure combing back again after gaining more knowledge.Great content thanx kirat sir and arpit
That's what everybody should be learning - First Principal Problem Solving, One step at a time and then suddenly you will be breaking the odds.
Really don't care. One could just patch up bunch of projects from GitHub and make clones or even make better versions. India's tiktok clone Moj was bootstrapped in just 90 hours. Literally, probably nobody in their dev team start with first principles.
Corporations aren't wasting time on that. Programs extend existing programs for a reason.
Timestamps:
4:30 Why Distributed DB reqd? + Monolith vs MicroService + 1st principle for system design patterns
-> If single node cannot handle the service, then distribute. Distribution requires data splitting or sharding
07:00 Routing basics -> static routing + storing metadata
- 08:00 Framework of opposites :
don't wanna store metadata -> use a mathematical function named hash
don't wanna use routers -> Either middleware knows databases .. or client knows databases
11:00 - CS design patterns are inspired by real-world basics
13:00 - Consistent Hashing
Love Arpit Sir's way of teaching, the way he simplified "Senior Engineer" concepts to basic principles / algorithms, truly shows his depth of knowledge in computer science topics. Please make more such videos, learnt a lot !!
Wow, this video is a literal gem, Loved it...harkirat please bring more videos like this
All the success story videos are great but videos like these are the ones I'd pay to watch.
One of the best videos I have watched so far, would love more videos like this
It's really nice to see how things actually work behind the fancy words and on such fundamental concepts which everyone knows about, the only difference is taking a step further to dig deep on the implementation. Loved the complete session 🤍🤍
Indian ke sare teachers ko replace kardo real engineer who work in industry who give us real world to implement solution without bluff 😊😂😂
Most of these guys are FAKE as well - they are more of social media gimmicks and fakes, real coders and designers do not talk much. Their aim is very high, very very high, in fact.
@@sbasu4748he worked at Google, Amazon stop thinking as a marketing gimmick lol😂
Would love more such discussions, loved it
amazing session about software engineering. the way he taught about thinking by first principle and breaking big problems into smaller ones and solving it is amazing. thanks Arpit sir for wonderful session🙏🏻
I think I understood only 10% of what Arpit Sir told but it was really insightful.
"Experience speaks lounder than actions"
Didn't expect the video to be this informative!
I am very happy to have had a very nice discussion
This is gold!!!! , thanks a lot for uploading this
These concepts are really not that hard to think of (of course excluding the epoll kernel syscall, you either know about it or don't). It's kinda surprising people can't put that 2 and 2 together. This is why I'll always die on the hill that more people need to care and learn about what's going on under-the-hood. You're so much confident when theorising or building stuff when you know what's _actually_ happening (what/how many connections are made, what syscalls might be happening, what part is too computationally/IO heavy, what are the libraries doing, etc.). It also naturally develops this first-principles thinking. All of this came to me naturally as a consequence of knowing my stuff, I did not get stuck at any of his questions. I'm not special or very smart. Just dive deep.
Bring him on podcast more man this is awesome
He is genius ❤️
I can watch his talks in loop.
Amazing Guy. Thank you very much.
Been following arpit for a while and his diceDB🙌
it's feels like a friend ek apna person - feels so connected
Become a fan of this person
became...well it dosen't matter
@@ramnikTDMbecome
@@ramnikTDM I think he was asking us all to become a fan of Arpit, but nvm 😅
Fanboyism is low IQ ideology.
Don't be fan of any person
My job is so much saturated that I keep doing the same things. After almost 6 months, I used my brain to the question Consistent Hashing implementation and got it right in the first attempt. Proud.
Arpit Bhayani is OG 🙏
Very Insight Full.
Loved It....
Thank god, you uploaded this gem!!!!
Wow kya shamja raha hai bhai ❤....
He is THE REAL ENGINEER.
Bros confidence and way of convey 🔥 🔥
Why did the note sharing stop after 24:00?
This lecture was so insightful !
Interesting class in System design, Arpit Bhaiya class like why in everythings
47:16 Sir will it not be an overhead if we will check every time if the array if full so that we can start referencing to second array.
I don't know why this seems a big problem you basically need to rebalance only 1 node and do bisect ordering when adding a new node. Use the shard key as array values, and when a new shard is added, use bisect ordering to figure out new pos and refills. The Shard key should not be monotonically incrementing. I don't have CSE background.
4:55 for god sake, THANK YOU! someone finally said that! -- they think its fancy and scalable but project shuru hua nahi microservice thokna hota hai newbies ko, cuz its cool for them -- its a fucking overhead -- go to market is most important
Great video. Glad that it was free. Gold content🙏💙
Was waiting for this 🎉
Love the fact, sirf system design mat karo, implement bhi karo
Great Session !!
name's super30 , folks are stupid. you guys have literally kirat and arpit like ppl around you... consider this as your golden time and learn without bitting around bush. you guys have no ideas how many of us wanted to be a part of this cohort 💔
SDE 3 level k chize hai
Cool guy! Will definitely check codebase of Dice DB now.
What all the youtubers lack is mentioned by Arpit Sir here!
Woao , woao ,he is the guy from whom we should learn engineering
Pickle hi interview me Arpit ne bataya his backup is staff engineer at google and not principle engineer
Please re-upload the video with screen share all throughout ,notes sharing stops at 24 minutes
exactlyy !!!
Why the screen sharing of notes is STOPPED after 24:50 mins?? ANY REASON (bold letter to gain attention only)
It's displayed on the screen behind him.
It's difficult to follow @@AaK012
I respect Arpit as an engineer, but this was not a good session . Maybe people who attended it offline might find it useful. The things seem to be bit rushed. Rather than sparking curiosity this will overwhelm a lot of students.
PS:- No hate to him, I think he is a good engineer, but teaching things is different.
Exactly bro.. thank god I found your comment. I felt so overwhelmed so worthless in the first 10 mins. I stopped watching. Will re-watch it maybe after year.
People who found it overwhelming has skill issue lol
@@priyanshrawat442 You can say that. But context matters. The students he is teaching are freshers and I don't think this is how you teach topics.
If it would have been for people who have more experience and already know most of the concepts on a high level it would be ok.
But still freshers or experienced, this is such a vague style of teaching. Everything seems to be all over the place.
It's normal to feel overwhelmed sometimes; not everyone has the same background or understanding. The key is to keep learning and pushing forward. One day, you'll look back and see how far you've come. Many concepts discussed here are part of CS syllabuses, though they're often not taught in ways that highlight their real-world value.
Arpit's approach might feel fast-paced, especially if you're not familiar with some of the topics yet, but that's part of the learning process. It's a great reminder of the importance of these concepts.
@@Abhinav_2060 you dont need a year bro
just listen to it you'll understand it as u watch
Where do we run consistent hasing?
I could not hear properly from cloud
Everyone is gangsta until the Arpit Bhawani arrives 😎
He is a legendary 🔥🔥
He is too good!
is ipad pass - 133113 ?
Really hate it when engineering is limited to software engineering only. There is a world beyond.
Chatak bhar code. I can't unnotice Marathi slag😂
i felt there is minor issue fro audio side ...slight vibration - which might be noticed - but i can hear ...needed to be sorted
Is epoll also a blocking call? Or we specify some timeout if none of the file descriptors have any events thats needs to be processed?
Woahhhhh 🔥
@Harkirat sir I wanted to ask ki what is the right time to join a cohort since I want to join an active and live one
Hi , I have one question , Do we keep this consistent topology [Array/LL] in-memory or in a persistent storage . Because if we only rely at in-memory service can die/boot-up at anytime .....?
Arpit bhai big fan
what are the prerequisits of this video ? i was not able to understand majority of it 🙃
this is system design thing. Just knowing this and some famous algorithm and system design principle would be the prerequisite. (this is my opinion, i dont know how much is absolutely correct)
basic computer science fundamentals - os, threads, networks.
@@poorvadityabehre thx, btw nice dp 😂
If socket is non blocking then how recv() is blocking?
Don't watch this if u are a newbie like me.Youll get overwhelmed. It's fine if you don't understand what he says rn we can always learn things step by step.Tbh I stopped watching after 10 mins into it.
P.s - no hate to Arpit
Lmao skill issue
@priyanshrawat442 agreed
Jo bhi ye skill issue likh rhe hai saalo tumne konsa facebook ya twitter ban diya kis liye hai overwhelming hai to hai Arpit sir 10+ experience hai unke liye kuch bhi nhi hai but jo newbie hai unke liye to ye nhi smjha aayga starting mein isliye gyan kam do aur video dekho
Bhai to agar kisko ek skill nahi aati to skill issue hi to kehkaya jayega?
Nope. Definitely a skill issue.
Its good to have information but did anyone here ask themselves that do they need it?
Asli Engineering 🎉
Very interesting lecture
Shirt dmart se liye...
same same but different..🥲
asli engineer
bhai saab kuch sar kay upar jaa raha hay 😭
Skill issue
sar ko uper karo Phir, niche nhi
First principles me socho
That’s why he is staff engineer!
It’s not about DSA
Be Asli Engineer🤝
Love it
great Pod
Now course seller champion harkirat and these newbies will teach the mass how to develop logical thinkin' 😂
He looks like sargam. Im i right?
Is this are SYstem desgin concept
His iPad password is 133113. Let him know that.
next level
open the offline center in puneee...
gem
Please start this in Banglore also
I was able to understand everything 💀☠🦨
\open the offline center in puneee...
fantablous
1:10:30 Chhatak bhar ka code. You can get out of Pune, but can't take Pune out of you :P🤣
Nice shirt. Link please ?
what I don't like about these sessions is that it is still not very hands on! real engineering is I write actual code for say a basic server in say C or Rust and show some tool like iptraf to see the sockets, use tcpdump/wireshark to capture packets sent & received, display threads etc on a unix system! can get more deep also! Write unit tests!!! He talks about epoll and io_uring but show some code in action for event loop! go deep and not talk 10 things, that real engineering for an hour tbh.
Pyramid theory🥶
Brother leaking password at 54:24
yes common sense is the most uncommon thing. Its really true. People don't know about climate change
Seems like dsa is important
Arpit biryani is amazing
"saale sab same hai".....😂😂
Need to see the screen, not the speaker.
write implementation
Why they hell are oyu sharing all this with folks? this is supposed to be learnt on the job. You guys have already screwed up the design interviews by sharing designs of various systems, even when they have not designed any system. Now, they know all these things in bits and pieces and behave as if they have built it, in the interview.
It only makes our lives difficult as interviewers. We have learnt all the concepts you mentioned on the job and implemented them in low-latency systems in C.
Hyderabadi accent jhalakri 😀
Add english caption
too much information to process in an hour