Razorpay's Journey to Microservices w/ Arjun | Ep 1
HTML-код
- Опубликовано: 13 июл 2024
- System Design for SDE-2 and above: arpitbhayani.me/masterclass
System Design for Beginners: arpitbhayani.me/sys-design
Redis Internals: arpitbhayani.me/redis
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafters.io/join?via=...
In the video, I discussed the challenges of maintaining data consistency in microservices, focusing on the payment system at Razorpay. Initially, they used a monolithic architecture with MySQL for strong data consistency. However, as traffic grew, they transitioned to microservices to handle the load. They faced issues with database connection limits and decided to split core entities like payments, orders, and ledger into separate microservices. The transition was gradual, involving dual writes to ensure data consistency before fully transitioning traffic.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.me/knowledge-base
Bookshelf: arpitbhayani.me/bookshelf
Papershelf: arpitbhayani.me/papershelf
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack.com
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff. 
Наука
You should continue this podcast series. Out of all the podcasts or technical tutorials, there is no one in this niche. Students or experienced professionals will learn a lot from this. Kudos to you.
Check out other videos on my channel. You will find them interesting.
Pp0
*timestamps:*
0:00 Introduction
2:48 Initial architecture/techstack of RazorPay
4:30 ensuring strong consistency on monolithic architecture
9:06 check constraints
10:47 deciding to move from monolith to microservices
15:23 splitting of databases/application logic when moving to microservices
18:25 dual writing of data into monolith as well as microservice dbs(during transition phase), transaction outbox pattern
21:19 databases in current microservice architecture
22:45 deciding on SQL vs NOSQL
25:10 ensuring consistency in microservice architecture
28:10 payment flow with eventual consistency
34:40 prioritization for settlements@T+0
37:20 chaos testing
39:00 integration tests on interface between 2 services
41:00 incidents when things went wrong
47:40 outro
Thank you so much for this.
The way you progress the communication through passively narrating and curiously listening made this podcast different from other existing system design podcasts.
Such a technical dense podcast, rare to find such podcasts, nothing less than a technical conference talk. Great job Arpit !!
Few things amazed me and I am not sure about whether its worth it.
1. Just for connection pooling they migrated monolith to microservice.
2. They remove foreign key constraints in their table and made custom orm.
Such an amazing and detailed conversation. Absolutely loved it 💯
These 49.48 mins went pass like some couple of minutes.
This is so much learning and that also in such such basic terminology. That's one heck of a enginnering.
Thanks for the podcast Arpit and Arjun. I wish this series never comes to an end ❤️
Loved every bit of it . Please continue this series .
Great conversation !! Covered almost end to end high level architecture, challenges etc.
This is one of the best Podcasts I have ever seen. Such deep and niche insights. Beauty of Engineering!
loved this episode, so much to learn from razorpay. The data consistency section and ledger settlement is such a creative insight for anybody who hasn't worked on it yet. Thanks for recording this session and happy 1 month @google Arpit!!!
Just one word.. Brilliant Podcast
This is really Asli Engineering Stuff. Worth watching every second of this podcast, can't skip anything. Very Insightful.
Looking forward!!
Amazing stuff! Looking forward to more such podcasts.
Awesome talk. Thanks for doing this.
Wow! what an amazing podcast. Looking forward for more such Videos.
Thank you so muchh arpit.... It is true knowledge that will help engineers in real time, who are newly taking you challenges.
Brilliant value addition to engineering contents, real stuff! ❤️
Pure gold content 👏
Thanks for the video to both of you, the content is really phaad!!
Great podcast Arpit. Best thing I loved was that you were paraphrasing most of the things that Arjun said. I think you already know most of the things and still you sounded like a newbie with excitement who's listening about systems for the first time :)
Brilliant podcast! Thank you arpit for doing this!!
Can’t state how insightful it was. It blew my mind. Discussion really took my understanding to next level.
You must continue this series.... Phenomenal insights... Waiting for more
Wow Arpit....very knowledgeable session, Thanks for bringing him and the insights
woow, my mind blows away, so much knowledge on a plate, lots of things to get to know about how the actual digital payment system works, which may appear simple from a normal person's perspective. Thank you, Arpit.
Informative session. Learnt so much!
Thanks for this amazing video.
loved It, amazing things are doing by Razorpay Engineers. 😍
Very interesting and informative. Loved it.
Thankyou for providing this content ❤, really insightful
Great discussion. Enjoyed and got to learn new things. 🔥
Keep this series going!!! So much knowledge in one video
I have a ton of videos on various topics that you'll find more interesting than this. Scroll through my channel once.
Best engineering interview out there. loved it.
Loved it.. insightful
i am very excited for this series.
#thanks#gratefulToArpit
Great start of the day.
Really insightful and learnt a lot.
As always superb content and also I had recently cracked offered from Razorpay and Its Interview Process is also Amazing.
Lots of engineering insights and technical points. Got to learn many new things.
Really loved this one, thanks Arpit for bringing such insightful engineering podcast
Thanks Arpit Bhaiya for this awesome podcast.
superb… really learned lot of things from this session.🤘🏻
Really Enjoyed it
thanks
Need More Sir
really insightful video , keep up the good work
Great video, we want more of these
Was waiting for this one.
Thanks arpit such a wonderful podcast
Learned a lot from these 49 minutes! Hats off to the engineers!!
Mind blown podcast 🤯Thanks Arpit. You really inspires many like me. 😁
This is Asli Hardcore Engineering! Thanks a lot Arpit for bringing this amazing episode. Really amazed by the engineering practices at Razorpay. The way Arjun explained each and every detail was enlightening; definitely a skill to be learned by every software engineer. It is one of the best engineering podcasts and I got to learn so much!
need more podcasts like this
A brilliant show. Thank you guys. One thing that I am still interested to know. I have seen so many people talks about a lot of requests we were getting. But there is no number. So if you could tell this in your next video. Then it would be really great.
gold content. I got to learn from this video a lot .Thanks Arpit .
So true!
Great podcast very informative, thanks
This is truly asli engineering . Seedhi Baat no bakwaas, Real domain challenges and solution . I know that no architectures is The BEST. OutBox CDC ... amazing stuff ... please continue making stuff like this.
Wow! What a knowledge packed podcast and this is just ep1! So glad this exists! Really enjoyed conversations between both of you!
Excited for future eps! Thanks Arpit!
Real engineering hits more than any dopamine.
Got facinated by the number of DBs razorpay is using depends on seevice.
Really cool stuff with cool Engineer
This style of system design discussion is lit and interesting af
There are bunch of videos on my channel on real-world System Design. You will definitely find them interesting.
Great watch. Couldn't understand few of the technology used as my experience is mostly related to MS stack. But it has generated lot of curiosity about the stack used at RazorPay. Looking for more such podcasts. Thanks Arjan/Arpit for this wonderful discussion.
Very nice podcast , Enjoying the technology
We really need these kind of podcasts. Thank you Arpit
Would recommend going through other videos of my channel.
It always amazes to see real world problems.
just one word: wow!
Very informative content
Pure Gold
maja agaya sir bas 🙏
ase podcast kar te rahiye
Really inspiring series of videos, arpit. Keep it going I am sure it is inspiring a lot of Indian techies to dive deep and try these things on, it will definitely make better quality made in india for the world systems at an unbelievable price point.
Amazing ♥
Great video, one interesting question would’ve been what have been some issues with microservice architecture which still they’re in process of solving or need to evolve more
Wow great video, such complex concepts put so simple without using a single diagram.
Good going arjun…
video aane se pehle hi dekh liya tumne
@@saumitrashukla591 ji shukla ji, it was open for sometime, you got late
Thanks Varun 😊
Please get more such videos
As Always Arjun is great at explaining!
Brilliant Engineering by razorpay.
Going good...
The interviewee is providing calm and composed answers. I liked the clarity and the interviewer has to improve and avoid rephrasing every answer for such.a long time
Good one bro
I think one more step before dual write is to backup the existing data so that traffic can be served
Would love a 20 mins video where you can make notes about this conversation and explain. There were some terms and flows which i couldnt follow. CDC was new, sounds like ddb streams. Also the part where they ensure the ledger update was pretty cool but I couldnt understand it fully.
Intial tech stack:Php, Larvel, Route 53, alb, mysql rds
It’s really difficult to think that a foreign key validation at PHP (application) level would be faster than what MySQL can do. Database are optimized for these things.
not about the speed. But having the ability to split.
Hi Arpit & Arjun,
I've been thoroughly enjoying your content and have been binge-watching your videos.
At 18:25, Arjun discussed implementing the outbox pattern for dual writes. While I understand the concept well (and have implemented something similar), I wanted to delve deeper into the actual implementation. The traditional use case for this pattern involves achieving ACD (a form of transactionality) for database writes and event publication in the message queue or a different DB.
Once the data is transactionally written to both the monolith table and the outbox table, an asynchronous job, using transactional log tailing (binlog consumption), allows us to consistently write to the microservice table.
However, let's consider a scenario where the broker is down (acting as a SPoF). How does the OUTBOX PATTERN ensure consistency in this situation?
In my implementation, instead of relying on binlog consumption, I've set up a cronjob (using a serverless service) to read from the outbox table and publish the event to the message queue. If it succeeds, it updates the status in the outbox table to prevent the next run of the cronjob from picking it up again.
I'd love to hear your thoughts.
Hi Arpit, top video with some great insights. One request though - could you add timestamps to the video covering different topics that were discussed in the podcast, would be a huge help?
I have observed that people skip most of the part when I add timestamps.
If they jump forward and there comes a point where some context is needed, people are more likely to drop at that point.
Hence I have stopped adding timestamps.
Hey Arpit,
Do read LedgerStore paper by Uber. With that in mind, you will love the beauty of both razorpay and Ubers ledger system, especially the way uber migrated their db from aws to in house and writing to both db parallels to test thing out.
Thanks a ton for sharing. Definitely going through it.
@@AsliEngineering
Please do make video on same if possible. I think it is good enough for everyone to learn something from a large db migration.
Love your content and open for any help you need (not for technical perspective as I am still learning and growing but any other task or any initiative) in growing your community.
I just knew I too using transaction out box pattern (by name) :)
Hi Arpit! Eager to listen to Dr. Kailash of Zerodha in such podcasts as well. Thanks!
You can add visual explanations of some technical concepts as every one might not understand/know all the concepts.
after learning DBMS and MySQL able to understand this podcast
wwoooooooooh amazing.....I would like to work on such an infra
same here
When we said MySQL is high on consistency. which consistency we r referring here..ACID one or CAP one.. it sound like acid one but then you removed all the constraints from db.. which is bit conflicting @arujn
Nice idea but validation on application will still need atleast the primary key on the table and transaction between app and db and will need to avoid any such case where there is m:n cardinality between entities if we have such cases then the only better way is to use relation tables where foreign key constraint is inevitable.
Additional way is to even manage that relation table manually through app instead of using foreign key on the relation table , but how efficient will it be is a question again. May be it more efficient as it has to do a lot less that db has to do.
Amazing discussion! :D
Being a part of debugging through the last issue mentioned would be lit af :D
Yeah. Even I got engrossed and tried to get max info on it 😀
Good one Arpit and Arjun learned a lot, I am so stuck with AWS that can only relate to amazon specific service be it queue or notification topics, this is triggering an alarm for me to look into other similar products used at scale worldwide.
PS:~ Not sure their UTs are with 95% code coverage; hahaha Arjun can reply and confirm.
Curious about how uniqueness & race conditions regd database writes were achieved by moving db validations & constraints to application layers.
Payments data can be huge is rdbms enough to support the data , are you removing payments data periodically to some cold store from the transactional rdbms?
Amazing 48:05
Amazing session, shut up and take my subscription!
Can you suggest books for design patterns in Eventual Consistency in Distributed Computing?
How can you decide to which bank you should route request its chooseen by customer
Amazing discussion.
If possible don't add advertisements randomly between the conversation. It breaks the flow.
YT adds it not me.
Please get this up on Spotify 🥺
AsliEngineering is already on Spotify. I think it's would be available on Spotify by default.
Will upload on it soon.
@@AsliEngineering also upload to other podcast platforms like Apple Podcasts, Amazon Music etc.