Thanks for the comment and congratulations 🙂. Please keep liking and commenting 😁
2 года назад+9
Nice video, thanks! Usually, stock exchange APIs are based on the FIX protocol, which is implemented on top of the TCP transport. But if I could pick one, I would go with gRPC, specifically with a server stream. It is easy to use and implement (you can automatically generate the client for most languages). It is a contract-based protocol. It performs very well, providing a good throughput with very low latency.
Doubt, when we're using queues, how dispatch server knows what are all stocks that i had to initiate connection with stock exchange ? Say, User is interested in Apple, how that's being communicated to dispatch server ?
Thanks for the detailed video. 1. Assuming we have 5000 symbols (not including commodity or options/future etc), instead of checking for each symbol if they are subscribed by the user, i would use stock groups as defined by stock server like Nifty 50, nifty bank etc. and only check against that group while adding/reading to stock server. For Tier1 groups, Nifty50 stock group, send notification without checking whether individual stock is subscribed or not. Though stock can also move between these groups as well with time, not considering at this moment. 2. If FE goes down, then we can have multiple backup servers. Using consistent hashing and using multiple virtual servers, we do the reassignment of FE servers from global cache or kafka. 3. I would use websockets (bidirectional) and kafka. We can have multiple topics in kafka and bind them using key as stock group and not each symbol.
Great video. I would have picked web-sockets and Kafka. If the FE server dies, the client can just reconnect to another FE. I don't really see many benefits of SSE since it cost almost the same as web-sockets, but if we wanted to add a bidirectional feature in the future, the ability is there for web-sockets. For example, the connection is bad and they want to reduce the rates of updates, maybe the client can request the server drops the updates by half.
Thanks for the comment. Yes, your reasoning is correct and if you check the existing APIs, most are websocket based. Also if you check the websocket approach picture in this video that I showed, it clearly shows that the client an actually send request on the websocket to change the list of stocks they want to listen to (I didn't discuss this in the video though). However, I want my viewer to be in the habit of questioning and thinking, so although I have chosen SSE, but asked the viewers what they will choose and why.
Say the client executed some transaction. The FE server dies before completing this transaction. If the client just reconnects to another FE server, isn't this transaction just lost and this would cause a problem?
I appeared for an interview yesterday and the same question was asked to me to develop a widget to show real time stock price updates. Wish i had watched your video before. Having worked mostly on Healthcare domain projects in my career i wasnt able to fully answer it. Thanks for the video Sir
I think rather than App server having a mapping of stock to list of subscribed customer, this information can be offloaded to a key-value store(or any other DB for that matter). We can also introduce a cache for this if Db is slow. This way we can make App servers stateless and hence more fault-tolerant(any App server can process any stock, so there is no single point of failure). This would also make the system more available. Also, to make Dispatch server fault tolerant, rather than on dispatch server listening to certain stock, we can have an ensemble of server and exposing them via load balancer. In the design shared in the video, we would have 2 such ensembles. Please share your thoughts. PS : Great content and very well explained
Thanks for the comment. The reply could become big. I would try to keep it short. First of all, there is no point offloading the list of customers connected to an app server to a key-value store or a distributed cache (i.e. on a different machine). Because if that app server dies, so as all the SSE connections that it is hosting. Secondly, regarding using an ensemble of servers (for dispatch servers), I didn't what you suggesting here? Can you please elaborate? Ideally we would have atleast two machines (preferably three) listening to a particular stock in dispatch servers. So if you are suggesting we have two sets of machines in dispatch servers listening to each stock then that is fine. But I don't think we need to have them behind a load balancer. They are just going to publish the stock values to distributed queues.
nice Video, but one question here is whenever there is any change for a stock at Stock exchange server it must come to get updated in Kafka, then why do we need to make so many SSE, let Stock Exchange Server publish the change to any stock to Kafka queue instead of making SSE from dispatch server request?
I think partion tolerance should be handled on both front end and dispatch servers with health monitoring system. On front end clients should reconnect. Dispatch side, We should maintain cache server -> stock mapping, and reconnect to stock exchange when dispatch server dies.
With a queue, it is enabling a pushing mode for latest price, so user can get know the price sooner, which could be faster than the cache method. Then I would support a websocket method from the begining from client side. The data is refreshed all the time if got connected.
I worked on learning java language. This is understandable. Now I’m building with pluto io as it is much more easier for me and faster to test automations without spending a ton of time. I also tried cube, etc. with subjective gains.
If one (or more) of the frontend (FE) servers crashes, the disconnected clients would have to re-establish connectivity with the FE service through the LB, so that the newly formed connections are all distributed as others have pointed out. We'd also need to persist the inverted mapping {client_ids - list_of_stockTickerSymbols} in a database/distributed disk-backed in-memory cache so that in the event of an FE crash, the affected clients that then get reconnected to a subset of the rest of the alive FE servers would have a seamless experience (The FEs would pull the mapping from the persistent store and merge it with the local mapping of stock_ticker -> client_ids). Basically the suggestion is to maintain a persistent user account with the list of stock tickers the user is interested in.
@@ThinkSoftware no problem. Great video, by the way. I really loved the way you incrementally expanded the complexity of the problem, suggesting multiple approaches at every stage with careful analysis of pros and cons while also motivating the viewers to engage in further thinking.
Really good video. Probably a dumb question though: the frontend app server => client connection I understand using SSE, but how does the Stock Exchange => backend app server work? I'm guessing I'm being limited by my PHP experience here, but I can't say I'm familiar with PHP being a client for Server Sent Events. EDIT: Nevermind. I guess there are libraries that allow you to do this
Thanks for the comment. Yes there are different libraries in different languages/platforms that provide such support. Everything is built on top of socket APIs ultimately.
This is at a very high level with lot of pitfalls and Gaps. Common usecase i see is to help others understand how would you get the list of user to send out notifications with Million users subscribed to a given topic? Will you end up querying those users which is very non-optimal. Will you get them in batches , that means processing them paginated way, but this has to happen for multiple events and these events are coming at a high rate. Think about those cases
Doubt, when we're using queues, how dispatch server knows what are all stocks that i had to initiate connection with stock exchange ? Say, User is interested in Apple, how that's being communicated to dispatch server ?
Great video! I got a question: what kind of database system is being used by the likes of Robinhood? You mentioned horizontal scaling in the video, does that mean a no-SQL like mongoDB?
It depends on the usage and has been discussed in the course in greater detail. In short, if you start new, you should choose a newSql database like Google spanner or cockroachdb etc.
Nice video... I would definitely choose kafka to implement pub-sub model . the dispatch servers would publish onto a topic (may be topic per stocK with compaction enabled as we always interested in the most recent price for a given stock.) and the FE can subscribe/listen to those topics for updates. I have very basic idea on web sockets, SSE, with my limited knowledge, it might be better option go for web sockets so that you can enable two way communication if required in future unless there are any major concerns like too much cost in implementing with web sockets compared to SSE.. or other approach is, for this use case, we can go for SSE since it's mostly one way communication and can have a adapter kind of pattern to minimize the change in case if we want switch to web sockets in future...
I think the Global cache is better in this system because the global cache has a piece of information that the frontend required the updated stock so, it will just provide that info to the particular Fronted server, not the entire Frontend server. I also have a question, how to manage the lookup table if the load balancer is responsible for the request, means if the C1 redirect to the F1 and in next call the load balancer redirect it to the F2 then how the F2 knows about the Lookup table of C1
These are websockets or SSE requests. Once they are established, the same SSE or websocket connection is used to send all the different stock updates to the client app. Only when a connection drops, then the new connection request will be initiated that could be forwarded by the load balancer to a different front-end server.
One issue with using diff. client and dispatch application servers is that we are introducing an extra hop in a real-time system + request to figure out the client server. Instead use a partition method based on stock type s.t. one machine handles load for not so popular stocks and dedicated machine per popular stocks like MSFT/APPL etc. The other issue is regarding HA. In a real time stock update system I wouldn't go for HA rather Strong consistency because of transactional requests related decisions being involved i.e. a user may want to buy or sell a stock.
I see one probable loop hole in the design. Front end servers can't listen to Kafka, they have to poll it. Kafka exhibit poll based mechanism at the consumer end. And when they poll the message from queue then firstly they should be aware that to whom all users they need to send notifications and secondly each one of them should be aware of user connection with Front end server. I clearly see 1 layer of app missing from the system. The design discussed may work but i doubt that it can scale.
thanks for the comment. If you watch the video in detail, you will find that there is lookup table in each frontend host (which is holding a SSE connection and also at the same time there are kafka consumer threads that are polling kafka). Every time the host read a record from kafka, it checks the lookup table to find how many connections are interested in that particular stock, so that the stock update can be forwarded to them.
@ThinkSoftware The moment we add Kafka or any pub-sub model, the definition of real time (SLA) must be clarified with the interviewer. There will be some delay depending on the request load, which could result in small difference in the buy/sell functions.
Hi - thanks for your videos. They're very interesting. With regard to this one - I didn't follow how the client got updated from the 'frontend servers' as if they were using HTTP to request the stocks, the frontend servers wouldn't be able to return updates unless the client was constantly polling the frontend servers. Thanks again.
Excellent video and liked the way you evolved the solution. I do have one question about the final design. In the absence of cache where FE registers the tickers to monitor, how do dispatchers know which tickers to monitor? Is there a way to communicate via kafka? Or is it all tickers are monitored and published to the queue?
Thanks for the comment. Multiple stocks can be mapped to different topics (eg via hash) and then listeners can listen to topics based on stocks they are interested.
Hi, No doubt information is useful but I have one question. In India we have stock broker like zerodha, upstock have watch list. There we can select no of stocks for what we want the updates. Here If we get the updates from exchange and put it into a kafka queue ( multiple partitions for high throughput) then from there how we will send the updates in the UI ? from frontend service to client again SSE or client will send request every 1 sec ? If it 1 sec then front server will be bombarded with lot of requests.
In this case, if the consumer is slow, the end user will receive a price with a delay. However, in that case the service is supposed to have alarms and matrics on the consumption delay to make sure proper mitigation is done if a consumer is lagging.
how can we decide what to use for Stock Exchange Service? It must already be implemented by some trading or government companies? Then they might be using their own protocol to interface with some mainframe or something right? Never worked a real stock exchange service, but I remember hearing that the real source of truth is usually in some of the mainframe systems like AS400 etc.
Thanks for the comment. This is I already discussed in the video that Robinhood backend cannot decide what mechanism the API provided by the Stock Exchange Service will use. It is the Stock Exchange Service that will provide the type of API. Now then I discussed what are different possibilities of those APIs and then decided to use one of them. The architecture will remain intact regardless of which mechanism you will use for that API. Now regarding, some AS400 server in stock exchange running all the stock data and source of truth, it still need to provide this data to all these online stock brokers and apps. Those apps cannot get that data directly accessing those mainframe computers. They can only call an API to get that data.
Kafka approach is better as it is more decoupled. Also I have a question. In case of a deployment on front end app servers, the connection with the client will be closed and it's local lookup ll be wiped out. How do we handle that?
Thanks for the comment. That's the similar question I have left for you guys right now in then that what will happen if a front-end server dies etc. However, it is not an issue for a periodic maintenance and if you think about it a bit more you should get it. I will let you guys figure out 🙂
I think once a server dies, all the clients connected to that server will not getting any new data. Either client side can re-initialize or user can refreshed the page, then the client can start a new connection and a new server will hold the connection.
I see one problem with SSE. It will notify all the clients one by one which is a problem for any exchange - it can not choose to notify one client first over any other client. I think a pub sub queue will be the right mechanism to notify all the clients from the exchange. Please let me know if I am thinking right or wrong.
@@ThinkSoftware yes, exchange is just responsible to pub the data to the queue. It's the clients responsibility to sub to the queue, and read it from there while market is open.
1. Global cache is a single point of failure. How can this be avoided? 2. Front end app server registers with global cache and updates itself, how can this be implemented, any frameworks that support achieving this?
How come the global/distributed cache be single point of failure? Have you seen my video on designing distributed cache? Only if you designing global cache in such a way that you store only in a single instance then it could be but not the way I designed in my video. I don't know about any framework because in my experience it was explicit. Also you should see the complete video because the final design does not use the cache.
Thank you for the nice informative videos. i have a request to you, if you can make a system design video on Forex Trading platform where the rates are refreshed every second, where the trades are booked and processed instantly like buy and sell based on the chart of the rates.
If a frontend server goes down, mobile/web clients (perhaps using a heartbeat msg) can detect this and re-issue a request for the stocks they're interested in. The load balancer should be able to support these extra requests following the existing load balancing strategy. Depending on scalability and availability requirements, further changes could me made to adapt the design.
@@ThinkSoftware If we create client that has 2 timeout value. connection timeout and connection retry timeout with min and max. Client randomly choose timeout number between min and max value. Say between 10-30 seconds. So we reduce the number of reconnection reaching LB at the same time. Also, client should have exponential backoff for reconnection, that helps in case many servers dies at the same time.
Here we are already using the queue between our backend servers and front-end servers. So are you suggesting the external customers/clients to call Kafka endpoints directly i.e. we expose our internal queue to the external customers? Those calls are http polling then BTW.
@@ThinkSoftware got it...we could spend more time on the final solution. Like retention to use, consumer groups needed, no of partitions on the topic, the partition key choice etc. Final solution was less than a minute
@@ThinkSoftware Not sure I follow your comment on why we should not use Kafka for communication between external systems Interested clients like Robinhood, Fidelity firms can subscribe to stock exchange's read only updates of orders, prices etc by subscribing to the topics. These clients can pull the data at their own pace and relay the updates to interested Robinhood/Fidelity users who are subscribing to updates for specific stocks. SSE would not scale given the volume of transactions, frequency of updates and the number of different stock brokers connecting to the stock exchange server. I think SSE is more practical for Robinhood client -> Robinhood backend that give clients updates in near real time and scalability challenges can be confined to all the Robinhood users subscribed that is more manageable Thoughts?
@@ThinkSoftware There will be no module in between our backend and stock exchange services? also what if different exchanges uses different communication protocols. I have one more question how can we setup an alert service let say a user will buy AAPL if it goes below $70 and sell if it goes above $300 ?
@@ThinkSoftware one more question, how can we maintain fairness for every client trying to buy or sell stock so that each of them are getting updates on prices simultaneously
It is a big topic to explain in comments. You should check for asking and bidding prices and matching algorithms. Fairness is maintained by randomness in assigning the sellers to buyers. Then there is a difference in market and limit price matches.
> if dispatch server 1 is listening to FB updates there's no need for other servers to listen to FB updates What if dispatch server 1 is unavailalbe, then we won't have FB updates. How does this guarantee availability?
It doesn't make sense from the very beginning. Robinhood should get price updates for all the stocks that can be traded on Robinhood. It's not like if nobody subscribes to the price of META, then Robinhood doesn't know the current price of META. Secondly, you don't overflow your exchange by one connection per stock name. Also I believe saying using Kafka is fine but you really need to explain how it works.
You not talk about the fault tolerance and performance. If your dispatch server down or Kafka broker down then what will happen. Consume queue also take time and stock market is work on real time. Your solution is not good in real cases.
Thanks for the comment. I have left some stuff for future videos :). Yes it is possible that a dispatch server can be down or a Kafka broker can be down but then there are ways to minimize the effect of it which would result in few clients getting affected for a small period of time. I have a chapter in my course about distributed queue design in which I have discussed what things are done to avoid the scenario of a broker node going down that you have mentioned. Similarly if a dispatch server is down there are ways to fix it. One way is redundancy :)
Thanks for the comment. Even in that case you won't be able to have access directly to the pub-sub queue in stock exchange service (this service being an external service to your robinhood service) and you may need to invoke an API to get the records.
Please let me know in the comments below if you find this video useful. Also please don't forget to like this video and subscribe to the channel.
0p
I was clueless about System design & design patterns, but With help of your videos, I've cracked Microsoft.
Thanks for the comment and congratulations 🙂. Please keep liking and commenting 😁
Nice video, thanks! Usually, stock exchange APIs are based on the FIX protocol, which is implemented on top of the TCP transport. But if I could pick one, I would go with gRPC, specifically with a server stream. It is easy to use and implement (you can automatically generate the client for most languages). It is a contract-based protocol. It performs very well, providing a good throughput with very low latency.
Thanks for the comment
Adding a queue instead of cache is a very elegant and appropriate design. Thanks for the video
Thanks for the comment :)
Doubt, when we're using queues, how dispatch server knows what are all stocks that i had to initiate connection with stock exchange ? Say, User is interested in Apple, how that's being communicated to dispatch server ?
Thanks for the detailed video.
1. Assuming we have 5000 symbols (not including commodity or options/future etc), instead of checking for each symbol if they are subscribed by the user, i would use stock groups as defined by stock server like Nifty 50, nifty bank etc. and only check against that group while adding/reading to stock server. For Tier1 groups, Nifty50 stock group, send notification without checking whether individual stock is subscribed or not. Though stock can also move between these groups as well with time, not considering at this moment.
2. If FE goes down, then we can have multiple backup servers. Using consistent hashing and using multiple virtual servers, we do the reassignment of FE servers from global cache or kafka.
3. I would use websockets (bidirectional) and kafka. We can have multiple topics in kafka and bind them using key as stock group and not each symbol.
Thanks for the comment
Thank you sir, this video got me an offer in coinbase 🙌 Keep doing the amazing work.
Great video. I would have picked web-sockets and Kafka. If the FE server dies, the client can just reconnect to another FE.
I don't really see many benefits of SSE since it cost almost the same as web-sockets, but if we wanted to add a bidirectional feature in the future, the ability is there for web-sockets. For example, the connection is bad and they want to reduce the rates of updates, maybe the client can request the server drops the updates by half.
Thanks for the comment. Yes, your reasoning is correct and if you check the existing APIs, most are websocket based. Also if you check the websocket approach picture in this video that I showed, it clearly shows that the client an actually send request on the websocket to change the list of stocks they want to listen to (I didn't discuss this in the video though). However, I want my viewer to be in the habit of questioning and thinking, so although I have chosen SSE, but asked the viewers what they will choose and why.
Say the client executed some transaction. The FE server dies before completing this transaction. If the client just reconnects to another FE server, isn't this transaction just lost and this would cause a problem?
If the transaction has not committed yet, there is no issue. If it is committed then there is a database entry for it.
Love it. You discribe it so easy for understanding and design is so nice and flexible. Great respect. 👍
Thanks for the comment :)
I appeared for an interview yesterday and the same question was asked to me to develop a widget to show real time stock price updates. Wish i had watched your video before. Having worked mostly on Healthcare domain projects in my career i wasnt able to fully answer it. Thanks for the video Sir
Thanks for the comment and good luck for your interviews.
Wonderful video and approach.
awesome.... so much to learn from a single video
Thanks for the comment 😊
it's been very helpful for me, thank you very much!!!
Welcome 🙂
Beautiful explanation
Excellent Video. Your explanation is crisp and clear
Thanks for the comment 🙂
I think rather than App server having a mapping of stock to list of subscribed customer, this information can be offloaded to a key-value store(or any other DB for that matter). We can also introduce a cache for this if Db is slow.
This way we can make App servers stateless and hence more fault-tolerant(any App server can process any stock, so there is no single point of failure). This would also make the system more available.
Also, to make Dispatch server fault tolerant, rather than on dispatch server listening to certain stock, we can have an ensemble of server and exposing them via load balancer. In the design shared in the video, we would have 2 such ensembles.
Please share your thoughts.
PS : Great content and very well explained
Thanks for the comment. The reply could become big. I would try to keep it short.
First of all, there is no point offloading the list of customers connected to an app server to a key-value store or a distributed cache (i.e. on a different machine). Because if that app server dies, so as all the SSE connections that it is hosting.
Secondly, regarding using an ensemble of servers (for dispatch servers), I didn't what you suggesting here? Can you please elaborate? Ideally we would have atleast two machines (preferably three) listening to a particular stock in dispatch servers. So if you are suggesting we have two sets of machines in dispatch servers listening to each stock then that is fine. But I don't think we need to have them behind a load balancer. They are just going to publish the stock values to distributed queues.
Could you clarify the need of lookup table ? Is connection between clients and frontend are websockets ?
nice Video, but one question here is whenever there is any change for a stock at Stock exchange server it must come to get updated in Kafka, then why do we need to make so many SSE, let Stock Exchange Server publish the change to any stock to Kafka queue instead of making SSE from dispatch server request?
Stock exchange server is an external service. Usually we use APIs to communicate with external systems.
I think partion tolerance should be handled on both front end and dispatch servers with health monitoring system. On front end clients should reconnect. Dispatch side, We should maintain cache server -> stock mapping, and reconnect to stock exchange when dispatch server dies.
With a queue, it is enabling a pushing mode for latest price, so user can get know the price sooner, which could be faster than the cache method. Then I would support a websocket method from the begining from client side. The data is refreshed all the time if got connected.
Thanks for the comment
I worked on learning java language. This is understandable. Now I’m building with pluto io as it is much more easier for me and faster to test automations without spending a ton of time. I also tried cube, etc. with subjective gains.
thanks for the comment but I didn't get the context of it...
great work
Thanks for the comment
If one (or more) of the frontend (FE) servers crashes, the disconnected clients would have to re-establish connectivity with the FE service through the LB, so that the newly formed connections are all distributed as others have pointed out. We'd also need to persist the inverted mapping {client_ids - list_of_stockTickerSymbols} in a database/distributed disk-backed in-memory cache so that in the event of an FE crash, the affected clients that then get reconnected to a subset of the rest of the alive FE servers would have a seamless experience (The FEs would pull the mapping from the persistent store and merge it with the local mapping of stock_ticker -> client_ids). Basically the suggestion is to maintain a persistent user account with the list of stock tickers the user is interested in.
Thanks for the comment
@@ThinkSoftware no problem. Great video, by the way. I really loved the way you incrementally expanded the complexity of the problem, suggesting multiple approaches at every stage with careful analysis of pros and cons while also motivating the viewers to engage in further thinking.
Really good video. Probably a dumb question though: the frontend app server => client connection I understand using SSE, but how does the Stock Exchange => backend app server work? I'm guessing I'm being limited by my PHP experience here, but I can't say I'm familiar with PHP being a client for Server Sent Events.
EDIT: Nevermind. I guess there are libraries that allow you to do this
Thanks for the comment. Yes there are different libraries in different languages/platforms that provide such support. Everything is built on top of socket APIs ultimately.
This is at a very high level with lot of pitfalls and Gaps.
Common usecase i see is to help others understand how would you get the list of user to send out notifications with Million users subscribed to a given topic? Will you end up querying those users which is very non-optimal. Will you get them in batches , that means processing them paginated way, but this has to happen for multiple events and these events are coming at a high rate. Think about those cases
very helpful
Doubt, when we're using queues, how dispatch server knows what are all stocks that i had to initiate connection with stock exchange ? Say, User is interested in Apple, how that's being communicated to dispatch server ?
Great video!
I got a question: what kind of database system is being used by the likes of Robinhood? You mentioned horizontal scaling in the video, does that mean a no-SQL like mongoDB?
It depends on the usage and has been discussed in the course in greater detail. In short, if you start new, you should choose a newSql database like Google spanner or cockroachdb etc.
@@ThinkSoftware thanks for responding..I'm currently trying out CockroachDB free-tier.
Nice video... I would definitely choose kafka to implement pub-sub model . the dispatch servers would publish onto a topic (may be topic per stocK with compaction enabled as we always interested in the most recent price for a given stock.) and the FE can subscribe/listen to those topics for updates. I have very basic idea on web sockets, SSE, with my limited knowledge, it might be better option go for web sockets so that you can enable two way communication if required in future unless there are any major concerns like too much cost in implementing with web sockets compared to SSE.. or other approach is, for this use case, we can go for SSE since it's mostly one way communication and can have a adapter kind of pattern to minimize the change in case if we want switch to web sockets in future...
Thanks for the comment :)
wouldn't a topic per stock result in 5k+ topics?
Why don't you use a database to write the updates from exchange. The front end servers may be stateless and just query (point read) from database.
There are only 65,410 ports available for general use. You don't mention this when outlining the system design.
I think the Global cache is better in this system because the global cache has a piece of information that the frontend required the updated stock so, it will just provide that info to the particular Fronted server, not the entire Frontend server. I also have a question, how to manage the lookup table if the load balancer is responsible for the request, means if the C1 redirect to the F1 and in next call the load balancer redirect it to the F2 then how the F2 knows about the Lookup table of C1
These are websockets or SSE requests. Once they are established, the same SSE or websocket connection is used to send all the different stock updates to the client app. Only when a connection drops, then the new connection request will be initiated that could be forwarded by the load balancer to a different front-end server.
Great video!
Thanks for the comment 😊
One issue with using diff. client and dispatch application servers is that we are introducing an extra hop in a real-time system + request to figure out the client server. Instead use a partition method based on stock type s.t. one machine handles load for not so popular stocks and dedicated machine per popular stocks like MSFT/APPL etc.
The other issue is regarding HA. In a real time stock update system I wouldn't go for HA rather Strong consistency because of transactional requests related decisions being involved i.e. a user may want to buy or sell a stock.
Thanks for the comment
Well done.
I see one probable loop hole in the design.
Front end servers can't listen to Kafka, they have to poll it.
Kafka exhibit poll based mechanism at the consumer end. And when they poll the message from queue then firstly they should be aware that to whom all users they need to send notifications and secondly each one of them should be aware of user connection with Front end server.
I clearly see 1 layer of app missing from the system.
The design discussed may work but i doubt that it can scale.
thanks for the comment. If you watch the video in detail, you will find that there is lookup table in each frontend host (which is holding a SSE connection and also at the same time there are kafka consumer threads that are polling kafka). Every time the host read a record from kafka, it checks the lookup table to find how many connections are interested in that particular stock, so that the stock update can be forwarded to them.
would it be a push model from kafka? i thought kafka will push the events to the consumer, instead of consumers pulling them
@ThinkSoftware The moment we add Kafka or any pub-sub model, the definition of real time (SLA) must be clarified with the interviewer. There will be some delay depending on the request load, which could result in small difference in the buy/sell functions.
Hi - thanks for your videos. They're very interesting. With regard to this one - I didn't follow how the client got updated from the 'frontend servers' as if they were using HTTP to request the stocks, the frontend servers wouldn't be able to return updates unless the client was constantly polling the frontend servers. Thanks again.
Thanks for the comment. Client side will also use SSE as discussed in the video.
Absolutely useful!
Thanks for the comment 🙂
Excellent video and liked the way you evolved the solution. I do have one question about the final design. In the absence of cache where FE registers the tickers to monitor, how do dispatchers know which tickers to monitor? Is there a way to communicate via kafka? Or is it all tickers are monitored and published to the queue?
Thanks for the comment. Multiple stocks can be mapped to different topics (eg via hash) and then listeners can listen to topics based on stocks they are interested.
like like you are great !!!
How would you design a system where users can request for quotes from a service personnel?
Hi,
No doubt information is useful but I have one question.
In India we have stock broker like zerodha, upstock have watch list. There we can select no of stocks for what we want the updates. Here If we get the updates from exchange and put it into a kafka queue ( multiple partitions for high throughput) then from there how we will send the updates in the UI ? from frontend service to client again SSE or client will send request every 1 sec ? If it 1 sec then front server will be bombarded with lot of requests.
This is discussed in the video SSE
Please add a video on auction/bidding system design
Noted.
WOW! This is a good arch! thanks, it is helping me
Thanks for the comment :)
Amazing
Thanks for the comment 😊
what if consumer is slow, share price of apple will be old in this case which pass to user?
In this case, if the consumer is slow, the end user will receive a price with a delay. However, in that case the service is supposed to have alarms and matrics on the consumption delay to make sure proper mitigation is done if a consumer is lagging.
how can we decide what to use for Stock Exchange Service? It must already be implemented by some trading or government companies? Then they might be using their own protocol to interface with some mainframe or something right? Never worked a real stock exchange service, but I remember hearing that the real source of truth is usually in some of the mainframe systems like AS400 etc.
Thanks for the comment. This is I already discussed in the video that Robinhood backend cannot decide what mechanism the API provided by the Stock Exchange Service will use. It is the Stock Exchange Service that will provide the type of API. Now then I discussed what are different possibilities of those APIs and then decided to use one of them. The architecture will remain intact regardless of which mechanism you will use for that API. Now regarding, some AS400 server in stock exchange running all the stock data and source of truth, it still need to provide this data to all these online stock brokers and apps. Those apps cannot get that data directly accessing those mainframe computers. They can only call an API to get that data.
Kafka approach is better as it is more decoupled.
Also I have a question. In case of a deployment on front end app servers, the connection with the client will be closed and it's local lookup ll be wiped out. How do we handle that?
Thanks for the comment. That's the similar question I have left for you guys right now in then that what will happen if a front-end server dies etc. However, it is not an issue for a periodic maintenance and if you think about it a bit more you should get it. I will let you guys figure out 🙂
@@ThinkSoftware you are awesome. Thanks for being there.
awesome!
Thanks 🙂
Thank you for the nice informative videos. i have a request to you, if you can make a system design video on Forex Trading platform
Thanks for the comment
I think once a server dies, all the clients connected to that server will not getting any new data. Either client side can re-initialize or user can refreshed the page, then the client can start a new connection and a new server will hold the connection.
Yes
I see one problem with SSE. It will notify all the clients one by one which is a problem for any exchange - it can not choose to notify one client first over any other client. I think a pub sub queue will be the right mechanism to notify all the clients from the exchange. Please let me know if I am thinking right or wrong.
How the pubsub model will be used with the client apps? Are you suggesting the client app calls (basically http poll) to read data from queue?
@@ThinkSoftware yes, exchange is just responsible to pub the data to the queue. It's the clients responsibility to sub to the queue, and read it from there while market is open.
1. Global cache is a single point of failure. How can this be avoided? 2. Front end app server registers with global cache and updates itself, how can this be implemented, any frameworks that support achieving this?
How come the global/distributed cache be single point of failure? Have you seen my video on designing distributed cache? Only if you designing global cache in such a way that you store only in a single instance then it could be but not the way I designed in my video. I don't know about any framework because in my experience it was explicit. Also you should see the complete video because the final design does not use the cache.
Thank you for the nice informative videos. i have a request to you, if you can make a system design video on Forex Trading platform where the rates are refreshed every second, where the trades are booked and processed instantly like buy and sell based on the chart of the rates.
If a frontend server goes down, mobile/web clients (perhaps using a heartbeat msg) can detect this and re-issue a request for the stocks they're interested in. The load balancer should be able to support these extra requests following the existing load balancing strategy. Depending on scalability and availability requirements, further changes could me made to adapt the design.
Thanks for the comment.. what if if the LB is unable to support the surge of 90K/100K requests coming at the same time?
@@ThinkSoftware how about adaptive reconnect timeout at client side to avoid thundering herd?
Can you elaborate?
@@ThinkSoftware If we create client that has 2 timeout value. connection timeout and connection retry timeout with min and max. Client randomly choose timeout number between min and max value. Say between 10-30 seconds. So we reduce the number of reconnection reaching LB at the same time. Also, client should have exponential backoff for reconnection, that helps in case many servers dies at the same time.
@@1986kunjan yes that is correct.
Looks like the presenter is one of my coworkers!, surprising !
A better approach would be to use a durable and distributed messaging system like kafka for such sourxe sink problems
Here we are already using the queue between our backend servers and front-end servers.
So are you suggesting the external customers/clients to call Kafka endpoints directly i.e. we expose our internal queue to the external customers? Those calls are http polling then BTW.
@@ThinkSoftware no. i meant between your servers.
@@balajipattabhiraman This is the final design if you check the video till end.
@@ThinkSoftware got it...we could spend more time on the final solution. Like retention to use, consumer groups needed, no of partitions on the topic, the partition key choice etc. Final solution was less than a minute
@@ThinkSoftware Not sure I follow your comment on why we should not use Kafka for communication between external systems
Interested clients like Robinhood, Fidelity firms can subscribe to stock exchange's read only updates of orders, prices etc by subscribing to the topics. These clients can pull the data at their own pace and relay the updates to interested Robinhood/Fidelity users who are subscribing to updates for specific stocks.
SSE would not scale given the volume of transactions, frequency of updates and the number of different stock brokers connecting to the stock exchange server.
I think SSE is more practical for Robinhood client -> Robinhood backend that give clients updates in near real time and scalability challenges can be confined to all the Robinhood users subscribed that is more manageable
Thoughts?
How can we engage multiple stock exchanges here?
The same way you engage with one.
@@ThinkSoftware There will be no module in between our backend and stock exchange services? also what if different exchanges uses different communication protocols.
I have one more question how can we setup an alert service let say a user will buy AAPL if it goes below $70 and sell if it goes above $300 ?
@@ThinkSoftware one more question, how can we maintain fairness for every client trying to buy or sell stock so that each of them are getting updates on prices simultaneously
It is a big topic to explain in comments. You should check for asking and bidding prices and matching algorithms. Fairness is maintained by randomness in assigning the sellers to buyers. Then there is a difference in market and limit price matches.
isn't robinhood a broker instead of an exchange?
Yes it is broker.
> if dispatch server 1 is listening to FB updates there's no need for other servers to listen to FB updates
What if dispatch server 1 is unavailalbe, then we won't have FB updates. How does this guarantee availability?
This is handled via redundancy and lease mechanism. A topic for future video.
@@ThinkSoftware Thanks. I really like your way of iterating each different designs. The rationale for the design is very clear as well.
Thanks
It doesn't make sense from the very beginning. Robinhood should get price updates for all the stocks that can be traded on Robinhood. It's not like if nobody subscribes to the price of META, then Robinhood doesn't know the current price of META. Secondly, you don't overflow your exchange by one connection per stock name. Also I believe saying using Kafka is fine but you really need to explain how it works.
You not talk about the fault tolerance and performance. If your dispatch server down or Kafka broker down then what will happen. Consume queue also take time and stock market is work on real time. Your solution is not good in real cases.
Thanks for the comment. I have left some stuff for future videos :). Yes it is possible that a dispatch server can be down or a Kafka broker can be down but then there are ways to minimize the effect of it which would result in few clients getting affected for a small period of time. I have a chapter in my course about distributed queue design in which I have discussed what things are done to avoid the scenario of a broker node going down that you have mentioned. Similarly if a dispatch server is down there are ways to fix it. One way is redundancy :)
I would use pub sub to have stock exchange service to send to the robinhood backend
Thanks for the comment. Even in that case you won't be able to have access directly to the pub-sub queue in stock exchange service (this service being an external service to your robinhood service) and you may need to invoke an API to get the records.
Websocket