Who is handling de seq number the producer or the application ? the sequence is incremented for a given PID continusoulsy? will work with in a multi thread rpocess? will it work well if more then 1 process use the same PID?
idempotence requires max.in.flight.requests.per.connection to be less than or equal to 5 ( to preserve the ordering of the messages ), retries to be greater than 0, and acks must be ‘all’
thanks for this video. I am currently in a project where Kafka is considered as a possible solution for Event Sourcing (not Event Processing in terms of Event Carried State Transfer or Event Notification). So the question boils down to: Can Kafka give the same hard guarantees as a flight recorder that events will be flushed directly and be sorted? I don't want to be bogged down in philosophical/religious discussions. Kafka is probably one of the best messaging systems out there, but would it also be the BEST solution for Event Sourcing? A simple yes or no would be great! :)
I suppose it can give you close to 100% guarantees (I'm not completely sure about 100%) if you are willing to make some compromises on throughput. I don't know if kafka flushes to disk before sending acknowledgements. It's possible that it might store messages in some memory mapped files and maybe data is not flushed from memory to disk right away. I suspect some delay in which...if your broker fails...you will lose some data.
you might be able to enforce those guarantees with some OS configurations (if you have the appropriate access). if anybody has more knowledge on this topic...please share
![Chino Pacas - Modo Capone (with Drake & Fuerza Regida) [Video Oficial]](http://i.ytimg.com/vi/zjv8TQ0gBkg/mqdefault.jpg)
Who is handling de seq number the producer or the application ? the sequence is incremented for a given PID continusoulsy? will work with in a multi thread rpocess? will it work well if more then 1 process use the same PID?
idempotence requires max.in.flight.requests.per.connection to be less than or equal to 5 ( to preserve the ordering of the messages ), retries to be greater than 0, and acks must be ‘all’
Informative and great visuals.
Great introduction.
Thanks jun rao
thanks for this video. I am currently in a project where Kafka is considered as a possible solution for Event Sourcing (not Event Processing in terms of Event Carried State Transfer or Event Notification). So the question boils down to: Can Kafka give the same hard guarantees as a flight recorder that events will be flushed directly and be sorted? I don't want to be bogged down in philosophical/religious discussions. Kafka is probably one of the best messaging systems out there, but would it also be the BEST solution for Event Sourcing? A simple yes or no would be great! :)
I suppose it can give you close to 100% guarantees (I'm not completely sure about 100%) if you are willing to make some compromises on throughput. I don't know if kafka flushes to disk before sending acknowledgements. It's possible that it might store messages in some memory mapped files and maybe data is not flushed from memory to disk right away. I suspect some delay in which...if your broker fails...you will lose some data.
you might be able to enforce those guarantees with some OS configurations (if you have the appropriate access). if anybody has more knowledge on this topic...please share