I'm currently working on a project in which I'm getting event time > process time; for example, in my database, the process time happened in April and my event time happened in May. How is this possible? Please help me. Or is this any kind of cyberattack?
Can there be errors when recording the event time? Maybe the events were sending the wrong timestamp to begin with? Or vice-versa, maybe the process time is recorded incorrectly.
If you are doing window-based aggregations, and your event arrives beyond the window "grace period", you will either ignore the data or need to recompute.
Good to see u back buddy !!
I will be back again haha.
have you used Kafka streams in production? If yes, what tool have you used to load test such application?
Hi! I work at a Product team, not an Infra one. So unfortunately I am not really sure what tools are used. They are abstracted away from us.
I'm currently working on a project in which I'm getting event time > process time; for example, in my database, the process time happened in April and my event time happened in May. How is this possible? Please help me. Or is this any kind of cyberattack?
Can there be errors when recording the event time? Maybe the events were sending the wrong timestamp to begin with? Or vice-versa, maybe the process time is recorded incorrectly.
Why are you over engineering
Why don’t you use timestamp to identify?
If you are doing window-based aggregations, and your event arrives beyond the window "grace period", you will either ignore the data or need to recompute.
For systems like ad aggregator, it makes sense to ignore the late data ,correct?@@irtizahafiz