Design: delayed task
The purpose of delayed task is to execute some task in the the future
Common solutions:
- DelayQueue
- ScheduledExecutorService
- zset in redis with activation time as key, need to check if deletion is successful to see if it is consumed multiple times
- Use key space notification (Default is off in redis)
- Timer wheel-based implementation
What if we can have up to 1 B tasks to be run the future
For example, we want to send different notifications to user 1 day after performing different types of actions. How do you design this?
Assume we have 100 mil users, each user can get up to 10 different delayed tasks per day, and we have to execute the task within 12 hours of the scheduled execution time
Then we have to persist with (virtual shard, timestamp, seq id) as the key prefix, and use timestamp as the checkpoint offset