The purpose of delayed task is to execute some task in the the future

Common solutions:

  • DelayQueue
  • ScheduledExecutorService
  • zset in redis with activation time as key, need to check if deletion is successful to see if it is consumed multiple times
  • Use key space notification (Default is off in redis)
  • Timer wheel-based implementation

What if we can have up to 1 B tasks to be run the future

For example, we want to send different notifications to user 1 day after performing different types of actions. How do you design this?

Assume we have 100 mil users, each user can get up to 10 different delayed tasks per day, and we have to execute the task within 12 hours of the scheduled execution time

Then we have to persist with (virtual shard, timestamp, seq id) as the key prefix, and use timestamp as the checkpoint offset