Reactor pattern in solving c10k problems
One example is how kafka handles it.
- Acceptor thread accepts connections, and pass them to processor threads
- Each processor thread select() on multiple connecitons, i.e., the sync event demultiplexer in the textbook reactor pattern
- On each event, processor sends the arriving requests to the request queue.
- Pooled worker threads will fetech from the requeust queue and process it. Note implicitly, the queue and its maintainance logic acts as the dispatcher.
- Woker thread puts the response to each processor’s response queue.
- Processor thread gets result from its own response queue and answer the client