boost · advanced

Boost.Asio: Scaling io_context Across a Thread Pool

To exploit multiple cores in Asio you have several threads call run() on the same io_context, so ready handlers are dispatched across all of them — a thread-pooled event loop that parallelizes handler execution while sharing a single queue. That concurrency, however, reintroduces races on shared and per-connection state, which is exactly what strands solve: binding each connection's handlers to its own strand serializes them while different connections run on different pool threads, making strands-plus-a-thread-pool the standard scalable Asio design. Size the pool around hardware_concurrency for CPU-bound handlers and keep every handler short and non-blocking, offloading any blocking or heavy computation to a separate pool so an I/O thread is never stalled. A popular high-performance alternative, common in HFT systems, is the thread-per-core model: give each core its own io_context in a shared-nothing arrangement, pin the thread to the core, and avoid cross-thread synchronization altogether.

The code

boost::asio::io_context io;
std::vector<std::thread> pool;
for (int i = 0; i < std::thread::hardware_concurrency(); ++i)
    pool.emplace_back([&]{ io.run(); });   // N threads share one context
// per-connection strands keep each connection's handlers serialized
for (auto& t : pool) t.join();

What this lesson walks through

1. 01One context, N threads on run()
2. 02Now you need strands
3. 03Sizing & the thread-per-core alternative

To use all your cores, have several threads call run() on the SAME io_context. Ready handlers are then dispatched across those threads — a thread-pooled event loop that parallelizes handler work while still sharing one queue.