lowlatency · advanced

Memory Pools & Arena Allocators

General-purpose allocators must serve any size, stay thread-safe, and resist fragmentation, so each new/malloc can cost hundreds of cycles, may take a global lock, and scatters objects across the heap causing cache misses and unpredictable (tail) latency — unacceptable on a hot path. The remedy is to pre-allocate and manage memory yourself with one of two workhorse designs. A fixed-size pool pre-allocates a slab of equal-size blocks linked into an intrusive free list, so alloc pops the head and free pushes it back (both O(1)) with zero fragmentation since all blocks are identical — ideal for churning one object type such as list nodes or messages. An arena (bump/monotonic) allocator simply advances a pointer per allocation and frees everything at once by resetting that pointer, which suits per-request or per-frame lifetimes where all objects are discarded together. Integrate pools through a custom Allocator, an overloaded operator new, or — cleanest in modern C++ — std::pmr (C++17), where monotonic_buffer_resource is an arena and the synchronized/unsynchronized pool_resources are pools usable by pmr containers. Respect alignment when returning blocks, keep pools thread-local or per-core to avoid lock contention, and remember the pitfalls: getting lifetimes wrong leaks the whole arena, and an arena reset reclaims memory but does not run destructors.

The code

// new/malloc are general-purpose: slow, lockable, fragmenting.
// Hot paths pre-allocate and hand out memory cheaply.

struct FixedPool {   // fixed-size blocks, O(1) alloc/free
    void* free_list; // intrusive singly-linked free list
    void* alloc() {
        void* p = free_list;
        free_list = *(void**)p;
        return p;
    }
    void free(void* p) {
        *(void**)p = free_list;
        free_list = p;
    }
};

// Arena/bump: ptr += size; free EVERYTHING at once (per request/frame).
// C++17: std::pmr::monotonic_buffer_resource / pool_resource

What this lesson walks through

1. 01Why not just new / malloc on the hot path?
2. 02Fixed-size pool — O(1) alloc & free
3. 03Arena / bump allocator — free everything at once
4. 04Pitfalls & C++ integration

General allocators are slow, take a global lock, and fragment the heap over time. Latency-critical code pre-allocates a pool once and hands out memory in O(1) with no lock and no fragmentation.