sockets · advanced

Sockets: Blocking, Non-blocking & epoll

By default a socket is blocking: recv() sleeps until data arrives, send() until buffer space frees, accept() until a client connects — simple, but it parks an entire thread per connection. Setting O_NONBLOCK makes these calls return immediately, yielding -1 with errno EAGAIN/EWOULDBLOCK when there is nothing to do (which must not be treated as an error), and this readiness model is the foundation of event loops. I/O multiplexing lets one thread watch many descriptors and act only on the ready ones: select() and poll() are portable but rescan the entire descriptor set on every call (O(n), with select additionally capped at FD_SETSIZE ~1024), whereas epoll on Linux (kqueue on BSD/macOS, IOCP on Windows) registers descriptors once and returns only those that are ready (O(number ready)), scaling to tens of thousands of connections — the 'C10k' solution behind servers like nginx and Redis. epoll offers two notification modes: level-triggered (the default, like poll) keeps reporting a descriptor while data remains and is forgiving, while edge-triggered (EPOLLET) reports only on a state change, so the socket must be drained in a loop until EAGAIN on a non-blocking fd or data is lost — faster but unforgiving. The canonical design is the reactor pattern: non-blocking sockets plus an epoll loop that dispatches read/write events to handlers, run single-threaded or as a pool of event loops.

The code

// Blocking (default): recv() sleeps until data arrives
n = recv(fd, buf, len, 0);          // one client at a time per thread

// Non-blocking: returns immediately; EAGAIN means 'nothing yet'
fcntl(fd, F_SETFL, O_NONBLOCK);

// Readiness notification — watch MANY fds with one thread:
select(...);  poll(...);            // O(n) scan each call
epoll_wait(epfd, events, ...);      // O(ready), scales to 100k fds

What this lesson walks through

1. 01Blocking — one client per thread
2. 02epoll — register many fds, wait once
3. 03Only the READY fds come back — O(ready)
4. 04Gotcha — edge-triggered must drain to EAGAIN

By default recv() SLEEPS until data arrives, so one thread can serve only one connection at a time. Thousands of clients would need thousands of threads — expensive and limited.