This change removes a bottleneck at the client/server protocol level for high concurrency workloads.
When reading a packet from a socket, the read can be performed either in non-blocking mode or in blocking mode. The non-blocking mode was originally chosen because it avoids the cost of setting up an alarm in case of a timeout: thus the first attempt to read is done in non-blocking mode, and only if it fails, the next attempts are done in blocking mode.
However, this behavior can hurt performance as the switch from non-blocking mode to blocking mode is expensive, requiring calls to the fcntl function and calls into the kernel.
The solution is to use socket timeouts, with the SO_RCVTIMEO and SO_SNDTIMEO options. This way, the timeouts are automatically handled by the operating system, which means that all reads can be done in blocking mode.
For general inquiries, please send us your question and someone will contact you.