Time Scales

For roughly two decades, CPU clock rates have hovered around 3 GHz while DRAM access latency has stayed near 60—100 ns. The consequence is arithmetic: a cache miss that cost a handful of cycles on early-1990s hardware now costs several hundred. The processor did not merely get faster than memory — it got faster than memory by a growing margin, and the entire cache hierarchy exists to paper over that gap. Note in the table above that the raw device-level latency of DDR5 is barely distinguishable from DDR4; the generational improvements went into bandwidth, not into latency.

Once several cores share one memory subsystem, a single uniform path to memory becomes the bottleneck. The answer was to move the memory controller onto the CPU die and attach memory per socket — and, with chiplet designs, effectively per die group. This is what NUMA (non-uniform memory access) describes: memory that “belongs” to the core you are running on is reached quickly, while memory belonging to a neighbour costs noticeably more. The practical upshot is that thread placement and memory placement are no longer independent concerns, and that a scheduler which migrates a worker between cores may silently convert local accesses into remote ones.

Cores also keep private caches, and a store performed on one core does not become visible to another core at the moment the instruction retires: it first lands in that core’s store buffer. Cache coherence guarantees that the cores eventually agree on the content of a cache line, but it says nothing about when, nor about the order in which several independent writes become visible. This is what memory fences (memory barriers) are for — they constrain the order, and thereby allow one core to reason about what an adjacent core has already done. The cost shows up in the table twice: an uncontended fence is roughly the price of an L3 hit, whereas a cache line actually bouncing between two cores costs about as much as going to DRAM. It should be noted that the expensive part is usually not the synchronisation primitive itself, but the cache coherence traffic it provokes.

And as far as memory synchronisation is concerned, there is another twist that has already led quite some programmers astray. Intel’s x86 implements a comparatively strong coherency model, Total Store Order (TSO): of the four possible reorderings, only StoreLoad is permitted. Acquire and release semantics are therefore free on x86 — they compile to plain MOV — and only sequential consistency requires an actual instruction. Quite to the contrary, ARM and POWER are weakly ordered and consequently offer a more differentiated set of barriers: ARM distinguishes DMB from the heavier DSB, and provides graded load acquire flavours such as LDAR and LDAPR; POWER separates the heavyweight sync from the cheaper lwsync. The trap for the unwary is that C++ memory_order distinctions which cost nothing on x86 have a real price on ARM — and, worse, that under-synchronised code can appear perfectly correct on x86 and fail on ARM.

A system call is not a function call: it is a controlled trap. The syscall instruction switches the processor to privilege level 0, and the kernel entry path then swaps to the kernel GS base, switches stacks, and saves the user register state. Since the Meltdown/Spectre family of vulnerabilities, that sequence also includes a page-table switch (KPTI writes CR3, with the attendant TLB consequences) and one or more branch-prediction barriers — retpolines, IBPB or IBRS depending on hardware and configuration. On return, the whole dance runs in reverse.

The direct cost is measurable: a minimal syscall such as getpid() amounts to little more than two mode switches, and lands in the region of a hundred nanoseconds on a fast host without mitigations, rising to several hundred once mitigations are enabled [gms]. The indirect cost is harder to see and frequently larger. Kernel entry code and the syscall handler evict user-space data from L1 and L2, and disturb the branch predictors; the application resumes with a cold working set and pays for it in misses that no syscall benchmark will attribute to the syscall. A simple read() that happens to hit data already in the kernel cache can incur costs of around 1 µs.

The interesting question thus is is rarely “how expensive is one syscall” but rather “how many syscalls does this design require per unit of work” — which is the question io_uring was built to answer.

A thread that asks to sleep for a specific duration will not, in general, wake at that instant. Part of this is unavoidable: an interrupt must be handled, the sleeping task must be moved to the run queue, and the scheduler must actually place it on a CPU, which together account for something in the range of 10—30 µs [lkmlwake].

The larger part, however, is deliberate. Linux maintains a per-thread timer slack value, and the kernel is free to delay a timed wake-up by up to that amount in order to coalesce several wake-ups into one and thereby let the CPU stay in a low-power state longer. For every thread not running under a real-time scheduling policy, the default is 50 µs [slack]. This is not imprecision; it is a power-versus-precision trade that the kernel makes on the application’s behalf, and it can be adjusted — down to 1 ns — via prctl(PR_SET_TIMERSLACK) or the per-process /proc/PID/timerslack_ns interface.

The key distinction is therefore that between resolution and accuracy. Since Linux 2.6.21 the high-resolution timer subsystem reports nanosecond resolution, and clock_getres() will happily confirm it. The accuracy with which a nanosleep() actually returns is a different story altogether, two to three orders of magnitude coarser, and dominated by slack, wake-up latency and — if the core had gone idle — C-state exit time. Any design that assumes it can steer a worker thread with microsecond-scale sleeps should be checked against these figures first.

The in-text markers such as [downs] link down to the full citation here. Collected as a reading list, roughly in order of usefulness for further work.

One caveat deserves a separate mention, as it explains why the published figures on synchronisation primitives differ so drastically from one another. A hot microbenchmark keeps every thread runnable and every core out of idle by its very construction. Under those conditions an uncontended atomic operation or a Mutex acquisition costs single-digit to low-three-digit nanoseconds, and a benchmark will faithfully report that. What such a benchmark structurally cannot observe is the cost of blocking — of a thread actually parking and later being woken — because pressing the benchmark is precisely what prevents that from happening.

The instruction-level cost of a synchronisation primitive is what a hot microbenchmark measures. Its systemic cost — coherence traffic under contention, and blocking and wake-up latency when it is not contended — is what production tail latency measures. The two values can differ by two to four orders of magnitude, and it is for this very reason that the table above covers both regimes.