When working with legacy or rigid codebases, performance bottlenecks can emerge from designs you can’t easily change—like interfaces that force inefficient map access by index. This article explores how a simple thread_local cache can dramatically improve performance in such cases, reducing repeated lookups from quadratic to near-constant time.

Speeding up C++ functions with a thread_local cache

by Daniel Lemire

From the article:

In large code bases, we are often stuck with unpleasant designs that are harming our performance. We might be looking for a non-intrusive method to improve the performance. For example, you may not want to change the function signatures.

Let us consider a concrete example. Maybe someone designed the programming interface so that you have to access the values from a map using an index.