NimbusKV Engineering Journal

Substantive technical articles on concurrency internals, mutex design, free-threaded CPython optimization, and memory reclamation.

Concurrency Architecture

October 11, 2026

6 min read

Hemanshu Vaidya

Eliminating CAS Retry Collapse: Why NimbusKV 3.1 Moved to a Single Short Mutex

In theory, lock-free compare-and-swap (CAS) loops sound like the gold standard for concurrency. In practice, under high-thread write contention on a hot key, CAS creates a devastating retry collapse. Here is how and why we fixed it.

The Illusion of Optimistic CAS

In LiveDict and NimbusKV v3.0.1, the mutate() implementation followed the classic optimistic concurrency pattern: 1. Load current root pointer. 2. Run user mutation callback outside any lock. 3. Construct new HAMT path. 4. Try to atomically swap the root pointer using compare-and-swap. 5. If another thread swapped first, discard work and retry from Step 1.

Under 1 to 4 threads, this worked reasonably well. But when we tested under 16 concurrent threads hammering a hot key, throughput collapsed catastrophically from 439K ops/s down to 55K ops/s. 15 losing threads were constantly throwing their work away, burning CPU cycles in tight loops without making forward progress.

The Solution: One Short Mutex

In NimbusKV v3.1.0, we replaced the retry loop with a single deterministic mutex for writes: - Readers never touch the mutex; they remain 100% lock-free. - Each write acquires the lock once, performs path-copying, updates the root, and releases the lock. - User callbacks run exactly once per mutation.

The results from b06_write_path.py on a Linux sandbox were immediate: - set() throughput on standard Python 3.13 increased by +48% (from 328K to 485K ops/s). - set() throughput on free-threaded Python 3.14t increased by +157% (from 188K to 483K ops/s). - Hot-key contention remained stable instead of collapsing.

Python Internals

October 10, 2026

5 min read

Hemanshu Vaidya

Benchmarking Free-Threaded Python: Running NimbusKV with Zero GIL on 3.13t and 3.14t

Free-threaded Python (PEP 703) enables true multi-core parallel execution in CPython. But third-party C-extensions can silently re-enable the GIL. Here is what we discovered while optimizing NimbusKV for 3.13t and 3.14t.

The Hidden Trap: Extension Modules Re-enabling the GIL

When CPython 3.13t starts with a free-threaded build, the GIL is disabled by default. However, Python has a safety mechanism: if any imported C-extension does not explicitly declare support for free-threading via Py_MOD_PER_INTERPRETER_GIL_SUPPORTED, CPython silently turns the GIL back on for the entire process.

In NimbusKV, we rely on the immutables package for persistent HAMT structures. Our CI test initially verified that sys._is_gil_enabled() was False before importing. After importing immutables, the GIL was suddenly enabled again!

The Fix: PYTHON_GIL=0

By configuring the environment variable PYTHON_GIL=0, CPython forces the GIL to remain disabled even when importing extensions that have not yet updated their declaration flags.

With true zero GIL in effect, our b06 write path benchmark showed NimbusKV hitting 483K ops/s on 3.14t, demonstrating genuine concurrent throughput without GIL bottlenecks.

Memory Management

October 9, 2026

4 min read

Hemanshu Vaidya

Generational Priority Queues: Fixing the TTL Scheduler Memory Leak

How an innocent-looking background priority queue scheduler leaked memory on repeated key overwrites, and how generational tracking solved it.

The Ghost in the Priority Queue

In a high-throughput key-value store, keys with Time-To-Live (TTL) deadlines are typically tracked using a min-heap priority queue. When key "sess:42" is set with a 10-second TTL, an entry is pushed to the heap with timestamp T + 10.

Now suppose a busy client updates "sess:42" 1,000 times before the 10 seconds expire. In older versions, each update pushed a new heap entry while the old entry remained orphaned in the heap. Over days of operation, millions of dead heap entries accumulated, causing memory footprint to balloon.

Generational Validation on Inspection

In v3.1.0, we overhauled the scheduler: 1. Every key maintains an integer generation counter that increments on every write. 2. Heap entries store { key, expire_at, generation }. 3. When the scheduler thread wakes up, it checks if the key's current store generation matches the heap entry generation. 4. If the generation is mismatched (meaning the key was rewritten or deleted), the entry is discarded instantly. 5. Generation records only exist in memory while a TTL is actively pending.

This ensures zero memory leaks regardless of write frequency.