NimbusKV v3.1.0 Benchmarks
Verified execution metrics, thread scalability, latency, and comparative analysis.
Verified Result Registry & Provenance
Every benchmark figure on this page is classified by its provenance record to ensure developer transparency and technical integrity.
Direct raw microbenchmark executions from bench/reports.json.
Cross-version comparative baselines logged during prior tagged release runs.
Computed speedup factors (vs 1T) and memory bytes-per-key overhead ratios.
Queue-theoretic single-lock contention model based on measured service times.
Hardware & Runtime Environment
1. Correctness First
Performance is meaningless if concurrency causes data corruption or lost updates. The correctness suite verifies compound atomicity, snapshot isolation, and race condition immunity.
mset_atomicity_naive_get_get (expected gotcha) is documented as an expected race condition in non-transactional multi-key reads. Use atomic() or explicit coordination when strict multi-key linearizability is required.
2. Throughput & Latency
Measures lock-free concurrent reads vs disjoint writes and hot-key atomic updates as thread counts scale.
Interactive Workload Estimator
Configure expected read/write ratio and worker threads. Estimated throughput derives from an analytical single-lock queuing model calibrated against lock-free read timings and write mutex serialization hold times.
python -m bench.b06_write_path --mixed 80-20 for measured figures.Blended rate under 80% lock-free reads and 20% mutex writes.
Write Path Architecture Delta (v3.1 vs v3.0.1)
Benchmark b06In v3.1.0, NimbusKV eliminated the optimistic compare-and-swap retry loop, replacing it with a single deterministic 2-microsecond mutex. Measured on Linux sandbox with bench/b06_write_path.py:
Single mutex eliminates path allocation thrash under write pressure.
With PYTHON_GIL=0, writers serialize with negligible mutex hold time.
Copy-on-write notifications dispatch without holding the store write lock.
3. Scalability, Memory, TTL Scheduler & Subscriber Overhead
Assesses HAMT persistent map memory footprint overhead relative to standard Python dictionaries, and validates TTL timestamp expiry accuracy under load.
Resident Set Size (RSS) measured under Python 3.12 64-bit Linux. Shows the lean ~7% to 17% Hash Array Mapped Trie (CHAMP) metadata overhead of immutable snapshots compared to raw hash tables. Negative measurements from uncalibrated garbage collection deltas are rejected by schema validation checks.
4. Async Honesty, Persistence Backends & Breaking Points
Measures asyncio wrapper latency vs synchronous execution, and write-through persistence overhead across SQLite and Redis backends.
5. Comparative: NimbusKV vs. dict+Lock, cachetools, aiocache, shelve
Side-by-side comparison under multi-threaded load against standard Python dict + threading.Lock and cachetools.TTLCache.
Compare Two Actual Benchmark Runs
Select two verified benchmark runs to compute differential speedup, latency changes, and architectural caveats.
| Benchmark Metric | Baseline (A) | Comparison (B) | Difference (B - A) | Delta % |
|---|---|---|---|---|
| Lock-Free Reads (8 Threads)Readers never acquire mutex; scales directly with thread execution. | 7,520,000 ops/s | 7,850,000 ops/s | +330,000 ops/s | +4.4% |
| Disjoint Writes (8 Threads)Independent keys writing concurrently through write coordination. | 340,000 ops/s | 495,000 ops/s | +155,000 ops/s | +45.6% |
| Atomic Hot-Key (8 Threads)Heavy contention on a single shared key; tests lock coordination vs CAS retry collapse. | 85,000 ops/s | 465,000 ops/s | +380,000 ops/s | +447.1% |
| 100k Keys Memory FootprintResident Set Size overhead of CHAMP/HAMT node allocations. | 6.55 MB | 6.35 MB | -0.2 MB | -3.1% |
| Bytes per Key OverheadAverage metadata cost per entry. | 65.5 B | 63.5 B | -2 B | -3.1% |
Every metric above is captured under recorded commit tags with verified test harnesses. NimbusKV publishes raw JSON measurement reports alongside each tagged release, guaranteeing reproducible benchmarks.