# NimbusKV v3.1.0 - Documentation > This documents the current release. For the original v3.0.0 architecture writeup, see [`v3.0.0`](v3.0.0.md). --- ## 1. Changes from v3.0.0 **Fixed** - **`atomic()` now notifies subscribers.** In v3.0.0, `NimbusKV.atomic()` called the store's internal update path directly, bypassing notification - so `subscribe()` and anything built on it, including `wait_for()`, silently never fired for changes made via `atomic()`, even though the value itself updated correctly. Fixed by giving the core store its own `atomic()` that performs the read-modify-write and the notification as a single, indivisible operation. If you use `atomic()` alongside `subscribe()` or `wait_for()`, this is the version you need. **Added** - A real `tests/` suite (36 tests) covering core CRUD, concurrency correctness, TTL, reactive subscriptions, async, and all persistence backends. - CI across standard Python 3.9–3.13 and free-threaded 3.13t/3.14t. - `examples/session_store.py` and `examples/worker_pool.py` - the latter is what caught the `atomic()` bug during development. - `dev/API_REFERENCE.md`, generated directly from the library's docstrings. - `CONTRIBUTING.md`, `.github/FUNDING.yml`. **Not changed:** every public API signature from v3.0.0 is identical. This is a bug-fix-plus-infrastructure release, not a breaking one - if you're already on v3.0.0, upgrading is a plain `pip install -U nimbuskv` with no code changes required (unless you were relying on the buggy non-notifying behavior of `atomic()`, which nothing reasonably would be). --- ## 2. What NimbusKV is NimbusKV is a reactive, TTL-aware, concurrent key/value store for Python. - **Reads never block, ever.** State lives as an immutable persistent snapshot. A read is just "look at the current snapshot" - no lock, no wait, regardless of what any writer is doing concurrently. - **Writes take one short lock.** A writer builds the new snapshot from the current one and publishes it, all under a brief lock, so no write is ever wasted or retried. (Up to v3.0.1 writers retried on conflict instead; under a hot key that became a retry storm, so 3.1.0 replaced it.) - **One code path, two runtimes.** The same code runs correctly on standard (GIL) Python and adapts automatically to free-threaded Python (3.13t/3.14t), where reads run truly in parallel. Note: NimbusKV's dependency `immutables` re-enables the GIL on import unless you run with `PYTHON_GIL=0` (or `python -X gil=0`). - **Reactive.** `subscribe()` pushes every change to you as it happens; `wait_for()` blocks a thread on a condition without polling. - **TTL built in.** Keys can expire automatically, firing their own `'expire'` event distinct from a manual delete. - **Optional persistence.** Memory (default), SQLite, or Redis, as a write-through mirror - the backend is never on the read path; memory is always the live source of truth. --- ## 3. Install ```bash pip install nimbuskv pip install "nimbuskv[redis]" # only if you want the Redis backend ``` Requires Python 3.9+. Works on standard Python and on free-threaded builds (3.13t/3.14t); on free-threaded builds, run with `PYTHON_GIL=0` to keep the GIL off. For contributing / running the test suite locally: ```bash git clone https://github.com/NimbusKV/NimbusKV.git cd NimbusKV pip install -e ".[dev]" pytest tests/ -v ``` --- ## 4. Quick start ```python from nimbuskv import NimbusKV ld = NimbusKV() # in-memory, default ld.set('foo', 123) print(ld.get('foo')) # 123 ld.stop() # stops the background TTL scheduler thread ``` Or as a context manager, so cleanup happens automatically even on exception: ```python with NimbusKV() as ld: ld.set('foo', 123) print(ld.get('foo')) ``` --- ## 5. Core API ### `NimbusKV(backend="memory")` Constructs a store. `backend` accepts: | Value | Meaning | |---|---| | `"memory"` (default) | No persistence - nothing survives a restart. | | `"sqlite:/path/to/file.db"` | SQLite-backed, file created if missing. | | `"redis://host:port/db"` | Redis-backed. Requires `pip install "nimbuskv[redis]"`. | | A `PersistenceBackend` instance | For custom backends or pre-configured clients (e.g. a `fakeredis` client for testing). | If the backend already has data, it's loaded into memory before the constructor returns (already-expired keys are skipped during rehydration). ### `set(key, value, ttl=None)` ```python ld.set('foo', 123) ld.set('session:abc', {'user': 'alvin'}, ttl=3600) # expires in 1 hour ``` Setting a key with no `ttl` cancels any previously pending expiry on that key. ### `get(key, default=None)` ```python ld.get('foo') # 123 ld.get('missing', 'n/a') # 'n/a' ``` Always reads from memory - never touches the persistence backend, so this is fast regardless of which backend is configured. ### `delete(key) -> bool` ```python ld.delete('foo') # True if it existed, False if already gone ``` Also cancels any pending TTL for that key, so a manual delete can't race with a later automatic expiry. ### `exists(key) -> bool`, `key in ld` ```python ld.exists('foo') 'foo' in ld ``` ### `keys()`, `items()`, `len(ld)`, iteration ```python ld.keys() # point-in-time list of keys ld.items() # point-in-time list of (key, value) pairs len(ld) for k in ld: ... ``` Snapshot-safe: the list you get back won't change under you even while writers are actively modifying the store. ### `repr(ld)` ```python >>> ld ``` --- ## 6. TTL ```python ld.set('temp', 'value', ttl=5) # expires in 5 seconds ``` Expiry fires its own `'expire'` event on `subscribe()`, distinct from `'delete'`: ```python def on_change(key, event, old, new): print(key, event, old, '->', new) ld.subscribe(on_change) ld.set('temp', 1, ttl=1) # ... after 1 second ... # temp expire 1 -> None ``` Re-setting a key's TTL before it fires correctly replaces the old deadline via an internal generation counter - the earlier schedule is invalidated, not left to fire alongside the new one, so you'll never see a spurious duplicate expiry event. --- ## 7. Reactive subscriptions ```python def on_change(key, event, old_value, new_value): print(f'{key}: {event} {old_value!r} -> {new_value!r}') sub = ld.subscribe(on_change) # every key sub_foo = ld.subscribe(on_change, key='foo') # just 'foo' ld.set('foo', 1) # foo: set None -> 1 ld.delete('foo') # foo: delete 1 -> None sub.cancel() # stop receiving notifications ``` `event` is one of `'set'`, `'delete'`, `'expire'`. Callbacks run **synchronously**, in the thread that made the change, immediately after that write succeeds - keep them fast, since a slow callback delays that `set()`/`delete()`/`atomic()` call from returning. A subscriber that raises is caught and logged (via `logging.getLogger("nimbuskv")`) rather than propagating into the write path. As of v3.0.1, **`atomic()` updates fire notifications correctly**, same as `set()`. --- ## 8. Atomic compound updates ```python ld.set('counter', 0) ld.atomic('counter', lambda v: v + 1) # -> 1, safe under concurrent writers ``` `atomic()` replaces manual locking. Your function runs **exactly once**, under the store's write lock, so keep it short: readers are never blocked, but other writers wait for it. Don't write to the same store from inside it; that raises `ReentrantWriteError` (in v3.0.1 it was allowed). Any TTL already on the key is kept. Every `atomic()` call fires a `subscribe()` notification, same as `set()`. ```python import threading ld.set('counter', 0) def worker(): for _ in range(1000): ld.atomic('counter', lambda v: v + 1) threads = [threading.Thread(target=worker) for _ in range(10)] [t.start() for t in threads] [t.join() for t in threads] print(ld.get('counter')) # 10000, exactly - no lost updates ``` --- ## 9. Batch operations ```python ld.mset({'a': 1, 'b': 2}) # one atomic swap for both keys ld.mget(['a', 'b', 'missing']) # {'a': 1, 'b': 2, 'missing': None} ``` `mset()` matters beyond convenience: setting several related keys in one swap means readers never observe a partially-applied update (some keys changed, others not yet). A loop of individual `set()` calls doesn't give you that guarantee. ```python ld.mset({'account_a': 90, 'account_b': 110}) # both change together, atomically ``` `mset()` also accepts a shared `ttl` applied to every key in the batch. --- ## 10. `wait_for` - thread coordination without polling ```python import threading def consumer(): result = ld.wait_for('job:status', lambda v: v == 'done', timeout=30) print('unblocked with:', result) threading.Thread(target=consumer).start() # ... elsewhere, another thread ... ld.set('job:status', 'done') # consumer's wait_for() returns right here ``` Blocks the calling thread until `predicate(current_value)` is true, driven by the subscription system - not a sleep-and-poll loop. Raises `TimeoutError` if `timeout` elapses first. Works correctly whether the triggering change comes from `set()`, `mset()`, or `atomic()` (as of v3.0.1 - this is exactly the code path the v3.0.0 bug affected). See [`examples/worker_pool.py`](https://github.com/NimbusKV/NimbusKV/blob/latest/examples/worker_pool.py) for a full producer/consumer example using this. --- ## 11. Async ```python import asyncio from nimbuskv import NimbusKV async def main(): ld = NimbusKV() await ld.aset('bar', 'hello') print(await ld.aget('bar')) ld.stop() asyncio.run(main()) ``` For the default in-memory backend, `aget`/`aset`/`adelete`/`aexists` are **genuinely non-blocking** - no thread spawned, because there's no real I/O to hide behind an executor. When a real backend (SQLite/Redis) is attached, *only* the persistence write is delegated to a thread-pool executor; the in-memory part of every operation still never blocks. --- ## 12. Persistence backends ```python ld = NimbusKV(backend='sqlite:/path/to/store.db') ld = NimbusKV(backend='redis://localhost:6379/0') ``` State is rehydrated from the backend on startup automatically; already-expired keys are skipped during rehydration. A backend is a **write-through mirror only** - it never sits on the read path. If you need a custom backend, subclass `PersistenceBackend`: ```python from nimbuskv.modules.persistence import PersistenceBackend class MyBackend(PersistenceBackend): def load_all(self): return [] # [(key, value, expire_at_or_None), ...] def persist_set(self, key, value, expire_at): ... def persist_delete(self, key): ... ld = NimbusKV(backend=MyBackend()) ``` See [`examples/session_store.py`](https://github.com/NimbusKV/NimbusKV/blob/latest/examples/session_store.py) for a realistic TTL + persistence usage pattern. --- ## 13. Architecture ```mermaid flowchart TD A["NimbusKV
(public API)"] A --> B["ReactiveStore
in-memory, immutable-snapshot,
concurrent core"] A --> C["ExpiryScheduler
background thread, min-heap,
fires TTL expiry against the store"] A --> D["PersistenceBackend
write-through mirror +
startup rehydration"] B --> B1["Readers: lock-free, always"] B --> B2["Writers: build new snapshot,
one short lock, publish"] B --> B3["Adapts to LOCK_FREE_MODE
(free-threaded vs GIL, detected once)"] D --> D1[NullBackend] D --> D2[SQLiteBackend] D --> D3[RedisBackend] C -.-> B D -.-> B ``` **Why a lock for writes?** CPython guarantees a single reference write isn't torn, so readers always see either the fully-old or fully-new snapshot and never need a lock. Up to v3.0.1 writers built their snapshot outside the lock and retried if another writer won; under many threads on one key, losing threads kept throwing work away. Since 3.1.0 each write takes the lock once and does its work once, which was measured equal or faster at every thread count. For the full generated API surface (every class, method, `Args`/`Returns`/`Example`), see [`dev/API_REFERENCE.md`](https://github.com/NimbusKV/NimbusKV/blob/latest/dev/API_REFERENCE.md). --- ## 14. Testing and CI ```bash pytest tests/ -v ``` runs the full suite (36 tests as of this release) covering: - Core CRUD and iteration - Concurrency correctness under contention (no lost updates across many threads) - TTL expiry, event semantics, and the reschedule-doesn't-leave-a-stale-entry guarantee - Reactive subscriptions, including the `atomic()` regression tests for the bug fixed in this release - Async behavior, including verifying the in-memory path spawns no thread - SQLite and Redis persistence, including rehydration after a simulated restart CI (`.github/workflows/tests.yml`) runs this matrix on every push/PR across Python 3.9–3.13 on Linux/macOS/Windows, plus a dedicated job on free-threaded 3.13t/3.14t (Linux) that also confirms the GIL is actually disabled before running, and runs the concurrency benchmark in `testfile.py`. --- ## 15. What's explicitly not in this version - **Sandboxed callback execution.** A pre-3.0 version of this library had a "sandbox" for callbacks; it was actually just a timeout wrapper, not real isolation, and calling it a sandbox was misleading. It was dropped rather than kept under a false name. - **Manual locking (`lock()`/`unlock()`).** `atomic()` is the replacement. --- ## 16. License Apache License 2.0, from 3.1.0 onwards. Versions up to and including 3.0.1 were GPL-3.0. See [`LICENSE`](../LICENSE), [`NOTICE`](../NOTICE), [`TRADEMARKS.md`](../TRADEMARKS.md) and [`COMMERCIAL-LICENSING.md`](../COMMERCIAL-LICENSING.md). --- ## 17. Contributing See [`CONTRIBUTING.md`](https://github.com/NimbusKV/NimbusKV/blob/latest/CONTRIBUTING.md). Free-threaded (3.13t/3.14t) testing and bug reports are the most valuable contribution right now - it's the newest, least battle-tested part of the runtime story.