Vector search benchmark
Full technical results for KiviDB's vector search engine, benchmarked against Redis Stack. Key-value throughput and latency against Dragonfly and Redis are…
Superseded. This is the July 2026 report for KiviDB v1.0.3 against Redis Stack 7.4, kept as a record. The current vector numbers, KiviDB 1.1.0 against Dragonfly 2.0.2 and Redis 8.10's query engine on the same machines, are on the Benchmarks page and in the KiviDB README.
Full technical results for KiviDB's vector search engine, benchmarked against Redis Stack. Key-value throughput and latency against Dragonfly and Redis are on the Benchmarks page.
Revision note (2026-07-27): this replaces the July 2026 (v1.0.1) report, which ran on a single Apple Silicon host pending a real AWS reproduction. That reproduction is done — twice: once on our own fork of the benchmark tool, and again on the official upstream repo (linked below) to confirm it wasn't an artifact of anything fork-specific. Dedicated AWS Graviton (
c7gn.12xlarge) instances, KiviDB and the benchmark driver on separate machines in the same cluster placement group — see Methodology below. The earlier single-host numbers undersold both sides of the comparison: KiviDB's ingest is now a clear win too (not parity), and search QPS is higher again for both engines with a dedicated, uncontended server.
TL;DR
- Search QPS 13.4×–16.9× Redis Stack at 100 parallel clients — the gap widens as
ef(and recall) increases. - Higher precision at every EF tier — at EF 512, KiviDB achieves 99.8% recall at 13,229 QPS (100 parallel clients), versus Redis Stack's 99.5% recall at 988 QPS.
- Ingest ~2.0× faster too — KiviDB indexes synchronously (no background indexing phase): 225.1 s vs Redis Stack's 454.6 s for the full 1.18M-vector corpus (5,262 vs 2,605 rec/s).
- Lower p50 and p95 latency at every EF tier — e.g. at EF 64 / 100 parallel clients, p50 1.9 ms vs Redis Stack's 30.4 ms; p95 4.5 ms vs 59.7 ms.
Part 1 — Vector search (ANN benchmark)
Methodology
| Parameter | Value |
|---|---|
| Harness | vector-db-benchmark, the official industry-standard benchmark tool, run at this commit |
| Dataset | glove-25-angular (1,183,514 vectors · 25-dim · cosine distance) |
| Index | HNSW, M=16, EF_CONSTRUCTION=256 |
| Upload | 100 threads, batch size 64, unpipelined HSET per vector |
| Search concurrency | 100 parallel clients |
| Repetitions | 3 per EF tier; best QPS reported |
| Redis Stack version | 7.4.0-v8 (Redis engine 7.4.7 + RediSearch 2.10.20) |
| Hardware | AWS, us-east-1, one cluster placement group. Server (KiviDB / Redis Stack, one at a time): c7gn.12xlarge (48 vCPU, Graviton, network-optimized). Client (benchmark driver): c7gn.12xlarge, a separate instance from the server — co-locating a 100-thread driver with the server under test understates the server's real throughput. |
| Date | 2026-07-27 |
| Reproduce | Run via our internal CI (see "Reproduce the benchmark" below for the public steps against the benchmark driver) |
Ingestion
KiviDB indexes synchronously: when a write returns, the vector is immediately searchable — there is no separate background indexing phase to wait out. On dedicated, uncontended hardware this also wins on raw wall-clock ingest.
| Metric | KiviDB | Redis Stack |
|---|---|---|
| Upload time (1,183,514 vectors) | 225.1 s | 454.6 s |
| Upload throughput | 5,262 rec/s | 2,605 rec/s |
| Post-upload wait to fully queryable | ~0 s (synchronous) | ~0 s |
| Speedup | 2.02× faster | baseline |
Search throughput vs precision (GloVe-25, 100 parallel clients)
Each row is a distinct ef_runtime sweep. Higher precision and higher QPS both mean better — KiviDB wins both at every tier.
| EF tier (ef_runtime) | KiviDB Precision | KiviDB QPS | KiviDB P50 | KiviDB P95 | Redis Stack Precision | Redis Stack QPS | Redis Stack P50 | Redis Stack P95 |
|---|---|---|---|---|---|---|---|---|
| 64 | 0.942 | 42,360 | 1.91 ms | 4.52 ms | 0.912 | 2,508 | 30.40 ms | 59.68 ms |
| 128 | 0.961 | 36,023 | 2.38 ms | 4.92 ms | 0.937 | 2,265 | 34.12 ms | 67.25 ms |
| 256 | 0.989 | 23,409 | 3.82 ms | 7.66 ms | 0.979 | 1,563 | 49.84 ms | 98.67 ms |
| 512 | 0.998 | 13,229 | 6.56 ms | 14.91 ms | 0.995 | 988 | 80.27 ms | 159.44 ms |
Headline comparisons (100 parallel clients):
| Operating point | KiviDB QPS | Redis Stack QPS | Advantage |
|---|---|---|---|
| High throughput (EF 64) | 42,360 | 2,508 | KiviDB 16.9× |
| Mid recall (EF 128) | 36,023 | 2,265 | KiviDB 15.9× |
| High recall (EF 256) | 23,409 | 1,563 | KiviDB 15.0× |
| Max recall (EF 512) | 13,229 | 988 | KiviDB 13.4× |
KiviDB's advantage is consistent across all recall targets — not an artifact of a single operating point. (A first run on our fork of the benchmark tool, before this one on official upstream, landed within a few percent of every number above — this isn't a one-off.)
Memory footprint
Not re-measured in this revision in a way we're confident enough to publish
a number for — the two engines' INFO/FT.INFO memory accounting isn't
directly comparable (see the reproduction notes). Removed pending a real,
apples-to-apples measurement rather than repeating old figures.
Part 2 — General KV throughput & latency
The key-value numbers, the commands they were taken with and how to read them are on the Benchmarks page. The runbook is Benchmarking KiviDB.
Reproduce the benchmark
The numbers above ran via our internal CI, which provisions a separate server and client AWS instance in one cluster placement group, builds KiviDB and the benchmark driver from source, runs both engines, and tears everything down. That's the authoritative source for these numbers.
The manual steps below run both engines and the driver on one host — fine for checking the tool works and for local dev, but co-locating the driver with the server under test will not reproduce the published numbers (a 100-thread driver competing with the server for the same cores understates the server's real throughput, which is exactly why the CI workflow above uses two separate instances).
Prerequisites
redis/vector-db-benchmark didn't have a kividb engine until we added one
and it merged upstream as PR #203
— these commands build straight from the official repo:
git clone https://github.com/redis/vector-db-benchmark
cd vector-db-benchmark && git checkout 31a28b5ae6d35da96de6d218ada209868a628b42
# HDF5 dev headers + dpkg-dev (for locating them) -- build-time only.
sudo apt-get install -y libhdf5-dev pkg-config dpkg-dev
HDF5_DIR="/usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/hdf5/serial" \
cargo build --release --bin vector-db-benchmark
No Python, no venv, no RESP3 version pin needed — the Rust tool's redis
crate defaults to RESP2 like KiviDB, which is exactly what the old Python
client's redis==4.6.0 pin was working around.
Dataset
No separate download step: the binary fetches glove-25-angular.hdf5
straight into datasets/glove-25-angular/ on first run, driven by
datasets/datasets.json in the checkout above.
Start servers
KiviDB (port 6380 so both can run at once):
./kividb --port 6380 --loglevel error
Redis Stack (port 6381; pin the version — see Methodology above):
redis-stack-server --port 6381 --bind 0.0.0.0 --protected-mode no
Run ANN benchmark
Two single-entry engines-files (one per port) — the tool rejects --engines
and --engines-file together, and an engines-file with no name filter runs
every entry in it, so a shared file can't select just one engine. kividb
uses the dedicated "kividb" engine (from PR #203 above); Redis Stack uses
the generic "redis" engine — both speak the same wire config:
cat > kividb-engine.json << 'EOF'
[{ "name": "kividb-glove25", "engine": "kividb", "connection_params": {},
"collection_params": { "hnsw_config": { "M": 16, "EF_CONSTRUCTION": 256 } },
"search_params": [
{ "parallel": 100, "search_params": { "ef": 64 } },
{ "parallel": 100, "search_params": { "ef": 128 } },
{ "parallel": 100, "search_params": { "ef": 256 } },
{ "parallel": 100, "search_params": { "ef": 512 } }
],
"upload_params": { "parallel": 100, "batch_size": 64 } }]
EOF
KIVIDB_PORT=6380 ./target/release/vector-db-benchmark --host localhost \
--engines-file kividb-engine.json --datasets glove-25-angular --parallels 100 --skip-if-exists false
# Same file with "engine": "redis" and REDIS_PORT=6381 for Redis Stack.
Limitations & notes
- Hardware matters. These numbers are on dedicated AWS Graviton
c7gn.12xlargeinstances with the server and the benchmark driver on separate machines. A smaller or shared instance (e.g. the driver co-located with the server) measures materially lower absolute numbers for both engines — we hit this directly while building the CI workflow behind this report, which is why the driver runs on its own instance now. - Single-server benchmark: both engines run as a single, unreplicated instance. Clustered or replicated deployments may produce different relative results.
- 100 parallel clients is the reported scenario. We haven't re-captured a
parallel=1breakdown on this topology; treat the 100-client numbers above as the representative comparison for a real serving workload rather than assuming a smaller-concurrency number would scale linearly.
Footnote — GloVe-100 (higher-dimensionality comparison)
The primary benchmark uses GloVe-25 (25-dimensional, 1.18 M vectors). We also ran a partial sweep on GloVe-100 (100-dimensional, same corpus). The full GloVe-100 dataset results will be published in a future report; early observations:
- Ingest throughput for 100-dim vectors is approximately 30–35% lower on both systems (larger vectors = more I/O per write).
- KiviDB's QPS advantage narrows slightly at 100-dim but remains positive at all tested EF tiers.
- Precision curves shift left — a given EF produces lower recall on 100-dim than on 25-dim, consistent with HNSW theory.
We recommend running your own sweep with the dataset dimensionality that matches your production embedding model.