Vector search benchmark

Full technical results for KiviDB's vector search engine, benchmarked against Redis Stack. Key-value throughput and latency against Dragonfly and Redis are…

Superseded. This is the July 2026 report for KiviDB v1.0.3 against Redis Stack 7.4, kept as a record. The current vector numbers, KiviDB 1.1.0 against Dragonfly 2.0.2 and Redis 8.10's query engine on the same machines, are on the Benchmarks page and in the KiviDB README.

Full technical results for KiviDB's vector search engine, benchmarked against Redis Stack. Key-value throughput and latency against Dragonfly and Redis are on the Benchmarks page.

Revision note (2026-07-27): this replaces the July 2026 (v1.0.1) report, which ran on a single Apple Silicon host pending a real AWS reproduction. That reproduction is done — twice: once on our own fork of the benchmark tool, and again on the official upstream repo (linked below) to confirm it wasn't an artifact of anything fork-specific. Dedicated AWS Graviton (c7gn.12xlarge) instances, KiviDB and the benchmark driver on separate machines in the same cluster placement group — see Methodology below. The earlier single-host numbers undersold both sides of the comparison: KiviDB's ingest is now a clear win too (not parity), and search QPS is higher again for both engines with a dedicated, uncontended server.


TL;DR

  • Search QPS 13.4×–16.9× Redis Stack at 100 parallel clients — the gap widens as ef (and recall) increases.
  • Higher precision at every EF tier — at EF 512, KiviDB achieves 99.8% recall at 13,229 QPS (100 parallel clients), versus Redis Stack's 99.5% recall at 988 QPS.
  • Ingest ~2.0× faster too — KiviDB indexes synchronously (no background indexing phase): 225.1 s vs Redis Stack's 454.6 s for the full 1.18M-vector corpus (5,262 vs 2,605 rec/s).
  • Lower p50 and p95 latency at every EF tier — e.g. at EF 64 / 100 parallel clients, p50 1.9 ms vs Redis Stack's 30.4 ms; p95 4.5 ms vs 59.7 ms.

Part 1 — Vector search (ANN benchmark)

Methodology

ParameterValue
Harnessvector-db-benchmark, the official industry-standard benchmark tool, run at this commit
Datasetglove-25-angular (1,183,514 vectors · 25-dim · cosine distance)
IndexHNSW, M=16, EF_CONSTRUCTION=256
Upload100 threads, batch size 64, unpipelined HSET per vector
Search concurrency100 parallel clients
Repetitions3 per EF tier; best QPS reported
Redis Stack version7.4.0-v8 (Redis engine 7.4.7 + RediSearch 2.10.20)
HardwareAWS, us-east-1, one cluster placement group. Server (KiviDB / Redis Stack, one at a time): c7gn.12xlarge (48 vCPU, Graviton, network-optimized). Client (benchmark driver): c7gn.12xlarge, a separate instance from the server — co-locating a 100-thread driver with the server under test understates the server's real throughput.
Date2026-07-27
ReproduceRun via our internal CI (see "Reproduce the benchmark" below for the public steps against the benchmark driver)

Ingestion

KiviDB indexes synchronously: when a write returns, the vector is immediately searchable — there is no separate background indexing phase to wait out. On dedicated, uncontended hardware this also wins on raw wall-clock ingest.

MetricKiviDBRedis Stack
Upload time (1,183,514 vectors)225.1 s454.6 s
Upload throughput5,262 rec/s2,605 rec/s
Post-upload wait to fully queryable~0 s (synchronous)~0 s
Speedup2.02× fasterbaseline

Search throughput vs precision (GloVe-25, 100 parallel clients)

Each row is a distinct ef_runtime sweep. Higher precision and higher QPS both mean better — KiviDB wins both at every tier.

EF tier (ef_runtime)KiviDB PrecisionKiviDB QPSKiviDB P50KiviDB P95Redis Stack PrecisionRedis Stack QPSRedis Stack P50Redis Stack P95
640.94242,3601.91 ms4.52 ms0.9122,50830.40 ms59.68 ms
1280.96136,0232.38 ms4.92 ms0.9372,26534.12 ms67.25 ms
2560.98923,4093.82 ms7.66 ms0.9791,56349.84 ms98.67 ms
5120.99813,2296.56 ms14.91 ms0.99598880.27 ms159.44 ms

Headline comparisons (100 parallel clients):

Operating pointKiviDB QPSRedis Stack QPSAdvantage
High throughput (EF 64)42,3602,508KiviDB 16.9×
Mid recall (EF 128)36,0232,265KiviDB 15.9×
High recall (EF 256)23,4091,563KiviDB 15.0×
Max recall (EF 512)13,229988KiviDB 13.4×

KiviDB's advantage is consistent across all recall targets — not an artifact of a single operating point. (A first run on our fork of the benchmark tool, before this one on official upstream, landed within a few percent of every number above — this isn't a one-off.)


Memory footprint

Not re-measured in this revision in a way we're confident enough to publish a number for — the two engines' INFO/FT.INFO memory accounting isn't directly comparable (see the reproduction notes). Removed pending a real, apples-to-apples measurement rather than repeating old figures.


Part 2 — General KV throughput & latency

The key-value numbers, the commands they were taken with and how to read them are on the Benchmarks page. The runbook is Benchmarking KiviDB.


Reproduce the benchmark

The numbers above ran via our internal CI, which provisions a separate server and client AWS instance in one cluster placement group, builds KiviDB and the benchmark driver from source, runs both engines, and tears everything down. That's the authoritative source for these numbers.

The manual steps below run both engines and the driver on one host — fine for checking the tool works and for local dev, but co-locating the driver with the server under test will not reproduce the published numbers (a 100-thread driver competing with the server for the same cores understates the server's real throughput, which is exactly why the CI workflow above uses two separate instances).

Prerequisites

redis/vector-db-benchmark didn't have a kividb engine until we added one and it merged upstream as PR #203 — these commands build straight from the official repo:

git clone https://github.com/redis/vector-db-benchmark
cd vector-db-benchmark && git checkout 31a28b5ae6d35da96de6d218ada209868a628b42

# HDF5 dev headers + dpkg-dev (for locating them) -- build-time only.
sudo apt-get install -y libhdf5-dev pkg-config dpkg-dev
HDF5_DIR="/usr/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/hdf5/serial" \
  cargo build --release --bin vector-db-benchmark

No Python, no venv, no RESP3 version pin needed — the Rust tool's redis crate defaults to RESP2 like KiviDB, which is exactly what the old Python client's redis==4.6.0 pin was working around.

Dataset

No separate download step: the binary fetches glove-25-angular.hdf5 straight into datasets/glove-25-angular/ on first run, driven by datasets/datasets.json in the checkout above.

Start servers

KiviDB (port 6380 so both can run at once):

./kividb --port 6380 --loglevel error

Redis Stack (port 6381; pin the version — see Methodology above):

redis-stack-server --port 6381 --bind 0.0.0.0 --protected-mode no

Run ANN benchmark

Two single-entry engines-files (one per port) — the tool rejects --engines and --engines-file together, and an engines-file with no name filter runs every entry in it, so a shared file can't select just one engine. kividb uses the dedicated "kividb" engine (from PR #203 above); Redis Stack uses the generic "redis" engine — both speak the same wire config:

cat > kividb-engine.json << 'EOF'
[{ "name": "kividb-glove25", "engine": "kividb", "connection_params": {},
   "collection_params": { "hnsw_config": { "M": 16, "EF_CONSTRUCTION": 256 } },
   "search_params": [
     { "parallel": 100, "search_params": { "ef": 64 } },
     { "parallel": 100, "search_params": { "ef": 128 } },
     { "parallel": 100, "search_params": { "ef": 256 } },
     { "parallel": 100, "search_params": { "ef": 512 } }
   ],
   "upload_params": { "parallel": 100, "batch_size": 64 } }]
EOF

KIVIDB_PORT=6380 ./target/release/vector-db-benchmark --host localhost \
  --engines-file kividb-engine.json --datasets glove-25-angular --parallels 100 --skip-if-exists false

# Same file with "engine": "redis" and REDIS_PORT=6381 for Redis Stack.

Limitations & notes

  • Hardware matters. These numbers are on dedicated AWS Graviton c7gn.12xlarge instances with the server and the benchmark driver on separate machines. A smaller or shared instance (e.g. the driver co-located with the server) measures materially lower absolute numbers for both engines — we hit this directly while building the CI workflow behind this report, which is why the driver runs on its own instance now.
  • Single-server benchmark: both engines run as a single, unreplicated instance. Clustered or replicated deployments may produce different relative results.
  • 100 parallel clients is the reported scenario. We haven't re-captured a parallel=1 breakdown on this topology; treat the 100-client numbers above as the representative comparison for a real serving workload rather than assuming a smaller-concurrency number would scale linearly.

Footnote — GloVe-100 (higher-dimensionality comparison)

The primary benchmark uses GloVe-25 (25-dimensional, 1.18 M vectors). We also ran a partial sweep on GloVe-100 (100-dimensional, same corpus). The full GloVe-100 dataset results will be published in a future report; early observations:

  • Ingest throughput for 100-dim vectors is approximately 30–35% lower on both systems (larger vectors = more I/O per write).
  • KiviDB's QPS advantage narrows slightly at 100-dim but remains positive at all tested EF tiers.
  • Precision curves shift left — a given EF produces lower recall on 100-dim than on 25-dim, consistent with HNSW theory.

We recommend running your own sweep with the dataset dimensionality that matches your production embedding model.