Get the data out: migrate from Valkey to KiviDB
These write Redis's RDB format, so every method in the Redis guide works as written, with valkey-cli in place of redis-cli if you like:
Valkey 7.2 and 8.x
These write Redis's RDB format, so every method in the
Redis guide works as written,
with valkey-cli in place of redis-cli if you like:
Replicate, then switch:
kividb --port 6380 --dir /var/lib/kividb \
--replicaof valkey.internal 6379 --masterauth "$VALKEY_PASSWORD"
RDB file:
valkey-cli -h valkey.internal BGSAVE
valkey-cli -h valkey.internal CONFIG GET dir # copy dump.rdb from here when the save is done
kividb --dir /var/lib/kividb --preload-file rdb:/backups/dump.rdb
RedisShake (sync_reader or scan_reader) and RIOT: as in the
Redis guide.
Valkey 9
KiviDB can't load Valkey 9's snapshot, so replication, RDB files and anything
that moves DUMP payloads stop with an error:
| Method | Error |
|---|---|
kividb --replicaof | full-sync payload is not a valid RDB: not a Redis RDB file in KiviDB's log; the link stays down |
dump.rdb in --dir | dump.rdb cannot be loaded: not a Redis RDB file; KiviDB doesn't start |
--preload-file rdb: | could not be loaded: bad magic; KiviDB doesn't start |
MIGRATE, RedisShake scan_reader, RIOT | ERR DUMP payload version or checksum are wrong |
Fetch a Redis-format RDB file (simplest)
Where the server allows SYNC (self-hosted Valkey does), valkey-cli --rdb
receives the snapshot in Redis's format, which KiviDB loads, including every
logical database and stream consumer groups:
valkey-cli -h valkey.internal -a "$VALKEY_PASSWORD" --rdb dump.rdb
head -c 9 dump.rdb; echo # REDIS0011
kividb --dir /var/lib/kividb --preload-file rdb:dump.rdb
For KiviDB Cloud, import the same file under Snapshots → Import a snapshot.
This fails if any hash has field expiry, because Redis's version 11 format has
no way to store it. Valkey's log says
Can't store key (db 0) in RDB version 11, valkey-cli reports
I/O Error reading RDB payload from socket, and it leaves a partial
dump.rdb behind: delete it. If keys_with_volatile_items in INFO keyspace
is above 0, use RedisShake below. Like any file, it holds no writes made after
it was taken, so stop writes first.
RedisShake, following writes
RedisShake's sync_reader works: it reads Valkey 9's snapshot, then the write
stream, and writes each key to KiviDB in Redis's format. Streams with their
consumer groups, TTLs and every other type come across. The one exception is a
hash with field expiry, which RedisShake can't re-encode: KiviDB refuses it
with ERR Bad data format and RedisShake stops. So:
-
Find the hashes with field expiry. If
keys_with_volatile_itemsis 0, there are none; skip to step 3. Otherwise save this script, which finds them and copies them, and run it with-as the target to list them. It reads every hash, so run it against a replica or off peak.#!/usr/bin/env python3 """Copy the hashes that have per-field expiry, keeping each field's expiry. pip install "redis>=5" python3 copy_field_ttl_hashes.py SOURCE_URL TARGET_URL [DB ...] python3 copy_field_ttl_hashes.py SOURCE_URL - [DB ...] # only list them """ import sys import redis src_url, dst_url = sys.argv[1], sys.argv[2] dbs = [int(d) for d in sys.argv[3:]] or [0] found = 0 for db in dbs: src = redis.Redis.from_url(src_url, db=db) dst = None if dst_url == "-" else redis.Redis.from_url(dst_url, db=db) for key in src.scan_iter(count=1000, _type="hash"): fields = list(src.hgetall(key).items()) if not fields: continue names = [f for f, _ in fields] when = src.execute_command("HPEXPIRETIME", key, "FIELDS", len(names), *names) if all(w == -1 for w in when): continue # no expiring field: RedisShake copies it found += 1 if dst is None: print(f"db{db} {key.decode(errors='replace')}") continue key_when = src.pexpiretime(key) p = dst.pipeline(transaction=True) p.delete(key) p.hset(key, mapping=dict(fields)) for (field, _), w in zip(fields, when): if w > 0: p.execute_command("HPEXPIREAT", key, w, "FIELDS", 1, field) if key_when > 0: p.pexpireat(key, key_when) p.execute() print(f"{'found' if dst_url == '-' else 'copied'} {found} hashes with field expiry")pip install "redis>=5" python3 copy_field_ttl_hashes.py redis://:$VALKEY_PASSWORD@valkey.internal:6379 - 0 # db0 session:8812 # found 1240 hashes with field expiry -
Leave their prefixes out of RedisShake:
[sync_reader] cluster = false address = "valkey.internal:6379" password = "valkey-password" [redis_writer] cluster = false address = "kividb.internal:6380" password = "kividb-password" [filter] block_key_prefix = ["session:"] # the prefixes from step 1 [advanced] dir = "shake-data" -
Run RedisShake and wait for
syncing aof, diff=[0]in its log. It keeps following writes from then on. -
Copy the hashes with field expiry with the same script, KiviDB as the target. It writes each hash with
HSET, then each field's expiry withHPEXPIREATand the key's own expiry withPEXPIREAT:python3 copy_field_ttl_hashes.py redis://:$VALKEY_PASSWORD@valkey.internal:6379 \ redis://:$KIVIDB_PASSWORD@kividb.internal:6380 0RedisShake does not follow writes to the keys it filters out, so run the script once more after you stop writes, just before the switch.
-
Cut over as in the Redis runbook:
CLIENT PAUSE <ms> WRITEon Valkey, wait fordiff=[0], run the script again, verify, switch the clients, stop RedisShake.
Into KiviDB Cloud, use the same steps with the database as the target
(your-name.cloud.kividb.io:6380, user kividbapp, tls = true; see
Loading into KiviDB Cloud).
RIOT, as a fallback
RIOT with --struct also works from Valkey 9, because it copies with ordinary
commands, but it leaves out stream consumer groups and hash field expiry. Use
it where neither of the above is possible.
Cluster mode
Use RedisShake with cluster = true on the reader, as in
From Redis Cluster. Into a self-hosted
KiviDB cluster, set cluster = true on the writer too; into one KiviDB server
or a non-sharded KiviDB Cloud database, leave it false. The KiviDB cluster
doesn't need the same number of shards.
Valkey 9 allows databases other than 0 in cluster mode. A KiviDB cluster has
database 0 only, so copy each other database into a separate KiviDB server
(RedisShake's allow_db filter selects one), or move its keys into database 0
under a prefix in the application first.