Get the data out: migrate from Valkey to KiviDB

These write Redis's RDB format, so every method in the Redis guide works as written, with valkey-cli in place of redis-cli if you like:

Valkey 7.2 and 8.x

These write Redis's RDB format, so every method in the Redis guide works as written, with valkey-cli in place of redis-cli if you like:

Replicate, then switch:

kividb --port 6380 --dir /var/lib/kividb \
  --replicaof valkey.internal 6379 --masterauth "$VALKEY_PASSWORD"

RDB file:

valkey-cli -h valkey.internal BGSAVE
valkey-cli -h valkey.internal CONFIG GET dir      # copy dump.rdb from here when the save is done
kividb --dir /var/lib/kividb --preload-file rdb:/backups/dump.rdb

RedisShake (sync_reader or scan_reader) and RIOT: as in the Redis guide.

Valkey 9

KiviDB can't load Valkey 9's snapshot, so replication, RDB files and anything that moves DUMP payloads stop with an error:

MethodError
kividb --replicaoffull-sync payload is not a valid RDB: not a Redis RDB file in KiviDB's log; the link stays down
dump.rdb in --dirdump.rdb cannot be loaded: not a Redis RDB file; KiviDB doesn't start
--preload-file rdb:could not be loaded: bad magic; KiviDB doesn't start
MIGRATE, RedisShake scan_reader, RIOTERR DUMP payload version or checksum are wrong

Fetch a Redis-format RDB file (simplest)

Where the server allows SYNC (self-hosted Valkey does), valkey-cli --rdb receives the snapshot in Redis's format, which KiviDB loads, including every logical database and stream consumer groups:

valkey-cli -h valkey.internal -a "$VALKEY_PASSWORD" --rdb dump.rdb
head -c 9 dump.rdb; echo          # REDIS0011
kividb --dir /var/lib/kividb --preload-file rdb:dump.rdb

For KiviDB Cloud, import the same file under Snapshots → Import a snapshot.

This fails if any hash has field expiry, because Redis's version 11 format has no way to store it. Valkey's log says Can't store key (db 0) in RDB version 11, valkey-cli reports I/O Error reading RDB payload from socket, and it leaves a partial dump.rdb behind: delete it. If keys_with_volatile_items in INFO keyspace is above 0, use RedisShake below. Like any file, it holds no writes made after it was taken, so stop writes first.

RedisShake, following writes

RedisShake's sync_reader works: it reads Valkey 9's snapshot, then the write stream, and writes each key to KiviDB in Redis's format. Streams with their consumer groups, TTLs and every other type come across. The one exception is a hash with field expiry, which RedisShake can't re-encode: KiviDB refuses it with ERR Bad data format and RedisShake stops. So:

  1. Find the hashes with field expiry. If keys_with_volatile_items is 0, there are none; skip to step 3. Otherwise save this script, which finds them and copies them, and run it with - as the target to list them. It reads every hash, so run it against a replica or off peak.

    #!/usr/bin/env python3
    """Copy the hashes that have per-field expiry, keeping each field's expiry.
    
        pip install "redis>=5"
        python3 copy_field_ttl_hashes.py SOURCE_URL TARGET_URL [DB ...]
        python3 copy_field_ttl_hashes.py SOURCE_URL - [DB ...]     # only list them
    """
    import sys
    import redis
    
    src_url, dst_url = sys.argv[1], sys.argv[2]
    dbs = [int(d) for d in sys.argv[3:]] or [0]
    
    found = 0
    for db in dbs:
        src = redis.Redis.from_url(src_url, db=db)
        dst = None if dst_url == "-" else redis.Redis.from_url(dst_url, db=db)
        for key in src.scan_iter(count=1000, _type="hash"):
            fields = list(src.hgetall(key).items())
            if not fields:
                continue
            names = [f for f, _ in fields]
            when = src.execute_command("HPEXPIRETIME", key, "FIELDS", len(names), *names)
            if all(w == -1 for w in when):
                continue  # no expiring field: RedisShake copies it
            found += 1
            if dst is None:
                print(f"db{db} {key.decode(errors='replace')}")
                continue
            key_when = src.pexpiretime(key)
            p = dst.pipeline(transaction=True)
            p.delete(key)
            p.hset(key, mapping=dict(fields))
            for (field, _), w in zip(fields, when):
                if w > 0:
                    p.execute_command("HPEXPIREAT", key, w, "FIELDS", 1, field)
            if key_when > 0:
                p.pexpireat(key, key_when)
            p.execute()
    print(f"{'found' if dst_url == '-' else 'copied'} {found} hashes with field expiry")
    
    pip install "redis>=5"
    python3 copy_field_ttl_hashes.py redis://:$VALKEY_PASSWORD@valkey.internal:6379 - 0
    # db0 session:8812
    # found 1240 hashes with field expiry
    
  2. Leave their prefixes out of RedisShake:

    [sync_reader]
    cluster = false
    address = "valkey.internal:6379"
    password = "valkey-password"
    
    [redis_writer]
    cluster = false
    address = "kividb.internal:6380"
    password = "kividb-password"
    
    [filter]
    block_key_prefix = ["session:"]   # the prefixes from step 1
    
    [advanced]
    dir = "shake-data"
    
  3. Run RedisShake and wait for syncing aof, diff=[0] in its log. It keeps following writes from then on.

  4. Copy the hashes with field expiry with the same script, KiviDB as the target. It writes each hash with HSET, then each field's expiry with HPEXPIREAT and the key's own expiry with PEXPIREAT:

    python3 copy_field_ttl_hashes.py redis://:$VALKEY_PASSWORD@valkey.internal:6379 \
      redis://:$KIVIDB_PASSWORD@kividb.internal:6380 0
    

    RedisShake does not follow writes to the keys it filters out, so run the script once more after you stop writes, just before the switch.

  5. Cut over as in the Redis runbook: CLIENT PAUSE <ms> WRITE on Valkey, wait for diff=[0], run the script again, verify, switch the clients, stop RedisShake.

Into KiviDB Cloud, use the same steps with the database as the target (your-name.cloud.kividb.io:6380, user kividbapp, tls = true; see Loading into KiviDB Cloud).

RIOT, as a fallback

RIOT with --struct also works from Valkey 9, because it copies with ordinary commands, but it leaves out stream consumer groups and hash field expiry. Use it where neither of the above is possible.

Cluster mode

Use RedisShake with cluster = true on the reader, as in From Redis Cluster. Into a self-hosted KiviDB cluster, set cluster = true on the writer too; into one KiviDB server or a non-sharded KiviDB Cloud database, leave it false. The KiviDB cluster doesn't need the same number of shards.

Valkey 9 allows databases other than 0 in cluster mode. A KiviDB cluster has database 0 only, so copy each other database into a separate KiviDB server (RedisShake's allow_db filter selects one), or move its keys into database 0 under a prefix in the application first.