Get the data out: migrate from Dragonfly to KiviDB

Because nothing follows Dragonfly's writes, plan a short write freeze: stop writes, take the copy, verify, switch.

MethodFrom DragonflyFollows writes
RDB file (SAVE RDB)YesNo
RedisShake scan_readerYesNo (Dragonfly sends only expiry notifications)
RIOT replicate (scan mode)YesNo
Replicate (--replicaof), RedisShake sync_readerNo: ERR unknown command 'PSYNC'
MIGRATENo: Dragonfly has no MIGRATE

Because nothing follows Dragonfly's writes, plan a short write freeze: stop writes, take the copy, verify, switch.

An RDB file

Dragonfly's SAVE writes its own format by default: several .dfs files and a -summary.dfs. KiviDB doesn't read those. Ask for an RDB file instead:

redis-cli -h dragonfly.internal SAVE RDB migrate
# writes migrate.rdb in Dragonfly's --dir

SAVE RDB <name> writes one Redis-format file without changing Dragonfly's settings. BGSAVE RDB <name> does the same in the background. To make every snapshot an RDB file, start Dragonfly with --df_snapshot_format=false.

Dragonfly can also write the snapshot straight to object storage:

redis-cli -h dragonfly.internal SAVE RDB s3://my-backups/dragonfly migrate

Check the file before you rely on it: an RDB file starts with REDIS, a Dragonfly one does not load.

head -c 9 migrate.rdb; echo     # REDIS0009

Don't give KiviDB a .dfs file. KiviDB 1.2.0 started from --preload-file rdb:dump-summary.dfs comes up with no keys and no error.

RedisShake (scan)

RedisShake's scan_reader reads with SCAN and DUMP; KiviDB accepts Dragonfly's DUMP payloads for every core type:

[scan_reader]
cluster = false
address = "dragonfly.internal:6379"
password = "dragonfly-password"
scan = true
ksn = false
count = 100

[redis_writer]
cluster = false
address = "kividb.internal:6380"
password = "kividb-password"

[advanced]
dir = "shake-data"
./redis-shake shake.toml

It copies every key once and exits. Leave ksn = false: Dragonfly only publishes expiry events, so it can't follow writes. Download and options are in the Redis guide. sync_reader does not work with Dragonfly.

RIOT

riot replicate redis://:$DRAGONFLY_PASSWORD@dragonfly.internal:6379 \
  redis://kividb.internal:6380 --target-pass "$KIVIDB_PASSWORD" --compare NONE

RIOT copies one logical database per run (add /1 to both URIs for database 1). --mode LIVE does not work against Dragonfly, for the same reason as ksn.

Keys with member expiry

A hash with a field set by HSETEX or FIELDEXPIRE, or a set with a member added by SADDEX, is stored in a Dragonfly-only form:

  • in an RDB file, KiviDB refuses the whole file with unsupported RDB type 22;
  • through DUMP (RedisShake scan, RIOT), KiviDB refuses that key with ERR Bad data format.

Move those keys separately, as plain values, and set the expiry again:

  1. Find them. If they follow a naming pattern, use it. Otherwise test each hash and set with FIELDTTL on its fields and members.

  2. Copy them with RIOT's --struct mode, which writes with HSET and SADD instead of RESTORE, limited to those keys:

    riot replicate redis://dragonfly.internal:6379 redis://kividb.internal:6380 \
      --struct --key-pattern 'session:*' --compare NONE
    
  3. Set the field expiry on KiviDB with HEXPIRE key <seconds> FIELDS 1 <field>. For set members there is no equivalent: move them into a hash or a sorted set (with the expiry time as the score).

  4. Copy everything else with an RDB file or RedisShake, leaving those keys out (block_key_prefix in RedisShake's [filter]), or delete them from the RDB copy's source first.