Load into KiviDB: migrate from Redis Cloud to KiviDB

Self-hosted KiviDB

One RDB file:

kividb --dir /var/lib/kividb --requirepass "$KIVIDB_PASSWORD" --preload-file rdb:/backups/backup-1.rdb

--preload-file loads that file instead of what is in --dir; if it cannot be read, KiviDB does not start. Leave the flag off on later restarts. A file with time series or probabilistic keys is refused as a whole: use Option 2 for that database.

One file per shard. --preload-file takes one file, so load each one over the network with RedisShake's rdb_reader, into a running KiviDB:

[rdb_reader]
filepath = "/backups/backup-1.rdb"

[redis_writer]
address = "kividb.internal:6380"
password = "..."
# cluster = true     # when the target is a KiviDB cluster: any node's address

./redis-shake shake.toml   # once per file, changing filepath; each run ends with "all done"

Over commands (Option 2): add the writer to the scan configuration and run it:

[redis_writer]
address = "kividb.internal:6380"
password = "..."

[advanced]
rdb_restore_command_behavior = "rewrite"   # a rerun overwrites keys already copied

./redis-shake shake.toml

Without ksn it stops after all done; with ksn it keeps running until you stop it.

Time series go across with RIOT, limited to that type:

curl -LO https://github.com/redis/riot/releases/download/v4.3.0/riot-standalone-4.3.0-linux-x86_64.zip
unzip riot-standalone-4.3.0-linux-x86_64.zip

riot-standalone-4.3.0-linux-x86_64/bin/riot replicate --struct --key-type TSDB-TYPE \
  --source-user default --source-pass "$REDIS_PASSWORD" \
  --target-pass "$KIVIDB_PASSWORD" \
  redis://redis-12345.c12345.us-east-1-mz.ec2.cloud.rlrcp.com:12345 \
  redis://kividb.internal:6380

--key-type takes the name TYPE returns. RIOT copies the samples; check each series' labels, retention and compaction rules afterwards with TS.INFO, and set them again with TS.ALTER key LABELS ... and TS.CREATERULE (in our test the labels did not come across). Add --source-tls and --source-cacert redis_ca.pem when the source uses TLS.

KiviDB Cloud

A KiviDB Cloud database cannot replicate from another server, so data arrives as a file or over normal commands. The endpoint, the username (kividbapp) and the password are on the database's overview; see Connect to your instance. Add the address of any machine that runs a copy tool to the database's allowed IPs.

One RDB file, from S3 or Cloud Storage. Make a link that stays valid for a few hours, for the database to fetch while it starts:

aws s3 presign s3://my-migration-bucket/kividb-move/backup-1.rdb --expires-in 43200
gcloud storage sign-url gs://my-migration-bucket/kividb-move/backup-1.rdb --duration=12h \
  --impersonate-service-account=signer@my-project.iam.gserviceaccount.com

Create the database; on the review page, under Seed data, choose HTTPS URL and RDB snapshot, and paste the link. The file must be uncompressed and no larger than the database's data size.

One RDB file, from your machine. Under Snapshots → Import a snapshot, upload it. Create a database from it, or restore it into an existing one.

Several files, or a copy over commands. Use RedisShake (and RIOT for time series) as for self-hosted, with the Cloud database as the target:

[redis_writer]
address = "my-db.cloud.kividb.io:6380"
username = "kividbapp"
password = "..."
tls = true

riot ... --target-tls --target-cacert kividb-ca.pem --target-user kividbapp --target-pass "$KIVIDB_PASSWORD" \
  ... rediss://my-db.cloud.kividb.io:6380

For a sharded database, set cluster = true in the writer (or --target-cluster for RIOT) and give my-db-s1.cloud.kividb.io:6380. A sharded database cannot be created from an imported file, so this is how to fill one.

Search indexes

After the data is in, run each FT.CREATE again. KiviDB indexes the keys that are already there. Text matching has no stemming or phonetic matching; check queries that relied on them.

Active-Active databases

An Active-Active database keeps its values as conflict-free replicated types, and Redis documents no export format for moving one elsewhere. Copy it over commands from one participating region's endpoint with RIOT, which reads values with ordinary commands:

riot replicate --struct --mode LIVE \
  --source-user default --source-pass "$REDIS_PASSWORD" \
  --target-tls --target-cacert kividb-ca.pem --target-user kividbapp --target-pass "$KIVIDB_PASSWORD" \
  redis://my-aa-db.example.redis-cloud.com:12345 rediss://my-db.cloud.kividb.io:6380

We have not tested this against an Active-Active database: do a trial copy and run the verify script before you plan the cutover. Counters (INCR on an Active-Active string) come across as their current value.