Get the data out: migrate from Upstash to KiviDB

In the Upstash console, open the database, then Backups → Backup & Export → Export. When the export finishes, a Download button appears in the backups…

Option 1: export an RDB file (simplest)

In the Upstash console, open the database, then Backups → Backup & Export → Export. When the export finishes, a Download button appears in the backups table. Download the .rdb file.

The file holds the database at the moment of the export: writes made after it are not in it. Upstash notes that Redis Functions are not included.

KiviDB reads RDB files written by Redis 6.2 to 8.10. If loading the file stops with an error about its format or version, use option 2 instead.

Option 2: a copy tool over the Redis protocol

Two open-source tools copy over normal commands and both work with KiviDB 1.2.0:

ToolReads withKeeps
RedisShake 4.x, scan modeSCAN, DUMP, PTTLEvery type, TTLs, stream consumer groups
RIOT 4.xSCAN, then Lua scripts (EVALSHA) that run TYPE, PTTL and DUMPEvery type and TTLs

Run the copy from a machine near the Upstash region, and allow that machine on KiviDB Cloud under Configuration → Network → Allowed IPs.

RedisShake. Download it:

curl -LO https://github.com/tair-opensource/RedisShake/releases/download/v4.7.0/redis-shake-v4.7.0-linux-amd64.tar.gz
tar xzf redis-shake-v4.7.0-linux-amd64.tar.gz

Write shake.toml:

[scan_reader]
address = "my-db-12345.upstash.io:6379"
username = "default"
password = "upstash-password"
tls = true
ksn = false     # true to keep following changes (see the cutover)
count = 1000    # keys per SCAN: fewer SCAN commands on the bill

[redis_writer]
address = "your-name.cloud.kividb.io:6380"
username = "kividbapp"
password = "kividb-password"
tls = true

[advanced]
dir = "shake-data"
log_file = "shake.log"
rdb_restore_command_behavior = "rewrite"   # replace keys already on KiviDB

For a self-hosted KiviDB, put its address in [redis_writer] (port 6380 by default, or its --tls-port), and tls = false if it has no TLS. Run it:

./redis-shake shake.toml

The log ends with scanStandaloneReader restore finished and all done when the copy is complete.

RIOT.

curl -LO https://github.com/redis/riot/releases/download/v4.3.0/riot-standalone-4.3.0-linux-x86_64.zip
unzip riot-standalone-4.3.0-linux-x86_64.zip

riot-standalone-4.3.0-linux-x86_64/bin/riot replicate \
  --source-tls --source-user default --source-pass "$UPSTASH_PASSWORD" \
  --target-tls --target-cacert kividb-ca.pem --target-user kividbapp --target-pass "$KIVIDB_PASSWORD" \
  --ttl-tolerance 10s \
  rediss://my-db-12345.upstash.io:6379 \
  rediss://your-name.cloud.kividb.io:6380

At the end RIOT compares the two sides, which reads every key again on Upstash. Add --compare NONE to skip it, and use the verify script on a sample instead.

The compare also reads KiviDB through Lua scripts, so on a KiviDB without Lua (the default build, or a Cloud database without the Lua add-on) it fails with ERR this build of KiviDB has no Lua after the copy has completed: another reason to use --compare NONE. Read RIOT's output rather than trusting its exit code, which can be 0 after a failed read.

What a copy costs on Upstash

On pay-as-you-go, Upstash bills every command the copy sends, including the ones inside a pipeline. Measured against a test server, the copy tools send:

ToolCommands on the source per key
RedisShake, scan mode (count = 1000)2 (DUMP, PTTL), plus one SCAN per 1,000 keys
RIOT replicate --compare NONE4 to 5: a Lua script (EVALSHA) running TYPE, PTTL, DUMP, and TIME for keys with a TTL. Upstash bills a script as one command plus one per command inside it
RIOT replicate, with its compare at the end7 to 8: the above, then a second script per key for the compare

At Upstash's pay-as-you-go price of $0.20 per 100,000 commands, a one-off copy costs about $4 per million keys with RedisShake, about $9 with RIOT and --compare NONE, and about $15 with RIOT's compare. A live copy (keyspace notifications on) adds two commands for every change your application makes while it runs. The values cross the network too, and count towards Upstash's bandwidth allowance.

  • Free plan: 500,000 commands a month cover about 250,000 keys with RedisShake, less what your application uses. Export the RDB file instead.
  • Fixed plans: commands are not billed; only the bandwidth limit applies.
  • Throughput: Upstash allows 10,000 commands a second on most plans, so a million keys take at least about three and a half minutes with RedisShake.

Check current prices on Upstash's pricing page before a large copy.

What does not work

  • Replication (SYNC, PSYNC, REPLICAOF): not in Upstash's command set, so redis-cli --rdb and kividb --replicaof can't read from it.
  • The REST API for copying: @upstash/redis does not support DUMP, and reading every key through REST costs the same commands with more overhead. Copy over the Redis protocol.