Get the data out: migrate from Upstash to KiviDB
In the Upstash console, open the database, then Backups → Backup & Export → Export. When the export finishes, a Download button appears in the backups…
Option 1: export an RDB file (simplest)
In the Upstash console, open the database, then Backups → Backup & Export →
Export. When the export finishes, a Download button appears in the
backups table. Download the .rdb file.
The file holds the database at the moment of the export: writes made after it are not in it. Upstash notes that Redis Functions are not included.
KiviDB reads RDB files written by Redis 6.2 to 8.10. If loading the file stops with an error about its format or version, use option 2 instead.
Option 2: a copy tool over the Redis protocol
Two open-source tools copy over normal commands and both work with KiviDB 1.2.0:
| Tool | Reads with | Keeps |
|---|---|---|
| RedisShake 4.x, scan mode | SCAN, DUMP, PTTL | Every type, TTLs, stream consumer groups |
| RIOT 4.x | SCAN, then Lua scripts (EVALSHA) that run TYPE, PTTL and DUMP | Every type and TTLs |
Run the copy from a machine near the Upstash region, and allow that machine on KiviDB Cloud under Configuration → Network → Allowed IPs.
RedisShake. Download it:
curl -LO https://github.com/tair-opensource/RedisShake/releases/download/v4.7.0/redis-shake-v4.7.0-linux-amd64.tar.gz
tar xzf redis-shake-v4.7.0-linux-amd64.tar.gz
Write shake.toml:
[scan_reader]
address = "my-db-12345.upstash.io:6379"
username = "default"
password = "upstash-password"
tls = true
ksn = false # true to keep following changes (see the cutover)
count = 1000 # keys per SCAN: fewer SCAN commands on the bill
[redis_writer]
address = "your-name.cloud.kividb.io:6380"
username = "kividbapp"
password = "kividb-password"
tls = true
[advanced]
dir = "shake-data"
log_file = "shake.log"
rdb_restore_command_behavior = "rewrite" # replace keys already on KiviDB
For a self-hosted KiviDB, put its address in [redis_writer] (port 6380 by
default, or its --tls-port), and tls = false if it has no TLS. Run it:
./redis-shake shake.toml
The log ends with scanStandaloneReader restore finished and all done when the
copy is complete.
RIOT.
curl -LO https://github.com/redis/riot/releases/download/v4.3.0/riot-standalone-4.3.0-linux-x86_64.zip
unzip riot-standalone-4.3.0-linux-x86_64.zip
riot-standalone-4.3.0-linux-x86_64/bin/riot replicate \
--source-tls --source-user default --source-pass "$UPSTASH_PASSWORD" \
--target-tls --target-cacert kividb-ca.pem --target-user kividbapp --target-pass "$KIVIDB_PASSWORD" \
--ttl-tolerance 10s \
rediss://my-db-12345.upstash.io:6379 \
rediss://your-name.cloud.kividb.io:6380
At the end RIOT compares the two sides, which reads every key again on Upstash.
Add --compare NONE to skip it, and use the verify script on a
sample instead.
The compare also reads KiviDB through Lua scripts, so on a KiviDB without Lua
(the default build, or a Cloud database without the Lua add-on) it fails with
ERR this build of KiviDB has no Lua after the copy has completed: another
reason to use --compare NONE. Read RIOT's output rather than trusting its exit
code, which can be 0 after a failed read.
What a copy costs on Upstash
On pay-as-you-go, Upstash bills every command the copy sends, including the ones inside a pipeline. Measured against a test server, the copy tools send:
| Tool | Commands on the source per key |
|---|---|
RedisShake, scan mode (count = 1000) | 2 (DUMP, PTTL), plus one SCAN per 1,000 keys |
RIOT replicate --compare NONE | 4 to 5: a Lua script (EVALSHA) running TYPE, PTTL, DUMP, and TIME for keys with a TTL. Upstash bills a script as one command plus one per command inside it |
RIOT replicate, with its compare at the end | 7 to 8: the above, then a second script per key for the compare |
At Upstash's pay-as-you-go price of $0.20 per 100,000 commands, a one-off copy
costs about $4 per million keys with RedisShake, about $9 with RIOT and
--compare NONE, and about $15 with RIOT's compare. A live copy (keyspace
notifications on) adds two commands for every change your application makes
while it runs. The values cross the network too, and count towards Upstash's
bandwidth allowance.
- Free plan: 500,000 commands a month cover about 250,000 keys with RedisShake, less what your application uses. Export the RDB file instead.
- Fixed plans: commands are not billed; only the bandwidth limit applies.
- Throughput: Upstash allows 10,000 commands a second on most plans, so a million keys take at least about three and a half minutes with RedisShake.
Check current prices on Upstash's pricing page before a large copy.
What does not work
- Replication (
SYNC,PSYNC,REPLICAOF): not in Upstash's command set, soredis-cli --rdbandkividb --replicaofcan't read from it. - The REST API for copying:
@upstash/redisdoes not supportDUMP, and reading every key through REST costs the same commands with more overhead. Copy over the Redis protocol.