Cut over: migrate from Aerospike to KiviDB
Aerospike Community Edition cannot stream its changes, so the safe order is: send new writes to both stores first, copy the rest, then fix up what the copy…
Aerospike Community Edition cannot stream its changes, so the safe order is: send new writes to both stores first, copy the rest, then fix up what the copy raced with.
-
Prepare KiviDB. Start it (with
--aofon a self-hosted server), or create the Cloud database. Create the search indexes. -
Dual-write. Deploy a version of the application that writes every change to Aerospike and to KiviDB (the same conversion as the loader: Changing your application), deletes from both, and still reads from Aerospike. Note the time it went live everywhere as
T0(Unix seconds). A failed KiviDB write should be logged, not fail the request: the catch-up pass repairs it. -
Bulk copy. Run the loader. Records written during the copy may be copied from an older read, after the application already wrote the newer value.
-
Catch up. Re-copy everything written since
T0; this puts back the newest value of every record touched during the copy:python as2kivi.py --aerospike 10.0.0.5:3000 --namespace app --kividb "$KIVIDB_URL" --mode json --since "$T0"It uses its own checkpoint file,
as2kivi-since-<T0>.done. -
Prune. Remove keys whose record was deleted while the copy ran (the copy could have read a record just before the application deleted it):
python as2kivi.py --aerospike 10.0.0.5:3000 --namespace app --kividb "$KIVIDB_URL" --prune -
Verify. Run the verify script. Counts should match per set, and the sample should show no problems.
-
Switch reads. Move reads to KiviDB, a service or a percentage at a time, while still writing to both. Compare results if you can.
-
Stop writing to Aerospike. Once reads have run on KiviDB long enough to trust it, remove the Aerospike writes. Keep Aerospike running, unwritten, for a while.
Rollback. Until step 8, Aerospike still has every write: move reads back. After step 8, rolling back means copying KiviDB back into Aerospike, so keep dual-writing until you are sure.
If you can stop writes instead. Put the application in maintenance, run the loader, run verify, and switch. No dual-write, catch-up or prune needed; the downtime is the copy time, so measure it on a copy of your data first.
Enterprise Edition with XDR or Kafka. If the changes already flow to Kafka through Aerospike Connect, start a consumer that applies them to KiviDB before step 3, and skip steps 2, 4 and 5.