Limits and gotchas: migrate from Redis to KiviDB

What does not come across

• Search indexes. Index definitions, dictionaries and aliases from the Redis search module are not in what KiviDB loads; the keys are. Run FT.CREATE again and it indexes the keys already there. KiviDB has no stemming or phonetic matching, and refuses GEO and GEOSHAPE fields.
• Other modules' data (time series, Bloom and the other probabilistic types, vector sets) does not load from an RDB file or RESTORE. A file that holds them is refused whole. Move them by script, and filter them out of the copy.
• Function libraries without Lua. A file with FUNCTION libraries loads them in a lua or full build; the default build loads the data and leaves the libraries out.
• Sentinel. There is none in KiviDB. See above.
• Configuration and users are not in an RDB file or a copy. Set them on KiviDB yourself, as above.
• Keyspace notification settings, CLIENT names and Pub/Sub messages are not data and do not move. Pub/Sub subscribers reconnect and resubscribe; a message published during the switch is lost, as with any Redis failover.

Gotchas

• HyperLogLogs. KiviDB 1.2.0 reads Redis's HyperLogLogs and counts them the same (PFCOUNT, PFADD and PFMERGE gave identical answers in our tests). Earlier KiviDB releases did not, so the verify script compares PFCOUNT on both sides.

• RedisShake and Redis 8.6+ streams. RedisShake 4.7.0's sync_reader and rdb_reader stop with unknown rdb value type byte … type=[27] on a stream written by Redis 8.6 or later; rdb_reader exits having copied only part of the file. Use scan_reader (with ksn = true to follow writes), replicate, or load the RDB file into KiviDB directly, for those sources.

• RIOT and Lua. --compare QUICK (the default) needs Lua on the target. Use --compare NONE and the verify script instead.

• RIOT and databases. RIOT copies one logical database per run.

• Hash field expiry times can come across 1 ms later than on Redis after an RDB load or replication. Field TTLs are otherwise kept; the verify script allows for it.

• Key names must be valid UTF-8. KiviDB refuses any other key name in SET, RESTORE and the commands RedisShake, RIOT and MIGRATE send (ERR key must be valid UTF-8), and RIOT stops at the first one. Binary key names (raw hashes, packed integers) need renaming at the source first. This lists them:

  python3 - "redis://:$PASSWORD@source.internal:6379" <<'EOF'
  import sys, redis
  bad = 0
  for db in range(16):  # the databases your source uses
      r = redis.Redis.from_url(sys.argv[1], db=db)
      for key in r.scan_iter(count=1000):
          try:
              key.decode("utf-8")
          except UnicodeDecodeError:
              bad += 1
              print(f"db{db} {key!r}")
  print(f"{bad} key names are not valid UTF-8")
  EOF
  

• Copying back to Redis has limits; see Rollback.

• replica-priority is accepted by KiviDB 1.2.0 but not reported in INFO replication, so it does not keep Sentinel from promoting a KiviDB replica.

• Run each KiviDB with its own config file (--configfile). Without one, it reads /etc/kividb/kividb.conf; a server that was started without that file and then told CONFIG REWRITE (as Sentinel does) writes it there, and every KiviDB on the machine started afterwards reads it.

• save points are accepted and ignored. Use --aof for durability, or BGSAVE on a schedule.

• Port. KiviDB listens on 6380 by default. Pass --port 6379 to take Redis's place on the same address.

• Protected mode is on: with no password and no --bind, only local clients are accepted. Set --requirepass or an ACL file before opening it up.