Change your application: migrate from Memcached to KiviDB

The address changes from memcached.internal:11211 to KiviDB's endpoint (port 6380 unless you set --port), and every client now sends a password.

The address changes from memcached.internal:11211 to KiviDB's endpoint (port 6380 unless you set --port), and every client now sends a password.

Python: pymemcache or pylibmc → redis-py

Before:

from pymemcache.client.base import Client

mc = Client(("memcached.internal", 11211))

mc.set("user:1", "alice", expire=300)
name = mc.get("user:1")                       # bytes, or None
found = mc.get_many(["user:1", "user:2"])     # dict of the keys found
mc.add("lock:report", "1", expire=60)         # only if missing
mc.incr("hits", 5)
mc.append("log:1", "more")
mc.touch("user:1", 900)
mc.delete("user:1")

After:

import os
import redis

r = redis.Redis(
    host="kividb.internal",
    port=6380,
    password=os.environ["KIVIDB_PASSWORD"],
    decode_responses=True,
)

r.set("user:1", "alice", ex=300)
name = r.get("user:1")                        # str, or None
values = r.mget(["user:1", "user:2"])         # a list in key order, None for missing
r.set("lock:report", "1", ex=60, nx=True)     # only if missing
r.incrby("hits", 5)
r.append("log:1", "more")
r.expire("user:1", 900)
r.delete("user:1")

get_many returns a dict of the keys it found; mget returns a list with None in the place of each missing key. pylibmc's get_multi, set, add, incr and touch translate the same way.

Java: spymemcached or XMemcached → Jedis

Before (spymemcached):

MemcachedClient mc = new MemcachedClient(new InetSocketAddress("memcached.internal", 11211));

mc.set("user:1", 300, "alice");
Object name = mc.get("user:1");
Map<String, Object> found = mc.getBulk("user:1", "user:2");
mc.add("lock:report", 60, "1");
mc.incr("hits", 5);
CASValue<Object> cv = mc.gets("config");
mc.cas("config", cv.getCas(), "v2");

After (Jedis 5):

JedisPooled r = new JedisPooled("kividb.internal", 6380, null, System.getenv("KIVIDB_PASSWORD"));

r.set("user:1", "alice", SetParams.setParams().ex(300));
String name = r.get("user:1");
List<String> values = r.mget("user:1", "user:2");
r.set("lock:report", "1", SetParams.setParams().nx().ex(60));
r.incrBy("hits", 5);
// gets + cas: see "Compare-and-set" below

XMemcached's set, get, add, incr and gets/cas map the same way.

spymemcached serializes Java objects for you. Jedis stores strings and byte arrays, so decide how each cached object becomes one: JSON with Jackson or Gson, or a hash (HSET) if you read single fields.

Node.js: memcached → ioredis

Before (the memcached package, callbacks):

const Memcached = require("memcached");
const mc = new Memcached("memcached.internal:11211");

mc.set("user:1", JSON.stringify({ name: "alice" }), 300, (err) => {});
mc.get("user:1", (err, data) => {});
mc.getMulti(["user:1", "user:2"], (err, found) => {});
mc.add("lock:report", "1", 60, (err) => {});
mc.incr("hits", 5, (err, value) => {});

After (ioredis, promises):

import Redis from "ioredis";

const redis = new Redis({
  host: "kividb.internal",
  port: 6380,
  password: process.env.KIVIDB_PASSWORD,
});

await redis.set("user:1", JSON.stringify({ name: "alice" }), "EX", 300);
const user = JSON.parse(await redis.get("user:1"));
const values = await redis.mget("user:1", "user:2");      // null for missing
await redis.set("lock:report", "1", "EX", 60, "NX");
await redis.incrby("hits", 5);

PHP: Memcached → phpredis

Before:

$mc = new Memcached();
$mc->addServer('memcached.internal', 11211);

$mc->set('user:1', ['name' => 'alice'], 300);
$user = $mc->get('user:1');
$found = $mc->getMulti(['user:1', 'user:2']);
$mc->add('lock:report', '1', 60);
$mc->increment('hits', 5);

After:

$r = new Redis();
$r->connect('kividb.internal', 6380);
$r->auth(getenv('KIVIDB_PASSWORD'));

$r->set('user:1', json_encode(['name' => 'alice']), ['ex' => 300]);
$user = json_decode($r->get('user:1'), true);
$values = $r->mget(['user:1', 'user:2']);                  // false for missing
$r->set('lock:report', '1', ['nx', 'ex' => 60]);
$r->incrBy('hits', 5);

PHP's Memcached serializes arrays and objects for you (and sets a client flag to say so); phpredis stores strings unless you set Redis::OPT_SERIALIZER. Encode explicitly, as above, or set the option. For PHP sessions, change session.save_handler = memcached to session.save_handler = redis and session.save_path to "tcp://kividb.internal:6380?auth=<password>".

TLS and passwords

Memcached usually runs with no authentication on a private network. KiviDB Cloud always requires a password, and self-hosted KiviDB should have one (--requirepass, or ACL users). With TLS on, pass the CA certificate:

redis-cli -h your-name.cloud.kividb.io -p 6380 -a "$KIVIDB_PASSWORD" --tls --cacert kividb-ca.pem

Connect to your instance shows the client settings for TLS in each language.

Expiry

Memcached reads an expiry up to 30 days (2,592,000 seconds) as seconds from now, and anything larger as an absolute Unix time. Redis clients keep the two apart:

Memcached callKiviDB
set(k, v, expire=300)SET k v EX 300
set(k, v, expire=1767225600) (a Unix time)SET k v EXAT 1767225600
set(k, v, expire=0)SET k v (no expiry)

Translating a Unix time to EX gives a key that lives for decades.

Counters

  • incr on a missing key fails in Memcached; INCR in KiviDB creates the key at 0 first. Code that relied on the failure to detect a missing counter needs an EXISTS check, or SET k 0 NX before it.
  • decr stops at 0 in Memcached; DECRBY goes negative.
  • Memcached counters are unsigned 64-bit; KiviDB's are signed 64-bit.

Compare-and-set

Memcached's gets returns a CAS token, and cas writes only if the item has not changed since. KiviDB has two ways to do the same, neither needing Lua:

WATCH / MULTI / EXEC. Watch the key, read it, then write in a transaction. If anyone wrote the key in between, EXEC returns nil and you retry.

def cas_update(r, key, change):
    with r.pipeline() as p:
        while True:
            try:
                p.watch(key)
                new = change(p.get(key))
                p.multi()
                p.set(key, new)
                p.execute()
                return new
            except redis.WatchError:
                continue          # someone else wrote it: read again

SET ... IFEQ (KiviDB 1.2 and later). Write only if the current value is the one you read. One round trip, and no token: the value is the token.

old = r.get("config")
if r.set("config", new_value, ifeq=old) is None:
    ...  # changed since we read it: read again

If values can repeat (A, then B, then A again) and that matters, keep a version number in the value or in a hash field, and compare that.

Client flags

Memcached stores a 32-bit number next to each value, and clients use it to say how they serialized the value (string, integer, pickled, compressed). A Redis value has no such field. When you copy items:

  • If every item has flags 0, the bytes are the value; nothing to do.
  • If not, the new client can't tell how to decode them. Either decode each item with the old client's serializer and write it the way the new client reads it, or don't copy those items and let them warm.

More than one server

Memcached clients spread keys over a list of servers by hashing (often ketama). There is nothing to replace on the server side:

  • One KiviDB server. KiviDB runs commands on every core, so one server often replaces several Memcached nodes. Remove the server list; connect to one address.
  • A KiviDB cluster. For more memory than one machine has, use cluster mode and a cluster-aware client (redis-py's RedisCluster, Jedis's JedisCluster, ioredis's Cluster). Keys are placed by hash slot, and the client follows the cluster's own map; there is no client-side hash ring to configure. Multi-key commands such as MGET need all their keys in one slot (use a {hash tag} in the key) or the client splits them.