Change your application: migrate from Memcached to KiviDB
The address changes from memcached.internal:11211 to KiviDB's endpoint (port 6380 unless you set --port), and every client now sends a password.
The address changes from memcached.internal:11211 to KiviDB's endpoint (port
6380 unless you set --port), and every client now sends a password.
Python: pymemcache or pylibmc → redis-py
Before:
from pymemcache.client.base import Client
mc = Client(("memcached.internal", 11211))
mc.set("user:1", "alice", expire=300)
name = mc.get("user:1") # bytes, or None
found = mc.get_many(["user:1", "user:2"]) # dict of the keys found
mc.add("lock:report", "1", expire=60) # only if missing
mc.incr("hits", 5)
mc.append("log:1", "more")
mc.touch("user:1", 900)
mc.delete("user:1")
After:
import os
import redis
r = redis.Redis(
host="kividb.internal",
port=6380,
password=os.environ["KIVIDB_PASSWORD"],
decode_responses=True,
)
r.set("user:1", "alice", ex=300)
name = r.get("user:1") # str, or None
values = r.mget(["user:1", "user:2"]) # a list in key order, None for missing
r.set("lock:report", "1", ex=60, nx=True) # only if missing
r.incrby("hits", 5)
r.append("log:1", "more")
r.expire("user:1", 900)
r.delete("user:1")
get_many returns a dict of the keys it found; mget returns a list with
None in the place of each missing key. pylibmc's get_multi, set, add,
incr and touch translate the same way.
Java: spymemcached or XMemcached → Jedis
Before (spymemcached):
MemcachedClient mc = new MemcachedClient(new InetSocketAddress("memcached.internal", 11211));
mc.set("user:1", 300, "alice");
Object name = mc.get("user:1");
Map<String, Object> found = mc.getBulk("user:1", "user:2");
mc.add("lock:report", 60, "1");
mc.incr("hits", 5);
CASValue<Object> cv = mc.gets("config");
mc.cas("config", cv.getCas(), "v2");
After (Jedis 5):
JedisPooled r = new JedisPooled("kividb.internal", 6380, null, System.getenv("KIVIDB_PASSWORD"));
r.set("user:1", "alice", SetParams.setParams().ex(300));
String name = r.get("user:1");
List<String> values = r.mget("user:1", "user:2");
r.set("lock:report", "1", SetParams.setParams().nx().ex(60));
r.incrBy("hits", 5);
// gets + cas: see "Compare-and-set" below
XMemcached's set, get, add, incr and gets/cas map the same way.
spymemcached serializes Java objects for you. Jedis stores strings and byte
arrays, so decide how each cached object becomes one: JSON with Jackson or
Gson, or a hash (HSET) if you read single fields.
Node.js: memcached → ioredis
Before (the memcached package, callbacks):
const Memcached = require("memcached");
const mc = new Memcached("memcached.internal:11211");
mc.set("user:1", JSON.stringify({ name: "alice" }), 300, (err) => {});
mc.get("user:1", (err, data) => {});
mc.getMulti(["user:1", "user:2"], (err, found) => {});
mc.add("lock:report", "1", 60, (err) => {});
mc.incr("hits", 5, (err, value) => {});
After (ioredis, promises):
import Redis from "ioredis";
const redis = new Redis({
host: "kividb.internal",
port: 6380,
password: process.env.KIVIDB_PASSWORD,
});
await redis.set("user:1", JSON.stringify({ name: "alice" }), "EX", 300);
const user = JSON.parse(await redis.get("user:1"));
const values = await redis.mget("user:1", "user:2"); // null for missing
await redis.set("lock:report", "1", "EX", 60, "NX");
await redis.incrby("hits", 5);
PHP: Memcached → phpredis
Before:
$mc = new Memcached();
$mc->addServer('memcached.internal', 11211);
$mc->set('user:1', ['name' => 'alice'], 300);
$user = $mc->get('user:1');
$found = $mc->getMulti(['user:1', 'user:2']);
$mc->add('lock:report', '1', 60);
$mc->increment('hits', 5);
After:
$r = new Redis();
$r->connect('kividb.internal', 6380);
$r->auth(getenv('KIVIDB_PASSWORD'));
$r->set('user:1', json_encode(['name' => 'alice']), ['ex' => 300]);
$user = json_decode($r->get('user:1'), true);
$values = $r->mget(['user:1', 'user:2']); // false for missing
$r->set('lock:report', '1', ['nx', 'ex' => 60]);
$r->incrBy('hits', 5);
PHP's Memcached serializes arrays and objects for you (and sets a client
flag to say so); phpredis stores strings unless you set
Redis::OPT_SERIALIZER. Encode explicitly, as above, or set the option. For
PHP sessions, change session.save_handler = memcached to
session.save_handler = redis and session.save_path to
"tcp://kividb.internal:6380?auth=<password>".
TLS and passwords
Memcached usually runs with no authentication on a private network. KiviDB
Cloud always requires a password, and self-hosted KiviDB should have one
(--requirepass, or ACL users). With TLS on, pass the CA certificate:
redis-cli -h your-name.cloud.kividb.io -p 6380 -a "$KIVIDB_PASSWORD" --tls --cacert kividb-ca.pem
Connect to your instance shows the client settings for TLS in each language.
Expiry
Memcached reads an expiry up to 30 days (2,592,000 seconds) as seconds from now, and anything larger as an absolute Unix time. Redis clients keep the two apart:
| Memcached call | KiviDB |
|---|---|
set(k, v, expire=300) | SET k v EX 300 |
set(k, v, expire=1767225600) (a Unix time) | SET k v EXAT 1767225600 |
set(k, v, expire=0) | SET k v (no expiry) |
Translating a Unix time to EX gives a key that lives for decades.
Counters
incron a missing key fails in Memcached;INCRin KiviDB creates the key at 0 first. Code that relied on the failure to detect a missing counter needs anEXISTScheck, orSET k 0 NXbefore it.decrstops at 0 in Memcached;DECRBYgoes negative.- Memcached counters are unsigned 64-bit; KiviDB's are signed 64-bit.
Compare-and-set
Memcached's gets returns a CAS token, and cas writes only if the item has
not changed since. KiviDB has two ways to do the same, neither needing Lua:
WATCH / MULTI / EXEC. Watch the key, read it, then write in a
transaction. If anyone wrote the key in between, EXEC returns nil and you
retry.
def cas_update(r, key, change):
with r.pipeline() as p:
while True:
try:
p.watch(key)
new = change(p.get(key))
p.multi()
p.set(key, new)
p.execute()
return new
except redis.WatchError:
continue # someone else wrote it: read again
SET ... IFEQ (KiviDB 1.2 and later). Write only if the current value is
the one you read. One round trip, and no token: the value is the token.
old = r.get("config")
if r.set("config", new_value, ifeq=old) is None:
... # changed since we read it: read again
If values can repeat (A, then B, then A again) and that matters, keep a version number in the value or in a hash field, and compare that.
Client flags
Memcached stores a 32-bit number next to each value, and clients use it to say how they serialized the value (string, integer, pickled, compressed). A Redis value has no such field. When you copy items:
- If every item has flags
0, the bytes are the value; nothing to do. - If not, the new client can't tell how to decode them. Either decode each item with the old client's serializer and write it the way the new client reads it, or don't copy those items and let them warm.
More than one server
Memcached clients spread keys over a list of servers by hashing (often ketama). There is nothing to replace on the server side:
- One KiviDB server. KiviDB runs commands on every core, so one server often replaces several Memcached nodes. Remove the server list; connect to one address.
- A KiviDB cluster. For more memory than one machine has, use cluster mode
and a cluster-aware client (redis-py's
RedisCluster, Jedis'sJedisCluster, ioredis'sCluster). Keys are placed by hash slot, and the client follows the cluster's own map; there is no client-side hash ring to configure. Multi-key commands such asMGETneed all their keys in one slot (use a{hash tag}in the key) or the client splits them.