Sharded databases

A sharded database spreads its data over several shards, each on its own machine, so it can grow past what one machine holds: up to 4 TB. It runs the engine's cluster mode, and your application talks to the shards directly through a cluster-aware client.

Where they are available

Sharded databases are available on AWS in us-east-1 (N. Virginia) and us-west-2 (Oregon). Other regions and clouds follow as capacity is reserved there. Checkout tells you when an arrangement is larger than a region can take today, and offers a capacity request instead of payment.

Choosing a size

You choose the total size. Checkout suggests an arrangement of shards and lets you pick another one:

• 3 to 64 shards, each 4, 8, 16, 32 or 64 GB;
• Essentials: one machine per shard, no replicas;
• Pro: each shard has its own replica (up to 3), in another availability zone, and fails over by itself.

Databases above 512 GB are provisioned once we have reserved the machines for them: send a capacity request from checkout and we reply by email. Nothing is charged until you launch.

A sharded database reserves at least one month of credit when you order it (see Credits and reservations). It cannot be stopped when your balance runs out, so the reservation keeps it running while you top up.

Sharded databases need KiviDB 1.1.0 or later and a name. The name becomes each shard's hostname, so names ending in -s and a number are reserved.

Connecting

Each shard has its own hostname: {name}-s1.cloud.kividb.io, {name}-s2.cloud.kividb.io and so on. Give your client any one of them; it reads the cluster's layout and finds the rest.

from redis.cluster import RedisCluster

r = RedisCluster(host="orders-s1.cloud.kividb.io", port=6379,
                 username="kividbapp", password="...")
r.set("user:42", "Ada")

import Redis from "ioredis";

const r = new Redis.Cluster([{ host: "orders-s1.cloud.kividb.io", port: 6379 }], {
  redisOptions: { username: "kividbapp", password: "..." },
});

With TLS, connect to port 6380 (rediss://) with your database's CA certificate, as for any other database.

A command that touches several keys works only when they are on the same shard. Keep related keys together with a hash tag: {user:42}:cart and {user:42}:orders share a shard because the part in braces decides where a key lives. Without one, a multi-key command answers CROSSSLOT.

Search works across the shards: create an index once and a search counts the documents on every shard. FT.SPELLCHECK, FT.PROFILE and FT.HYBRID answer from one shard for now.

If one shard is down, the others keep serving the keys they hold.

Not available yet

These answer with a clear refusal on a sharded database rather than acting on one shard:

• stop and resume;
• resizing, and adding or removing shards;
• turning TLS on or off after creation (choose it when you order);
• private connectivity;
• disk snapshots, and creating a sharded database from an import;
• downloading a sharded KDB snapshot, moving it between regions, or restoring it into an existing database. A KDB snapshot can seed a new sharded database with the same number of shards;
• databases in your own cloud account (BYOC);
• the console's Schedules, Configuration and Persistence sections. Overview, Metrics, Explorer, Search, Logs and the Danger Zone work.