Hosted models

KiviDB can run the models behind its AI features for you, so you need no account with a model provider and no API key. Choose KiviDB hosted (kividb in the API) as the provider, and the tokens used are billed from your credit.

Model	Used by	Price
Amazon Titan Text Embeddings V2, 256, 512 or 1024 dimensions (titan-embed-text-v2)	the semantic cache, agent memory's search, search by meaning, and the embeddings API below	$0.05 per million tokens
Amazon Nova Lite (nova-lite)	agent memory's extraction and summaries	$0.15 per million tokens read, $0.60 per million written

A token is roughly four characters of English. A semantic cache answering 10,000 lookups a day of 200-character prompts uses about 15 million tokens a month: $0.75. Usage shows on the Usage page as Hosted embeddings, Hosted model input and Hosted model output, by database, and is billed by the hour.

Your own keys keep working: each semantic cache namespace and each agent memory store chooses its providers, and a store can use a hosted model for one job and your own for the other.

Where your text goes

Text is sent to Amazon Bedrock in AWS's us-east-1 region, whatever region your database is in, and the vectors and replies come back to be stored in your database. Bedrock does not store prompts or replies, and does not use them to train models. If your data must stay in one region, use your own provider.

Speed

Every embedding is a call to Bedrock in us-east-1 from KiviDB's API in eu-central-1, about 210 ms. The rest of a semantic cache lookup is the API's own round trips to your database, so it depends on where the database is. We measured a whole hosted lookup at about 250 ms for a database in eu-central-1 and about 750 ms for one in us-east-1. Sending your own vectors skips the embedding call. Either is still a fraction of a language model's answer, which is what a cache hit saves.

Limits

• Credit. Hosted models answer an organization only while its credit covers a month of its running databases and pipelines, plus $1. Otherwise calls are refused with insufficient_credits (402), and so is choosing kividb in a namespace or store.
• Rate. Calls are rate-limited per organization, at roughly 20 a second. Past that they are refused with hosted_ai_rate_limited (429); a refusal from the model itself is hosted_ai_busy (429). Retry after a short wait.
• Size. A text to embed can be up to 50,000 characters (text_too_long, 400).
• Dimensions. The embedding model makes 256, 512 or 1024 dimensions; any other size is refused when a namespace or store is saved.

Semantic cache

curl -X PUT https://api.kividb.io/instances/$DB/semantic-cache/support-bot \
  -H "Authorization: Bearer $KIVIDB_API_KEY" -H "Content-Type: application/json" \
  -d '{"provider": "kividb", "dimensions": 1024}'

A hosted namespace's threshold starts at 0.45: Titan scores reworded questions lower than OpenAI's models do (0.29 to 0.78 in our measurements, with unrelated questions at most 0.29). The semantic cache page has the measurements, and covers lookups, puts and tuning the threshold.

Agent memory

curl -X PUT https://api.kividb.io/instances/$DB/agent-memory/support-agent \
  -H "Authorization: Bearer $KIVIDB_API_KEY" -H "Content-Type: application/json" \
  -d '{"llm_provider": "kividb", "embed_provider": "kividb", "dimensions": 1024}'

See agent memory.

Embeddings API

Embed your own texts, for example documents to store beside a search index:

curl -X POST https://api.kividb.io/instances/$DB/embeddings \
  -H "Authorization: Bearer $KIVIDB_API_KEY" -H "Content-Type: application/json" \
  -d '{"input": ["A red waterproof jacket", "Hiking boots, size 42"], "dimensions": 1024}'

{
  "model": "titan-embed-text-v2",
  "dimensions": 1024,
  "data": [
    { "index": 0, "embedding": [0.0213, -0.0441, ...] },
    { "index": 1, "embedding": [-0.0102, 0.0378, ...] }
  ]
}

input is one text or up to 64. The vectors are normalised, so cosine similarity and inner product rank them the same way. The tokens are billed to the database in the path. The key needs the Member role or higher.

Search by meaning

An index whose vectors came from the hosted model (from the embeddings API above, at one size) can be searched by typing in the console's Search tab, or with query_text and an embed_model of titan-embed-text-v2-256, titan-embed-text-v2-512 or titan-embed-text-v2-1024 in the search API. The query is embedded with the same model and size, so the scores compare like with like. The Search tab picks the model whose size matches the index. Queries embedded with the local all-MiniLM-L6-v2 model (384 dimensions) are not billed.

Changing models

Vectors from two models cannot be compared, even at the same size. So a cache or store that already holds vectors refuses a change of embedding model or of dimensions (not_empty, 409): flush the cache, or delete the store's memories, first. Changing to none (you send your own vectors) is allowed.