Kivi Lens
Kivi Lens records each turn of an AI agent as a trace: the model calls and the tokens they used, tool calls, retrievals, and the semantic cache, agent…
Kivi Lens records each turn of an AI agent as a trace: the model calls and the tokens they used, tool calls, retrievals, and the semantic cache, agent memory and context tool calls KiviDB answered along the way. Your application sends spans with an OpenTelemetry SDK; Lens keeps them in one of your KiviDB databases and shows them in the console as a list of traces and, for each one, a waterfall.
Lens is part of Kivi Synapse. There is no extra charge beyond the database: traces use the database's own memory, up to a ceiling you set.
Set it up
- In the console, open Lens (or a database's Lens tab) and create a lens on a running database. Choose how long traces are kept (1 to 30 days, 7 by default) and the two ceilings described under Limits.
- On the lens's Setup tab, make an ingest key. It starts
kvlens_, is shown once, and is stored only as a hash. A key writes to that one lens and cannot read anything. - Point an OpenTelemetry exporter at the lens's ingest URL,
https://api.kividb.io/lens/<lens id>/v1/traces, with the headerAuthorization: Bearer kvlens_....
The ingest URL is shown on the Setup tab. https://api.kividb.io/lens/v1/traces
also works: the key says which lens it is for.
Lens takes OTLP/HTTP in both encodings, protobuf (application/x-protobuf,
what most SDKs send by default) and JSON (application/json), gzip-compressed
or not. The stock OpenTelemetry exporters work as they are.
Python
# pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
exporter = OTLPSpanExporter(
endpoint="https://api.kividb.io/lens/<lens id>/v1/traces",
headers={"Authorization": "Bearer kvlens_..."},
)
provider = TracerProvider(resource=Resource.create({"service.name": "support-agent"}))
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("support-agent")
with tracer.start_as_current_span("agent.turn"):
with tracer.start_as_current_span("chat gpt-4o-mini") as span:
span.set_attribute("gen_ai.operation.name", "chat")
span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
span.set_attribute("gen_ai.usage.input_tokens", 812)
span.set_attribute("gen_ai.usage.output_tokens", 64)
Node.js
// npm i @opentelemetry/api @opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-http @opentelemetry/resources
import { BasicTracerProvider, BatchSpanProcessor } from "@opentelemetry/sdk-trace-base";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { resourceFromAttributes } from "@opentelemetry/resources";
import { trace } from "@opentelemetry/api";
const exporter = new OTLPTraceExporter({
url: "https://api.kividb.io/lens/<lens id>/v1/traces",
headers: { Authorization: "Bearer kvlens_..." },
});
const provider = new BasicTracerProvider({
resource: resourceFromAttributes({ "service.name": "support-agent" }),
spanProcessors: [new BatchSpanProcessor(exporter)],
});
trace.setGlobalTracerProvider(provider);
Go, Java, .NET and the rest
Any SDK's OTLP/HTTP exporter works with the standard environment variables:
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.kividb.io/lens/<lens id>/v1/traces
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_TRACES_HEADERS=Authorization=Bearer%20kvlens_...
Use the HTTP exporter (otlptracehttp in Go), not the gRPC one: Lens has no
gRPC endpoint.
Through an OpenTelemetry Collector
Forward from an OpenTelemetry Collector
with the otlphttp exporter:
exporters:
otlphttp/lens:
traces_endpoint: https://api.kividb.io/lens/<lens id>/v1/traces
headers:
Authorization: Bearer kvlens_...
service:
pipelines:
traces:
receivers: [otlp]
exporters: [otlphttp/lens]
The reply comes in the request's encoding: an ExportTraceServiceResponse
(with partial_success when spans were not kept), or a google.rpc.Status on
an error.
What is kept
Each span keeps its trace id, span id and parent, name, kind, start and end
times, status and status message, attributes, events, the service.name of its
resource and its instrumentation scope.
Spans with the OpenTelemetry GenAI conventions show as model calls:
| Attribute | Shown as |
|---|---|
gen_ai.request.model, gen_ai.response.model | the model |
gen_ai.usage.input_tokens (or gen_ai.usage.prompt_tokens) | input tokens |
gen_ai.usage.output_tokens (or gen_ai.usage.completion_tokens) | output tokens |
gen_ai.operation.name | the operation, such as chat |
gen_ai.system, gen_ai.provider.name | the provider |
Each trace has a summary: its root span's name and service, every service in it, its start and duration, its span and error counts, its token totals and the models it called. The trace list shows these, newest first, and filters by service, by status (errors or none), by time, and by words in span names and attribute values.
A trace's spans can arrive in any order and in any number of requests (a
SimpleSpanProcessor sends each one alone; several services send parts of one
trace). They are put together as they arrive. A span sent again, as an
exporter does when it retries, replaces itself and is not counted twice.
In the database
Everything is kept under keys starting kividb:lens:{<lens id>}: in the
database the lens is on, with a search index kividb:lens:{<lens id>}:idx over
the trace summaries. Each trace's keys expire with the lens's retention, counted
from the last span that arrived. Deleting a lens removes all of them.
Self-tracing
Send a W3C traceparent header with a call to KiviDB's Synapse APIs and, when
the organization has a lens on that same database with Trace KiviDB's own
calls on (the default), KiviDB records a span for the call as a child of your
span, in your trace:
Each span is named kividb <operation>:
| Operation | What it records |
|---|---|
semantic_cache.lookup | hit or miss, the score (or the best score on a miss), the threshold, the entry id |
semantic_cache.put | the entry id |
agent_memory.recall | messages and memories returned, whether there was a summary, the memory ids |
agent_memory.search | mode, results, top score, the memory ids |
agent_memory.add | the memory id, and whether it was a duplicate |
agent_memory.add_messages | the number of messages |
context.tool_call | the model, the tool, records returned |
intent_router.match | the router, the intent matched (if any) and its score, the threshold |
intent_router.match_batch | the router, how many texts were sent and how many matched, the threshold |
playbook.discover | the playbook, the kinds asked for, results, top score, the FAQ ids and skill names returned |
Every span also has the store, cache, model, router or playbook name, the actor and session where there is one, and how long the call took. A prompt, query, task or memory text is kept only as a preview of its first 120 characters; responses and records are never kept.
These spans are written in the background through a queue of at most 2,000 spans. A span that finds the queue full, or whose write fails, is dropped and counted in the lens's KiviDB spans dropped figure. The call itself is never slowed or failed by it.
Most OpenTelemetry HTTP instrumentations (for fetch, requests, httpx)
send traceparent on outgoing requests already.
Memory inspector
A lens's Memory tab lists the agent memory stores on the same database. Each opens in Agent memory, where long-term memories are searched by actor, by words and by meaning, and sessions are listed. A memory span in a trace lists the memory ids it read or wrote, and its side panel opens the store.
From an assistant
The KiviDB MCP server has two read-only tools: lens_search_traces (the list,
with the same filters) and lens_get_trace (one trace with every span). They
are not offered in the ChatGPT app, because traces hold whatever your
application put in them, user content included.
Limits
| Retention | 1 to 30 days after a trace's last span (7 by default) |
| Traces per lens | 100 to 100,000 (10,000 by default); past it the oldest are removed first |
| Memory per lens | 1 to 1,024 MB of stored spans (64 by default); past it the oldest traces are removed first |
| Spans per trace | 10,000; later spans are dropped and counted |
| Bytes per trace | 16 MB; later spans are dropped and counted |
| Attribute value | 64 KB; a longer value is cut and ends with …[truncated by KiviDB Lens: N bytes] |
| Attributes per span and per event | 256; more are dropped and counted |
| Events per span | 128 |
| Request body | 4 MB, after gzip or deflate is undone |
| Requests per ingest key | 600 a minute |
| Lenses per database | 5 |
| Ingest keys per lens | 20 |
An export that loses spans to these limits, or that has invalid spans, gets
200 with OTLP's partialSuccess (rejectedSpans and the reason); the rest is
kept. The lens's Traces tab counts every span dropped, rejected or cut, and
every trace removed by a ceiling.
A stopped database answers exports with 503, which OpenTelemetry exporters
retry.
Lens is not available on sharded databases yet.
API
With an API key or a console session, as for the rest of the API:
GET /lens | every lens in the organization |
PUT /instances/{id}/lens/{name} | create or change one: retention_days, max_traces, max_megabytes, self_trace |
GET /instances/{id}/lens/{name} | its settings, counts and ingest keys |
DELETE /instances/{id}/lens/{name} | delete it and every trace it keeps |
POST /instances/{id}/lens/{name}/keys | make an ingest key {name} |
DELETE /instances/{id}/lens/{name}/keys/{key id} | revoke one |
GET /instances/{id}/lens/{name}/traces | traces, newest first: q, service, status, name, from, to (ms), limit, offset |
GET /instances/{id}/lens/{name}/traces/{trace id} | one trace: summary and spans |
Reading traces needs a role that may read data, as reading keys does.