Semantic cache

Large-language-model calls are slow and cost money, and many of the prompts an application sends mean the same thing in different words. A semantic cache…