System 05

Semantic Cache Layer

Type two questions and see whether the cache reuses the first answer for the second, and whether it should.

What it does

A cache in front of a language model saves money by reusing an old answer when a new question means the same thing. The risk is that two questions can look almost identical and mean opposite things. Free trial against paid trial. Enable against disable. After 90 days against after 30 days. Serve the wrong one and the visitor gets a confident, fluent, wrong answer with nothing to warn them.

Type one question per line. The first is answered and stored. The second is checked against everything stored, and the page shows you the similarity score, the closest stored question, and the decision. Twelve answers are already stored here, so a single line works too.

Every question runs through two caches side by side. One serves on similarity alone. The shipped one adds a second check over the words that change an answer: numbers, names, negations, and opposite pairs. Where the two disagree, the page says which check fired and on which words. At the bottom you get the running hit rate and tokens saved, plus the same comparison measured live over a labelled set of twenty six question pairs.

Try it

How do I rotate a token after 90 days? How do I rotate a token after 3On list endpoints, what is the maximum page size? What is the minimum Which role is allowed to delete a workspace account? Which role is notWhat is the default per workspace rate limit?