An optimization technique that caches LLM responses based on the semantic meaning of the prompt rather than exact string matching.
Semantic Caching intercepts user queries, converts them into vector embeddings, and checks them for conceptual similarity against previously cached requests. If a user asks 'How do I reset my password?' and the cache contains an answer for 'Steps to change password', it returns the cached response. Each cache hit avoids a model call, which lowers inference cost and latency.
See also Data Mobilization, Data Lakehouse and Vector Database.