A server that acts as an API front-end, receiving API requests, enforcing throttling and security policies, and routing them to the back-end.
The API Gateway is a central enforcement point for Control Boundaries in a microservices architecture. In a Private AI deployment, it inspects inbound prompts and outbound LLM responses, applies rate limits, enforces authorization, and provides a single choke point for detecting and blocking Prompt Injection attempts before they reach the model.
See also Control Boundaries, Microservices, Private AI, Prompt Injection, Air-Gapped Environment and Role-Based Access Control.