The short version
  • Apply least privilege to data retrieval and tool execution.
  • Assume model input and retrieved content can be hostile.
  • Log decisions and test realistic misuse paths before launch.

01

Map the attack surface

Document where prompts enter, which data sources can be retrieved, what tools the system can call, where outputs are stored, and which downstream actions can create business impact.

02

Prioritize practical controls

Use identity-aware retrieval, input isolation, output validation, tool allowlists, rate limits, secret handling, and human approval for consequential actions. Controls should fail closed where possible.

03

Test and monitor

Build evaluations for prompt injection, sensitive data leakage, unsafe tool use, misleading output, and bypass attempts. Monitor both technical events and user-reported failures.

Common questions

Questions worth asking

Is prompt filtering enough to secure a generative AI app?

No. Filtering is one layer. Strong systems also isolate permissions, constrain tools, validate outputs, protect data sources, and monitor behavior.