Continuous red teaming
A static security review goes stale the moment a model is upgraded. AI.Guard runs continuous, scheduled adversarial probes against your endpoints, scores their resistance, and detects regression as upstream models change underneath you.
Attack catalog & generation
The catalog is continuously updated from published research, observed in-the-wild attacks, and findings from customer pilots. Base attacks (like bias, PII extraction, and toxicity attacks) are generated synthetically, then modified and expanded using multiple strategies for hardening the attacking prompts. Once the LLM answers all red-teaming prompts, its responses are evaluated against metrics. Responses that fall below the defined standards expose the model's weak spots and set the direction for further hardening and defense.
| Attack family | Representative examples | Severity | Detection approach |
|---|---|---|---|
| Direct prompt injection | DAN, role-override, instruction-leak, system-prompt extraction | Critical | Classifier + structural heuristics |
| Indirect injection via RAG | Poisoned document, hidden HTML, encoded instructions in retrieved content | Critical | Source-tagged content boundary checks |
| Jailbreak variants | Roleplay framings, hypothetical reframings, low-resource language attacks | High | Multilingual classifier ensemble |
| Tool / function-call abuse | Coercing tool calls with unsafe arguments, parameter smuggling | High | Schema validation + intent scoring on arguments |
| Training-data extraction | Divergence prompts, repetition attacks, completion-leak probes | Medium | Output entropy + memorization fingerprints |
| Data exfiltration via output | Encoded PII, side-channel formatting, markdown image beacons | High | Output DLP + outbound link policy |
| Denial-of-wallet / cost abuse | Long context floods, model-shopping, retry storms | Medium | Token quotas + per-tenant rate limits |
Scheduled probe cadence
Probes run against your staging endpoints continuously and against production on a controlled schedule with synthetic-traffic markers, so real users are never affected and findings never pollute your business metrics.
1. Baseline
Catalog every reachable LLM endpoint, capture current policy posture, and run an initial 1,200-probe adversarial sweep across staging.
2. Continuous probes
Lightweight scheduled attacks against non-prod traffic. Detect drift after model upgrades or prompt-template changes within minutes.
3. Deep campaigns
Full attack-catalog sweep with multilingual, indirect-injection, and tool-abuse variants. Compares pass-rate against baseline.
4. Drift & report
Score deltas, regressions, and policy-tuning recommendations delivered as a signed report. Tickets opened automatically for new gaps.
Scoring framework
Each endpoint receives a per-family resistance score plus an aggregate posture grade. Scores are weighted by severity and by realistic attacker access.
Resistance score
Per attack family, the share of probes correctly blocked or safely neutralized. Reported with confidence intervals.
Drift detection
Any drop in resistance after an upstream model revision, prompt template change, or policy update is flagged within the next probe cycle.
Signed reports
Monthly artifact suitable for board / regulator review. Cryptographically signed, references the exact policy and model versions tested.
Sample report excerpt
A stylized excerpt from a monthly assurance report. Real reports include per-app breakdowns, regression deltas, and remediation guidance.
Indirect-injection score dropped 1.6 pts after the 2026-04-12 upstream model revision. Recommended action: enable the rag.boundary.strict rule for affected apps. Estimated recovery: +1.4 pts.