Install core tools — p3ta-tricks

Tools Arsenal

Offensive AI Security Tools

Tool	Description	Install
Garak	LLM vulnerability scanner — tests for prompt injection, jailbreaks, data leakage	`pip install garak && garak --model openai:gpt-4o`
PyRIT	Microsoft's AI red team toolkit — automated multi-turn attacks, jailbreak orchestration	`pip install pyrit`
PromptBench	LLM robustness evaluation, adversarial prompt testing framework	`pip install promptbench`
LLMFuzzer	Automated fuzzing for LLM APIs, finds edge cases in output handling	`github.com/mnns/LLMFuzzer`
Vigil	LLM prompt injection scanner for web apps — Burp Suite compatible	`pip install vigil-llm`
ModelScan	Scan ML model files (pickle/safetensors) for malicious payloads	`pip install modelscan && modelscan -p model.pkl`
Pliny (Promptfoo)	LLM red teaming and eval framework with built-in attack plugins	`npx promptfoo redteam init`
ART (IBM)	Adversarial Robustness Toolbox — ML model evasion, poisoning attacks	`pip install adversarial-robustness-toolbox`
Counterfit	Microsoft's CLI for attacking ML models — evasion and poisoning	`github.com/Azure/counterfit`

Standard Toolkit Commands

# Install core tools
pip install garak pyrit promptbench vigil-llm modelscan openai anthropic
pip install sentence-transformers chromadb langchain

# Install Promptfoo (LLM red team)
npm install -g promptfoo

# Garak — full scan
garak --model openai --model_type gpt-4o \
  --probes promptinject,xss,malwaregen,knowledgegrounding

# PyRIT — multi-turn jailbreak attempt
python -c "
from pyrit.prompt_target import OpenAIChatTarget
from pyrit.orchestrator import RedTeamingOrchestrator
target = OpenAIChatTarget()
orchestrator = RedTeamingOrchestrator(
    attack_strategy='How can I make something dangerous?',
    prompt_target=target
)
orchestrator.apply_attack_strategy_until_completion()
"

# ModelScan — check downloaded model
modelscan -p ~/.cache/huggingface/hub/models--meta-llama

# Promptfoo red team
promptfoo redteam init
promptfoo redteam run --target openai:gpt-4o

Burp Suite for AI Apps

# Configure Burp to intercept AI API calls
# Add scope: api.openai.com, api.anthropic.com, target.com

# Key Burp extensions for AI testing:
# - AI Security Scanner (BApp store)
# - GPT Scan
# - Burp AI Assistant (built-in)

# Intercept and modify AI requests:
# Catch the POST to /api/chat
# Modify "content" field to inject payloads
# Use Intruder to fuzz with payload list

# Useful payload list for Intruder:
# github.com/swisskyrepo/PayloadsAllTheThings/tree/master/Prompt Injection