How It Works
Teams use tools like Garak to test their AI-powered applications before shipping updates. These tools help detect unsafe AI models and apps before they ever reach users, a protective security layer of product development.
You can see Garak in action with a few minutes of setup. Choose a path to start:
Use Existing API Endpoint (Fastest)
Test your deployed models directly via API integration
Jump to API TestingRun Locally on Your Machine
Follow the steps below to run Google's open-source model, Gemma, on NVIDIA's testing framework, Garak
Select your operating system:
Install Ollama
Easy Installation
Ollama is a free tool that runs AI models locally on your computer
Download Test Model
Model Download
Secure, verified AI model from Google
Setup Python and pip
Python Setup
Install Python (the programming language that powers most AI tools) and pip (installs Python packages). Garak is written in Python, so we need these to run the security tests on your AI model.
Setup Python Environment & Install Dependencies
Virtual Environment
Creates a separate Python workspace that won't interfere with other projects on your computer. This keeps Garak's dependencies organized and prevents conflicts with your existing software.
Verify Setup
System Verification
Confirm all components are working before testing
Run Security Tests
Comprehensive Scanning
35+ security tests across multiple attack vectors
API Integration Testing
Test your deployed models via API integration before releasing updates to your customers. Ensure your production systems remain secure.
Test Your Custom AI Models
Secure your deployed models with two simple commands using NVIDIA's free, open-source Garak framework.
Three-Step Security Testing
Test your custom API endpoints with these ready-to-use commands
Quick Security Test
Run a fundamental jailbreak vulnerability test on your custom API endpoint
# Quick jailbreak test
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" -p dan.DAN_Jailbreak -d dan.DAN --generations 1Comprehensive Security Test
Run all available security probes for thorough vulnerability assessment
# Comprehensive testing with all probes
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" --config broadProduction Security Test
Test with custom headers and authentication for production-ready security
# With custom headers and authentication
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" --config broad --rest_headers '{"Authorization": "Bearer your-token", "X-Custom-Header": "value"}'Command Center
Ready-to-deploy security testing commands for your production models
# Quick jailbreak test
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" -p dan.DAN_Jailbreak -d dan.DAN --generations 1
# Comprehensive security testing
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" --config broad
# Extended testing with multiple payloads
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" --config full
# With custom headers and authentication
python -m garak -m rest -n "https://your-api.com/v1/chat/completions" --config broad --rest_headers '{"Authorization": "Bearer your-token", "X-Custom-Header": "value"}'
# List all available security tests
python -m garak --list_probes# Set your OpenAI API key
export OPENAI_API_KEY="your-api-key-here"
# Quick jailbreak test
python -m garak -m openai -n gpt-4 -p dan.DAN_Jailbreak -d dan.DAN --generations 1
# Comprehensive security testing
python -m garak -m openai -n gpt-4 --config broad
# Extended testing with multiple payloads
python -m garak -m openai -n gpt-4 --config full
# Test specific vulnerability categories
python -m garak -m openai -n gpt-4 -p promptinject,encoding,leakreplay --generations 2
# List all available security tests
python -m garak --list_probes# Set your Anthropic API key
export ANTHROPIC_API_KEY="your-api-key-here"
# Quick jailbreak test
python -m garak -m anthropic -n claude-3-sonnet -p dan.DAN_Jailbreak -d dan.DAN --generations 1
# Comprehensive security testing
python -m garak -m anthropic -n claude-3-sonnet --config broad
# Extended testing with multiple payloads
python -m garak -m anthropic -n claude-3-sonnet --config full
# Test specific vulnerability categories
python -m garak -m anthropic -n claude-3-sonnet -p promptinject,encoding,leakreplay --generations 2
# List all available security tests
python -m garak --list_probesVulnerability Scanning
Garak probes your models for real-world attack scenarios across multiple vulnerability categories. Choose the right testing approach for your needs.
Vulnerability Categories Tested
Jailbreak Attacks
Tests if your model can be tricked into ignoring safety constraints and generating harmful content through various attack techniques.
Prompt Injection
Attempts to manipulate your model's behavior through carefully crafted input prompts and encoding techniques.
Data Leakage
Checks if your model accidentally reveals sensitive training data, internal information, or system prompts.
Content Safety
Evaluates your model's ability to avoid generating toxic, harmful, or inappropriate content across various contexts.
Malware Generation
Tests whether your model can be prompted to generate malicious code, scripts, or other harmful software.
System Manipulation
Checks for vulnerabilities that could allow manipulation of system behavior, file access, or security boundaries.
Run python -m garak --list_probes to see all available tests