Inkog vs Promptfoo
Promptfoo tests model outputs. Inkog secures the agent.
Promptfoo is an eval and red-teaming framework for testing LLM outputs — running prompts, measuring quality, and simulating adversarial inputs. Now acquired by OpenAI and being integrated into OpenAI Frontier. Inkog is an AI agent security platform that analyzes and attacks agent logic — architecture, tool interactions, data flows, delegation chains, runtime behavior, and compliance — across any LLM provider.
Feature Comparison
| Feature | Inkog | Promptfoo |
|---|---|---|
| Agent architecture analysis | ||
| Agent logic & data flow tracing | ||
| Dynamic agent testing (DAST) | ||
| LLM output evaluation (evals) | ||
| Red-team prompt generation | ||
| MCP server auditing | ||
| Tool poisoning detection | ||
| Multi-agent delegation analysis | ||
| Agent loop & recursion detection | ||
| AGENTS.md governance verification | ||
| Agent framework support (15+) | ||
| EU AI Act compliance reports | ||
| SARIF output for CI/CD | ||
| Prompt injection detection | Static + dynamic | Dynamic (adversarial testing) |
| Vendor independence | Works with any LLM provider | Now part of OpenAI |
| Open source | Apache 2.0 | MIT |
When to Use Each Tool
Use Promptfoo when...
Use Promptfoo for evaluating LLM output quality — measuring accuracy, consistency, and robustness of model responses through test suites and red-team simulations. Promptfoo excels at testing what the model says.
Use Inkog when...
Use Inkog for comprehensive agent security — analyzing architecture, attacking agent logic to find vulnerabilities, detecting unsafe tool interactions, missing oversight, tainted data flows, and compliance gaps. Inkog secures the agent, not just the model.
Frequently Asked Questions
What does the OpenAI acquisition mean for Promptfoo users?
Promptfoo is being integrated into OpenAI Frontier. The open-source project continues, but the long-term roadmap for teams using non-OpenAI providers is an open question. Inkog is vendor-independent and works with any LLM provider.
Does Promptfoo analyze agent architecture?
No. Promptfoo tests model outputs — what the LLM responds to given prompts. It does not analyze agent code structure, tool interactions, delegation chains, or data flows. Those require understanding agent logic, which is what Inkog does.
Can I use both Inkog and Promptfoo?
Yes. They cover different layers. Promptfoo tests what the model produces. Inkog tests what your agent does with it — how data flows through tools, where oversight is missing, and whether the architecture is sound.
Is Promptfoo still open source?
The codebase is MIT-licensed and remains on GitHub. As part of OpenAI, the development direction will align with OpenAI Frontier. Inkog is Apache 2.0 and independently maintained.