LLM Agents Can Autonomously Hack Websites
Free while signed in. Answers cite the passages they came from.
The paper shows GPT-4 agents with tool use and long context can autonomously exploit real websites, including performing blind SQL injection and schema extraction.
Attack capabilities: GPT-4 agents autonomously perform SQL injection, extract database schemas blindly, and chain together multi-step exploits without any prior knowledge of the specific vulnerability.
Frontier-only: Only GPT-4 demonstrates this capability; tested open-source models and GPT-3.5 fail, suggesting a clear capability threshold enabled by frontier scale.
Real-world vulnerabilities: GPT-4 successfully discovered vulnerabilities in real websites in the wild, not just in lab-constructed targets.
Safety implications: Provides concrete evidence that frontier LLM capabilities can directly translate into offensive cyber capability, informing open-release and deployment decisions.
Get next week’s papers.
The same picks and the same summaries, in your inbox. Free, and 176 issues deep.
Subscribe on Substack