# AI-Driven Penetration Testing: The Shift from Automation to Agentic Workflows

Published: 2025-01-15 · Last reviewed: 2026-10-25 · By HackerGPT Team
Canonical: https://hackergpt.org/blog/ai-driven-penetration-testing-the-shift-from-automation-to-agentic-workflows/

> How an agentic AI pentest works (observe, decide, act, remember), traced through the real route table of CyberGym, and the three things an agent still misses: business logic, chained context and scope judgement.

**TL;DR:** Agentic AI penetration testing puts a language model in a loop: it observes the target (pages, traffic, responses), decides what is worth testing next, acts with a tool such as a browser or an HTTP request, and keeps what it learned in a memory it carries into the next turn. That loop is good at breadth and at following leads, but it still misses business-logic flaws, checks and uses that live far apart in the code, and judgement about scope and impact, which is why its output is a set of findings for a person to verify, not the equivalent of a manual pentest.

This article is for developers and technical owners of a web app who are deciding whether an AI agent can test it, and for pentesters who want to know what such an agent is doing under the hood. It explains the mechanism, walks the loop through the real source of [CyberGym](https://cybergym.hackergpt.app), our deliberately vulnerable practice app, and names what the loop does not do. If you want the step-by-step method for testing your own site, that is [How to pentest your own website](https://hackergpt.org/blog/how-to-pentest-your-own-website/); this article is the part underneath it. Everything here applies only to targets you own or have written permission to test.

## How does agentic AI penetration testing work?

An agentic pentest runs a loop of **observe → decide → act → remember** until it runs out of leads, budget or scope. The model does not just answer a question; it chooses the next action, runs it through a tool, reads the result and chooses again.

The pattern comes from research on language-model agents. [ReAct (Yao et al., 2022)](https://arxiv.org/abs/2210.03629) showed that a model which interleaves reasoning steps with actions plans and gathers information better than one that only reasons or only acts. Security tooling adopted the same shape:

1. **Observe.** The agent reads what the target exposes: rendered pages, the HTTP traffic behind them, response codes, error messages, and in a white-box test the source code and route table.
2. **Decide.** The model reasons over what it has seen: this endpoint takes an object id, that one returned a stack trace, this form reflects input. It picks the most promising next step.
3. **Act.** It runs that step with a tool: a browser action, a crafted request, a scanner, a script. The action must stay inside the scope a person set before the run.
4. **Remember.** The result goes into the agent's working memory, which is finite, so long runs are summarised as they go.

![AI pentest agent loop (observe, decide, act, memory) and its limits, traced on CyberGym's basket routes](https://hackergpt.org/blog/ai-driven-penetration-testing-the-shift-from-automation-to-agentic-workflows/agent-loop.webp) *The loop and where it is weak, applied to three basket routes in CyberGym's server.ts. Green is what the loop does well; amber is where a person decides.*

### Why is that different from a scanner?

A classic dynamic scanner (DAST) sends a fixed set of payloads to every input it can reach and matches the responses against signatures. It is fast and repeatable, and it does not change plan when the application answers in an unexpected way. An agent does change plan: a `403` sends it to look at how the request was rejected, a new endpoint in the traffic gets added to the list, and an error message becomes the next hypothesis. That adaptivity is the whole difference, and it is also where the new failure modes come from.

### Agentic workflows and tool orchestration

The model is the reasoning part; the tools do the work. The [ReAct](https://arxiv.org/abs/2210.03629) pattern (Yao et al., 2022) interleaves the two for exactly this reason: the model reasons about what to do, a tool does it, and the result feeds the next step of reasoning. In HackerGPT, [CTF Mode](https://hackergpt.org/features/ctf-mode/) drives a headless browser, watches the HTTP traffic and adjusts what it tries next to how the application responds, explaining each step, and keeps what it learned about the target in a Memory tab. [AppMap](https://hackergpt.org/features/appmap/) does the observe step at scale: it crawls an app into a state graph of pages and logins and lists every API endpoint with its auth requirement. We cover that mapping step in depth in [Map a web app's attack surface before you test it](https://hackergpt.org/blog/map-web-app-attack-surface/).

## What does the loop look like on a real app?

Here is the loop traced through CyberGym's source rather than against the live site: we read the code, nothing was sent to `cybergym.hackergpt.app`. CyberGym is a re-branded derivative of [OWASP Juice Shop](https://owasp.org/www-project-juice-shop/) (MIT licence), so every flaw below is there on purpose, for practice. We read a local checkout at commit `dfee8b3`.

### Observe: read the route table

A white-box agent starts where a pentester would: the list of routes and the middleware in front of each. We ran this against our checkout:

```bash
grep -nE "^\s*app\.(use|get|post|put)\('/(rest/basket|api/BasketItems)" server.ts
```

```text
350:  app.use('/rest/basket', security.isAuthorized(), security.appendUserId())
352:  app.use('/api/BasketItems', security.isAuthorized())
353:  app.use('/api/BasketItems/:id', security.isAuthorized())
393:  app.use('/rest/basket/:id', security.isAuthorized())
394:  app.use('/rest/basket/:id/order', security.isAuthorized())
420:  app.put('/api/BasketItems/:id', security.appendUserId(), basketItems.quantityCheckBeforeBasketItemUpdate())
421:  app.post('/api/BasketItems', security.appendUserId(), basketItems.quantityCheckBeforeBasketItemAddition(), basketItems.addBasketItem())
596:  app.get('/rest/basket/:id', retrieveBasket())
597:  app.post('/rest/basket/:id/checkout', placeOrder())
598:  app.put('/rest/basket/:id/coupon/:coupon', applyCoupon())
```

Every basket route sits behind `isAuthorized()`. A rule-based check that asks "is there auth middleware on this route?" stops here and reports nothing.

### Decide: authentication is not ownership

In `lib/insecurity.ts`, `isAuthorized()` is a JWT check: it confirms the caller holds a valid token. It says nothing about *whose* basket `:id` is. So the useful decision is not "auth present, move on" but "read the handler and see whether it ties the id to the caller". That is the step that separates an agent from a checklist.

### Act: read the three handlers

Reading the handlers gives three different outcomes, and they show where the loop is strong and where it is not.

- **`routes/basket.ts`** looks the basket up with `findOne({ where: { id } })`, with no user in the query. That is a straightforward IDOR lead, and the kind of pattern an agent recognises well, because it is local: the id comes in and the lookup is on the next line. We take that bug apart, with a local reproduction and the fix, in [IDOR vulnerability example: find it and fix it](https://hackergpt.org/blog/idor-vulnerability-example/).
- **`routes/basketItems.ts`** *does* have an ownership check: it compares the caller's basket with `basketIds[0]`. But the item it saves uses `basketIds[basketIds.length - 1]`. An agent that asks "is there an ownership check?" finds one and moves on. Seeing that the check and the write read different elements of the same list is the **chained-context** part, and it is the kind of detail that gets lost.
- **`quantityCheck`** in the same file compares the requested quantity with the per-user limit and the stock, and nothing else. Whether a quantity below one is a bug depends on a rule that is not written in the code: a shop does not pay you to take its products. That is **business logic**, and it is decided by what the business meant, not by what the code says.

None of this needed a live request. In a black-box test the same loop runs on traffic instead of source, and the same three outcomes appear: the local pattern is found readily, the split check needs the agent to connect two requests, and the business rule needs a person to say what should be allowed.

## What can't an AI pentest agent do?

An AI agent does not replace the judgement a pentest depends on. Three gaps show up again and again.

### Business logic

The [OWASP Web Security Testing Guide](https://wstg.owasp.org/v4.2/4-Web_Application_Security_Testing/10-Business_Logic_Testing/00-Introduction_to_Business_Logic/) says business-logic flaws "cannot be detected by a vulnerability scanner" and that testing them relies on knowing "the complete business process and its rules". An agent is better than a scanner at reading intent from names and flows, but it still infers the rules; it was not in the meeting where they were set. Refund flows, coupon stacking, approval steps and negative quantities belong here.

### Chained context

The basket check above is the small version of a big problem: the check is in one place, the use is in another, and in a real engagement the two may be hours of exploration apart. Research shows how much the agent depends on context it is handed: in [Fang et al., 2024](https://arxiv.org/abs/2404.08144) the same GPT-4 agent exploited 87% of 15 one-day vulnerabilities when it was given the CVE description, and 7% without it. Memory and summaries help, but a summary keeps what the agent thought mattered at the time.

### Scope and judgement

Whether a host is in scope, whether a destructive test is acceptable on production, whether a finding matters to this business: those are decisions, not observations. [NIST SP 800-115](https://csrc.nist.gov/pubs/sp/800/115/final), the US government's technical guide to security testing (2008), treats planning and rules of engagement as part of the test itself. An agent follows the scope it is given; it does not negotiate it.

### The confident-liar problem

Models also report things that are not there. Anthropic's November 2025 report on an attack campaign that misused its Claude Code tool notes that the model "occasionally hallucinated credentials or claimed to have extracted secret information that was in fact publicly-available" ([Anthropic, 2025](https://www.anthropic.com/news/disrupting-AI-espionage)). That is why a finding without a reproduction is a lead, not a result.

## Is an AI pentest as good as a manual pentest?

No, and it is not trying to be the same thing. An AI pass covers breadth quickly and follows the leads it can see; a manual pentest brings scoping, business context, chaining across a whole engagement and accountability for the conclusions. The honest comparison:

|  | AI agent pass | Manual pentest |
| --- | --- | --- |
| Coverage of known vulnerability classes | broad, repeatable, can run often | depends on the time booked |
| Following leads and adapting | yes, within one run's memory | yes, across the whole engagement |
| Business-logic flaws | weak: it infers the rules | strong: the tester asks for them |
| Scope and risk decisions | follows the scope it is given | sets and negotiates it |
| Output | findings to verify | findings with a tester's judgement behind them |

Use the agent between engagements and on every change, and use a person when the stakes or the logic call for one. The two are complements, not substitutes.

## How far has autonomous attack automation gone?

Further than it had when this article was first published in January 2025, which is why the defensive case for running the loop on your own app first has grown. The research came first. [ReAct](https://arxiv.org/abs/2210.03629) was posted on 6 October 2022. [Fang et al.](https://arxiv.org/abs/2404.08144), posted on 11 April 2024, gave a GPT-4 agent built on ReAct the CVE descriptions for 15 one-day vulnerabilities and it exploited 87% of them, but only 7% without the description. Then came use in the wild. In a [report published on 13 November 2025](https://www.anthropic.com/news/disrupting-AI-espionage) on a campaign it detected in mid-September 2025, Anthropic says the threat actor "was able to use AI to perform 80-90% of the campaign", with humans stepping in at a few decision points, against roughly thirty targets. Read together, the reports say the same thing: given direction and context, an agent can do a lot of the mechanical work; without them it does much less, and it still needs checking.

![Timeline: ReAct 2022, one-day exploit agent Apr 2024, Anthropic campaign report Nov 2025](https://hackergpt.org/blog/ai-driven-penetration-testing-the-shift-from-automation-to-agentic-workflows/timeline.webp)

*Each date is the publication date of a source cited in this section; the sources are listed under the chart.*

## Key takeaways

- Treat an AI pentest as a loop of observe, decide, act, remember, and ask any tool you evaluate how it handles each step, especially memory.
- Check that an agent's findings go past "auth present": authentication middleware on a route says nothing about who owns the object.
- Write down your business rules (refunds, limits, approvals) and test them yourself or with a person; that is where agents are weakest.
- Treat every AI finding as a lead until it has a reproduction you have run.
- Set the scope before the run, and practise the whole loop on [CyberGym](https://cybergym.hackergpt.app) before pointing it at your own app.

## FAQ

### What is an autonomous pentest agent?

It is a language model connected to security tools (a browser, an HTTP client, scanners) that chooses its own next step: it observes the target, decides what to test, runs the test and learns from the result, in a loop. It works within a scope a person sets, and its output is a list of findings with evidence for a person to verify.

### Can an AI agent pentest my web app?

It can run a useful first pass on an app you own or have written permission to test: mapping the attack surface, testing known vulnerability classes and following leads. It does not cover business-logic flaws, long chains of context or scope decisions as well as a person, so treat it as a frequent, broad check rather than a replacement for a manual test.

### Is an AI pentest as good as a manual pentest?

No. An AI pass is broader and faster and can run on every change; a manual pentest adds scoping, business context, chaining across the whole engagement and a tester's judgement on what matters. They answer different needs, and many teams use both.

### What vulnerabilities do AI pentest agents miss most often?

Business-logic flaws, such as refunds, discounts, quantities and approval steps that work as coded but not as intended, are the most common gap, because the rule is not in the code. Next come flaws where a check and the value it protects are far apart, which require the agent to keep a long chain of context.

### Do AI pentest agents hallucinate findings?

They can. Models sometimes report credentials, secrets or vulnerabilities that do not exist, or overstate what they found. Require a reproduction for every finding and rerun it yourself before acting on it.

## Sources

- [Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022)](https://arxiv.org/abs/2210.03629)
- [Fang et al., LLM Agents can Autonomously Exploit One-day Vulnerabilities (2024)](https://arxiv.org/abs/2404.08144)
- [Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign (November 2025)](https://www.anthropic.com/news/disrupting-AI-espionage)
- [OWASP Web Security Testing Guide v4.2, Introduction to Business Logic](https://wstg.owasp.org/v4.2/4-Web_Application_Security_Testing/10-Business_Logic_Testing/00-Introduction_to_Business_Logic/)
- [NIST SP 800-115, Technical Guide to Information Security Testing and Assessment (2008)](https://csrc.nist.gov/pubs/sp/800/115/final)
- [OWASP Juice Shop](https://owasp.org/www-project-juice-shop/), the project CyberGym is derived from

Want to see the loop on your own app? Point [HackerGPT](https://hackergpt.app) at a site you own, or practise first on [CyberGym](https://cybergym.hackergpt.app).