Back to research

RESEARCH · 03 · AI-Assisted Red Teaming

AI-Assisted Red Teaming: From a Security Task to a Continuous Workflow

ABSTRACT

Red teaming a real application is mostly mechanical. An engineer moves between a browser, an HTTP history, a source tree, a terminal and a tracker, and every move costs a copy and an explanation. This article asks how much of that work an agent can actually perform — and where the engineer's judgement has to stay.

POSITION

The limit on a security engineer is rarely a missing tool. It is that the tools do not share a working context — and that is the part an agent can hold for them.

Updated October 28, 202615 min readOxo Operator
Discover
  1. Explore the targethttps://session-lab.local/login
  2. Inspect the requestPOST200/api/session
  3. Locate candidate codesrc/server/routes/session.ts:42
Operate
  1. Select a toolSemgrep · static analysis, runs locally
  2. Run local rulesSemgrep · bundled rule pack
  3. Read the resultrule matches · needs validation
Verify
  1. Verifyexit 0 · bound to revision 03
  2. Collect evidencetransaction #142 · revision 03
  3. Prepare a findingAuthentication boundary · needs review

One workspace context

  • browser
  • request
  • code
  • tool output
  • evidence

The request the agent captured in step two is the request it correlates in step six; the command it ran in step five is the receipt the finding cites.

Before: nine steps across five tools, each handoff a manual copy.

The workflow a security engineer already follows, in three stages — each step writing into the same record instead of into a different application.

KEY FINDINGS

  1. 01

    The cost of a security task is not the tools. It is the handoffs between them, each of which asks the engineer to carry state that no tool holds.

  2. 02

    Thirteen steps sit between a security question and a tracked finding. Judgement is required at three; the other ten are mechanical, which is what makes them worth delegating.

  3. 03

    An agent can perform the mechanical steps — navigate, capture, search, execute, correlate — while the engineer keeps the three that decide whether the work is any good.

  4. 04

    Automation is only tolerable when side effects are bounded. A step that can change a target needs an approval path, a record, and a way to tell an interrupted run from a failed one.

  5. 05

    What makes this a workflow rather than a sequence of prompts is continuity: the request, the file, the command output and the conclusion stay attached to the same task.

01

The mechanical cost of security work

Ask an engineer what they did yesterday on a web target and the answer is rarely a technique. It is a sequence. They found a feature in the browser, found the request it sent, found the handler that served it, looked up how a scanner option works, copied parameters into a terminal, saved the output, explained the whole thing again to a model, tidied the evidence, wrote the finding, filed the ticket — and then went looking for something they had already read two hours earlier.

StepWhat it producesNeeds judgement
Find the feature in the browserA page and its entry pointsPartly — what to probe
Locate the request it sendsOne HTTP transactionNo
Find the handler in sourceOne or more candidate filesPartly — which candidate matters
Search for a toolA shortlist of scanners or utilitiesNo
Look up how the tool is usedFlags, arguments, exit behaviourNo
Copy parameters acrossThe same request in another programNo
Run it in a terminalRaw output and an exit statusNo
Save the outputA file nobody will find againNo
Re-explain context to an assistantA prompt restating what tools already knewNo
Tidy the evidenceA record a reviewer can followNo
Write the findingA description and an impact claimYes — this is the work
File the ticketA tracked itemNo
Re-find something read earlierThe same page, a second timeNo
One ordinary task, decomposed. The third column is the argument: only three of these steps need a security decision.

The ten mechanical steps are not trivial. Each is easy to get subtly wrong — the wrong request variant, a scan pointed at the wrong host, an output file that loses which target it came from. But none of them requires a security decision, and that is precisely what makes them the part worth handing over.

02

Why a toolchain does not share context

The usual diagnosis is that security engineers need better tools. In practice they have good tools, and the difficulty is that the tools do not share state. The browser knows the page; the proxy knows the transaction; the editor knows the file; the terminal knows the command; the tracker knows the ticket. None of them knows what the others know.

  • The browser holds a rendered page, a DOM, and a session — and nothing about the request it produced.
  • The HTTP history holds transactions and bodies — and nothing about the handler that served them.
  • The editor holds the source, and nothing about which request reached it.
  • The terminal holds output and an exit status, and nothing about which finding it supports.
  • The tracker holds the ticket, and nothing about the evidence behind it.

So the connective tissue is the engineer. Every handoff is a manual serialisation: reading one tool's state and writing a summary of it into the next one. That is why the same investigation feels long even when no single step is slow.

It also explains why adding an assistant to the chain changes less than expected. The assistant receives what the engineer typed: a hand-made summary of six tools' state. The model is not short of capability. The chain is short of shared state.

03

Explore the target

Most web work starts with exploration: reaching an application, finding the features worth looking at, and understanding what the page is doing before deciding what to try. It is the step where judgement appears first, so it is the step to automate most carefully.

In a task-bound workspace, the agent can drive the browser itself: open the scoped application, navigate, read the page, and act on a specific element — including one the engineer pointed at with a comment attached. The engineer can watch the surface change as it works rather than receiving a summary afterwards.

  • What the agent does well here: reach the entry points, follow links, submit a form, and let the resulting traffic be recorded without being asked.
  • What stays with the engineer: deciding which features are in scope, which are worth probing, and when a behaviour is interesting rather than normal.

One boundary is worth stating plainly: an agent explores the scope you gave it. If the target boundary is wrong, delegation does not make it less wrong — it makes the same mistake faster and more thoroughly, which is an argument for defining scope before exploration rather than after.

04

Inspect the request that matters

For most application work the request is the pivot. It is the point where the application's behaviour becomes observable, editable and repeatable, and it is the artifact the rest of the investigation refers back to.

The workspace keeps a traffic ledger scoped to the task, and the agent searches its metadata first — method, host, path, status, type — retrieving bodies only for the transactions it needs. That ordering is not only about speed. It is a limit on how much of a real application reaches the model's context: an agent handed every request body by default has been given far more of the target than the question required.

From there, a selected transaction can be kept as a working copy. The capture stays untouched and the revision is retained separately, which matters because the two mean different things: the capture is what the application did, and the revision is what the engineer intends to try. Sending a revision is a side effect, not a read — it can be approval-gated, and it does not follow redirects automatically, because following a redirect silently is how a controlled test reaches a system nobody meant to touch.

05

Correlate a request with candidate source

The next step is the one engineers most often describe as the tedious one: going from a request to the code that served it. In a workspace, the selected method and path can be carried into a bounded search of the local project, and the search returns candidate files to inspect.

It is worth being precise about what that is. Method and path narrow the search space; they do not establish that a file is the code that ran. A route can be registered elsewhere, generated, or reached through a rewrite. Presenting results as candidates is therefore not hedging, it is accuracy — and it is also more useful, because the engineer's judgement is exactly the instrument that closes the gap.

  • The search is bounded to the workspace and excludes build output, dependencies and repository internals, so the result list reflects the project's own code.
  • Files carry their language and revision, so a candidate is a location in a specific version rather than a filename.
  • A candidate is a starting point for reading, and the record keeps the distinction between what was observed and what was inferred.
06

Operate: tools, terminal, and knowing how to use them

The tool step is really two problems. The first is finding the right tool; the second, larger one is knowing how to use it — which flags, which scope, what a malformed result looks like, and what the tool expects to be pointed at.

In this workspace, tools arrive as plugins that package an MCP server, a set of instructions, or a connected application. Each declares what it can do, and separately what it can change. That second list is the important one: a scanner that reads a workspace is a different object from a tracker that files work items, and the difference should be visible in the tool's own definition rather than learned from an incident.

ToolReadChange
SemgrepStatic analysis with bundled local rules; no account or registry requestNo autofix; no registry or supply-chain tools
OWASP ZAPAlerts, history, scan statusSpider and active-scan operations wait for approval
JiraSearch and read work itemsCreate and update work items, with approval
Three entries from the plugin catalog, with each plugin's own description of what it can and cannot do. The last column is what a workflow has to respect.

The terminal is the second half of the step, and it is where a workspace earns its keep by keeping what usually evaporates: the exact command, its output, and its exit status, attached to the task rather than to a scrollback buffer. It is also the step where a tool's output has to be handled as data. A scanner returns findings to explain, not instructions to follow — an output file is untrusted input, whatever produced it.

07

Preserve what the task produced

Delegating the mechanical steps is only valuable if their output survives. Otherwise the engineer saves a few minutes per step and pays for it later, reconstructing what happened when the ticket is questioned.

  • The original transaction, and the request revision that was derived from it.
  • The command, its output and its exit status.
  • The candidate file and the line that was actually read.
  • The verification that was run, and the revision it applied to.
  • The normalised activity record: which action happened, who or what caused it, and what it changed.

Material that should remain available beyond this investigation can be saved in a selected Space. Keep the returned source identity and version, check processing status, then retrieve the relevant original lines. Saving a source is not proof that indexing completed, and navigation is not evidence for a factual claim. The resulting answer can identify the source location and show which material supports its conclusion.

Normalising that activity matters more than it sounds. When the record is a canonical shape rather than a transcript of one runtime's log format, the investigation does not become unreadable when the execution path changes — and an action that was interrupted with an unprovable outcome stays visible as unresolved instead of being flattened into a clean success or failure.

08

Report: from evidence to a tracked finding

The end of the chain is a finding someone else can act on, and it is the step where the human decision is largest. A record can be complete — request, candidate source, command output — and still not be a vulnerability, because whether a behaviour matters depends on impact, exploitability and what the application was supposed to do.

What a workspace can do here is keep the finding connected to its material: the occurrence, the scan or the repository it came from, and the transactions that demonstrate it, so a reviewer follows the same path the engineer did instead of trusting a summary.

The last mechanical step is filing. Tracker integrations exist as plugins, and creating or updating a work item is declared as a mutating capability, which means it goes through the same approval path as any other action that changes something outside the workspace. An agent can draft the description; whether the description is true remains the engineer's problem, and should.

09

What a harness adds

Stated as capabilities, the difference between a chain of tools and a workspace is eight verbs. Each of them is something the engineer currently performs by hand at a boundary between two tools.

Eight things an agent does inside one task context, instead of eight things an engineer does at eight boundaries.
  1. 01Understandthe objective and its scope
  2. 02Navigatethe target application
  3. 03Operatethe surfaces that act
  4. 04Searchproject and original sources
  5. 05Executetools and commands
  6. 06Connectobservations to code
  7. 07Rememberwhat the task established
  8. 08Reportevidence to a finding

None of these verbs is new. What is new is that they execute against one context. The request the agent captured in step two is the request it correlates in step six; the command it ran in step five is the receipt the finding cites in step ten. That continuity, not the model, is what turns a set of tool invocations into a workflow.

10

Trade-offs

DecisionWhat it buysWhat it costs
Request bodies retrieved on demandThe model sees far less of the target by defaultOne more step before the agent can reason about a payload
Search results presented as candidatesAn honest record the engineer can closeA read that ends in 'probably' rather than a mapping
Revisions kept separate from capturesNothing destructive happens to the observationTwo artifacts to understand, and a reviewer must know the difference
Mutating tools behind approvalNo silent change to a target or a trackerThe smooth path is interrupted by a decision
Task context and selected SpacesWorking context stays attached to the task; selected sources can be reusedReused material needs its source identity, version and scope checked
11

Limitations

  • An agent performs steps; it does not choose the investigation. Deciding what is worth attacking, and when to stop, is not delegated by any of this.
  • Automating a scanner run is not automating an assessment. Tools produce candidates for an engineer to reason about, and their output is untrusted input.
  • Approval behaviour is not uniform across runtimes. Where the runtime cannot ask, an action is refused rather than attempted.
  • Some steps resist automation because they depend on context no tool has: business intent, client expectations, and whether a behaviour would be accepted as a finding.
  • Automation makes side effects easier, and that cuts both ways. A workflow that is smooth at sending requests is also smooth at sending the wrong ones.
12

Future work

  • Measuring the handoff cost directly — how often a task re-derives something it already established — so that workflow changes can be compared on evidence.
  • Making a task resumable across days, including the ability to hand the same context to a different engineer.
  • Studying which steps stay manual even when they could be automated, since the answer is the strongest signal about what engineers still do not trust.
  • Extending the finding handoff so a reviewer can follow the evidence path without the product installed.
  • Deciding, for each tool class, what evidence a result must carry before an agent may present it as support for a conclusion.
13

Conclusion

Red teaming a real application is mostly mechanical, and the mechanics are spread across tools that do not share state. The engineer supplies the missing integration layer by hand, one copy at a time, and pays for it in attention that the security question needed.

An agent can carry that layer instead: exploring the target, keeping the request, correlating it with candidate code, running the tool, retaining what came back, and filing a finding with its evidence attached. What it cannot carry is the judgement — which is the point, because that is the part the workflow exists to protect.

REFERENCES

  1. Model Context Protocol

    The interface plugins use to expose tools and instructions to the agent runtime.

  2. OWASP Web Security Testing Guide

    The methodology whose steps this article is about automating: what an engineer does, and in what order.

  3. Penetration Testing Execution Standard

    An earlier attempt to describe the same workflow end to end, including the reporting stage.

  4. OWASP ZAP MCP Integration

    A concrete example of a tool surface where read operations are free and scan operations require approval.