AI Vulnerability Scanner vs. Agent Audit: Choose by the Risk

The build finishes, the security scan posts a page of alerts, and the release owner asks one question: which alerts deserve engineering time today? Scanners find candidate vulnerabilities; audits test reachability, execution, and impact when static output cannot settle them.

An AI vulnerability scanner searches code, dependencies, or analysis data for possible weaknesses. An agent audit takes a candidate, reads its surrounding system, runs scoped checks, and reports whether the evidence supports escalation, rejection, or further review.

A scanner reruns the same rules on every revision and preserves raw findings, while an audit reads callers and configuration, runs scoped tests, and records the observed state change.

This comparison describes the Code Auditor direction in Tangle’s public writing. It does not imply that a public Tangle audit endpoint is live, and it does not treat an agent’s conclusion as a security guarantee.

Scanner output and agent-audit evidence answer different questions

The word “scanner” can make a modern static-analysis system sound like a regular-expression search. That description is too weak. Static analysis examines source, dependencies, or a derived code representation without running the complete application against live traffic.

CodeQL extracts a representation of a codebase, runs queries against it, and interprets results that may include data-flow or control-flow paths. CodeQL calls using a known vulnerability as a seed to find similar issues across codebases “variant analysis,” meaning a search for other instances of the same underlying flaw.

Semgrep defines rules that match code and can perform forms of semantic and data-flow analysis depending on the engine and configuration. Its triage documentation shows how teams preserve states and reasons while reviewing findings over time.

Both are structured analysis systems rather than simple text searches. A sink is the sensitive operation where untrusted data could cause harm, such as a database query or a privileged write. Their output still needs a project-specific answer to questions such as “Can an attacker reach that operation?” and “What state changes if the path runs?”

An agent audit starts where the scanner’s candidate ends. The agent may use the scanner’s raw output, but its quality comes from the additional work it performs and the evidence it preserves.

Question	Scanner	Agent audit
Where does a suspicious pattern occur?	Primary job	Reads the scanner result
Which callers and configurations reach it?	Can model some paths	Examines repository context and runs checks
Does the suspected action execute?	Often outside the scan	Attempts a test, request, simulation, or execution record
Is the impact material?	Depends on configured analysis	Connects the path to data, money, permissions, or state
Are several alerts one bug?	May report separate matches	Groups candidates by root cause
What should an engineer change?	Rule or generic remediation	Repository-specific change and regression proof
What happens when setup fails?	Reports an analysis error	Must separate setup failure from exploit evidence

The two columns describe roles, not product limits shared by every vendor. Some scanners include reachability analysis or AI-assisted triage, and some agent workflows stop at static review. Ask what was executed and what evidence was captured before comparing labels.

A scanner is the right tool for repeated coverage

Use a scanner when the question has a stable rule and the cost of checking every change should remain low.

Need	Why scanning fits
Find banned APIs or insecure coding patterns	Rules can run on every change with consistent output
Detect hard-coded secrets	Pattern and entropy checks, which look for strings that resemble secret values, can cover the repository continuously
Search for variants of a known flaw	A query can apply one security idea across many files or repositories
Enforce dependency policy	Package metadata and vulnerability data can be checked before merge
Produce a baseline for later review	Every candidate carries a source location and tool provenance

The scan should preserve its version, rules, configuration, source revision, and raw output. GitHub’s code scanning documentation describes how scan results become alerts that teams can assess and resolve. Preserving the scan version, rules, configuration, source revision, and raw output gives a later audit the provenance of every candidate finding.

Coverage has a different failure mode from an agent audit. A scanner may miss a vulnerability because no rule models it, the build did not expose the source correctly, or the relevant configuration was outside the scan. It may also report a harmless match because a wrapper, permission, or deployment setting changes the behavior.

The scanner should say what it analyzed. The release owner should not infer that a clean scan means the system has no exploitable path.

An agent audit buys context with time and execution risk

Use an agent audit when callers are unclear, state changes cross files, or reachability evidence is missing. The workflow is an investigation with a budget, not a longer scan.

scanner candidate
  -> locate callers, guards, configuration, and affected state
  -> build or run the smallest relevant test
  -> capture inputs, output, exit status, and state change
  -> keep, merge, downgrade, or reject the candidate
  -> write a fix and rerun the proof when possible

The model can propose a test or a path. The execution environment must show whether the command ran and what it returned.

Here is a deliberately small pseudocode shape for the decision. It is an illustrative interface, not a Tangle or scanner SDK.

type Candidate = {
  id: string
  location: string
  rawSignal: string
}

type AuditResult = {
  status: 'reproduced' | 'reachable' | 'rejected' | 'incomplete'
  evidence: string[]
  nextAction: string
}

async function inspect(candidate: Candidate): Promise<AuditResult> {
  const context = await readCallersGuardsAndConfig(candidate.location)
  const run = await executeSmallestSafeProof(context)

  if (run.setupFailed) {
    return {
      status: 'incomplete',
      evidence: [run.error],
      nextAction: 'repair the fixture or record the missing environment',
    }
  }

  if (run.impactObserved) {
    return {
      status: 'reproduced',
      evidence: [run.command, run.output, run.stateChange],
      nextAction: 'fix the path and rerun the proof',
    }
  }

  return {
    status: context.reachable ? 'reachable' : 'rejected',
    evidence: [context.reason, run.output],
    nextAction: context.reachable ? 'request deeper review' : 'retain negative evidence',
  }
}

The important fields are not the function names. The important fields distinguish a candidate, a setup failure, an observed impact, and a negative result.

An agent can also create new risk. It may run an expensive test without a limit, mutate the working tree before the proof is captured, expose a secret to a command, or accept a model-generated fixture that does not represent the real application. The audit design must constrain those actions and record them.

Record the handoff explicitly

The scanner-to-agent boundary needs its own evidence record. Without it, a missing agent result can look like a clean review, and a rejected candidate can look like a candidate that was never examined.

Handoff field	What it answers
Candidate ID and source revision	Which alert and code state moved between tools?
Raw rule output and scan configuration	Why did the scanner create the candidate?
Agent profile and selected checks	What instructions, permissions, and proof plan shaped the investigation?
Execution evidence and setup status	Did the command run, and did it produce the expected output?
Disposition and next action	Was the candidate reproduced, rejected, incomplete, or not escalated?

“Not escalated” means the scanner lane resolved the question without an agent run. “Rejected” means the agent or another reviewer tested the candidate and found a blocking guard or assumption. “Incomplete” means the intended check could not finish. Keeping those states separate lets a release owner measure coverage without treating unexamined alerts as safe.

Trace an authorization alert from match to proof

Imagine a scanner reports that GET /exports/:accountId loads records without an authorization check. The scanner has found a call pattern or missing guard, not yet a cross-account disclosure.

The agent should inspect four pieces of context:

1. How does the request identity enter the handler?
2. Does middleware, a request-processing layer that runs before the handler, authorize the requested account before the data query?
3. Can a normal signed-in user choose another account identifier?
4. Does the response contain data the caller should not see?

A narrow proof can use two test accounts and one request. The test logs the identity, requested resource, response status, and a redacted assertion about the returned record owner. It should not copy customer data into the report.

The result changes the decision:

Result	Scanner conclusion	Agent-audit conclusion
Middleware rejects the second account	Candidate remains a match	Reject or downgrade with the guard location and negative test
Request returns the second account’s record	Candidate becomes a reproduced issue	Escalate with request, identity, response assertion, and fix path
The test fixture lacks the authorization layer	Candidate remains unresolved	Mark incomplete and name the missing fixture
Three rules report the same handler	Three alerts	One root-cause finding with linked raw signals

This is why “AI scanner versus agent audit” is a poor winner-takes-all question. The scanner found a place to look, and the audit spent time deciding whether the place matters.

Spend agent time where the scan cannot decide

An agent audit has a variable cost that a rule-based scan usually does not. The cost comes from model calls, workspace time, dependency setup, test execution, and human review of the resulting packet.

That cost should be allocated by uncertainty. Candidates with a clear policy answer can remain in the scanner lane, while candidates with uncertain callers, state, or impact can receive the execution budget.

An evaluation is a fixed collection of tasks and expected outcomes used to compare runs under the same conditions. It makes that allocation measurable. For this comparison, include at least these cases:

Evaluation case	What it tests
Known true vulnerability	Whether the workflow can carry a real issue from signal to proof
Known safe wrapper	Whether repository context can close a tempting match
Duplicate rules on one cause	Whether the report stays concise without losing raw signals
Missing dependency or broken build	Whether setup failure is labeled honestly
Logic issue with no scanner rule	Whether scanner-first coverage leaves a blind spot
Fix that blocks intended behavior	Whether the proposed repair preserves the product contract

Track the number of candidates entering each lane, the fraction with captured execution evidence, the fraction correctly rejected, seeded issues missed, and reviewer minutes per accepted finding. Keep scanner and agent runs on the same source revisions and task set when comparing them. Otherwise a lower report count may reflect fewer attempts rather than better judgment.

The agent profile also belongs in the comparison. An agent profile is the declared model, instructions, available tools, permissions, and resource limits for a run. Changing the profile can change which files the agent reads, which commands it selects, and whether it attempts a reproduction. Record it as part of the experimental condition rather than attributing every difference to the scanner or audit design.

The useful metric is a reviewer outcome. If an agent turns ten candidates into two accepted fixes with five reproducible rejections, it may have reduced work even though its output still contains seven records. If it turns ten candidates into ten confident paragraphs with no runnable evidence, the workflow has added prose rather than reduced risk.

Choose the smallest tool that can settle the question

Keep stable policy checks in scanning, and audit candidates that require reachability or impact evidence.

Use the scanner alone when the result is a policy check, a known pattern, a secret match, or a baseline that a human will inspect later. The scan should block or warn according to an explicit rule, not a vague confidence score.

Add an agent audit when the release depends on exploitability, cross-file context, a business-logic path, a state transition, or a project-specific fix. The audit should receive a narrowed candidate set rather than the whole repository by default.

Use a human specialist when the system handles significant assets, relies on production-only behavior, crosses trust boundaries that cannot be represented in the fixture, or needs a judgment about the threat model, meaning the assumptions about who can attack and what they can reach. An agent can prepare the evidence packet without becoming the final approver.

For smart contracts, the choice is sharper because a static pattern can look severe while a reachable exploit depends on callers, transaction ordering, balances, and deployed dependencies. Continue with Automated Smart Contract Audit With PoC Validation for that case.

Put the execution boundary next to the decision

Tangle’s public Sandbox is an isolated machine for an agent. It gives a run a filesystem, processes, ports, resource controls, and captured results through an SDK or other client surface. The Sandbox quickstart shows the public create, execute, inspect, and delete lifecycle.

If the audit workflow becomes a Tangle service, a Blueprint is the package that declares its jobs, metadata, contracts, and runtime rules. An operator is the provider that runs a service instance for users. A service instance is the live registration and execution context for one service, not the reusable Blueprint package. The runtime is the environment that executes the scanner, build, tests, and report steps. The public AI Agent Sandbox Blueprint overview describes the split between protocol lifecycle and live machine I/O.

An agent profile records the model, instructions, tools, permissions, and resource limits used by a run. Two audits with different profiles are different experiments even when they examine the same source revision.

A trace records the ordered actions, inputs, tool outputs, and results from that run. It can show that the agent invoked a test and what the test returned. It cannot prove that the test covered the right threat model.

The AI Code Audit With Sandboxed Agents article shows how the execution boundary and finding states fit together. The AI Security Audit With Reproducible Findings article defines the report packet in more detail.

Limits belong beside the comparison

Grouping duplicates and recording rejection evidence can reduce unresolved candidates, but an agent audit can also create a second layer of unsupported reasoning. It may miss a flaw that was never proposed by the scanner, call an unreachable fixture exploitable, or produce a plausible fix that changes intended behavior.

A scanner can miss a novel logic bug, while an agent can spend time on a pattern that a precise rule could reject in milliseconds. Neither tool sees a production incident response path unless the test scope includes it.

Compare reviewer minutes, reproduced findings, rejected candidates, missed seeded issues, and execution evidence. Do not compare a scanner’s raw alert count with an agent’s polished report and call the lower number a better security result.

Choose the lane by evidence

Run scanners continuously for stable patterns, dependency policy, secrets, and broad coverage. Add an agent audit when a candidate needs execution, repository context, duplicate merging, or a repository-specific fix. Escalate to a human when the threat model or impact cannot be represented by the available evidence.

For a concrete execution example, open the Tangle Sandbox quickstart. For the report requirement, read AI Security Audit With Reproducible Findings. For the smart-contract branch, use Automated Smart Contract Audit With PoC Validation.

What is an AI vulnerability scanner?

It is software that uses rules, static analysis, dependency information, or model assistance to identify possible security issues in code.

What is an agent audit?

It is a workflow in which an agent examines a candidate, runs scoped commands or tests, preserves evidence, and writes a decision with limits.

Is an agent audit more accurate than a scanner?

Accuracy depends on the task, source, configuration, and proof method. The scanner usually offers broader repeated coverage, while the agent can resolve context that static output leaves open.

When is a scanner enough?

Use a scanner alone when the issue has a stable rule, the result is a policy signal, or a human will inspect the candidate before release.

When should a team add an agent audit?

Add one when exploitability, cross-file behavior, business logic, state transitions, or a concrete fix determines the release decision.

Does either tool prove a codebase has no exploitable vulnerabilities?

No. Both provide evidence within a stated scope, and both can miss paths outside their rules, fixtures, configuration, or threat model.