A security scan reports that a withdrawal function sends funds to an untrusted address. The alert may be useful, but it does not say whether an attacker can reach the function, whether state changes happen before the external call, or whether a test can reproduce a loss.
An AI code audit turns that alert into a bounded engineering decision. The agent reads the project, runs its tools in an isolated workspace, tests the suspected path, and returns either evidence for the claim or evidence that narrows or rejects it.
At release time, an unreproduced warning extends review, while a tested finding identifies a failure and a fix target. An unexplained warning adds review work, while a finding with a reproducible test gives an engineer a concrete failure to fix.
This article uses a withdrawal example to show the workflow, then maps it to Tangle’s public Sandbox, an isolated machine for agent work, and related service vocabulary. The Code Auditor described here is a design direction for agent-assisted review, not a claim that a public audit endpoint is available.
Start with the path an attacker would need
A scanner is a program that examines source code, dependencies, or generated analysis data for patterns associated with bugs. Static analysis performs that examination without running the complete application against live traffic.
CodeQL builds a queryable database, runs queries, and interprets the results as possible issues or data-flow paths. Semgrep’s rule glossary describes a finding as the result produced when a rule matches code.
Their rules and queries inspect every matching file, path, or data-flow edge on each run. Their output is still a candidate until someone establishes the path from an external input to an affected asset.
| Audit state | Question answered | Evidence worth keeping |
|---|---|---|
| Candidate | What pattern or query produced the alert? | Rule name, tool version, location, and raw output |
| Reachable | Can the relevant caller reach the code under the stated setup? | Call path, permissions, configuration, and input |
| Reproduced | Does the suspected action create the claimed state change? | Test, command output, request, or transaction trace |
| Rejected | Which check, wrapper, or assumption blocks the claim? | Negative test and the reason the candidate was closed |
The state is a reporting aid, not a verdict supplied by the language model. It makes the evidence that changed the conclusion visible to the next reviewer.
Work one withdrawal alert all the way through
Consider this small Solidity example. Solidity is the programming language, and the Ethereum Virtual Machine, or EVM, is the execution environment used by Ethereum-compatible contracts.
pragma solidity ^0.8.0;
contract Vault {
mapping(address => uint256) public credit;
function withdraw() external {
uint256 amount = credit[msg.sender];
require(amount > 0, "no credit");
(bool sent, ) = payable(msg.sender).call{value: amount}("");
require(sent, "send failed");
credit[msg.sender] = 0;
}
}This is a teaching example with no deposit path or deployment configuration. The suspicious sequence is the external call followed by the balance update.
Reentrancy is the failure mode in which code called by a contract calls back into that contract before the first invocation has finished updating its state. The scanner can point at the call, but the audit still has to show whether an attacker-controlled recipient can enter again and whether the second call changes the balance.
A disciplined agent follows the same questions a human reviewer would ask:
- Which function is public, and which caller controls the input?
- What storage value represents the asset or permission at risk?
- What must be true before the external call can run?
- What happens if the recipient calls
withdrawagain beforecreditis cleared? - Does the test end with a balance, permission, or ownership state that should be impossible?
The minimum proof is one fixture, one command, and assertions for the attacker’s credit and the vault balance.
For this example, fund the local vault with 20 ether, give the attacker 10 ether of recorded credit, and ask it to withdraw while its receiver calls withdraw once more.
One ordinary withdrawal should remove 10 ether and leave zero credit.
If both calls read the stale credit before either update completes, the observed result is a 20-ether decrease even though the attacker had only 10 ether of credit.
The test needs assertions for the initial credit, final credit, and vault balance, not a model-generated label.
The test should record the call depth or a transaction trace when the toolchain makes that information available.
The report should not write “reentrancy confirmed” because a model recognized a familiar pattern. It should state the caller, amount, initial state, sequence of calls, observed state change, and the code path that caused it.
The same process catches the false positive. If an owner check rejects the attacker or a non-reentrant lock blocks the callback, the agent should preserve that negative test and the reason the path stopped.
| Test outcome | Safe report language |
|---|---|
| The attacker reaches the function and changes protected state | “Reproduced under these inputs and setup” |
| The attacker reaches the function but the state change does not occur | “Reachable, impact not reproduced in this test” |
| Authorization or initialization blocks the path | “Candidate rejected by the tested guard” |
| The build or fixture fails before the path runs | “Reproduction incomplete because setup failed” |
That language keeps setup failure separate from security evidence.
Keep the experiment away from the host
An audit agent needs to read files, install dependencies, run compilers, and execute commands that were written by someone else. A sandbox is a separate computer or process boundary for that work, with its own filesystem, processes, resource limits, and network policy.
Tangle Sandbox documents this model as an isolated machine for an agent, implemented with a container or microVM and driven through an SDK, command-line interface, or dashboard. The public AI Agent Sandbox guide explains the same lifecycle in Tangle terms.
The smallest public SDK flow looks like this:
npm install @tangle-network/sandboximport { Sandbox } from '@tangle-network/sandbox'
const client = new Sandbox({
apiKey: process.env.TANGLE_API_KEY!,
baseUrl: process.env.SANDBOX_BASE_URL ?? 'https://sandbox.tangle.tools',
})
const box = await client.create({
image: 'universal',
name: 'audit-smoke',
resources: { cpuCores: 2, memoryMB: 4096, diskGB: 20 },
})
try {
const result = await box.exec('npm ci && npm test', {
cwd: '/workspace',
timeoutMs: 300000,
env: { CI: 'true' },
})
console.log(JSON.stringify({
exitCode: result.exitCode,
stdout: result.stdout,
}))
} finally {
await box.delete()
}The API names and cleanup pattern come from the Tangle Sandbox quickstart and SDK reference.
The example assumes the reviewed project is already present at /workspace; it does not pretend that the public snippet uploads source code for you.
The isolation boundary reduces the damage a test can do to the machine that holds credentials or unrelated source. It does not make the project trustworthy, and it does not make the agent’s conclusion true.
Keep secrets out of the workspace whenever a fixture or mocked service can replace them. If a live account, private registry, or chain fork is required, record that fact, grant the narrowest temporary permission, and state which production behavior remained untested.
Retain the artifacts before deleting the workspace:
| Artifact | Why the reviewer needs it |
|---|---|
| Source revision or archive digest | Identifies the exact code under review |
| Scanner name, version, and configuration | Explains how the candidate was created |
| Commands, inputs, exit codes, and output | Lets another engineer repeat the attempt |
| Patch or suggested change | Connects the finding to engineering work |
| Rejected candidates and failed attempts | Prevents the same question returning without new evidence |
Let the Sandbox run the test without becoming the verdict
Tangle’s public Sandbox is the execution boundary for this workflow. Its quickstart shows a client creating a machine, running a command, capturing its output, and deleting the machine afterward. That lifecycle gives the audit a place to run untrusted build and test commands, but it does not make the project or the conclusion trustworthy by itself.
A Blueprint is Tangle’s service package that defines jobs, metadata, contracts, and runtime rules. An operator is the person or service that runs the actual instance for other users. A service instance is the live registration and execution context for one service, not the reusable Blueprint package. The runtime is the environment that starts processes, exposes tools, and returns their results. The public AI Agent Sandbox Blueprint overview describes how those pieces split protocol lifecycle from live machine I/O.
An agent profile is the declared combination of model, instructions, tools, permissions, and resource limits used for one audit run. Pinning the profile helps a reviewer understand why two runs may differ, but it is not a security proof.
A trace is the chronological record of a run, including actions, inputs, outputs, and results. Tangle’s Sandbox docs say a run can emit a trace so a failed run remains inspectable instead of disappearing with the client connection.
An evaluation is a repeatable set of audit tasks with expected properties used to compare runs. For an audit workflow, useful evaluation measures include the fraction of candidates with reproducible evidence, the fraction of rejected candidates with a recorded reason, time to a reviewer decision, and findings missed by the workflow.
Give every specialist the same evidence shape
Different codebases need different tools, but the report should not change its meaning when the language changes. The specialist’s job is to produce evidence for a narrow question, then return it in the same record shape as every other specialist.
Solana programs are smart contracts for the Solana blockchain, Move packages are projects written in the Move contract language, and a zero-knowledge circuit is a set of constraints that checks a statement without revealing its private witness.
| Code area | Narrow question | Useful evidence |
|---|---|---|
| Web application | Can an unauthenticated request reach the protected action? | Request, response, identity, route, and configuration |
| EVM contract | Can a caller change funds or permissions outside the intended rules? | Test, transaction trace, caller, and state diff |
| Solana program | Do account ownership and instruction constraints hold for a hostile account set? | Program test, account inputs, and failure output |
| Move package | Do resource and ability rules reject the invalid transition? | Unit test, expected failure, and package state |
| Zero-knowledge circuit | Does an invalid witness satisfy the circuit constraints? | Constraint test, witness, and proof result |
The coordinator should route by detected language and build system, not by a fixed set of agent names. The coordinator also needs to merge duplicates by root cause. Three alerts that point to the same missing authorization check are one engineering problem, even if three tools reported different lines.
Severity follows the observed impact
Severity is a communication about impact, prerequisites, and scope. It is not a decoration added after a model writes an alarming paragraph.
| Observed evidence | Claim it supports |
|---|---|
| A test changes another user’s balance | The tested input and caller can cause that state change |
| A request reaches an administrative action without authorization | The tested environment permits that unauthorized route |
| A scanner matches a dangerous API but the path is blocked | The pattern exists, but the tested attack path did not reach it |
| The build fails before the proof runs | The security claim remains untested |
The report should name the affected asset, attacker prerequisites, tested configurations, and untested configurations. “High severity” without those details asks the reviewer to supply the missing reasoning.
Know where the workflow stops
An AI code audit can miss a dependency defect, a production-only configuration, a race between services, an economic attack, or a path absent from the test fixtures. It can also produce a plausible explanation for a test that never ran if the runtime does not capture exit status and output separately from model text.
A clean sandbox does not cover a missing network integration. A passing unit test does not cover a different compiler version or deployment configuration. A protected execution claim does not establish contract correctness. A paid request does not establish reviewer agreement.
Human review, monitoring, incident response, and independent testing remain necessary for systems handling money, credentials, or personal data. The audit narrows the next review by preserving the source digest, command, output, state difference, and remaining uncertainty.
Make each command explainable after cleanup
The most useful audit record is not a transcript of every token the model produced. It is a small map from an assertion to the command, input, output, and state that support it.
For the withdrawal example, the record can be organized by phase:
| Phase | Input to preserve | Observable result |
|---|---|---|
| Discovery | Source revision, scanner rule, tool version, and configuration | Candidate location and raw match |
| Reachability | Caller identity, route or entry point, permissions, and relevant settings | Path reached or guard rejected it |
| Reproduction | Test name, arguments, initial state, and dependency versions | Exit code, assertion, response, or transaction result |
| Impact | Before-and-after balances, permissions, records, or files | Specific state change and affected scope |
| Repair | Patch revision and original proof | Exploit proof fails while intended behavior remains |
This shape gives a reviewer a way to challenge one claim without rerunning the entire agent session. It also prevents a common reporting error in which the model’s summary survives while the command that should support it has been discarded.
Keep model messages, tool output, and test assertions in separate fields. Model text can explain why the agent chose a test, but the exit code and captured output show whether the test ran. An assertion can show that a balance changed, but the source revision and initial state show which code and conditions produced it.
When the workspace is deleted, preserve the source digest, dependency lockfile digest, tool versions, command list, redacted inputs, and artifact checksums. Do not preserve secrets as proof. If a command can print a token, scrub the value while retaining the fact that the command accessed a secret-dependent path.
Replay also needs a boundary. If the original run used a local chain fork at one block, a later run against a different block is a new experiment even if the command text is identical. If the run used a database fixture, record its schema and seed version rather than calling it production behavior. If a reviewer changes the agent profile, source revision, or network policy, the new result should receive a new run identifier.
That discipline helps an evaluation distinguish a stronger audit from a lucky answer. The system can be measured on whether it kept the evidence required for replay, not on how persuasive its explanation sounded.
The practical decision
Run an AI code audit when a release needs more than scanner coverage and less than a full formal review. Require each escalated finding to name the code path, caller, inputs, reproduction result, affected state, proposed fix, and remaining uncertainty. Keep rejected candidates with their negative evidence.
For the execution boundary, start with the Tangle Sandbox quickstart. For the reporting standard, continue to AI Security Audit With Reproducible Findings. For contract-specific proof, use Automated Smart Contract Audit With PoC Validation.
What is an AI code audit?
An AI code audit is an AI-assisted review that combines code analysis, repository context, executable tests, and an evidence-backed report.
How does an AI code audit differ from a scanner?
A scanner produces candidates from rules or queries. An audit tests reachability, runs a scoped reproduction, records negative evidence, and explains the next fix.
Does a sandbox make an audit safe?
No. A sandbox limits where commands run, while permissions, network policy, secret handling, and review scope determine what the test can still affect.
What should a finding contain?
It should contain the source revision, location, attacker path, inputs, command or test, observed result, impact, fix, and residual risk.
When should a team stop trusting the automated result?
Stop escalating when the build did not complete, the required environment was unavailable, the test did not reach the suspected path, or the report cannot separate model text from captured tool output.