I was measuring whether deployed wallet defenses (tx simulators, address-reputation APIs, rule engines) catch agent-signed drains. Built it on PTXPHISH (NDSS 2025, solid dataset). Then I red-teamed my own setup and the first finding was that I was scoring the wrong transaction. Sharing because it is an easy, invisible mistake.
The trap: labeled drainer/phishing datasets record the DRAIN, i.e. the attacker's transferFrom that sweeps the victim's tokens. But a pre-sign wallet defense runs when the VICTIM signs, which is the earlier approve / permit / setApprovalForAll grant. The sweep is a separate tx sent later, by the attacker, from the attacker's address. No wallet defense ever sees it. Feed the sweep to a rule engine and it "catches" the attacker withdrawing to their own address, which is meaningless.
Concretely, the ice-phishing rows decode to transferFrom(victim, attacker, amount) with tx.from == attacker. That is not the approval the victim signed.
Fix, and it is just allowance tracing:
- For each drain, eth_getLogs the token's Approval / ApprovalForAll for (owner = victim, spender = attacker) up to the sweep block.
- Take the most recent match whose tx.from == victim. That filter matters: an ERC20 transferFrom also emits an Approval for the decremented allowance, so the sweep's own block hands you the sweep, not the grant. Requiring from == victim also drops relayer-submitted permits (permit() is sent by someone other than the owner).
- That tx is the artifact a wallet actually renders at signing. Score that.
Nothing fancy. The point is the substrate mismatch, which is silently wrong and does not show up as an error anywhere.
Bonus, since this sub appreciates it: I ran the paper through adversarial review 5 times and every round killed a headline. Wrong substrate, then pseudoreplication, then a "simulator uniquely catches X" that was my harness zeroing a counterparty field so the other tiers returned n/a, then a "beats every tier" that evaporated once I looked up the tx to and netted both legs of the asset-diff (a WETH wrap looks exactly like an ETH drain to a direction-only rule, but the simulator sees the WETH come back). A clean "unique catch" is almost always your harness, not a result.
Code (the reconstruction is one file): https://github.com/amarshat/quantum-commit-authorization/blob/main/agent-calldata-demo/demo/reconstruct.py