Essay · Fraud Operations · Data-Informed

The Human in the Approval Gate

Your attacker now has an AI assistant too. That one sentence should change how you design a payment-approval workflow — and the instinct to remove the human is the wrong optimization.

By Amira Meza · August 21, 2026 · Chandler, Arizona

I run first-line operations, which in plain language means I sit upstream of the money. Every invoice, every vendor portal change, every outbound wire crosses a desk before it lands. For years, the threat model was simple: a bad actor tries to trick a human into approving something wrong, and the humans catch most of it because there are three of them and they distrust each other just enough. That model is now obsolete, and most teams are optimizing the wrong variable.

This month's news carried two stories that belong in the same paragraph. The first is a security demonstration on how an attacker can stand up an AI assistant that intervenes at the exact moment you approve a payment — patiently learning your patterns, then inserting itself into the control loop. The second is Manic Android malware, which uses offline device-relay networks to drain bank accounts without a direct internet trace. Different attack surfaces, identical logic: the fraudster is no longer speeding up. They are matching your gait and walking beside you.

§ 1 Why the "remove the human" instinct is a trap

The clearest KPI I track is approval-cycle time. Any ops lead sees a payment sitting in a queue for three people and thinks: that's the bottleneck, automate it. And for legitimate, repeatable, low-ticket flows — yes. Kill the human there. But the moment you optimize the exception gate the same way, you are handing your attacker the one asset they cannot fabricate: a pattern of consistent, trustless, second-guessing human judgment.

The human is not the bottleneck. The human is the audit trail that walks and breathes and refuses a wire at 4:47 on a Friday.

Here is what I've observed on the floor that no classifier has matched: humans are terrible at catching the first fraudulent invoice, and exceptional at catching the second. The first one looks like a normal Tuesday. The red flag is rarely in the email — it's in the delta, the tiny difference from Tuesday-last-week, and that delta only exists inside a person who processed Tuesday-last-week. Automation resets that memory every release. A stable approval crew is a living baseline.

§ 2 The attacker's new cost structure

An AI assistant changes the fraudster's economics the same way every tooling change always has: it collapses the cost of the mundane work. A human scammer can run three convincing payment-interruption scripts a day before the seams show. An AI assistant runs three hundred, each tuned to a different vendor's invoice timetable, each phrased for a different approver's hobbies and recent travel.

The defensive response is not to add four more approval steps — a taller gate is just a taller target, and more steps means more authorized checkpoints to socially engineer. The response is to make the gate cheaper to run honestly and expensive to defeat, which is the opposite of what most org charts do. You want few gates, high trust in each, and a human who is explicitly rewarded for one specific behavior: saying "not today" and being able to justify it in one sentence.

§ 3 A field protocol, not a policy

Policies get laminated and ignored. Protocols get rehearsed. Three rules I put on the desk that survived contact with the real Monday:

1. The out-of-band rule. Any request to change a payment destination confirmed only in-email is, by definition, not confirmed. The approver calls a number already on file — never a number in the email. Cost: ninety seconds. Failure mode eliminated: the exact one the AI-assistant demo weaponizes.
2. The word-jar rule. Every approver keeps a small private jar of the vendor's known phrasings. A new word, a new salutation, a shift in urgency — that's the delta. If a human can't name what changed, the request waits for morning.
3. The morning verdict. No payment is killed by automation's confidence score alone. A rejection must survive a human reread with a clear head after coffee. Rise-and-review catches what 2 a.m. automation flags as "minor variance" — and it catches the false positives that, uncontrolled, will make your crew rubber-stamp the real ones.

These are cheap, and that's the point. A security control you resent is a security control you'll shortcut the moment you're behind schedule.

§ 4 The optimization that matters

Don't optimize the gate out of the path. Optimize the signal at the gate. Give each approver a machine-readable digest of exactly what changed since the last approval — the delta rendered as a one-page receipt. The human then does what humans do best: reads the delta, feels whether it's off, and owns the answer out loud.

"Numbers are honest about what they count, and silent about everything else." — the bake-sale ledger school of thought that keeps circling our boards, and it's exactly right. A line my neighbor linda-miller put on her canning-ledger film. It applies to payment systems with more teeth than to preserves.

Two years ago I would have drawn you a swimlane with the human circled, tagged "eliminate." Today I circle the same human and tag "insist." The attacker gained an assistant; so the human in the gate — the one who keeps the word-jar, who says not-today and can justify it — just became the only control that doesn't scale, and the only one worth scaling around.

← Back to Amira's field notes · Workflow terms