LLMs and the economics of Evil, Inc., with Manish Goregaokar

Patrick McKenzie (patio11) is joined by Manish Goregaokar, a senior software engineer and member of the Rust security response team, to discuss what LLMs do to the cost side of running a sophisticated con. They walk through a scam built for programmers: a plausible company, a few rounds of interviews, and an NDA signed via "sign in with email" that quietly hands an attacker the identity account everything else resets against. A version of it hit at least five people in the Rust community within days of Manish publishing on the subject, and Patrick describes being targeted himself by someone posing as a Bloomberg reporter who wanted him on a Zoom call with a codec problem. They also cover money mules and the three stages of laundering, the $25 million Hong Kong transfer authorized on a fully deepfaked video call, and why the only durable individual defense is refusing to trust any communication you did not initiate.

–
Full transcript available here: https://www.complexsystemspodcast.com/what-llms-do-to-the-economics-of-scams-with-manish-goregaokar/

–
Presenting Sponsors: Mercury, MongoDB & Granola

Complex Systems is presented by Mercury—radically better banking for founders. Mercury's new feature Command brings an LLM directly into your banking interface, so checking balances, finding invoices, or sending a wire is as easy as asking. Apply online in minutes at https://mercury.com/.

If meetings consistently leave you with hazy action items and lost context, Granola handles the transcription so you can actually participate and gives you searchable notes afterward. Try it free at granola.ai/complexsystems with code COMPLEXSYSTEMS

What's the point of building faster with AI if your database can't keep up? MongoDB's native data model mirrors the language LLMs already speak. Ship at the speed of AI while staying ACID compliant at Fortune 500 scale. Start building at https://mongodb.com/ai.

–

Links:

• Manish Goregaokar’s blog: https://manishearth.github.io/
• Manish on X: https://x.com/ManishEarth

–

Timestamps:

(00:00) Intro
(00:27) The future of the con is already here
(01:40) Scams are run like businesses
(03:30) A scam aimed at programmers
(05:12) Why single sign-on is the one ring to rule them all
(06:50) Persistent threats against individuals
(08:58) What LLMs change about cost
(11:40) Manual scams and business email compromise
(14:30) How money movement constrains scammers
(17:39) The industry that helps you pay ransoms
(19:32) Bespoke attacks on individual targets
(21:41) The Rust community gets targeted
(23:02) Supply chain attacks and long-lived impersonation
(24:58) Money mules and the three stages of laundering
(28:35) Why law enforcement struggles with internet crime
(30:01) Sponsors: Mercury │ Granola
(33:03) Patriotic hackers and the post-Soviet talent pool
(35:27) What individuals can do
(36:27) Only trust communication you initiate
(38:21) Wire fraud in real estate closings
(41:07) Why legitimate institutions look like scams
(45:22) Who eats the loss
(47:21) Sponsor: MongoDB
(48:12) Why banks can't just block new payees
(50:45) Verification protocols and family passwords
(53:07) Urgency and secrecy
(54:44) Systemic fixes and who pays
(57:55) Open weights and the coming wave
(01:02:17) Wrap