Pluralistic: This is your brain on fraud apologetics (24 Feb 2023)

doctorow · 24 February 2023 15:04

Originally published at: Pluralistic: This is your brain on fraud apologetics (24 Feb 2023) – Pluralistic: Daily links from Cory Doctorow

Today's links

This is your brain on fraud apologetics: The curious comfort of victim-blaming.
Hey look at this: Delights to delectate.
This day in history: 2003, 2008, 2013, 2018, 2022
Colophon: Recent publications, upcoming/recent appearances, current writing projects, current reading

A modified version of Hieronymus Bosch's painting 'The Conjurer,' which depicts a scam artist playing a shell-game for a group of gawking rubes. The image has been modified so that the scam artist's table has a Google logo and the pea he is triumphantly holding aloft bears the 'Sponsored' wordmark that appears alongside Google search results.

This is your brain on fraud apologetics (permalink)

In 1998, two Stanford students published a paper in Computer Networks entitled "The Anatomy of a Large-Scale Hypertextual Web Search Engine," in which they wrote, "Advertising funded search engines will be inherently biased towards the advertisers and away from the needs of consumers."

https://research.google/pubs/pub334/

The co-authors were Lawrence Page and Sergey Brin, and the "large-scale hypertextual web search-engine" they were describing was their new project, which they called "Google." They were 100% correct – prescient, even!

On Wednesday night, a friend came over to watch some TV with us. We ordered out. We got scammed. We searched for a great local Thai place we like called Kiin and clicked a sponsored link for a Wix site called "Kiinthaila.com." We should have clicked the third link down (kiinthaiburbank.com).

We got scammed. The Wix site was a lookalike for Kiin Thai, which marked up their prices by 15% and relayed the order to our local, mom-and-pop, one-branch restaurant. The restaurant knew it, too – they called us and told us they were canceling the order, and said we could still come get our food, but we'd have to call Amex to reverse the charge.

As it turned out, the scammers double-billed us for our order. I called Amex, who advised us to call back in a couple days when the charge posted to cancel it – in other words, they were treating it as a regular customer dispute, and not a systemic, widespread fraud (there's no way this scammer is just doing this for one restaurant).

In the grand scheme of things, this is a minor hassle, but boy, it's haunting to watch the quarter-century old prophecy of Brin and Page coming true. Search Google for carpenters, plumbers, gas-stations, locksmiths, concert tickets, entry visas, jobs at the US Post Office or (not making this up) tech support for Google products, and the top result will be a paid ad for a scam. Sometimes it's several of the top ads.

This kind of "intermediation" business is actually revered in business-schools. As Douglas Rushkoff has written, the modern business wisdom reveres "going meta – not doing anything useful, but rather, creating a chokepoint between people who do useful things and people who want to pay for those things, and squatting there, collecting rent:

https://rushkoff.medium.com/going-meta-d42c6a09225e

It's the ultimate passive income/rise and grind side-hustle: It wouldn't surprised me in the least to discover a whole festering nest of creeps on Tiktok talking about how they pay Mechanical Turks to produce these lookalike sites at scale.

This mindset is so pervasive that people running companies with billions in revenue and massive hoards of venture capital run exactly the same scam. During lockdown, companies like Doordash, Grubhub and Uber Eats stood up predatory lookalike websites for local restaurants, without their consent, and played monster-in-the-middle, tricking diners into ordering through them:

https://pluralistic.net/2020/09/19/we-are-beautiful/#man-in-the-middle

These delivery app companies were playing a classic enshittification game: first they directed surpluses to customers to lock them in (heavily discounting food), then they directed surplus to restaurants (preferential search results, free delivery, low commissions) – then, having locked in both consumers and producers, they harvested the surplus for themselves.

Today, delivery apps charge massive premiums to both eaters and restaurants, load up every order with junk fees, and clone the most successful restaurants out of ghost kitchens – shipping containers in parking lots crammed with low-waged workers cranking out orders for 15 different fake "virtual restaurants":

https://pluralistic.net/2020/12/01/autophagic-buckeyes/#subsidized-autophagia

Delivery apps speedran the enshittification cycle, but Google took a slower path to get there. The company has locked in billions of users (e.g. by paying billions to be the default search on Safari and Firefox and using legal bullying to block third party Android device-makers from pre-installing browsers other than Chrome). For years, it's been leveraging our lock-in to prey on small businesses, getting them to set up Google Business Profiles.

These profiles are supposed to help Google distinguish between real sellers and scammers. But Kiin Thai has a Google Business Profile, and searching for "kiin thai burbank" brings up a "Knowledge Panel" with the correct website address – on a page that is headed with a link to a scam website for the same business. Google, in other words, has everything it needs to flag lookalike sites and confirm them with their registered owners. It would cost Google money to do this – engineer-time to build and maintain the system, content moderator time to manually check flagged listings, and lost ad-revenue from scammers – but letting the scams flourish makes Google money, at the expense of Google users and Google business customers.

Now, Google has an answer for this: they tell merchants who are being impersonated by ad-buying scammers that all they need to do is outbid them for the top ad-spot. This is a common approach – Amazon has a $31b/year "ad business" that's mostly its own platform sellers bidding against each other to show you fake results for your query. The first five screens of Amazon search results are 50% ads:

https://pluralistic.net/2022/11/28/enshittification/#relentless-payola

This is "going meta," so naturally, Meta is doing it too: Facebook and Instagram have announced a $12/month "verification" badge that will let you report impersonation and tweak the algorithm to make it more likely that the posts you make are shown to the people who explicitly asked to see them:

https://www.vox.com/recode/2023/2/21/23609375/meta-verified-twitter-blue-checkmark-badge-instagram-facebook

The corollary of this, of course, is that if you don't pay, they won't police your impersonators, and they won't show your posts to the people who asked to see them. This is pure enshittification – the surplus from users and business customers is harvested for the benefit of the platform owners:

https://pluralistic.net/2023/01/21/potemkin-ai/#hey-guys

The idea that merchants should master the platforms as a means of keeping us safe from their impersonators is a hollow joke. For one thing, the rules change all the time, as the platforms endlessly twiddle the knobs that determine what gets shown to whom:

https://doctorow.medium.com/twiddler-1b5c9690cce6

And they refuse to tell anyone what the rules are, because if they told you what the rules were, you'd be able to bypass them. content moderation is the only infosec domain where "security through obscurity" doesn't get laughed out of the room:

https://doctorow.medium.com/como-is-infosec-307f87004563

Worse: the one thing the platforms do hunt down and exterminate with extreme prejudice is anything that users or business-customers use to twiddle back – add-ons and plugins and jailbreaks that override their poor choices with better ones:

https://www.theverge.com/2022/9/29/23378541/the-og-app-instagram-clone-pulled-from-app-store

As I was submitting complaints about the fake Kiin scam-site (and Amex's handling of my fraud call) to the FTC, the California Attorney General, the Consumer Finance Protection Bureau and Wix, I wrote a little Twitter thread about what a gross scam this is:

https://twitter.com/doctorow/status/1628948906657878016

The thread got more than two million reads and got picked up by Hacker News and other sites. While most of the responses evinced solidarity and frustration and recounted similar incidents in other domains, a significant plurality of the replies were scam apologetics – messages from people who wanted to explain why this wasn't a problem after all.

The most common of these was victim-blaming: "you should have used an adblocker" or "never click the sponsored link." Of course, I do use an ad-blocker – but this order was placed with a mobile browser, after an absentminded query into the Google search-box permanently placed on the home screen, which opens results in Chrome (where I don't have an ad-blocker, so I can see material behind an ad-blocker-blocker), not Firefox (which does have an ad-blocker).

Now, I also have a PiHole on my home LAN, which blocks most ads even in a default browser – but earlier this day, I'd been on a public wifi network that was erroneously blocking a website (the always excellent superpunch.net) so I'd turned my wifi off, which meant the connection came over my phone's 5G connection, bypassing the PiHole:

https://pluralistic.net/2022/04/28/shut-yer-pi-hole/

"Don't click a sponsored link" – well, the irony here is that if you habitually use a browser with an ad-blocker, and you backstop it with a PiHole, you never see sponsored links, so it's easy to miss the tiny "Sponsored" notification beside the search result. That goes double if you're relaxing with a dinner guest on the sofa and ordering dinner while chatting.

There's a name for this kind of security failure: the Swiss Cheese Model. We all have multiple defenses (in my case: foreknowledge of Google's ad-scam problem, an ad-blocker in my browser, LAN-wide ad sinkholing). We also have multiple vulnerabilities (in my case: forgetting I was on 5G, being distracted by conversation, using a mobile device with a permanent insecure search bar on the homescreen, and being so accustomed to ad-blocked results that I got out of the habit of checking whether a result was an ad).

If you think you aren't vulnerable to scams, you're wrong – and your confidence in your invulnerability actually increases your risk. This isn't the first time I've been scammed, and it won't be the last – and every time, it's been a Swiss Cheese failure, where all the holes in all my defenses lined up for a brief instant and left me vulnerable:

https://locusmag.com/2010/05/cory-doctorow-persistence-pays-parasites/

Other apologetics: "just call the restaurant rather than using its website." Look, I know the people who say this don't think I have a time-machine I can use to travel back to the 1980s and retrieve a Yellow Pages, but it's hard not to snark at them, just the same. Scammers don't just set up fake websites for your local businesses – they staff them with fake call-centers, too. The same search that takes you to a fake website will also take you to a fake phone number.

Finally, there's "What do you expect Google to do? They can't possibly detect this kind of scam." But they can. Indeed, they are better situated to discover these scams than anyone else, because they have their business profiles, with verified contact information for the merchants being impersonated. When they get an ad that seems to be for the same business but to a different website, they could interrupt the ad process to confirm it with their verified contact info.

Instead, they choose to avoid the expense, and pocket the ad revenue. If a company promises to "to organize the world's information and make it universally accessible and useful," I think we have the right to demand these kinds of basic countermeasures:

https://www.google.com/search/howsearchworks/our-approach/

The same goes for Amex: when a merchant is scamming customers, they shouldn't treat complaints as "chargebacks" – they should treat them as reports of a crime in progress. Amex has the bird's eye view of their transaction flow and when a customer reports a scam, they can backtrack it to see if the same scammer is doing this with other merchants – but the credit card companies make money by not chasing down fraud:

https://www.buzzfeednews.com/article/rosalindadams/mastercard-visa-fraud

Wix also has platform-scale analytics that they could use to detect and interdict this kind of fraud – when a scammer creates a hundred lookalike websites for restaurants and uses Wix's merchant services to process payments for them, that could trigger human review – but it didn't.

Where do all of these apologetics come from? Why are people so eager to leap to the defense of scammers and their adtech and fintech enablers? Why is there such an impulse to victim-blame?

I think it's fear: in their hearts, people – especially techies – know that they, too, are vulnerable to these ripoffs, but they don't want to admit it. They want to convince themselves that the person who got scammed made an easily avoidable mistake, and that they themselves will never make a similar mistake.

This is doubly true for readerships on tech-heavy forums like Twitter or (especially) Hacker News. These readers know just how many vulnerabilities there are – how many holes are in their Swiss cheese – and they are also overexposed to rise-and-grind/passive income rhetoric.

This produces a powerful cognitive dissonance: "If all the 'entrepreneurs' I worship are just laying traps for the unwary, and if I am sometimes unwary, then I'm cheering on the authors of my future enduring misery." The only way to resolve this dissonance – short of re-evaluating your view of platform capitalism or questioning your own immunity to scams – is to blame the victim.

The median Hacker News reader has to somehow resolve the tension between "just install an adblocker" and "Chrome's extension sandbox is a dumpster fire and it's basically impossible to know whether any add-on you install can steal every keystroke and all your other data":

https://mattfrisbie.substack.com/p/spy-chrome-extension

In my Twitter thread, I called this "the worst of all possible timelines." Everything we do is mediated by gigantic, surveillant monopolists that spy on us comprehensively from asshole to appetite – but none of them, not a 20th century payment giant nor a 21st century search giant – can bestir itself to use that data to keep us safe from scams.

Hey look at this (permalink)

Infinite-Storage-Glitch https://github.com/DvorakDwarf/Infinite-Storage-Glitch (h/t Joshua Bradley)
Network Usage Fees Will Harm European Consumers and Businesses https://www.eff.org/deeplinks/2022/12/network-usage-fees-will-harm-european-consumers-and-businesses
I asked Popehat: Why isn't SBF in jail? https://newsletter.mollywhite.net/p/i-asked-popehat-why-isnt-sbf-in-jail

A Wayback Machine banner.

This day in history (permalink)

#20yrsago Ghanan gov’t shuts down ISPs to keep email from displacing long-distance https://web.archive.org/web/20030424090724/http://lists.elistx.com/archives/interesting-people/200302/msg00189.html

#20yrsago Doctorow on Gibson in Mindjack https://www.mindjack.com/books/gibsonpr.html

#15yrsago Government and corporate employees engage in an “epidemic” of snooping into databases https://web.archive.org/web/20080229003253/http://ap.google.com/article/ALeqM5ghPenZUJTE7BfSfgQbj6RX597DEAD8V019TG0

#15yrsago Science Fiction Writers of America election is a referendum on copyright extremism https://whatever.scalzi.com/2008/02/18/a-gut-check-moment-for-sfwa/

#15yrsago Knowledge isn’t property https://www.theguardian.com/technology/2008/feb/21/intellectual.property

#10yrsago Marriage proposal in the form of a physics paper https://www.reddit.com/r/funny/comments/194ilz/my_boyfriend_of_7_years_and_i_are_both_physicists/

#10yrsago Pastafarian denied religious freedom in New Jersey driver’s license scandal https://web.archive.org/web/20130225093656/http://www.nydailynews.com/news/national/colander-religious-headwear-police-article-1.1271134

#5yrsago Marjory Stoneman Douglas High School was surrounded by cowering “good guys with guns” https://edition.cnn.com/2018/02/23/politics/parkland-school-shooting-broward-deputies/index.html

#5yrsago Gothamist unionized and its evil Trumpist billionaire owner shut it down; now public radio is bringing it back https://www.wired.com/story/gothamist-dcist-laist-return-wnyc-public-radio/

#5yrsago Paul Manafort’s inability to save Word files as PDFs provided the evidence necessary to indict him for fraud https://arstechnica.com/tech-policy/2018/02/how-manaforts-inability-to-convert-a-word-doc-to-pdf-helped-prosecutors/

#1yrago Mass arbitration attack could bring Intuit to its knees https://pluralistic.net/2022/02/24/uber-for-arbitration/#nibbled-to-death-by-ducks

Colophon (permalink)

Currently writing:

Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. Yesterday's progress: 554 words (109029 words total)
The Bezzle, a Martin Hench noir thriller novel about the prison-tech industry. FIRST DRAFT COMPLETE, WAITING FOR EDITORIAL REVIEW
A Little Brother short story about DIY insulin PLANNING
Vigilant, Little Brother short story about remote invigilation. ON SUBMISSION
Moral Hazard, a short story for MIT Tech Review's 12 Tomorrows. FIRST DRAFT COMPLETE, ACCEPTED FOR PUBLICATION
Spill, a Little Brother short story about pipeline protests. ON SUBMISSION

Latest podcast: Tiktok's Enshittification https://craphound.com/news/2023/02/20/tiktoks-enshittification/

Upcoming appearances:

Antitrust, Regulation and the Political Economy (Brussels), Mar 2
https://www.brusselsconference.com/registration
Elevate Festival (Graz), Mar 3
https://elevate.at/diskurs/programm/event/e23doctorow/
UT School of Design and Creative Technologies (Austin), Mar 9
https://www.eventbrite.com/e/the-buckman-center-presents-cory-doctorow-tickets-526627556197
Ethics of Emerging Tech Lecture (U Manitoba), Mar 9
https://eventscalendar.umanitoba.ca/site/science/event/ethics-of-emerging-technology-lecture—rebecca-giblin-and-cory-doctorow/
SXSW Chokepoint Capitalism reading (Austin), Mar 10
https://schedule.sxsw.com/2023/events/PP1143284

Recent appearances:

ANU/Canberra Times Meet The Author
https://soundcloud.com/experience_anu/in-conversation-with-rebecca-giblin-and-cory-doctorow?si=67bcf6db3c4742bd973152e4b1b4e313
The Gould Standard
https://pod.link/thegouldstandard/episode/8648b67fb7a6ccbf49a7d5406f8582c7
ABC Radio National:
https://www.abc.net.au/radionational/programs/drawingroom/chokepoint-capitalism/101973108

Latest books:

"Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin", on how to unrig the markets for creative labor, Beacon Press/Scribe 2022 https://chokepointcapitalism.com
"Attack Surface": The third Little Brother novel, a standalone technothriller for adults. The Washington Post called it "a political cyberthriller, vigorous, bold and savvy about the limits of revolution and resistance." Order signed, personalized copies from Dark Delicacies https://www.darkdel.com/store/p1840/Available_Now%3A_Attack_Surface.html
"How to Destroy Surveillance Capitalism": an anti-monopoly pamphlet analyzing the true harms of surveillance capitalism and proposing a solution. https://onezero.medium.com/how-to-destroy-surveillance-capitalism-8135e6744d59 (print edition: https://bookshop.org/books/how-to-destroy-surveillance-capitalism/9781736205907) (signed copies: https://www.darkdel.com/store/p2024/Available_Now%3A__How_to_Destroy_Surveillance_Capitalism.html)
"Little Brother/Homeland": A reissue omnibus edition with a new introduction by Edward Snowden: https://us.macmillan.com/books/9781250774583; personalized/signed copies here: https://www.darkdel.com/store/p1750/July%3A__Little_Brother_%26_Homeland.html
"Poesy the Monster Slayer" a picture book about monsters, bedtime, gender, and kicking ass. Order here: https://us.macmillan.com/books/9781626723627. Get a personalized, signed copy here: https://www.darkdel.com/store/p2682/Corey_Doctorow%3A_Poesy_the_Monster_Slayer_HB.html#/.

Upcoming books:

Red Team Blues: "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books, April 2023
The Internet Con: A nonfiction book about interoperability and Big Tech, Verso, September 2023
The Lost Cause: a post-Green New Deal eco-topian novel about truth and reconciliation with white nationalist militias, Tor Books, November 2023

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.