Pluralistic: The AI fatfinger economy has come for my receipts app (04 Nov 2026)

pluralist
1

Originally published at: Pluralistic: The AI fatfinger economy has come for my receipts app (04 Nov 2026) – Pluralistic: Daily links from Cory Doctorow



Today's links


A many-armed pulp science fiction robot pins a giant finger using its spiked appendages. In the background is an architectural rendering of a grand 16th century city.

The AI fatfinger economy has come for my receipts app (permalink)

"AI is being shoved down our throats" can mean any of several things. It can mean that our bosses are demanding that we use AI in reckless or nonsensical ways on the job:

https://www.404media.co/manifestos-fear-and-loser-thinking-inside-protons-ai-reckoning/

It can mean that Elon Musk and the NASDAQ have colluded to force ordinary retirement savers to bail out his emirati and tech oligarch buddies:

https://www.nytimes.com/2026/06/05/technology/spacex-indexes-401k.html

These are arguably the most important ways in which AI is shoved down our throats, but the most visible form of conscription into the number-go-up AI project is happening at the user interface level. This UI fuckery predates AI. For more than a decade, tech companies and their executives have colluded to trick, coerce and force you into using new features that they wanted to sell to their investors as the future of their companies.

Remember Google Plus, the Facebook-killing social layer that was going to give Google access to a new, multi-trillion-dollar market? Bradley Horowitz, then a VP at Google, charged every product team with cramming G+ into their offerings, tying their bonuses to G+ usage (at companies like Google, your bonus can dwarf your base salary).

Now, googlers are bright and resourceful people, and if you tell them that they can double their earnings by getting users to click on Google Plus buttons, they'll come up with all kinds of ways to get you to do this. Some of those gambits will be the kind of thing that Horowitz doubtless hoped for: innovative and creative ways to add value to Google's core offerings with G+ features.

But when the stakes are that high, Goodhart's Law inevitably comes into play: "When a measure becomes a target, it ceases to be a good measure":

https://en.wikipedia.org/wiki/Goodhart%27s_law

The easiest way to get a user to click on a G+ button is to replace whatever button that user hits most often with a G+ version. Rather than introducing users to Google Plus in a way that turns them into G+ fans, you just frustrate them by tricking them into clicking on a useless button.

From a product manager's perspective, it's worth making whatever changes as are needed to ensure that your salary is doubled because you hit your "key performance indicator" (KPI), even if it's at the expense of your users' loyalty to your product. Preserving user loyalty at the expense of half your wages is a bad trade.

When a machine learning system does this, we call it "reward hacking." AI programmers create an "objective function" for their system, a single number that the system will try every tactic to increase; then, more often than not, the AI program finds perverse shortcuts to make the number go up whether or not it achieves the programmer's goal:

https://en.wikipedia.org/wiki/Reward_hacking

For example, if you charge an AI with coming up with the shortest run through a video game, it might well discover a bug that makes it impervious to enemy attacks or just lets it teleport to the end of the game. To avoid these perverse outcomes, programmers have to add more and more specificity to their objective functions: "speed run the game without teleporting or discovering an immortality hack."

Not for nothing, many science fiction writers (including me) have noted that tech bosses' fear of their own AI systems is a kind of emotional spillover from their frustrations with the corporations they run, where underlings engage in the same kinds of reward hacking that we see from chatbots:

https://pluralistic.net/2025/04/14/timmy-share/#a-superior-moral-justification-for-selfishness

Despite the ease with which metrics can be gamed, tech bosses remained committed to the use of KPIs to shift the priorities of their organizations, and they use the same objective function refinement technique that programmers use for reward-hacking AIs in a bid to prevent cheating by employees who hit their KPIs any way they can.

This explains a lot about the experience of watching a video on your phone. Any time you tune into a show on a streaming platform, you are forced to handle your phone with the delicacy of a photonegative. If any part of your hand brushes any part of the screen, you will immediately be whisked away from the video you're watching to an entirely different one:

https://pluralistic.net/2022/05/15/the-fatfinger-economy/

Worse, after your program has been replaced with this new one, there's no obvious way to get back to the show you were just watching. This is very weird, considering that the "back button" has been a part of computer interfaces since 1981:

https://www.merriam-webster.com/dictionary/back%20button

The reason you don't get a back button is reward hacking. Streaming video platforms are under enormous pressure to successfully recommend their back-catalog to their users, because the alternative (commissioning or purchasing a steady stream of popular new shows) is very expensive. Investors put a lot of stock in streaming services' "successful recommendation" rate.

So streaming service bosses charge their product teams with making "successful recommendations." The easiest way to "recommend a show" is to booby-trap the streaming service's UI so that touching any part of the screen takes you to a video you never asked to see.

This is obvious reward hacking, so streaming bosses refined their objective function: they made their teams' bonuses contingent on getting you to watch a new video for a minimum time-frame. This in turn triggered the next volley in the reward-hacking wars, in which product teams took away the back button. This gives users any number of greased slides that whisk them away to a new video. To get back to the old video, users are forced to climb a greased pole.

This is the fatfinger economy: created in the 2010s by teams at large tech companies with KPI-obsessed bosses; refined in the 2020s by teams at bottom-line obsessed streaming media companies; and now perfected by teams being charged with cramming AI into every corner of our lives:

https://pluralistic.net/2025/05/02/kpis-off/#principal-agentic-ai-problem

The fatfinger economy is an extraordinary beneficiary of the drive to add "telemetry" to all of our apps. Today, an app vendor can call up a dashboard and find out which buttons and fields and sliders see the most use, and they can use that ranked list to figure out which buttons to replace with AI-demon-summoning lookalikes that help product teams "prove" to their bosses that they've "successfully integrated AI" and lets their bosses prove to their investors that "users value AI features."

I recently switched from Android to Graphene, a de-googled phone operating system. Graphene's maintainers are blissfully free from the imperative to show that "users value AI features," and that's why I have never accidentally summoned up an undismissable AI by tapping a Graphene UI element.

It's a different story when it comes to apps. Graphene has many different ways to install apps, including via Google Play (Google's default Android store). That's useful because it lets you install your old Android apps after switching to Graphene, but it does leave you exposed to the fatfinger economy.

Last week, I got a painful reminder of the progress of the fatfinger economy outside of Graphene's utopian pocket-universe. I make extensive use of an app called "Genius Scanner," which lets me scan receipts and email them to myself (I do this a lot because, as a freelancer, I have to track a lot of different expenses). I bought a lifetime subscription to Genius Scanner in 2023, after getting burned by Camscanner (my previous, now enshittified scanning app), and I've been perfectly happy with it…until now.

Genius Scanner is obviously under pressure to drive AI adoption. I can tell because they've taken away the most important UI element and swapped it for a worse-than-useless AI button.

Here's how I use Genius Scanner: I press the "make new scan" button and line up the receipt on the screen. Then I press the "retitle" button, which brings up a field that is populated with the current date and time (16 characters in all). This text used to be selected so that hitting the little "x" next to it deleted it, making it easy to title a new scan. After that, I'd just email it to myself.

As of last week, though, there is no button that deletes the date and time so you can enter a useful title. That little "x" has been replaced with a button that brings up an ad for the company's new AI tools, which cost $40/year. There's no way to tell the system that you're not interested in buying a new subscription for the tool that you've already bought a lifetime sub for, and there's no way to get that little "x" back.

To be clear: that little "x" represents one quarter of my receipt scanning workflow. They have swapped it out for an ad for a new subscription AI feature, and gave me 16 new keypresses on the backspace button to replace that single tap on the "x." I have seen that ad about 100 times at this point, because I keep tapping the spot where the "x" used to be.

I'm certain that the company knew that its paying customers relied on the "x" to accomplish a vital function in any scanning app (giving good, descriptive titles to scans). They need the AI number to go up, so they replaced the "x" with the AI ad, which will let them tell their investors that they've got "strong consumer interest in their AI features."

Meanwhile, they already have my money, so there's no way I can punish them for their decision to screw me over to juke a meaningless stat. It's as neat (and depressing) an example of the principle that "even if you're paying for the product, you're still the product" that you could ask for.

It's also a perfect little object-lesson in the ways that companies can report increasing AI interest and usage without generating any interest or usage. In the fatfinger economy, every metric is a target, and is thereby rendered meaningless.

Hey look at this (permalink)


A shelf of leatherbound history books with a gilt-stamped series title, 'The World's Famous Events.'

Object permanence (permalink)

#20yrsago Army Times: Rumsfeld must go! https://web.archive.org/web/20061107220242/http://www.truthout.org/docs_2006/110406Z.shtml

#20yrsago UK is a surveillance society http://news.bbc.co.uk/1/hi/uk/6108496.stm

#20yrsago Record companies sue soccer mom’s kids https://web.archive.org/web/20061104145550/https://www.washingtonpost.com/wp-dyn/content/article/2006/11/02/AR2006110200114.html

#15yrsago HOWTO make your own skateboard truck lazy-susan table https://www.instructables.com/Skateboard-Table/

#15yrsago Home Secretary to OccupyLondon: you’re scaring the tourists https://www.bbc.co.uk/news/uk-england-london-15588987

#15yrsago Judge whose daughter released video of him beating her suspended from bench during judicial investigation https://web.archive.org/web/20111105234141/http://hosted.ap.org/dynamic/stories/U/US_VIDEOTAPED_BEATING_TEXAS

#15yrsago Prof who keeps announcing links between the Internet, childhood dementia and autism should publish theories in a scientific journal https://www.badscience.net/2011/11/why-wont-professor-greenfield-publish-this-theory-in-a-scientific-journal/

#10yrsago Internet of Things botnet threatens to knock the entire country of Liberia offline https://doublepulsar.com/shadows-kill-mirai-ddos-botnet-testing-large-scale-attacks-sending-threatening-messages-about-6a61553d1c7#.zg4448j8w

#10yrsago Black Lives Matter declares its opposition to TPP https://www.nakedcapitalism.com/2016/11/gauis-publius-tpp-has-picked-up-a-powerful-enemy-black-lives-matter.html

Upcoming appearances (permalink)

A photo of me onstage, giving a speech, pounding the podium.


A screenshot of me at my desk, doing a livecast.

Recent appearances (permalink)


A grid of my books with Will Stahle covers..

Latest books (permalink)


A cardboard book box with the Macmillan logo.

Upcoming books (permalink)

  • "The Post-American Internet," a geopolitical sequel of sorts to Enshittification, Farrar, Straus and Giroux, 2027

  • "Unauthorized Bread": a middle-grades graphic novel adapted from my novella about refugees, toasters and DRM, FirstSecond, April 20, 2027

  • "Enshittification, Why Everything Suddenly Got Worse and What to Do About It" (the graphic novel), Firstsecond, 2027

  • "The Memex Method," Farrar, Straus, Giroux, 2027


Colophon (permalink)

Today's top sources:

Currently writing:

  • “Once Is Enemy Action,” a science fiction novel about the origins of modern technofascism. Today's words: 544 (32413 total).

  • "The Post-American Internet," a sequel to "Enshittification," about the better world the rest of us get to have now that Trump has torched America. Fourth draft completed. Submitted to editor.

  • A Little Brother short story about DIY insulin PLANNING

This work – excluding any serialized fiction – is licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/@pluralistic

Bluesky (no ads, possible tracking and data-collection):

https://bsky.app/profile/doctorow.pluralistic.net

Medium (no ads, paywalled):

https://doctorow.medium.com/

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

READ CAREFULLY: By reading this, you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

ISSN: 3066-764X