Pluralistic: 22 Aug 2022 Gmail will call the cops on you based on the content of your emails

doctorow · 2022-08-22T14:47:32.980Z

Originally published at: Pluralistic: 22 Aug 2022 Gmail will call the cops on you based on the content of your emails – Pluralistic: Daily links from Cory Doctorow

Today's links

Gmail falsely told the police that a father was a molesting his son: Filternets fail.
Hey look at this: Delights to delectate.
This day in history: 2002, 2012, 2017
Colophon: Recent publications, upcoming/recent appearances, current writing projects, current reading

A desk with an open laptop on it. On the laptop's screen is a doctor's torso and folded arms. In the top right corner is a CCTV camera labelled with the Gmail logo. The camera's lens has been replaced with the staring red eye of HAL9000 from 2001: A Space Odyssey.

Gmail falsely told the police that a father was a molesting his son (permalink)

Mark's toddler had a painful, swollen penis. His wife contacted their doctor, whose nurse asked Mark to send him a picture of the toddler's penis, because the pandemic was raging and the doctor wasn't seeing patients in person. Gmail automatically detected the picture of a child's penis and turned Mark into the SFPD, accusing him of molesting his son.

https://www.nytimes.com/2022/08/21/technology/google-surveillance-toddler-photo.html

Mark and his wife took several pictures of their son's penis, including one that contained Mark's hand. The child had a bacterial infection, which was quickly alleviated with antibiotics that the doctor prescribed via telemedicine.

Google refused to listen to Mark's explanation. Instead, they terminated his account, seizing more than a decade's worth of personal and business email, cloud files, and calendar entries. He lost all the family photos he'd synched with Google Photos (including all the photos of his toddler from birth, on). He even lost his mobile plan, because he's a Google Fi user. He lost access to Google Authenticator and couldn't sign into any of his other online accounts to tell them that he had a new, non-Gmail email address.

Mark received an envelope from the SFPD telling him that Google had contacted the police department, accusing him of producing child sexual abuse material (#CSAM), and that the company had secretly given the police full access to all of his files and data, including his location and search history, as well as all his photos and videos.

The reason the police had to mail him all this stuff? Google had shut down his phone number and so they couldn't reach him.

To SFPD's credit, they'd figured out what was going on and decided Mark wasn't a child molester. To Google's shame, they continue to hold all his data hostage – including his address book with the contact info for everyone he is personally or professionally connected to, denying him access to it.

Google says they won't give Mark his account back because they found another "problematic" image in his files: "a young child lying in bed with an unclothed woman." Mark doesn't know which picture they mean (he no longer has access to any of his photos), but he thinks it was probably an intimate photo he captured of his son and wife together in bed one morning ("If only we slept with pajamas on, this all could have been avoided.").

Writing for the @NYtimes, @kashhill discusses another, similar case, involving a Houston dad called Cassio, whose doctor asked him to send in photos of his child's genitals for diagnostic purposes. Like Mark, Cassio was cleared by police, and, like Mark, Cassio is locked out of his Gmail account, along with all the services associated with it.

Hill spoke with my @EFF colleague @JonCallas, who criticized Google, saying that private family photos should be a "private sphere" and not subject to routine scanning by algorithms or review by moderators. Google claims that they only scan your photos when you take an "affirmative action" related to them, but this includes automatically uploading your photos to Google Photos, which is the default behavior on Android devices.

Also cited in the article is Kate @Klonick, a cyberlaw prof and expert on content moderation. Klonick pointed out that this was "doubly dangerous in that it also results in someone being reported to law enforcement," suggesting that this could have resulted in a loss of custody if the police had been a little less measured.

Klonick criticized Google for the lack of a "robust process" for handling this kind of automated filter error. Hill describes the "AI" tools Google uses to automatically flag potential CSAM. As is so often the case with automated filtering tools, the flagging takes place in a nanosecond, while the process for questioning its judgment takes months or years, or forever.

Last summer, I called Google and its Big Tech competitors "utilities governed like empires." The companies deliberately pursued a strategy of becoming indispensable to us, declaring mission statements like "organize all the world's information" and backing them up with vertical stacks of products designed to capture your whole digital life.

https://www.eff.org/deeplinks/2021/08/utilities-governed-empires

That is, the tech giants set out to become utilities, as important to your life as your electricity and water – and they succeeded. However, they continue to behave as though they are simply another business, whose commercial imperatives – including the arbitrary cancellation of your services without appeal – are private matters.

Some people say this means we should just turn these companies into actual utilities, but I think that's the wrong impulse. The problem with (say) Facebook, isn't merely that Zuck is monumentally unqualified to be the unaccountable self-appointed dictator of three billion peoples' digital lives. The problem is that no one should have that job. We should abolish that job.

Which is why I'm so interested in interoperability – including a mix of state-imposed interop obligations and protecting interoperators' self-help measures like reverse-engineering, scraping and bots.

https://www.eff.org/wp/interoperability-and-privacy

That is a path to pluralizing power over the necessities of our lives – use the power of the state to set limits on the conduct of online platforms (say, by passing strong privacy laws with a private right of action), which makes sure that no matter which choice a user makes, they won't be exploited by online companies. Then use the power of the state to safeguard interoperability, so that users who don't like the way an online host uses its discretion can easily leave, without surrendering their data or their social connections:

https://www.eff.org/deeplinks/2021/08/facebooks-secret-war-switching-costs

Rather than entrusting the US government – including its policing and espionage arms – to run our digital lives, and the digital lives of non-Americans around the world whom the US government explicitly disclaims any duty to, we can ask the government to do a much narrower job. We can ask them to prevent companies from harming us, and we can ask them to force companies not to take our data and social connections hostage. That way, we don't have to ask the government – which might be run by e.g. Ron Desantis in a couple years – to decide which conversations are lawful to have:

https://www.eff.org/deeplinks/2021/07/right-or-left-you-should-be-worried-about-big-tech-censorship

Instead, we can create our own, community run and community managed online spaces and services.

Image:
Cryteria (modified)
https://commons.wikimedia.org/wiki/File:HAL9000.svg

CC BY 3.0:
https://creativecommons.org/licenses/by/3.0/deed.en

(Image: Cryteria, CC BY 3.0, modified)

Hey look at this (permalink)

Inside the World’s Biggest Hacker Rickroll https://www.wired.com/story/biggest-hacker-rickroll-high-school-prank/
Who better than Liz Truss to lead a country whose own sewage laps at its shores? https://www.theguardian.com/commentisfree/2022/aug/19/liz-truss-lead-uk-sewage-leadership-marina-hyde (h/t John Naughton)
We need to start routinely tracking the ways in which insurance companies’ red tape affects patient care https://www.bmj.com/content/378/bmj.o2049 (h/t Naked Capitalism)

This day in history (permalink)

#20yrsago How the EU's DMCA will change the UK https://web.archive.org/web/20020824105943/http://www.stand.org.uk/weblog/archive/2002/08/21/000243.php

#10yrsago Epic doggerel about bad blog-comment behavior http://dreamcafe.com/2012/08/21/john-scalzis-blog/

#5yrsago OCTOBER: China Mieville’s novelistic history of the Russian revolution https://memex.craphound.com/2017/08/22/october-china-mievilles-novelistic-history-of-the-russian-revolution/

#5yrsago Researchers can take over domestic and industrial robots to spy and maim https://www.wired.com/story/watch-robot-hacks-spy-sabotage/

#5yrsago Calgary airport regrets converting disabled parking spaces into “Lexus only” parking spaces https://www.cbc.ca/amp/1.4255651

#5yrsago The Trump family’s holidays and business trips have used up the entire Secret Service budget https://www.vice.com/en/article/nee7xm/the-trumps-have-maxed-out-the-secret-services-budget-vgtrn

Colophon (permalink)

Today's top sources:

Currently writing:

The Bezzle, a Martin Hench noir thriller novel about the prison-tech industry. Friday's progress: 505 words (34829 words total)
The Internet Con: How to Seize the Means of Computation, a nonfiction book about interoperability for Verso. Friday's progress: 525 words (31181 words total)
Picks and Shovels, a Martin Hench noir thriller about the heroic era of the PC. (92849 words total) – ON PAUSE
A Little Brother short story about DIY insulin PLANNING
Vigilant, Little Brother short story about remote invigilation. FIRST DRAFT COMPLETE, WAITING FOR EXPERT REVIEW
Moral Hazard, a short story for MIT Tech Review's 12 Tomorrows. FIRST DRAFT COMPLETE, ACCEPTED FOR PUBLICATION
Spill, a Little Brother short story about pipeline protests. FINAL DRAFT COMPLETE
A post-GND utopian novel, "The Lost Cause." FINISHED
A cyberpunk noir thriller novel, "Red Team Blues." FINISHED

Currently reading: Analogia by George Dyson.

Latest podcast: What is Chokepoint Capitalism? https://craphound.com/news/2022/08/21/what-is-chokepoint-capitalism/

Upcoming appearances:

Radical Interoperability: An Internet Disassembly Manual, Aug 29 15h (Burning Man Center Camp)
Chokepoint Capitalism: a Better Deal for Creative Labor, Aug 30, 13h (Burning Man Palenque Norte, 915 and E)
Interview with Bunnie Huang on the Precursor and Trusting Trust, Aug 31, 12h (Burning Man Liminal Labs, 945 and Rod's)
Canadians Connected (Ottawa), Sept 15
https://member.cira.ca/Events/CanadiansConnected/About.aspx
Unfinished Live (NYC), Sept 21-24
https://live.unfinished.com/

Recent appearances:

DRM Secretly Polices You (Daily Tech News Show)
https://dailytechnewsshow.com/2022/07/29/drm-secretly-polices-you-dtns-4327/
Bricking Tractors (Inside Agri-Turf)
https://inside-agriturf.captivate.fm/episode/bricking-tractors-with-cory-doctorow
Blockchain, Bitcoin & Selling The Brooklyn Bridge (MMT Podcast):
https://pileusmmt.libsyn.com/135-cory-doctorow-blockchain-bitcoin-selling-the-brooklyn-bridge

Latest book:

"Attack Surface": The third Little Brother novel, a standalone technothriller for adults. The Washington Post called it "a political cyberthriller, vigorous, bold and savvy about the limits of revolution and resistance." Order signed, personalized copies from Dark Delicacies https://www.darkdel.com/store/p1840/Available_Now%3A_Attack_Surface.html
"How to Destroy Surveillance Capitalism": an anti-monopoly pamphlet analyzing the true harms of surveillance capitalism and proposing a solution. https://onezero.medium.com/how-to-destroy-surveillance-capitalism-8135e6744d59 (print edition: https://bookshop.org/books/how-to-destroy-surveillance-capitalism/9781736205907) (signed copies: https://www.darkdel.com/store/p2024/Available_Now%3A__How_to_Destroy_Surveillance_Capitalism.html)
"Little Brother/Homeland": A reissue omnibus edition with a new introduction by Edward Snowden: https://us.macmillan.com/books/9781250774583; personalized/signed copies here: https://www.darkdel.com/store/p1750/July%3A__Little_Brother_%26_Homeland.html
"Poesy the Monster Slayer" a picture book about monsters, bedtime, gender, and kicking ass. Order here: https://us.macmillan.com/books/9781626723627. Get a personalized, signed copy here: https://www.darkdel.com/store/p2682/Corey_Doctorow%3A_Poesy_the_Monster_Slayer_HB.html#/.

Upcoming books:

Chokepoint Capitalism: How to Beat Big Tech, Tame Big Content, and Get Artists Paid, with Rebecca Giblin, nonfiction/business/politics, Beacon Press, September 2022
Red Team Blues: "A grabby, compulsive thriller that will leave you knowing more about how the world works than you did before." Tor Books, April 2023

This work licensed under a Creative Commons Attribution 4.0 license. That means you can use it any way you like, including commercially, provided that you attribute it to me, Cory Doctorow, and include a link to pluralistic.net.

https://creativecommons.org/licenses/by/4.0/

Quotations and images are not included in this license; they are included either under a limitation or exception to copyright, or on the basis of a separate license. Please exercise caution.

How to get Pluralistic:

Blog (no ads, tracking, or data-collection):

Pluralistic.net

Newsletter (no ads, tracking, or data-collection):

https://pluralistic.net/plura-list

Mastodon (no ads, tracking, or data-collection):

https://mamot.fr/web/accounts/303320

Medium (no ads, paywalled):

https://doctorow.medium.com/

(Latest Medium column: "What is Chokepoint Capitalism? https://doctorow.medium.com/what-is-chokepoint-capitalism-b885c4cb2719)

Twitter (mass-scale, unrestricted, third-party surveillance and advertising):

https://twitter.com/doctorow

Tumblr (mass-scale, unrestricted, third-party surveillance and advertising):

https://mostlysignssomeportents.tumblr.com/tagged/pluralistic

"When life gives you SARS, you make sarsaparilla" -Joey "Accordion Guy" DeVilla

james_riley · 2022-08-22T16:00:05.332Z

Assuming they still hold the info, has he tried making a GDPR request for his data? Or the California near-equivalent? I forget how deep the Californian GDPR-almost-clone goes, but Alphabet won’t know for certain whether he holds a 2nd passport that’s EU.

Karellen · 2022-08-24T00:47:07.848Z

doctorow:

Some people say this means we should just turn these companies into actual utilities, but I think that’s the wrong impulse. […] I’m so interested in interoperability – including a mix of state-imposed interop obligations and protecting interoperators’ self-help measures like reverse-engineering, scraping and bots.

Would that help in a case like this?

In order for that to be useful, you’d have to have tried moving to your new provider before your old provider locked your account and denied access to all of its contents via every API access method. You might say that you should make backups, but the one plausible genuine selling point of storing your data on Other People’s Computers is that they take care of that so you don’t have to.

But also, it kind of suggests that customers would be able to make meaningful decisions over which Clown provider to choose, based on them not booting you out if they find innocuous pictures of your kid that you took for the doctor, or in the bath, or whatever. But it seems more likely to me that those providers would compete with each other on how aggressively they’d protect your kids online from predators, which sounds like something you want… until you realise too late that that actually means them yes actually booting you off when they find those pictures, and now you’re screwed again.

I think some kind of utility-like regulation regarding protection against having your service cancelled without warning, before you have a chance to line up another provider, is a necessity. That’s not to say mandatory interoperability isn’t a good idea - I think it really is. But I don’t think it’s enough.

system · 2022-09-06T14:48:24.159Z

This topic was automatically closed after 15 days. New replies are no longer allowed.