{"id":6872,"date":"2018-01-11T21:35:00","date_gmt":"2018-01-12T01:35:00","guid":{"rendered":"http:\/\/www.zephoria.org\/thoughts\/?p=6872"},"modified":"2018-07-21T21:35:48","modified_gmt":"2018-07-22T01:35:48","slug":"beyond-the-rhetoric-of-algorithmic-solutionism","status":"publish","type":"post","link":"https:\/\/www.zephoria.org\/thoughts\/archives\/2018\/01\/11\/beyond-the-rhetoric-of-algorithmic-solutionism.html","title":{"rendered":"Beyond the Rhetoric of Algorithmic Solutionism"},"content":{"rendered":"

(This was originally posted on Medium<\/a>)<\/em><\/p>\n

If you ever hear that implementing algorithmic decision-making tools to enable social services or other high stakes government decision-making will increase efficiency or reduce the cost to taxpayers, know that you\u2019re\u00a0<\/span>being<\/span>\u00a0lied to.\u00a0<\/span>When implemented ethically, these systems cost more. And they should.<\/span><\/p>\n

Whether we\u2019re\u00a0<\/span>talking about\u00a0<\/span>judicial<\/span>\u00a0decision making (e.g., \u201crisk assessment scoring\u201d) or modeling who is at risk for homelessness,\u00a0<\/span>algorithmic systems don\u2019t simply cost money to\u00a0implement<\/em>. They cost money to maintain.<\/span>\u00a0They cost money to audit.<\/span>\u00a0<\/span>They cost money to evolve with the domain<\/span>\u00a0that they\u2019re designed to serve.<\/span>\u00a0They cost money to train their users to use the data responsibly<\/span>.\u00a0<\/span>Above all,\u00a0<\/span>they make visible the brutal pain points and root causes in existing systems that<\/span>\u00a0<\/span>require an increase of services.<\/span><\/p>\n

Otherwise,\u00a0<\/span>all that these systems are doing is helping divert taxpayer money from direct services, to lining the pockets of for-profit entities under the illusion of helping people.<\/span>\u00a0<\/span>Worse,\u00a0<\/span>they\u2019re helping usher in a diversion of liability because\u00a0<\/span>time and time again,\u00a0<\/span>those in powerful positions blame the algorithms<\/span>.<\/span><\/p>\n

This doesn\u2019t mean that these tools can\u2019t be used responsibly. They can. And they should. The insights that large-scale data analysis can offer is inspiring.\u00a0The opportunity to help people by understanding the complex interplay of contextual information is invigorating.\u00a0<\/span>Any social scientist with a heart desperately wants to understand how to relieve inequality and create a more fair and equitable system<\/span>.<\/span>\u00a0So of course there\u2019s a desire to jump in and try to make sense of the data out there to make a difference in people\u2019s lives.\u00a0<\/span>But\u00a0<\/span>to treat data analysis as a savior to a broken system is woefully naive<\/span>.<\/span><\/p>\n

Doing so obfuscates the financial incentives of\u00a0<\/span>those who are building these services, the deterministic rhetoric that they use to justify their implementation, the opacity that results from having non-technical actors try to understand technical jiu-jitsu, and the stark reality of how\u00a0<\/span>technology is used as a political bludgeoning<\/span>\u00a0tool<\/span>.<\/span>\u00a0<\/span>Even more frustratingly,<\/span>\u00a0<\/span>what data analysis does well is open up opportunities for experimentation and deeper exploration<\/span>.\u00a0<\/span>But in a zero-sum context, that means that\u00a0<\/span>the resources to\u00a0do something<\/em>\u00a0about the information that is learned is siphoned off to the technology<\/span>.<\/span>\u00a0<\/span>And, worse, because the technology is supposed to save money, there is no budget for using that data to actually help people<\/span>. Instead,<\/span>technology becomes a mirage.\u00a0<\/span>Not because the technology is inherently bad, but because of how it is deployed and used.<\/span><\/p>\n

\n
\n
<\/div>\n
<\/canvas><\/div>\n<\/div>
READ THIS\u00a0BOOK!<\/a><\/figcaption><\/figure>\n

Next week,\u00a0a new book<\/strong>\u00a0that shows the true cost of these systems is being published.\u00a0Virginia Eubanks\u2019 book\u201cAutomating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor\u201d<\/strong><\/a><\/span>\u00a0is a deeply researched accounting of how algorithmic tools are integrated into services for welfare, homelessness, and child protection. Eubanks goes deep with the people and families who are targets of these systems, telling their stories and experiences in rich detail. Further, drawing on interviews with social services clients and service providers alongside the information provided by technology vendors and government officials,\u00a0Eubanks offers\u00a0a clear portrait of just how algorithmic systems actually play out on the ground<\/strong>, despite all of the hope that goes into their implementation.<\/span><\/p>\n

Eubanks eschews the term \u201cethnography\u201d because she argues that this book is immersive journalism, not ethnography. Yet, from my perspective as a scholar and a reader,\u00a0this is the best ethnography I\u2019ve read in years<\/strong>.\u00a0\u201c<\/a><\/span>Automating Inequality\u201d<\/a>\u00a0does exactly what a good ethnography should do\u200a\u2014\u200ait offers a compelling account of the cultural logics surrounding a particular dynamic, and invites the reader to truly grok what\u2019s at stake through the eyes of a diverse array of relevant people.<\/span>\u00a0Eubanks brings you into the world of technologically mediated social services and helps you see what this really looks like on the ground.\u00a0<\/span>She showcases the frustration and anxiety that these implementations produce<\/span>; the ways in which\u00a0<\/span>both social services recipientsand<\/em>\u00a0taxpayers are screwed by the false promises of these technologies<\/span>.\u00a0<\/span>She makes visible the politics and the stakes, the costs and the hope. Above all, she brings the reader into the stark and troubling reality of what it really means to be poor in America today.<\/span><\/p>\n

\u201cAutomating Inequality\u201d is on par with\u00a0<\/span>Barbara Ehrenreich\u2019s\u00a0\u201cNickel and Dimed\u201d<\/a>\u00a0or Matthew Desmond\u2019s\u00a0\u201cEvicted.<\/a><\/span>\u201d<\/a>\u00a0It\u2019s rigorously researched, phenomenally accessible, and utterly humbling. While there are a lot of important books that touch on the costs and consequences of technology through case studies and well-reasoned logic, this book is the first one that I\u2019ve read that really pulls you into the world of algorithmic decision-making and inequality, like a good ethnography should.<\/span><\/p>\n

I don\u2019t know how Eubanks chose her title, but one of the subtle things about her choice is that she\u2019s (unintentionally?) offering a fantastic backronym for AI.\u00a0<\/span>Rather than thinking of AI as \u201cartificial intelligence,\u201d Eubanks effectively builds the case for how\u00a0<\/span>we should think that\u00a0<\/span>AI often means \u201cautomating inequality\u201d in practice<\/span>.<\/span><\/p>\n

This book should be mandatory for anyone who works in social services, government, or the technology sector because it forces you to really think about what algorithmic decision-making tools are doing to our public sector, and the costs that this has on the people that are supposedly being served.<\/span>\u00a0<\/span>It\u2019s also\u00a0<\/span>essential reading for taxpayers and voters who need to understand why technology is not the panacea that it\u2019s often purported to be.<\/span>\u00a0<\/span>Or rather, how capitalizing on the benefits of technology will require serious investment and a deep commitment to improving the quality of social services, rather than a tax cut.<\/span><\/p>\n

Please please please\u00a0read this book<\/a>. It\u2019s too important not to.<\/p>\n

Data & Society will also be hosting Virginia Eubanks to talk about her book on January 17th at 4PM ET. She will be in conversation with Julia Angwin and Alondra Nelson. The event is sold out, but it will be\u00a0<\/em>livestreamed<\/em><\/a>\u00a0online. Please feel free to join us there!<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

(This was originally posted on Medium) If you ever hear that implementing algorithmic decision-making tools to enable social services or other high stakes government decision-making will increase efficiency or reduce the cost to taxpayers, know that you\u2019re\u00a0being\u00a0lied to.\u00a0When implemented ethically, these systems cost more. And they should. Whether we\u2019re\u00a0talking about\u00a0judicial\u00a0decision making (e.g., \u201crisk assessment scoring\u201d) […]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_s2mail":"yes","footnotes":""},"categories":[1],"tags":[],"class_list":["post-6872","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/posts\/6872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/comments?post=6872"}],"version-history":[{"count":1,"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/posts\/6872\/revisions"}],"predecessor-version":[{"id":6873,"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/posts\/6872\/revisions\/6873"}],"wp:attachment":[{"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/media?parent=6872"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/categories?post=6872"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.zephoria.org\/thoughts\/wp-json\/wp\/v2\/tags?post=6872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}