Tag Archives: algorithms

Behind every algorithm, there be politics.

In my first class in computer science, I was taught that an algorithm is simply a way of expressing formal rules given to a computer. Computers like rules. They follow them. Turns out that bureaucracy and legal systems like rules too. The big difference is that, in the world of computing, we call those who are trying to find ways to circumvent the rules “hackers” but in the world of government, this is simply the mundane work of politicking and lawyering. 

When Dan Bouk (and I, as an earnest student of his) embarked on a journey to understand the history of the 1920 census, we both expected to encounter all sorts of politicking and lawyering. As scholars fascinated by the census, we’d heard the basics of the story: Congress failed to reapportion itself after receiving data from the Census Bureau because of racist and xenophobic attitudes mixed with political self-interest. In other words, politics. 

As we dove into this history, the first thing we realized was that one justification for non-apportionment centered on a fight about math. Politicians seemed to be arguing with each other over which algorithm was the right algorithm with which to apportion the House. In the end, they basically said that apportionment should wait until mathematicians could figure out what the “right” algorithm was. (Ha!) The House didn’t manage to pass an apportionment bill until 1929 when political negotiations had made this possible. (This story anchors our essay on “Democracy’s Data Infrastructure.”)

Dan kept going, starting what seemed like a simple question: what makes Congress need an algorithm in the first place? I bet you can’t guess what the answer is! Wait for it… wait for it… Politics! Yes, that’s right, Congress wanted to cement an algorithm into its processes in a feint attempt to de-politicize the reapportionment process. With a century of extra experience with algorithms, this is patently hysterical. Algorithms as a tool to de-politicize something!?!? Hahahah. But, that’s where they had gotten to. And now the real question was: why? 

In Dan’s newest piece – “House Arrest: How an Automated Algorithm Constrained Congress for a Century” – Dan peels back the layers of history with beautiful storytelling and skilled analysis to reveal why our contemporary debates about algorithmic systems aren’t so very new. Turns out that there were a variety of political actors deeply invested in ensuring that the People’s House stopped growing. Some of their logics were rooted in ideas about efficiency, but some were rooted in much older ideas of power and control. (Don’t forget that the electoral college is tethered to the size of the House too!) I like to imagine power-players sitting around playing with their hands and saying mwah-ha-ha-ha as they strategize over constraining the growth of the size of the House. They wanted to do this long before 1920, but it didn’t get locked in then because they couldn’t agree, which is why they fought over the algorithm. By 1929, everyone was fed up and just wanted Congress to properly apportion and so they passed a law, a law that did two things: it stabilized the size of the House at 435 and it automated the apportionment process. Those two things – the size of the House and the algorithm – were totally entangled. After all, an automated apportionment couldn’t happen without the key variables being defined. 

Of course, that’s not the whole story. That 1929 bill was just a law. Up until then, Congress had passed a new law every decade to determine how apportionment would work for that decade. But when the 1940 census came around, they were focused on other things. And then, in effect, Congress forgot. They forgot that they have the power to determine the size of the House. They forgot that they have control over that one critical variable. The algorithm became infrastructure and the variable was summarily ignored.

Every decade, when the Census data are delivered, there are people who speak out about the need to increase the size of the House. After all, George Washington only spoke once during the Constitutional Convention. He spoke up to say that we couldn’t possibly have Congresspeople represent 40,000 people because then they wouldn’t trust government! The constitutional writers listened to him and set the minimum at 30,000; today, our representatives each represent more than 720,000 of us. 

After the 1790 census, there were 105 representatives in Congress. Every decade, that would increase. Even though it wasn’t exact, there was an implicit algorithm in that size increase. In short, increase the size of the House so that no sitting member would lose his seat. After all, Congress had to pass that bill and this was the best way to get everyone to vote on it. The House didn’t increase at the same ratio as the size of the population, but it did increase every decade until 1910. And then it stopped (with extra seats given to new states before being brought back to the zero-sum game at the next census). 

One of the recommendations of the Commission on the Practice of Democratic Citizenship (for which I was a commissioner) was to increase the size of the House. When we were discussing this as a commission, everyone spoke of how radical this proposition was, how completely impossible it would be politically. This wasn’t one of my proposals – I wasn’t even on that subcommittee – so I listened with rapt curiosity. Why was it so radical? Dan taught me the answer to that. The key to political power is to turn politicking into infrastructure. After all, those who try to break a technical system, to work around an algorithm, they’re called hackers. And hackers are radical. 

Want more like this?

  1. Read “House Arrest: How an Automated Algorithm Constrained Congress for a Century” by Dan Bouk. There’s drama! And intrigue! And algorithms!
  2. Read “Democracy’s Data Infrastructure” by Dan Bouk and me. It might shape your view about public fights over math.
  3. Sign up for my newsletter. More will be coming, I promise!

Facebook Must Be Accountable to the Public

A pair of Gizmodo stories have prompted journalists to ask questions about Facebook’s power to manipulate political opinion in an already heated election year. If the claims are accurate, Facebook contractors have depressed some conservative news, and their curatorial hand affects the Facebook Trending list more than the public realizes. Mark Zuckerberg took to his Facebook page yesterday to argue that Facebook does everything possible to be neutral and that there are significant procedures in place to minimize biased coverage. He also promises to look into the accusations.

Watercolor by John Orlando Parry, “A London Street Scene” 1835, in the Alfred Dunhill Collection.

As this conversation swirls around intentions and explicit manipulation, there are some significant issues missing. First, all systems are biased. There is no such thing as neutrality when it comes to media. That has long been a fiction, one that traditional news media needs and insists on, even as scholars highlight that journalists reveal their biases through everything from small facial twitches to choice of frames and topics of interests. It’s also dangerous to assume that the “solution” is to make sure that “both” sides of an argument are heard equally. This is the source of tremendous conflict around how heated topics like climate change and evolution are covered. Itis even more dangerous, however, to think that removing humans and relying more on algorithms and automation will remove this bias.

Recognizing bias and enabling processes to grapple with it must be part of any curatorial process, algorithmic or otherwise. As we move into the development of algorithmic models to shape editorial decisions and curation, we need to find a sophisticated way of grappling with the biases that shape development, training sets, quality assurance, and error correction, not to mention an explicit act of “human” judgment.

There never was neutrality, and there never will be.

This issue goes far beyond the Trending box in the corner of your Facebook profile, and this latest wave of concerns is only the tip of the iceberg around how powerful actors can affect or shape political discourse. What is of concern right now is not that human beings are playing a role in shaping the news — they always have — it is the veneer of objectivity provided by Facebook’s interface, the claims of neutrality enabled by the integration of algorithmic processes, and the assumption that what is prioritized reflects only the interests and actions of the users (the “public sphere”) and not those of Facebook, advertisers, or other powerful entities.

The key challenge that emerges out of this debate concerns accountability.In theory, news media is accountable to the public. Like neutrality, this is more of a desired goal than something that’s consistently realized. While traditional news media has aspired to — but not always realized — meaningful accountability, there are a host of processes in place to address the possibility of manipulation: ombudspeople, whistleblowers, public editors, and myriad alternate media organizations. Facebook and other technology companies have not, historically, been included in that conversation.

I have tremendous respect for Mark Zuckerberg, but I think his stance that Facebook will be neutral as long as he’s in charge is a dangerous statement.This is what it means to be a benevolent dictator, and there are plenty of people around the world who disagree with his values, commitments, and logics. As a progressive American, I have a lot more in common with Mark than not, but I am painfully aware of the neoliberal American value systems that are baked into the very architecture of Facebook and our society as a whole.

Who Controls the Public Sphere in an Era of Algorithms?

In light of this public conversation, I’m delighted to announce that Data & Society has been developing a project that asks who controls the public sphere in an era of algorithms. As part of this process, we convened a workshop and have produced a series of documents that we think are valuable to the conversation:

These documents provide historical context, highlight how media has always been engaged in power struggles, showcase the challenges that new media face, and offer case studies that reveal the complexities going forward.

This conversation is by no means over. It is only just beginning. My hope is that we quickly leave the state of fear and start imagining mechanisms of accountability that we, as a society, can live with. Institutions like Facebook have tremendous power and they can wield that power for good or evil. Butfor society to function responsibly, there must be checks and balances regardless of the intentions of any one institution or its leader.

This work is a part of Data & Society’s developing Algorithms and Publics project, including a set of documents occasioned by the Who Controls the Public Sphere in an Era of Algorithms? workshop. More posts from workshop participants:

Are We Training Our Students to be Robots?

Excited about the possibility that he would project his creativity onto paper, I handed my 1-year-old son a crayon. He tried to eat it. I held his hand to show him how to draw, and he broke the crayon in half. I went to open the door and when I came back, he had figured out how to scribble… all over the wooden floor.

Crayons are pretty magical and versatile technologies. They can be used as educational tools — or alternatively, as projectiles. And in the process of exploring their properties, children learn to make sense of both their physical affordances and the social norms that surround them. “No, you can’t poke your brother’s eye with that crayon!” is a common refrain in my house. Learning to draw — on paper and with some sense of meaning — has a lot to do with the context, a context in which I help create, a context that is learned outside of the crayon itself.

From crayons to compasses, we’ve learned to incorporate all sorts of different tools into our lives and educational practices. Why, then, do computing and networked devices consistently stump us? Why do we imagine technology to be our educational savior, but also the demon undermining learning through distraction? Why are we so unable to see it as a tool whose value is most notably discovered situated in its context?

The arguments that Peg Tyre makes in “iPads < Teachers” are dead on. Personalized learning technologies won’t magically on their own solve our education crisis. The issues we are facing in education are social and political, reflective of our conflicting societal values. Our societal attitudes toward teachers are deeply destructive, a contemporary manifestation of historical attitudes towards women’s labor.

But rather than seeing learning as a process and valuing educators as an important part of a healthy society, we keep looking for easy ways out of our current predicament, solutions that don’t involve respecting the hard work that goes into educating our young.
In doing so, we glom onto technologies that will only exacerbate many existing issues of inequity and mistrust. What’s at stake isn’t the technology itself, but the future of learning.

An empty classroom at the Carpe Diem school in Indianapolis.
Education shouldn’t be just about reading, writing, and arithmetic. Students need to learn how to be a part of our society. And increasingly, that society is technologically mediated. As a result, excluding technology from the classroom makes little sense; it produces an unnecessary disconnect between school and contemporary life.

This forces us to consider two interwoven — and deeply political — societal goals of education: to create an informed citizenry and to develop the skills for a workforce.

With this in mind, there are different ways of interpreting the personalized learning agenda, which makes me feel simultaneously optimistic and outright terrified. If you take personalized learning to its logical positive extreme, technology will educate every student as efficiently as possible. This individual-centric agenda is very much rooted in American neoliberalism.

But what if there’s a darker story? What if we’re really training our students to be robots?

Let me go cynical for a moment. In the late 1800s, the goal of education in America was not particularly altruistic. Sure, there were reformers who imagined that a more educated populace would create an informed citizenry. But what made widespread education possible was that American business needed workers. Industrialization required a populace socialized into very particular frames of interaction and behavior. In other words, factories needed workers who could sit still.

Many of tomorrow’s workers aren’t going to be empowered creatives subscribed to the mantra of, “Do what you love!” Many will be slotted into systems of automation that are hybrid human and computer. Not in the sexy cyborg way, but in the ugly call center way.
Like today’s retail laborers who have to greet every potential customer with a smile, many humans in tomorrow’s economy will do the unrewarding tasks that are too expensive for robots to replace. We’re automating so many parts of our society that, to be employable, the majority of the workforce needs to be trained to be engaged with automated systems.

All of this begs one important question: who benefits, and who loses, from a technologically mediated world?

Education has long been held up as the solution to economic disparity (though some reports suggest that education doesn’t remedy inequity). While the rhetoric around personalized learning emphasizes the potential for addressing inequity, Tyre suggests that good teachers are key for personalized learning to work.

Not only are privileged students more likely to have great teachers, they are also more likely to have teachers who have been trained to use technology — and how to integrate it into the classroom’s pedagogy. If these technologies do indeed “enhance the teacher’s effect,” this does not bode well for low-status students, who are far less likely to have great teachers.

Technology also costs money. Increasingly, low-income schools are pouring large sums of money into new technologies in the hopes that those tools can fix the various problems that low-status students face. As a result, there’s less money for good teachers and other resources that schools need.

I wish I had a solution to our education woes, but I’ve been stumped time and again, mostly by the politics surrounding any possible intervention. Historically, education was the province of local schools making local decisions. Over the last 30 years, the federal government and corporations alike have worked to centralize education.

From textbooks to grading systems, large companies have standardized educational offerings, while making schools beholden to their design logic. This is how Texas values get baked into Minnesota classrooms. Simultaneously, over legitimate concern about the variation in students’ experiences, federal efforts have attempted to implement learning standards. They use funding as the stick for conformity, even as local politics and limited on-the-ground resources get in the way.

Personalized learning has the potential to introduce an entirely new factor into the education landscape: network effects. Even as ranking systems have compared schools to one another, we’ve never really had a system where one students’ learning opportunities truly depend on another’s. And yet, that’s core to how personalized learning works. These systems don’t evolve based on the individual, but based on what’s learned about students writ large.

Personalized learning is, somewhat ironically, far more socialist than it may first appear. You can’t “personalize” technology without building models that are deeply dependent on others. In other words, it is all about creating networks of people in a hyper-individualized world. It’s a strange hybrid of neoliberal and socialist ideologies.

An instructor works with a student in the learning center at the Carpe Diem school in Indianapolis.
Just as recommendation systems result in differentiated experiences online, creating dynamics where one person’s view of the internet radically differs from another’s, so too will personalized learning platforms.

More than anything, what personalized learning brings to the table for me is the stark reality that our society must start grappling with the ways we are both interconnected and differentiated. We are individuals and we are part of networks.

In the realm of education, we cannot and should not separate these two. By recognizing our interconnected nature, we might begin to fulfill the promises that technology can offer our students.

This post was originally published to Bright at Medium on April 7, 2015. Bright is made possible by funding from the New Venture Fund, and is supported by The Bill & Melinda Gates Foundation.

Guilt Through Algorithmic Association

You’re a 16-year-old Muslim kid in America. Say your name is Mohammad Abdullah. Your schoolmates are convinced that you’re a terrorist. They keep typing in Google queries likes “is Mohammad Abdullah a terrorist?” and “Mohammad Abdullah al Qaeda.” Google’s search engine learns. All of a sudden, auto-complete starts suggesting terms like “Al Qaeda” as the next term in relation to your name. You know that colleges are looking up your name and you’re afraid of the impression that they might get based on that auto-complete. You are already getting hostile comments in your hometown, a decidedly anti-Muslim environment. You know that you have nothing to do with Al Qaeda, but Google gives the impression that you do. And people are drawing that conclusion. You write to Google but nothing comes of it. What do you do?

This is guilt through algorithmic association. And while this example is not a real case, I keep hearing about real cases. Cases where people are algorithmically associated with practices, organizations, and concepts that paint them in a problematic light even though there’s nothing on the web that associates them with that term. Cases where people are getting accused of affiliations that get produced by Google’s auto-complete. Reputation hits that stem from what people _search_ not what they _write_.

It’s one thing to be slandered by another person on a website, on a blog, in comments. It’s another to have your reputation slandered by computer algorithms. The algorithmic associations do reveal the attitudes and practices of people, but those people are invisible; all that’s visible is the product of the algorithm, without any context of how or why the search engine conveyed that information. What becomes visible is the data point of the algorithmic association. But what gets interpreted is the “fact” implied by said data point, and that gives an impression of guilt. The damage comes from creating the algorithmic association. It gets magnified by conveying it.

  1. What are the consequences of guilt through algorithmic association?
  2. What are the correction mechanisms?
  3. Who is accountable?
  4. What can or should be done?

Note: The image used here is Photoshopped. I did not use real examples so as to protect the reputations of people who told me their story.

Update: Guilt through algorithmic association is not constrained to Google. This is an issue for any and all systems that learn from people and convey collective “intelligence” back to users. All of the examples that I was given from people involved Google because Google is the dominant search engine. I’m not blaming Google. Rather, I think that this is a serious issue for all of us in the tech industry to consider. And the questions that I’m asking are genuine questions, not rhetorical ones.