Category Archives: Uncategorized

Put an End to Reporting on Election Polls

We now know that the US election polls were wrong. Just like they were in Brexit. Over the last few months, I’ve told numerous reporters and people in the media industry that they should be wary of the polling data they’re seeing, but I was generally ignored and dismissed. I wasn’t alone — two computer scientists whom I deeply respect — Jenn Wortman Vaughan and Hanna Wallach — were trying to get an op-ed on prediction and uncertainty into major newspapers, but were repeatedly told that the outcome was obvious. It was not. And election polls will be increasingly problematic if we continue to approach them the way we currently do.

It’s now time for the media to put a moratorium on reporting on election polls and fancy visualizations of statistical data. And for data scientists and pollsters to stop feeding the media hype cycle with statistics that they know have flaws or will be misinterpreted as fact.

Why Political Polling Will Never Be Right Again

Polling and survey research has a beautiful history, one that most people who obsess over the numbers don’t know. In The Averaged American, Sarah Igo documents three survey projects that unfolded in the mid-20th century that set the stage for contemporary polling: the Middletown studies, Gallup, and Kinsey. As a researcher, it’s mindblowing to see just how naive folks were about statistics and data collection in the early development of this field, how much the field has learned and developed. But there’s another striking message in this book: Americans were willing to contribute to these kinds of studies at unparalleled levels compared to their peers worldwide because they saw themselves as contributing to the making of public life. They were willing to reveal their thoughts, beliefs, and ideas because they saw doing so as productive for them individually and collectively.

As folks unpack the inaccuracies of contemporary polling data, they’re going to focus on technical limitations. Some of these are real. Cell phones have changed polling — many people don’t pick up unknown numbers. The FCC’s ruling that limited robocalls to protect consumers in late 2015 meant that this year’s sampling process got skewed, that polling became more expensive, and that pollsters took shortcuts. We’ve heard about how efforts to extrapolate representativeness from small samples messes with the data — such as the NYTimes report on a single person distorting national polling averages.

But there’s a more insidious problem with the polling data that is often unacknowledged. Everyone and their mother wants to collect data from the public. And the public is tired of being asked, which they perceive as being nagged. In swing states, registered voters were overwhelmed with calls from real pollsters, fake pollsters, political campaigns, fundraising groups, special interest groups, and their neighbors. We know that people often lie to pollsters (confirmation bias), but when people don’t trust information collection processes, normal respondent bias becomes downright deceptive. You cannot collect reasonable data when the public doesn’t believe in the data collection project. And political pollsters have pretty much killed off their ability to do reasonable polling because they’ve undermined trust. It’s like what happens when you plant the same crop over and over again until the land can no longer sustain that crop.

Election polling is dead, and we need to accept that.

Why Reporting on Election Polling Is Dangerous

To most people, even those who know better, statistics look like facts. And polling results look like truth serum, even when pollsters responsibly report margin of error information. It’s just so reassuring or motivating to see stark numbers because you feel like you can do something about those numbers, and then, when the numbers change, you feel good. This plays into basic human psychology. And this is why we use numbers as an incentive in both education and the workplace.

Political campaigns use numbers to drive actions on their teams. They push people to go to particular geographies, they use numbers to galvanize supporters. And this is important, which is why campaigns invest in pollsters and polling processes.

Unfortunately, this psychology and logic gets messed up when you’re talking about reporting on election polls in the public. When the numbers look like your team is winning, you relax and stop fretting, often into complacency.When the numbers look like your team is losing, you feel more motivated to take steps and do something. This is part of why the media likes the horse race — they push people to action by reporting on numbers, which in effect pushes different groups to take action. They like the attention that they get as the mood swings across the country in a hotly contested race.

But there is number burnout and exhaustion. As people feel pushed and swayed, as the horse race goes on and on, they get more and more disenchanted. Rather than galvanizing people to act, reporting on political polling over a long period of time with flashy visuals and constantly shifting needles prompts people to disengage from the process. In short, when it comes to the election, this prompts people to not show up to vote. Or to be so disgusted that voting practices become emotionally negative actions rather than productively informed ones.

This is a terrible outcome. The media’s responsibility is to inform the public and contribute to a productive democratic process. By covering political polls as though they are facts in an obsessive way, they are not only being statistically irresponsible, but they are also being psychologically irresponsible.

The news media are trying to create an addictive product through their news coverage, and, in doing so, they are pushing people into a state of overdose.

Yesterday, I wrote about how the media is being gamed and not taking moral responsibility for its participation in the spectacle of this year’s election. One of its major flaws is how it’s covering data and engaging in polling coverage. This is, in many ways, the easiest part of the process to fix. So I call on the news media to put a moratorium on political polling coverage, to radically reduce the frequency with which they reference polls during an election season, and to be super critical of the data that they receive. If they want to be a check to power, they need to have the structures in place to be a check to math.

(This was first posted on Points.)

I blame the media. Reality check time.

For months I have been concerned about how what I was seeing on the ground and in various networks was not at all aligned with what pundits were saying. I knew the polling infrastructure had broken, but whenever I told people about the problems with the sampling structure, they looked at me like an alien and told me to stop worrying. Over the last week, I started to accept that I was wrong. I wasn’t.

And I blame the media.

The media is supposed to be a check to power, but, for years now, it has basked in becoming power in its own right. What worries me right now is that, as it continues to report out the spectacle, it has no structure for self-reflection, for understanding its weaknesses, its potential for manipulation.

I believe in data, but data itself has become spectacle. I cannot believe that it has become acceptable for media entities to throw around polling data without any critique of the limits of that data, to produce fancy visualizations which suggest that numbers are magical information. Every pollster got it wrong. And there’s a reason. They weren’t paying attention to the various structural forces that made their sample flawed, the various reasons why a disgusted nation wasn’t going to contribute useful information to inform a media spectacle. This abuse of data has to stop. We need data to be responsible, not entertainment.

This election has been a spectacle because the media has enjoyed making it as such. And in doing so, they showcased just how easily they could be gamed. I refer to the sector as a whole because individual journalists and editors are operating within a structural frame, unmotivated to change the status quo even as they see similar structural problems to the ones I do. They feel as though they “have” to tell a story because others are doing so, because their readers can’t resist reading. They live in the world pressured by clicks and other elements of the attention economy. They need attention in order to survive financially. And they need a spectacle, a close race.

We all know that story. It’s not new. What is new is that they got played.
Over the last year, I’ve watched as a wide variety of decentralized pro-Trump actors first focused on getting the media to play into his candidacy as spectacle, feeding their desire for a show. In the last four months, I watched those same networks focus on depressing turnout, using the media to trigger the populace to feel so disgusted and frustrated as to disengage. It really wasn’t hard because the media was so easy to mess with. And they were more than happy to spend a ridiculous amount of digital ink circling round and round into a frenzy.

Around the world, people have been looking at us in a state of confusion and shock, unsure how we turned our democracy into a new media spectacle. What hath 24/7 news, reality TV, and social media wrought? They were right to ask. We were irresponsible to ignore.

In the tech sector, we imagined that decentralized networks would bring people together for a healthier democracy. We hung onto this belief even as we saw that this wasn’t playing out. We built the structures for hate to flow along the same pathways as knowledge, but we kept hoping that this wasn’t really what was happening. We aided and abetted the media’s suicide.
The red pill is here. And it ain’t pretty.

We live in a world shaped by fear and hype, not because it has to be that way, but because this is the obvious paradigm that can fuel the capitalist information architectures we have produced.

Many critics think that the answer is to tear down capitalism, make communal information systems, or get rid of social media. I disagree. But I do think that we need to actively work to understand complexity, respectfully engage people where they’re at, and build the infrastructure to enable people to hear and appreciate different perspectives. This is what it means to be truly informed.

There are many reasons why we’ve fragmented as a country. From the privatization of the military (which undermined the development of diverse social networks) to our information architectures, we live in a moment where people do not know how to hear or understand one another. And our obsession with quantitative data means that we think we understand when we hear numbers in polls, which we use to judge people whose views are different than our own. This is not productive.

Most people are not apathetic, but they are disgusted and exhausted. We have unprecedented levels of anxiety and fear in our country. The feelings of insecurity and inequality cannot be written off by economists who want to say that the world is better today than it ever was. It doesn’t feel that way. And it doesn’t feel that way because, all around us, the story is one of disenfranchisement, difference, and uncertainty.

All of us who work in the production and dissemination of information need to engage in a serious reality check.

The media industry needs to take responsibility for its role in producing spectacle for selfish purposes. There is a reason that the public doesn’t trust institutions in this country. And what the media has chosen to do is far from producing information. It has chosen to produce anxiety in the hopes that we will obsessively come back for more. That is unhealthy. And it’s making us an unhealthy country.

Spectacle has a cost. It always has. And we are about to see what that cost will be.

(This was first posted at Points.)

Columbus Day!?!? What the f* are we celebrating?

Today is Columbus Day, a celebration of colonialism wrapped up under the guise of exploration. Children around the US are taught that European settlers came in 1492 and found a whole new land magically free for occupation. In November, they will be told that there were small and disperse savage populations who opened their arms to white settlers fleeing oppression. Some of those students may eventually learn on their own about violence, genocide, infection, containment, relocation, humiliation, family separation, and cultural devaluation which millions of Native peoples experienced over centuries.

Hello, cultural appropriation!

Later this month, when everyone is excited about goblins and ghosts, thousands of sexy Indian costumes will be sold, prompting young Native Americans to cringe at the depictions of their culture and community. Part of the problem is that most young Americans think that Indians are dead or fictitious. Schools don’t help — children are taught to build teepees and wear headdresses as though this is a story of the past, not a living culture. And racist attitudes towards Native people are baked into every aspect of our culture. Why is it OK for Washington’s football team to be named the Redskins? Can you imagine a football team being named after the N-word?

Historically, Native people sit out Columbus Day in silence. This year, I hope you join me and thousands others by making a more active protest to Change what people learn!

In 2004, the Smithsonian’s National Museum of the American Indian was opened on the Mall in Washington DC as a cultural heritage institution to celebrate the stories of Native people and tell their story. I’m a proud trustee of this esteemed institution. I’m even more excited by upcoming projects that are focused on educating the public more holistically about the lives and experiences of Native peoples.

As a country, we’re struggling with racism and prejudice, hate that is woven deep into our cultural fabric. Injustice is at the core of our country’s creation, whether we’re talking about the original sin of slavery or the genocide of Native peoples. Addressing inequities in the present requires us to come to terms with our past. We need to educate ourselves about the limits of our understanding about our own country’s history. And we need to stop creating myths for our children that justify contemporary prejudice.

On this day, a day that we should not be celebrating, I have an ask for you. Please help me and NMAI build an educational effort that will change the next generation’s thinking about Native culture, past and present. Please donate a multiple of $14.91 to NMAI: http://nmai.si.edu/support/membership/ in honor of how much life existed on these lands before colonialist expansion. Help Indian nations achieve their rightful place of respect among the world’s nations and communities.

There was a bomb on my block.

I live in Manhattan, in Chelsea, on 27th Street between 6th and 7th, the same block in which the second IED was found. It was a surreal weekend, but it is increasingly becoming depressing as the media moves from providing information to stoking fear, the exact response that makes these events so effective. I’m not afraid of bombs. I’m afraid of cars. And I’m increasingly becoming afraid of American media.

After hearing the bomb go off on 23rd and getting flooded with texts on Saturday night, I decided to send a few notes that I was OK and turn off my phone. My partner is Israeli. We’ve been there for two wars and he’s been there through countless bombs. We both knew that getting riled up was of no help to anyone. So we went to sleep. I woke up on Sunday, opened my blinds, and was surprised to see an obscene number of men in black with identical body types, identical haircuts, and identical cars. It looked like the weirdest casting call I’ve ever seen. And no one else. No cars, no people. As always, Twitter had an explanation so we settled into our PJs and realized it was going to be a strange day.

Flickr / Sean MacEntree

As other people woke up, one thing became quickly apparent — because folks knew we were in the middle of it, they wanted to reach out to us because they were worried, and scared. We kept shrugging everything off, focusing on getting back to normal and reading the news for updates about how we could maneuver our neighborhood. But ever since a suspect was identified, the coverage has gone into hyperventilation mode. And I just want to scream in frustration.

The worst part about having statistical training is that it’s hard to hear people get anxious about fears without putting them into perspective. ~100 people die every day in car crashes in the United States. That’s 33,804 deaths in a year. Thousands of people are injured every day by cars. Cars terrify me.And anyone who says that you have control over a car accident is full of shit; most car deaths and injuries are not the harmed person’s fault.

The worst part about being a parent is having to cope with the uncontrollable, irrational, everyday fears that creep up, unwarranted, just to plague a moment of happiness. Will he choke on that food? What if he runs away and gets hit by a car? What if he topples over that chair? The best that I can do is breathe in, breathe out, and remind myself to find my center, washing away those fears with each breath.

And the worst part about being a social scientist is understanding where others’ fears come from, understanding the power of those fears, and understanding the cost of those fears on the well-being of a society. And this is where I get angry because this is where control and power lies.

Traditional news media has a lot of say in what it publishes. This is one ofthe major things that distinguishes it from social media, which propagates the fears and anxieties of the public. And yet, time and time again, news media shows itself to be irresponsible, motivated more by the attention and money that it can obtain by stoking people’s fears than by a moral responsibility to help ground an anxious public.

I grew up on the internet. I grew up with the mantra “don’t feed the trolls.” I always saw this as a healthy meditation for navigating the internet, for focusing on the parts of the internet that are empowering and delightful.Increasingly, I keep thinking that this is a meditation that needs to be injected into the news ecosystem. We all know that the whole concept of terrorism is to provoke fear in the public. So why are we not holding news media accountable for opportunistically aiding and abetting terroristic acts?Our cultural obsession with reading news that makes us afraid parallels our cultural obsession with crises.

There’s a reason that hate is growing in this country. And, in moments like this, I’m painfully reminded that we’re all contributing to the culture of hate.When we turn events like what happened this weekend in NY/NJ into spectacle, when we encourage media to write stories about how afraid people are, when we read the stories of how the suspect was an average person until something changed, we give the news media license to stoke up fear. And when they are encouraged to stoke fear, they help turn our election cycle into reality TV and enable candidates to spew hate for public entertainment. We need to stop blaming what’s happening on other people and start taking responsibility.

In short, we all need to stop feeding the trolls.

Be Careful What You Code For

Most people who don’t code don’t appreciate how hard it is to do right.Plenty of developers are perfectly functional, but to watch a master weave code into silken beauty is utterly inspiring. Unfortunately, most of the code that underpins the tools that we use on a daily basis isn’t so pretty. There isa lot of digital duct tape.

CC BY-NC 2.0-licensed photo by Dino Latoga.

I’m a terrible programmer. Don’t get me wrong — I’m perfectly capable of mashing together code to get a sorta-kinda-somewhat reasonable outcome.But the product is inevitably a Frankensteinesque monstrosity. I’m not alone. This is why I’m concerned about the code that is being built. Not all code is created equally.

If you want to understand what we’re facing, consider what this would mean if we were constructing cities. In the digital world, we are simultaneously building bridges, sewage systems, and skyscrapers. Some of the bridge builders have civil engineering degrees, some of our sewage contractors have been plumbers in past lives, but most of the people building skyscrapers have previously only built tree houses and taken a few math classes. Oh, and there aren’t any inspectors to assess whether or not it’s all going to fall apart.

Code is key to civic life, but we need to start looking under the hood and thinking about the externalities of our coding practices, especially as we’re building code as fast as possible with few checks and balances.

Area One: Environmental Consequences

Let’s play a game of math. Almost 1 billion people use Gmail. More than that are active on Facebook each month. Over 300 million are active on Twitter each month. All social media — including Facebook and Twitter — send out notifications to tell you that you have new friend requests, likes, updates, etc. Each one of those notifications is roughly 50KB. If you’re relatively active, you might get 1MB of notifications a day. That doesn’t seem to be that much. But if a quarter of Gmail users get that, this means that Google hosts over 90 petabytes of notifications per year. All of that is sitting live on server so that any user can search their email and find past emails, including the new followers they received in 2007. Is this really a good use of resources? Is this really what we want when we talk about keeping data around?

The tech industry uses crazy metaphors. Artificial intelligence. Files and folders. They often have really funny roots that make any good geek giggle. (UNIX geeks, did you know that the finger command is named as such because that word meant someone is a “snitch” in the 1970s? You probably had a dirtier idea in mind.

CC BY 2.0-licensed photo by Pattys-photos.

We don’t know who started calling the cloud the cloud, but he (and it’s inevitably a he) didus all a disservice. When the public hears about the cloud, they think about the fluffy white things in the sky. What were the skies like when you were young? They went on forever…And the skies always had little fluffy clouds.” Those clouds giveth. They offer rain, which gives us water, which is the source of life.

But what about the clouds we techies make? Those clouds take. They require rare earth metals and soak up land, power, and water. Many big companies are working hard to think about the environmental impact of data centers, to think about the carbon implications. (I’m proud to work forone of them.) Big companies still have a long way to go, but at least they’re trying. But how many developers out there are trying to write green code?At best, folks are thinking about the cost-per-computation, but most developers are pretty sloppy with code and data. And there’s no LEED-certified code. Who is going to start certifying LEED code!?

In the same sense, how many product designers are thinking about the environmental impact of every product design decision they make? Product folks are talking about how notifications might annoy or engage users but not the environmental impact of them. And for all those open data zealots, is the world really better off having petabytes of data sitting on live servers just to make sure it’s open and accessible just in case? It’s painful to think about how many terabytes of data are sitting in open data repositories that have never been accessed.

And don’t get me started about the blockchain or 3D printing or the Internet of Things. At least bitcoin got one thing right: this really is about mining.

Area Two: Social Consequences

In the early 2000s, Google thought that I was a truck driver. I got the bestadvertisements. I didn’t even know how many variations of trucker speed there were! All because I did fieldwork in parts of the country that only truckers visit. Consider how many people have received online advertisements that clearly got them wrong. Funny, huh?

Now…Have you ever been arrested? Have you ever been incarcerated?

Take a moment to think about the accuracy of our advertising ecosystem — the amount of money and data that goes into making ads right. Now think about what it means that the same techniques that advertisers are using to “predict” what you want to buy are also being used to predict the criminality of a neighborhood or a person. And those that work in law enforcement and criminal justice have less money, oversight mechanisms, and technical skills.

Inaccuracy and bias are often a given in advertising. But is it OK that we’re using extraordinarily biased data about previous arrests to predict future arrests and determine where police are stationed? Is it OK that we assess someone’s risk at the point of arrest and give judges recommendations for bail, probation, and sentencing? Is it OK that local law enforcement agencies are asking tech vendors to predict which children are going to commit a crime before they’re 21? Who is deciding, and who is holding them accountable?

We might have different political commitments when it comes to policing and criminal justice. But when it comes to tech and data analysis, I hope that we can all agree that accuracy matters. Yet, we’re turning a blind eye to all of the biases that are baked into the data and, thus, the models that we build.

CC BY-NC 2.0-licensed photo by Thomas Hawk.

Take a moment to consider that 96% of cases are plead out. Those defendants never see a jury of their peers. At a minimum, 10% — but most likely much more — of those who take a plea are innocent. Why? Last I saw, the average inmate at Riker’s waits ~600 days for their trial to begin. Average. And who is more likely to end up not making bail? Certainly not rich white folks.

Researchers have long known that whites are more likely to use and sell drugs. And yet, who is arrested for drugs? Blacks. 13% of the US population is black, but over 60% of those in prison are black. Mostly for drug crimes.

Because blacks are more likely to be arrested — and more likely to be prosecuted and serve time, guess what our algorithms tell us about who is most likely to commit a drug crime? About where drug crimes occur? Police aren’t sent by predictive policing tools to college campuses. They’re sent to the hood.

Engineers argue that judges and police officers should know the limits of the data they use. Some do — they’re simply ignoring these expensive, tax-payer-costing civic technologies. But in a world of public accountability, where police are punished for not knowing someone was a risk before they shoot up a church, many feel obliged to follow the recommendations for fear of reprisal. This is how racism gets built into the structures of our systems. And civic tech is implicated in this.

I don’t care what your politics are. If you’re building a data-driven system and you’re not actively seeking to combat prejudice, you’re building a discriminatory system.

Solution: Audits and Inspection

Decisions made involving tech can have serious ramifications that are outside of the mind’s eye of development. We need to wake up. Our technology is powerful, and we need to be aware of the consequences of our code.

Before our industry went all perpetual beta, we used to live in a world where Test or Quality Assurance meant something. Rooted in those domains is a practice that can be understood as an internal technical audit. We need to get back to this. We need to be able to answer simple questions like:

  • Does the system that we built produce the right output given the known constraints?
  • Do we understand the biases and limitations of the system and the output?
  • Are those clear to the user so that our tool cannot enable poor decision-making or inaccurate impressions?
  • What are the true social and environmental costs of the service?

We need to start making more meaningful trade-offs. And that requires asking hard questions.

Audits don’t have to be adversarial. They can be a way of honestly assessing the limitations of a system and benchmarking for improvement. This approach is not without problems and limitations, but, if you cannot understand whether a model is helping or hurting, discriminating or resulting in false positives, then you should not be implementing that technology in a high stakes area where freedom and liberty are at stake.Stick to advertising.

Technology can be amazingly empowering. But only when it is implemented in a responsible manner. Code doesn’t create magic. Without the right checks and balances, it can easily be misused. In the world of civic tech, we need to conscientiously think about the social and environmental costs, just as urban planners do.