Earlier this week, the CDC paused the roll-out of the Johnson & Johnson COVID-19 vaccination after 6 women experienced serious blood clots. Their caution has merit, given that the FDA has been approving vaccinations in advance of the typical large-scale evaluations because speed is seen as so crucial. Reasonably, there is a desire to know more about these blood clots before more might appear. Yet, there was also sheer frustration from many in the medical community because the choice to pause the roll-out suggested that there was a serious issue, that the vaccine was dangerous. In a context in which vaccine hesitancy is likely to undermine herd immunity, any suggestion that the vaccine might have consequences can be twisted and contorted. 

Across many mailing lists and Twitter streams, I kept seeing data points trying to ground the seriousness of the blood clots in the J&J vaccine. Most referenced the frequency of blood clots that women experience while taking the birth control pill, roughly 1/1000. People also highlighted how common blood clots are for those who are in the throes of COVID-19. These were meant to highlight just how rare and statistically insignificant blood clots are when taking the J&J vaccine. 

Yet, as these attempts to ground the conversation unfolded, a different kind of outrage formed. A handful of people highlighted women they knew who had died of blood clots most likely related to birth control. Many more women who took hormonal birth control expressed frustration that they had no idea that they were at increased risk of a blood clot. Sure, it’s part of the fine print of that printout you get from CVS when picking up your pill, but this wasn’t something doctors emphasized. Unlike the J&J vaccine situation, the relationship between birth control and blood clots – or even COVID-19 and blood clots – hasn’t been front page news.

As I was processing the back-and-forth about statistical risk and who was responsible for sharing what with whom, and at what level of amplitude, I couldn’t help but think about all of the scholarship into the politics of numbers. We’re living at a time when politicians are simultaneously espousing the need for “evidence-based policymaking” and working to diligently undermine, contort, or weaponize evidence. This is what scholars of “agnotology” mean when they talk about the manufacturing of ignorance through the seeding of doubt. Or what other scholars highlight as the “weaponization of transparency.” 

I couldn’t help but feel empathy for the scientists at J&J and the FDA who have been working around the clock trying to make a vaccine available to the public, trying to be responsible stewards of information and statistical risk in a context where their desire for caution can be turned on its head to undermine the legitimacy of their work. I also found myself feeling empathy for journalists who recognize the importance of reporting on this development, even as they know that their reporting is easily evolving into misinformation that’s undermining the vaccine roll-out. Working with numbers is itself political.

To work in the world of medicine and science, statistics and probabilities is to grapple with trade-offs at a macro level, which present ethical conundrums even in the best of times. After all, that one terrible death from a blood clot could perhaps have been prevented by not taking the vaccine. But this is where we enter into the world of trade-offs, of unknowns, of morality. Without a vaccine rollout, many more people will die of blood clots from COVID-19. Had that woman been infected with COVID-19, she might have still succumbed to a blood clot. Medicine alters the dimensionality of risk. So how do ethics get negotiated? And by whom? This is the story of public health. 

Those complexities underpinning the advancement of science are complicated further by a politicized context such as that which surrounds the COVID-19 vaccine. Each act of communication can be twisted and contorted to convey different agendas, different values, different goals. Amplified transparency of risk is itself a political act. Sprinkle in the expectation in our current society that individuals are expected to make informed decisions for themselves, their families, and their communities, and we have a recipe for disaster. This is what the production of ignorance – aka misinformation, information disorder, agnotology, etc… – looks like in practice. The very acts of scientific transparency, which are intended to help inform decision-making, are twisted on their head, serving to undermining the legitimacy of scientific work and the coordination of a public that must work together to address a deadly disease. 

I keep wondering what it will take for the public to trust scientific information. But, perhaps, a better question might be: What kind of information is needed to help a fragmented public work together to solve societal-level challenges?

Note to the reader: These are questions that I’m struggling with. If you have thoughts, ideas (or even reading recommendations!), don’t hesitate to reach out: zephoria [at] zephoria [dot] org.

If you look at the roster of the Biden-Harris transition team, it’s quickly apparent that the incoming administration is tech-forward. Given the systematic dismantlement of the federal government over the last four years, and the significant logistical and scientific needs underpinning a large-scale vaccine roll-out, it is unsurprising to hear that the new team is looking to bring in tech talent. Under the Obama Administration, the White House invested significantly in shoring up the Office of Science and Technology Policy, an office that has for all intents and purposes laid dormant for four years under the current Administration. The Obama Administration also hired the first Chief Technology Officer (CTO) to help envision what a tech-forward US government might look like. As the Biden-Harris transition builds its plans for January 20th, many people in my networks are abuzz, wondering who might be the next CTO.

My advice to the transition team is this: You need a VP of Engineering even more than you need a CTO.

To the non-geeks of the world, these two titles might be meaningless or perhaps even interchangeable. The roles and responsibilities associated with each are often co-mingled, especially in start-ups. But in more mature tech companies, they signal distinct qualifications and responsibilities. Moreover, they signal different ideas for what is top priority. In their ideal incarnation, a CTO is a visionary, a thought leader, a big picture thinker. The right CTO sees how tech can fit into the big picture of a complex organization, sits in the C-suite to integrate tech into the strategy. A tech-forward White House would want such a person precisely to help envision a technocratic government structure that could do great things. Yet, a CTO is nothing more than a figurehead if the organizational infrastructure is dysfunctional. This can prompts organizations to want to build new tech separately inside an “office of the CTO” rather than doing the hard work of fixing the core organizational infrastructure to ensure that larger visions can work. When it comes to government, we’ve learned the hard way how easily a tech-forward effort located exclusively inside the White House can be swept away.

Inside tech companies, there is often a more important but less visible role when it comes to getting things done. To those on the outside, a VP title appears far less powerful, far less important than a C-Suite title. If you’re not a tech geek, a VP of Engineering might appear less important than a CTO. But in my experience, finding the right VP of Engineering is more essential than getting a high profile CTO when a system is broken. A VP-Eng is a fixer, someone who looks at broken infrastructure with a debugger’s eye and recognizes that the key to success is ensuring that the organizational and technical systems function hand-in-hand. While CTOs are often public figures in industry, a VP-Eng tends to shy away from public attention, focusing most of their effort on empowering their team to do great things. VP-Engs have technical chops, but their superpower comes from their ability to manage large technical teams, to really understand the forest and see what’s getting in the way of achieving a goal so that they can unblock that and ensure that their team thrives. A VP-Eng also understands that finding and nurturing the right talent is key to success, which is why they tend to spend an extraordinary amount of time recruiting, hiring, training, and mentoring.

When structured well, the CTO faces outwards while the VP-Eng faces inwards. They can and should be extraordinarily complementary roles. Yet, even though the Obama Administration invested in a CTO and built numerous programs to bring tech talent into the White House and sprinkle tech workers throughout all of the agencies, that tech-forward team never invested in a VP-Eng. They never invested in people whose job it was to truly debug the underlying problems that prevent government agencies from successfully building and deploying technical systems.

…

As I listen to friends and peers in Silicon Valley talk about all of the ways in which tech people are going to go east to “fix government,” I must admit that I’m cringing. Government functions very differently than industry, by design. In industry, our job is to serve customers. Yes, our companies might want more customers, but we have the luxury of focusing on those who have money and those who want to use our tools. Government must serve everyone. Much to the chagrin of capitalists, the vast majority of government resources goes to the hardest problems, to ensuring that whatever the government implements can serve everyone.

I have spent 20 years calling bullshit on “the pipeline problem” as industry’s excuse for its under-investment in hiring and retaining BIPOC and non-male talent. Even as tech workers are slowly starting to wake up to the realization that justice, equity, diversity, and inclusion are essential to the long-term health of tech, I’m watching the flawed logics that underpin the narrative about pipeline problems infuse the conversation about why government tech is broken. Government tech isn’t broken because government lacks talent. Government tech is broken because there are a range of stakeholders who are actively invested in ensuring that the federal government cannot execute, who are actively working to ensure that when the government is required to execute, it does so through upholding capitalist interests. Moreover, there are a range of stakeholders who would rather systematically undermine and hurt the extraordinarily diverse federal talent than invest in them.

If Silicon Valley waltzes into the federal government in January with its “I’ve got a submarine for that” mindset thinking that it can sprinkle tech fairy dust all over the agencies, we’re screwed. The undermining of the federal government’s tech infrastructure began decades ago. What has happened in the last four years has just sped up a trend that was well underway before this administration. And it’s getting worse by the day. The issue at play isn’t the lack of tech-forward vision. It’s the lack of organizational, human capital, and communications infrastructure that’s necessary for a complex “must-reach-everyone” organization to transform. Rather than coming in with hubris and focusing on grand vision, we need a new administration who is willing to dive deep and understand the cracks in the infrastructure that make a tech-forward agenda impossible. And this is why we need a federal VP-Eng whose job it is to engage in deep debugging. Cuz the bugs aren’t in the newest layer of code; they’re down deep in the libraries that no one has examined for years.

…

If the new administration is willing to invest in infrastructural repair, my ethnographic work in and around government has led me to three core areas that I would prioritize first. Two are esoteric structural barriers that prevent basic functioning. The third is a political weakness.

1. Procurement. Government outsourcing to industry is modern-day patronage. You don’t need Tammany Hall when you have a swarm of governmental contractors buzzing about. When politicians talk about about “small government,” what they really mean is “no federal employees.” Don’t let talk of “efficiency” fool you either. The cost of greasing the hands of Big Business through procurement procedures theoretically designed for efficiency is extraordinarily expensive. Not only is the financial cost of outsourcing to industry mind-boggling and bloated, but there are additional cost to morale, institutional memory, and mission that are not captured in the economic models. Government procurement infrastructure is also designed for failure, to ensure that government agencies are unable to deliver which, in turn, prompts Congress (regardless of who is in power) to reduce funding and increase scrutiny, tightening the screws on a tightly coupled system to increase the scale and speed of failure. It is a vicious cycle. Government procurement infrastructure is filled with strategically designed inefficiencies, frictions, and insanely corrupt incentives that undermine every aspect of government. They key here is not to replicate industry; the structures of contracting, outsourcing, and supply chains within a capitalist system do not make sense in government — and for good reason. A VP-Eng and a tech-forward government should begin by understanding the damage and ripple effects caused by OMB Directive A-76, which fundamentally shapes tech procurement.

2. Human Resources. Too many people in the tech industry think that HR is a waste of space…. that is, until they find that recruiter who makes everything easier. As such, in industry, we often talk about “people operations” or “talent management” instead of HR. We recognize the importance of investing in talent over the long-term, even if we reject HR. In government, HR is the lifeblood of how work happens in government and it was redesigned by progressives in the 20th century to ensure a more equitable approach to hiring and talent development. For decades, government created opportunities for women and Black communities when industry did not. Unfortunately, this aspect of the “deep state” was not at all appreciated by those invested in maintaining America’s caste system. Those invested in racist hierarchies didn’t need to be explicit about their agendas; they could rely on the language of capitalism to systematically undermine the talent of the federal government. Just as outsourcing in government has statistically taken jobs from Black federal workers and given them to white contractors, a range of HR policies have been designed to make working in government hellacious. Those who have stuck around — out of duty, out of necessity — have become enrolled in an existentially broken system. Some have chosen to sit back and not do their jobs, waiting to be fired. Others took the opposite approach, masochistically throwing themselves at the problem. People come from the outside and complain that government workers are lazy, stupid, incompetent. But it is the system that has produced these conditions. The system has been starved, the policies and protocols are corroded. It is through the purposeful torturing of HR that an executive branch hellbent on destroying federal government can wage the greatest damage; this has been underway for 40 years but the proverbial frogs are now sitting in boiling water. HR will require a lot of repair-work, not quick-fix policy changes. An untended HR system in government becomes a bottleneck unimaginable to those in industry and that’s where we are. Existing talent will require nurturing, and this investment is crucial because their institutional knowledge is profound. Any administration who wants to build a government that can respond to crises as grand as a pandemic or climate change will need to create the conditions for government to be a healthy workplace not just for the next four years but for decades to come. They will need a “people ops” mindset to HR. A VP-Eng should start with a listening tour of those who work on tech projects in agencies.

3. Communications. It never ceases to amaze me that the top communications professional in every federal agency is a political appointee. And every incoming administration — regardless of partisan affiliation — tends to fill these positions with campaign comms people who helped them win the election. Unfortunately, the type of comms that’s needed to win an election (which requires appealing to only some people) is not the same as the type of comms that’s needed to be accountable to the public as a whole for 365 days per year. Over and over, the comms people that White Houses install focus on speaking to their political base and to Congress. This is all fine and well if the only comms need is to negotiate policy outcomes. But the partisan perversion of comms within agencies has another outcome — it delegitimizes the agency among members of the public who are not affiliated with that political party, not to mention the wide majority of the public who is outright disgusted by all partisan tomfoolery. If your political interest is to eliminate the federal government, undermining the legitimacy of federal agencies benefits you. If that’s not your goal, you need to rethink your approach to communications. Right now, every agency needs a crisis comms expert at the helm to regain control over the agency’s narrative. When things are more stable, they need strategic comms professionals who can build a plan for re-legitimization. Each agency also needs an org comms expert whose job, like a VP-Eng, is to repair internal communications infrastructure so that information can effectively flow. Most politicians and government watchdogs think that the key to greater transparency is to increase oversight, just as progressives did after Nixon. But given how broken comms is in all of these agencies, turning up the heat through FOIA, GAO, and Congressional hearings will not increase accountability right now; it will increase breakage. Inside tech companies, comms is often seen as soft, squishy, irrational work, an afterthought that should not be prioritized. But comms, like HR, is the infrastructure that makes other things possible. A VP-Eng needs a comms counterpart working alongside them to achieve any organizational transformation.

…

Addressing these three seemingly non-tech issues would do more to enable a tech-forward government than any new-fangled shiny tech object. There is so much repair work to be done inside government. Yet, as I listen to those I know in Silicon Valley talk about all of the ways they wish to “fix government,” I fear that we will see a significant flood of solutionism when what’s needed most is humility and curiosity. Humility to understand that the structure of governmental agencies exists in response to the never-ending flow of solutionist interventions. And curiosity to understand how and why road blocks and barriers exist — and which ones to strategically eradicate to empower civil servants who are devoted to ensuring that government functions for the long-term, regardless of who is in power. Grand visioning has its role, but when infrastructure is breaking all around us, we need debuggers and maintenance people first and foremost. We need people who find joy in the invisible work of just making a system function, of recognizing that technical systems require the right organizational structures to thrive. This is the mindset a VP-Eng brings to the table.

In conclusion… If you are working on the transition or planning to jump into government in January, please spend some time understanding why the system is the way it is. If you are a tech person, do not presume you know based on your experience with other broken systems or based on what you read in the news; take the time to learn. If you are not a tech person, do not assume that tech can fix what politics can’t; this is a classic mistake with a long history. If the goal is truly to “build back better,” it requires starting with repairing the infrastructure. Without this, you are building on quicksand.

Screenagers in the time of coronavirus.

(This was originally written for OneZero.)

If you’re a parent trying to corral your children into attending “school” online, you’ve probably had the joy of witnessing a complete meltdown. Tantrums are no longer the domain of two-year-olds; 15-year-olds are also kicking and screaming. Needless to say, so are the fortysomethings. Children are begging to go outside. Teenagers desperately want to share physical space with their friends. And parents are begging their kids to go online so that they themselves can get some downtime. These are just some of the ways in which today’s reality seems upside down.

I started studying teenagers’ use of social media in the early 2000s when Xanga and LiveJournal were cool. I watched as they rode the waves of MySpace and Facebook, into the realms of Snap and Instagram. My book It’s Complicated: The Social Lives of Networked Teens unpacks some of the most prevalent anxieties adults have about children’s use of technology, including the nonstop fear-inducing message that children are “addicted” to their phones, computers, and the internet. Needless to say, I never imagined how conditions might change when a global pandemic unfolded.

I cannot remember a period in my research when parents weren’t wringing their hands about kids’ use of screens. The tone that parents took paralleled the tone their parents took over heavy metal and rock music, the same one their grandparents had when they spoke of the evils of comic books. Moral panics are consistent — but the medium that the panic centers on changes. Still, as with each wave of moral panic, there’s supposedly something intrinsic to the new medium that makes it especially horrible for young people. Cognizant of this history and having gone deep on social media activities with hundreds of teenagers, I pushed back and said that it wasn’t the technology teens were addicted to; it was their friends. Adults rolled their eyes at me, just as their teens rolled their eyes at them.

Now, nearly a month into screen-based schooling en masse, I’ve gotten to witness a global natural experiment like none I ever expected. What have we learned? The majority of young people are going batshit crazy living a life wholly online. I can’t help but think that Covid-19 will end up teaching all of us how important human interaction in physical space is. If this goes on long enough, might this cohort end up going further and hating screens?

Until the world started sheltering in place, most teens spent the majority of their days in school, playing sports, and participating in other activities, almost always in physical spaces with lots of humans co-present. True physical privacy is a luxury for most young people whose location in space is heavily monitored and controlled. Screens represented a break from the mass social. They also represented privacy from parents, an opportunity to socialize without parents lurking even when their physical bodies were forced to be at home. Parents hated the portals that kids held in their hands because their children seemed to disappear from the living room into some unknown void. That unknown void was those children’s happy place — the place where they could hang out with their friends, play games, and negotiate a life of their own.

Now, with Covid-19, schools are being taught through video. Friends are through video. Activities are through video. There are even videos for gym and physical sport. Religious gatherings are through video. Well-intended adults are volunteering to step in and provide more video-based opportunities for young people. TV may have killed the radio star, but Zoom and Google Hangouts are going to kill the delight and joy in spending all day in front of screens.

The majority of young people are going batshit crazy living a life wholly online.

Fatigue is setting in. Sure, making a TikTok video with friends is still fun, but there’s a limit to how much time anyone can spend on any app — even teens. Give it another month and there will be kids dropping out of school or throwing their computers against the wall. (Well, I know of two teens who have already done the latter with their iPads.) Young people are begging to go outside, even if that means playing sports with their parents. Such things might not be surprising for a seven-year-old, but when your 15-year-old asks to play soccer with you, do it! As a child of the ‘80s, I was stunned during my fieldwork to learn that most contemporary kids didn’t find ways to sneak out of the house once their parents were asleep because going online was so much easier. I can’t help but wonder if sneaking out is becoming a thing once again.

As we’re all stuck at home, teens are still doing everything possible to escape into their devices to maintain relationships, socialize, and have fun. Their shell-shocked parents are ignoring any and all screen time limitations as they too crave escapism (people who study fortysomethings: explain Animal Crossing to me!!?). But when physical distancing is no longer required, we’ll get to see that social closeness often involves meaningful co-presence with other humans. Adults took this for granted, but teens had few other options outside of spaces heavily controlled by adults. They went online not because the technology is especially alluring, but because it has long been the most viable option for having meaningful connections with friends given the way that their lives have been structured. Maybe now adults will start recognizing what my research showed: youth are “addicted” to sociality, not technology for technology’s sake.

On April 17, 2019, I gave a talk at the Digital Public Library of America conference (DPLAfest). This is the transcript of that talk.

I love the librarian community. You all are deeply committed to producing, curating, and enabling access to knowledge. Many of you embraced the internet with glee, recognizing the potential to help so many more people access critical information. Many of you also saw the democratic and civic potential of this new technology, not to mention the importance of an informed citizenry in a democratic world. Yet, slowly, and systematically, a virus has spread, using technology to systematically tear at the social fabric of public life.

This shouldn’t be surprising. After all, most of Silicon Valley in the late 90s and early aughts was obsessed with Neal Stephenson’s Snow Crash. How did they not recognize that this book was dystopian?

Slowly, and systematically, a virus has spread, using technology to systematically tear at the social fabric of public life.

Epistemology is the term that describes how we know what we know. Most people who think about knowledge think about the processes of obtaining it. Ignorance is often assumed to be not-yet-knowledgeable. But what if ignorance is strategically manufactured? What if the tools of knowledge production are perverted to enable ignorance? In 1995, Robert Proctor and Iain Boal coined the term “agnotology” to describe the strategic and purposeful production of ignorance. In an edited volume called Agnotology, Proctor and Londa Schiebinger collect essays detailing how agnotology is achieved. Whether we’re talking about the erasure of history or the undoing of scientific knowledge, agnotology is a tool of oppression by the powerful.

Swirling all around us are conversations about how social media platforms must get better at content management. Last week, Congress held hearings on the dynamics of white supremacy online and the perception that technology companies engage in anti-conservative bias. Many people who are steeped in history and committed to evidence-based decision-making are experiencing a collective sense of being gaslit—the concept that emerges from a film on domestic violence to explain how someone’s sense of reality can be intentionally destabilized by an abuser. How do you process a black conservative commentator testifying before the House that the Southern strategy never happened and that white nationalism is an invention of the Democrats to “scare black people”? Keep in mind that this commentator was intentionally trolled by the terrorist in Christchurch; she responded to this atrocity with tweets containing “LOL” and “HAHA.” Speaking of Christchurch, let’s talk about Christchurch. We all know the basic narrative. A terrorist espousing white nationalist messages livestreamed himself brutally murdering 50 people worshipping in a New Zealand mosque. The video was framed like a first-person shooter from a video game. Beyond the atrocity itself, what else was happening?

He produced a media spectacle. And he learned how to do it by exploiting the information ecosystem we’re currently in.

This terrorist understood the vulnerabilities of both social media and news media. The message he posted on 8chan announcing his intention included links to his manifesto and other sites, but it did not include a direct link to Facebook; he didn’t want Facebook to know that the traffic came from 8chan. The video included many minutes of him driving around, presumably to build audience but also, quite likely, in an effort to evade any content moderators that might be looking. He titled his manifesto with a well-known white nationalist call sign, knowing that the news media would cover the name of the manifesto, which in turn, would prompt people to search for that concept. And when they did, they’d find a treasure trove of anti-Semitic and white nationalist propaganda. This is the exploitation of what’s called a “data void.” He also trolled numerous people in his manifesto, knowing full well that the media would shine a spotlight on them and create distractions and retractions and more news cycles. He produced a media spectacle. And he learned how to do it by exploiting the information ecosystem we’re currently in. Afterwards, every social platform was inundated with millions and millions of copies and alterations of the video uploaded through a range of fake accounts, either to burn the resources of technology companies, shame them, or test their guardrails for future exploits.

What’s most notable about this terrorist is that he’s explicit in his white nationalist commitments. Most of those who are propagating white supremacist logics are not. Whether we’re talking about the so-called “alt-right” who simply ask questions like “Are jews people?” or the range of people who argue online for racial realism based on long-debunked fabricated science, there’s an increasing number of people who are propagating conspiracy theories or simply asking questions as a way of enabling and magnifying white supremacy. This is agnotology at work.

What’s at stake right now is not simply about hate speech vs. free speech or the role of state-sponsored bots in political activity. It’s much more basic. It’s about purposefully and intentionally seeding doubt to fragment society. To fragment epistemologies. This is a tactic that was well-honed by propagandists. Consider this Russia Today poster.

But what’s most profound is how it’s being done en masse now. Teenagers aren’t only radicalized by extreme sites on the web. It now starts with a simple YouTube query. Perhaps you’re a college student trying to learn a concept like “social justice” that you’ve heard in a classroom. The first result you encounter is from PragerU, a conservative organization that is committed to undoing so-called “leftist” ideas that are taught at universities. You watch the beautifully produced video, which promotes many of the tenets of media literacy. Ask hard questions. Follow the money. The video offers a biased and slightly conspiratorial take on what “social justice” is, suggesting that it’s not real, but instead a manufactured attempt to suppress you. After you watch this, you watch more videos of this kind from people who are professors and other apparent experts. This all makes you think differently about this term in your reading. You ask your professor a question raised by one of the YouTube influencers. She reacts in horror and silences you. The videos all told you to expect this. So now you want to learn more. You go deeper into a world of people who are actively anti-“social justice warriors.” You’re introduced to anti-feminism and racial realism. How far does the rabbit hole go?

One of the best ways to seed agnotology is to make sure that doubtful and conspiratorial content is easier to reach than scientific material.

YouTube is the primary search engine for people under 25. It’s where high school and college students go to do research. Digital Public Library of America works with many phenomenal partners who are all working to curate and make available their archives. Yet, how much of that work is available on YouTube? Most of DPLA’s partners want their content on their site. They want to be a destination site that people visit. Much of this is visual and textual, but are there explainers made about this content that are on YouTube? How many scientific articles have video explainers associated with them?

Herein lies the problem. One of the best ways to seed agnotology is to make sure that doubtful and conspiratorial content is easier to reach than scientific material. And then to make sure that what scientific information is available, is undermined. One tactic is to exploit “data voids.” These are areas within a search ecosystem where there’s no relevant data; those who want to manipulate media purposefully exploit these. Breaking news is one example of this. Another is to co-opt a term that was left behind, like social justice. But let me offer you another. Some terms are strategically created to achieve epistemological fragmentation. In the 1990s, Frank Luntz was the king of doing this with terms like partial-birth abortion, climate change, and death tax. Every week, he coordinated congressional staffers and told them to focus on the term of the week and push it through the news media. All to create a drumbeat.

Illustration by Jim Cooke Today’s drumbeat happens online. The goal is no longer just to go straight to the news media. It’s to first create a world of content and then to push the term through to the news media at the right time so that people search for that term and receive specific content. Terms like caravan, incel, crisis actor. By exploiting the data void, or the lack of viable information, media manipulators can help fragment knowledge and seed doubt.

Media manipulators are also very good at messing with structure. Yes, they optimize search engines, just like marketers. But they also look to create networks that are hard to undo. YouTube has great scientific videos about the value of vaccination, but countless anti-vaxxers have systematically trained YouTube to make sure that people who watch the Center for Disease Control and Prevention’s videos also watch videos asking questions about vaccinations or videos of parents who are talking emotionally about what they believe to be the result of vaccination. They comment on both of these videos, they watch them together, they link them together. This is the structural manipulation of media. Journalists often get caught up in telling “both sides,” but the creation of sides is a political project.

The creation of sides is a political project.

And this is where you come in. You all believe in knowledge. You believe in making sure the public is informed. You understand that knowledge emerges out of contestation, debate, scientific pursuit, and new knowledge replacing old knowledge. Scholars are obsessed with nuance. Producers of knowledge are often obsessed with credit and ownership. All of this is being exploited to undo knowledge today. You will not achieve an informed public simply by making sure that high quality content is publicly available and presuming that credibility is enough while you wait for people to come find it. You have to understand the networked nature of the information war we’re in, actively be there when people are looking, and blanket the information ecosystem with the information people need to make informed decisions.

Thank you!

(Joan Donovan and I wrote the following op-ed for The Guardian.) 

When confronted with white supremacists, newspaper editors should consider ‘strategic silence’

The below original text was the basis for Data & Society Founder and President danah boyd’s March 2018 SXSW Edu keynote,“What Hath We Wrought?” — Ed.

Growing up, I took certain truths to be self evident. Democracy is good. War is bad. And of course, all men are created equal.

My mother was a teacher who encouraged me to question everything. But I quickly learned that some questions were taboo. Is democracy inherently good? Is the military ethical? Does God exist?

I loved pushing people’s buttons with these philosophical questions, but they weren’t nearly as existentially destabilizing as the moments in my life in which my experiences didn’t line up with frames that were sacred cows in my community. Police were revered, so my boss didn’t believe me when I told him that cops were forcing me to give them free food, which is why there was food missing. Pastors were moral authorities and so our pastor’s infidelities were not to be discussed, at least not among us youth. Forgiveness is a beautiful thing, but hypocrisy is destabilizing. Nothing can radicalize someone more than feeling like you’re being lied to. Or when the world order you’ve adopted comes crumbling down.

The funny thing about education is that we ask our students to challenge their assumptions. And that process can be enlightening.

The funny thing about education is that we ask our students to challenge their assumptions. And that process can be enlightening. I will never forget being a teenager and reading “The People’s History of the United States.” The idea that there could be multiple histories, multiple truths blew my mind.Realizing that history is written by the winners shook me to my core. This is the power of education. But the hole that opens up, that invites people to look for new explanations…that hole can be filled in deeply problematic ways.When we ask students to challenge their sacred cows but don’t give them a new framework through which to make sense of the world, others are often there to do it for us.

For the last year, I’ve been struggling with media literacy. I have a deep level of respect for the primary goal. As Renee Hobbs has written, media literacy is the “active inquiry and critical thinking about the messages we receive and create.” The field talks about the development of competencies or skills to help people analyze, evaluate, and even create media. Media literacy is imagined to be empowering, enabling individuals to have agency and giving them the tools to help create a democratic society. But fundamentally, it is a form of critical thinking that asks people to doubt what they see. And that makes me nervous.

Most media literacy proponents tell me that media literacy doesn’t exist in schools. And it’s true that the ideal version that they’re aiming for definitely doesn’t. But I spent a decade in and out of all sorts of schools in the US, where I quickly learned that a perverted version of media literacy does already exist.Students are asked to distinguish between CNN and Fox. Or to identify bias in a news story. When tech is involved, it often comes in the form of “don’t trust Wikipedia; use Google.” We might collectively dismiss these practices as not-media-literacy, but these activities are often couched in those terms.

I’m painfully aware of this, in part because media literacy is regularly proposed as the “solution” to the so-called “fake news” problem. I hear this from funders and journalists, social media companies and elected officials. My colleagues Monica Bulger and Patrick Davison just released a report on media literacy in light of “fake news” given the gaps in current conversations. I don’t know what version of media literacy they’re imagining but I’m pretty certain it’s not the CNN vs Fox News version. Yet, when I drill in, they often argue for the need to combat propaganda, to get students to ask where the money is coming from, to ask who is writing the stories for what purposes, to know how to fact-check, etcetera. And when I push them further, I often hear decidedly liberal narratives. They talk about the Mercers or about InfoWars or about the Russians. They mock “alternative facts.” While I identify as a progressive, I am deeply concerned by how people understand these different conservative phenomena and what they see media literacy as solving.

I get that many progressive communities are panicked about conservative media, but we live in a polarized society and I worry about how people judge those they don’t understand or respect. It also seems to me that the narrow version of media literacy that I hear as the “solution” is supposed to magically solve our political divide. It won’t. More importantly, as I’m watching social media and news media get weaponized, I’m deeply concerned that the well-intended interventions I hear people propose will backfire, because I’m fairly certain that the crass versions of critical thinking already have.

My talk today is intended to interrogate some of the foundations upon which educating people about the media landscape depends. Rather than coming at this from the idealized perspective, I am trying to come at this from the perspective of where good intentions might go awry, especially in a moment in which narrow versions of media literacy and critical thinking are being proposed as the solution to major socio-cultural issues. I want to examine the instability of our current media ecosystem to then return to the question of:what kind of media literacy should we be working towards? So let’s dig in.

Epistemological Warfare

In 2017, sociologist Francesca Tripodi was trying to understand how conservative communities made sense of the seemingly contradictory words coming out of the mouth of the US President. Along her path, she encountered people talking about making sense of The Word when referencing his speeches. She began accompanying people in her study to their bible study groups. Then it clicked. Trained on critically interrogating biblical texts, evangelical conservative communities were not taking Trump’s messages as literal text. They were interpreting their meanings using the sameepistemological framework as they approached the bible. Metaphors and constructs matter more than the precision of words.

Why do we value precision in language? I sat down for breakfast with Gillian Tett, a Financial Times journalist and anthropologist. She told me that when she first moved to the States from the UK, she was confounded by our inability to talk about class. She was trying to make sense of what distinguished class in America. In her mind, it wasn’t race. Or education. It came down to what construction of language was respected and valued by whom. People became elite by mastering the language marked as elite. Academics, journalists, corporate executives, traditional politicians: they all master the art of communication. I did too. I will never forget being accused of speaking like an elite by my high school classmates when I returned home after a semester of college. More importantly, although it’s taboo in America to be explicitly condescending towards people on the basis of race or education, there’s no social cost among elites to mock someone for an inability to master language.For using terms like “shithole.”

Linguistic and communications skills are not universally valued. Those who do not define themselves through this skill loathe hearing the never-ending parade of rich and powerful people suggesting that they’re stupid, backwards, and otherwise lesser. Embracing being anti-PC has become a source of pride, a tactic of resistance. Anger boils over as people who reject “the establishment” are happy to watch the elites quiver over their institutions being dismantled. This is why this is a culture war. Everyone believes they are part of the resistance.

But what’s at the root of this cultural war? Cory Doctorow got me thinkingwhen he wrote the following:

We’re not living through a crisis about what is true, we’re living through a crisis about how we know whether something is true. We’re not disagreeing about facts,we’re disagreeing about epistemology. The “establishment” version of epistemology is, “We use evidence to arrive at the truth, vetted by independent verification (but trust us when we tell you that it’s all been independently verified by people who were properly skeptical and not the bosom buddies of the people they were supposed to be fact-checking).”

The “alternative facts” epistemological method goes like this: “The ‘independent’ experts who were supposed to be verifying the ‘evidence-based’ truth were actually in bed with the people they were supposed to be fact-checking. In the end, it’s all a matter of faith, then: you either have faith that ‘their’ experts are being truthful, or you have faith that we are. Ask your gut, what version feels more truthful?”

Let’s be honest — most of us educators are deeply committed to a way of knowing that is rooted in evidence, reason, and fact. But who gets to decide what constitutes a fact? In philosophy circles, social constructivists challenge basic tenets like fact, truth, reason, and evidence. Yet, it doesn’t take a doctorate of philosophy to challenge the dominant way of constructing knowledge. Heck, 75 years ago, evidence suggesting black people were biologically inferior was regularly used to justify discrimination. And this was called science!

In many Native communities, experience trumps Western science as the key to knowledge. These communities have a different way of understanding topics like weather or climate or medicine. Experience is also used in activist circles as a way of seeking truth and challenging the status quo. Experience-based epistemologies also rely on evidence, but not the kind of evidence that would be recognized or accepted by those in Western scientific communities.

Those whose worldview is rooted in religious faith, particularly Abrahamic religions, draw on different types of information to construct knowledge. Resolving scientific knowledge and faith-based knowledge has never been easy; this tension has countless political and social ramifications. As a result, American society has long danced around this yawning gulf and tried to find solutions that can appease everyone. But you can’t resolve fundamental epistemological differences through compromise.

No matter what worldview or way of knowing someone holds dear, they always believe that they are engaging in critical thinking when developing a sense of what is right and wrong, true and false, honest and deceptive. But much of what they conclude may be more rooted in their way of knowing than any specific source of information.

If we’re not careful, “media literacy” and “critical thinking”will simply be deployed as an assertion of authority over epistemology.

Right now, the conversation around fact-checking has already devolved to suggest that there’s only one truth. And we have to recognize that there are plenty of students who are taught that there’s only one legitimate way of knowing, one accepted worldview. This is particularly dicey at the collegiate level, where us professors have been taught nothing about how to teach across epistemologies.

Personally, it took me a long time to recognize the limits of my teachers. Like many Americans in less-than-ideal classrooms, I was taught that history was a set of facts to be memorized. When I questioned those facts, I was sent to the principal’s office for disruption. Frustrated and confused, I thought that I was being force-fed information for someone else’s agenda. Now I can recognize that that teacher was simply exhausted, underpaid, and waiting for retirement. But it took me a long time to realize that there was value in history and that history is a powerful tool.

Weaponizing Critical Thinking

The political scientist Deen Freelon was trying to make sense of the role of critical thinking to address “fake news.” He ended up looking back at a fascinating campaign by Russian Today (known as RT). Their motto for a while was “question more.” They produced a series of advertisements as teasers for their channel. These advertisements were promptly banned in the US and UK, resulting in RT putting up additional ads about how they were banned and getting tremendous mainstream media coverage about being banned. What was so controversial? Here’s an example:

“Just how reliable is the evidence that suggests human activity impacts on climate change? The answer isn’t always clear-cut. And it’s only possible to make a balanced judgement if you are better informed. By challenging the accepted view, we reveal a side of the news that you wouldn’t normally see. Because we believe that the more you question, the more you know.”

If you don’t start from a place where you’re confident that climate change is real, this sounds quite reasonable. Why wouldn’t you want more information? Why shouldn’t you be engaged in critical thinking? Isn’t this what you’re encouraged to do at school? So why is asking this so taboo? And lest you think that this is a moment to be condescending towards climate deniers, let me offer another one of their ads.

“Is terror only committed by terrorists? The answer isn’t always clear-cut. And it’s only possible to make a balanced judgement if you are better informed. By challenging the accepted view, we reveal a side of the news that you wouldn’t normally see. Because we believe that the more you question, the more you know.”

Many progressive activists ask whether or not the US government commits terrorism in other countries. The ads all came down because they were too political, but RT got what they wanted: an effective ad campaign. They didn’t come across as conservative or liberal, but rather a media entity that was “censored” for asking questions. Furthermore, by covering the fact that they were banned, major news media legitimized their frame under the rubric of “free speech.” Under the assumption that everyone should have the right to know and to decide for themselves.

We live in a world now where we equate free speech with the right to be amplified. Does everyone have the right to be amplified? Social media gave us that infrastructure under the false imagination that if we were all gathered in one place, we’d find common ground and eliminate conflict. We’ve seen this logic before. After World War II, the world thought that connecting the globe through financial interdependence would prevent World War III. It’s not clear that this logic will hold.

For better and worse, by connecting the world through social media and allowing anyone to be amplified, information can spread at record speed.There is no true curation or editorial control. The onus is on the public to interpret what they see. To self-investigate. Since we live in a neoliberal society that prioritizes individual agency, we double down on media literacy as the “solution” to misinformation. It’s up to each of us as individuals to decide for ourselves whether or not what we’re getting is true.

Yet, if you talk with someone who has posted clear, unquestionable misinformation, more often than not, they know it’s bullshit. Or they don’t care whether or not it’s true. Why do they post it then? Because they’re making a statement. The people who posted this meme (figure 1) didn’t bother to fact check this claim. They didn’t care. What they wanted to signal loud and clear is that they hated Hillary Clinton. And that message was indeed heard loud and clear. As a result, they are very offended if you tell them that they’ve been duped by Russians into spreading propaganda. They don’t believe you for one second.

Misinformation is contextual. Most people believe that people they know are gullible to false information, but that they themselves are equipped to separate the wheat from the chaff. There’s widespread sentiment that we can fact check and moderate our way out of this conundrum. This will fail. Don’t forget that for many people in this country, both education and the media are seen as the enemy — two institutions who are trying to have power over how people think. Two institutions that are trying to assert authority over epistemology.

Finding the Red Pill

Growing up on Usenet, Godwin’s Law was more than an adage to me. I spent countless nights lured into conversation by the idea that someone was wrong on the internet. And I long ago lost count about how many of them ended up with someone invoking Hitler or the Holocaust. I might have even been to blame in some of these conversations.

Fast forward 15 years to the point when Nathan Poe wrote a poignant comment on an online forum dedicated to Christianity: “Without a winking smiley or other blatant display of humor, it is utterly impossible to parody a Creationist in such a way that someone won’t mistake for the genuine article.”Poe’s Law, as it became known, signals that it’s hard to tell the difference between an extreme view and a parody of an extreme view on the internet.

In their book, “The Ambivalent Internet,”media studies scholars Whitney Phillips and Ryan Milner highlight how a segment of society has become so well-versed at digital communications — memes, GIFs, videos, etc. — that they can use these tools of expression to fundamentally destabilize others’communication structures and worldviews. It’s hard to tell what’s real and what’s fiction, what’s cruel and what’s a joke. But that’s the point. That is howirony and ambiguity can be weaponized. And for some, the goal is simple:dismantle the very foundations of elite epistemological structures that are so deeply rooted in fact and evidence.

Many people, especially young people, turn to online communities to make sense of the world around them. They want to ask uncomfortable questions, interrogate assumptions, and poke holes at things they’ve heard. Welcome to youth. There are some questions that are unacceptable to ask in public and they’ve learned that. But in many online fora, no question or intellectual exploration is seen as unacceptable. To restrict the freedom of thought is to censor. And so all sorts of communities have popped up for people to explore questions of race and gender and other topics in the most extreme ways possible. And these communities have become slippery. Are those taking on such hateful views real? Or are they being ironic?

In the 1999 film The Matrix, Morpheus says to Neo: “You take the blue pill,the story ends. You wake up in your bed and believe whatever you want. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.” Most youth aren’t interested in having the wool pulled over their head, even if blind faith might be a very calming way of living. Restricted in mobility and stressed to holy hell, they want to have access to what’s inaccessible, know what’s taboo, and say what’s politically incorrect. So who wouldn’t want to take the red pill?

In some online communities, taking the red pill refers to the idea of waking up to how education and media are designed to deceive you into progressive propaganda. In these environments, visitors are asked to question more. They’re invited to rid themselves of their politically correct shackles. There’s an entire online university designed to undo accepted ideas about diversity, climate, and history. Some communities are even more extreme in their agenda. These are all meant to fill in the gaps for those who are opening to questioning what they’ve been taught.

In 2012, it was hard not to avoid the names Trayvon Martin and George Zimmerman, but that didn’t mean that most people understood the storyline.In South Carolina, a white teenager who wasn’t interested in the news felt like he needed to know what the fuss was all about. He decided to go to Wikipedia to understand more. He was left with the impression that Zimmerman was clearly in the right and disgusted that everyone was defending Martin. While reading up on this case, he ran across the term “black on white crime” on Wikipedia and decided to throw that term into Google where he encountered a deeply racist website inviting him to wake up to a reality that he had never considered. He took that red pill and dove deep into a worldview whose theory of power positioned white people as victims. Over a matter of years, he began to embrace those views, to be radicalized towards extreme thinking. On June 17, 2015, he sat down for an hour with a group of African-American church-goers in Charleston South Carolina before opening fire on them, killing 9 and injuring 1. His goal was simple: he wanted to start a race war.

It’s easy to say that this domestic terrorist was insane or irrational, but he began his exploration trying to critically interrogate the media coverage of a story he didn’t understand. That led him to online fora filled with people who have spent decades working to indoctrinate people into a deeply troubling, racist worldview. They draw on countless amounts of “evidence,” engage in deeply persuasive discursive practices, and have the mechanisms to challenge countless assumptions. The difference between what is deemed missionary work, education, and radicalization depends a lot on your worldview. And your understanding of power.

Who Do You Trust?

The majority of Americans do not trust the news media. There are many explanations for this — loss of local news, financial incentives, hard to distinguish between opinion and reporting, etc. But what does it mean to encourage people to be critical of the media’s narratives when they are already predisposed against the news media?

Perhaps you want to encourage people to think critically about how information is constructed, who is paying for it, and what is being left out. Yet, among those whose prior is to not trust a news media institution, among those who see CNN and The New York Times as “fake news,” they’re already there. They’re looking for flaws. It’s not hard to find them. After all, the news industry is made of people in institutions in a society. So when youth are encouraged to be critical of the news media, they come away thinking that the media is lying. Depending on someone’s prior, they may even take what they learn to be proof that the media is in on the conspiracy. That’s where things get very dicey.

Many of my digital media and learning colleagues encourage people to make media to help understand how information is produced. Realistically, many young people have learned these skills outside the classroom as they seek to represent themselves on Instagram, get their friends excited about a meme, or gain followers on YouTube. Many are quite skilled at using media, but to what end? Every day, I watch teenagers produce anti-Semitic and misogynistic content using the same tools that activists use to combat prejudice. It’s notable that many of those who are espousing extreme viewpoints are extraordinarily skilled at using media. Today’s neo-Nazis are a digital propaganda machine. Developing media making skills doesn’t guarantee that someone will use them for good. This is the hard part.

Most of my peers think that if more people are skilled and more people are asking hard questions, goodness will see the light. In talking about misunderstandings of the First Amendment, Nabiha Syed of Buzzfeedhighlights that the frame of the “marketplace of ideas” sounds great, but is extremely naive. Doubling down on investing in individuals as a solution to a systemic abuse of power is very American. But the best ideas don’t always surface to the top. Nervously, many of us tracking manipulation of media are starting to think that adversarial messages are far more likely to surface than well-intended ones.

This is not to say that we shouldn’t try to educate people. Or that producing critical thinkers is inherently a bad thing. I don’t want a world full of sheeple.But I also don’t want to naively assume what media literacy could do in responding to a culture war that is already underway. I want us to grapple with reality, not just the ideals that we imagine we could maybe one day build.

It’s one thing to talk about interrogating assumptions when a person can keep emotional distance from the object of study. It’s an entirely different thing to talk about these issues when the very act of asking questions is what’s being weaponized. This isn’t historical propaganda distributed through mass media. Or an exercise in understanding state power. This is about making sense of an information landscape where the very tools that people use to make sense of the world around them have been strategically perverted by other people who believe themselves to be resisting the same powerful actors that we normally seek to critique.

Take a look at the graph above. Can you guess what search term this is? This is the search query for “crisis actors.” This concept emerged as a conspiracy theory after Sandy Hook. Online communities worked hard to get this to land with the major news media after each shooting. With Parkland, they finally succeeded. Every major news outlet is now talking about crisis actors, as though it’s a real thing, or something to be debunked. When teenage witnesses of the mass shooting in Parkland speak to journalists these days, they have to now say that they are not crisis actors. They must negate a conspiracy theory that was created to dismiss them. A conspiracy theory that undermines their message from the get-go. And because of this, many people have turned to Google and Bing to ask what a crisis actor is. They quickly get to the Snopes page. Snopes provides a clear explanation of why this is a conspiracy. But you are now asked to not think of an elephant.

You may just dismiss this as craziness, but getting this narrative into the media was designed to help radicalize more people. Some number of people will keep researching, trying to understand what the fuss is all about. They’ll find online fora discussing the images of a brunette woman and ask themselves if it might be the same person. They will try to understand the fight between David Hogg and Infowars or question why Infowars is being restricted by YouTube. They may think this is censorship. Seeds of doubt will start to form. And they’ll ask whether or not any of the articulate people they see on TV might actually be crisis actors. That’s the power of weaponized narratives.

One of the main goals for those who are trying to manipulate media is to pervert the public’s thinking. It’s called gaslighting. Do you trust what is real?One of the best ways to gaslight the public is to troll the media. By getting the news media to be forced into negating frames, they can rely on the fact that people who distrust the media often respond by self-investigating. This is the power of the boomerang effect. And it has a history. After all, the CDC realized that the more news media negated the connection between autism and vaccination, the more the public believed there was something real there.

In 2016, I watched networks of online participants test this theory through an incident now known as Pizzagate. They worked hard to get the news media to negate the conspiracy theory, believing that this would prompt more people to try to research if there was something real there. They were effective. The news media covered the story to negate it. Lots of people decided to self-investigate. One guy even showed up with a gun.

The term “gaslighting” originates in the context of domestic violence. The term refers back to an 1944 movie called Gas Light where a woman is manipulated by her husband in a way that leaves her thinking she’s crazy. It’sa very effective technique of control. It makes someone submissive and disoriented, unable to respond to a relationship productively. While many anti-domestic violence activists argue that the first step is to understand that gaslighting exists, the “solution” is not to fight back against the person doing the gaslighting. Instead, it’s to get out. Furthermore, anti-domestic violence experts argue that recovery from gaslighting is a long and arduous process, requiring therapy. They recognize that once instilled, self-doubt is hard to overcome.

While we have many problems in our media landscape, the most dangerous is how it is being weaponized to gaslight people.

And unlike the domestic violence context, there is no “getting out” that is really possible in a media ecosystem. Sure, we can talk about going off the grid and opting out of social media and news media, but c’mon now.

The Cost of Triggering

In 2017, Netflix released a show called 13 Reasons Why. Before parents and educators had even heard of the darn show, millions of teenagers had watched it. For most viewers, it was a fascinating show. The storyline was enticing, the acting was phenomenal. But I’m on the board of Crisis Text Line, an amazing service where people around this country talk with trained counselors via text message when they’re in a crisis. Before the news media even began talking about the show, we started to see the impact. After all, the premise of the show is that a teen girl died by suicide and left behind 13 tapes explaining how people had bullied her to justify her decision.

At Crisis Text Line, we do active rescues every night. This means that we send emergency personnel to the homes of someone who is in the middle of a suicide attempt in an effort to save their lives. Sometimes, we succeed. Sometimes, we don’t. It’s heartbreaking work. As word of 13 Reasons Why got out and people started watching the show, our numbers went through the roof. We were drowning in young people referencing the show, signaling how it had given them a framework for ending their lives. We panicked. All hands on deck. As we got things under control, I got angry. What the hell was Netflix thinking?

Researchers know the data on suicide and media. The more the media normalizes suicide, the more suicide is put into people’s head as a possibility,the more people who are on the edge start to take it seriously and consider it for themselves. After early media effects research was published, journalists developed best practices to minimize their coverage of suicide. As Joan Donovan often discusses, this form of “strategic silence” was viable in earlier media landscapes; it’s a lot harder now. Today, journalists and media makers feel as though the fact that anyone could talk about suicide on the internet means that they should have a right to do so too.

We know that you can’t combat depression through rational discourse.Addressing depression is hard work. And I’m deeply concerned that we don’t have the foggiest clue how to approach the media landscape today. I’m confident that giving grounded people tools to think smarter can be effective.But I’m not convinced that we know how to educate people who do not share our epistemological frame. I’m not convinced that we know how to undo gaslighting. I’m not convinced that we understand how engaging people about the media intersects with those struggling with mental health issues.And I’m not convinced that we’ve even begun to think about the unintended consequences of our good — let alone naive — intentions.

In other words, I think that there are a lot of assumptions baked into how we approach educating people about sensitive issues and our current media crisis has made those painfully visible.

Oh, and by the way, the Netflix TV show ends by setting up Season 2 to start with a school shooting. WTF, Netflix?

Pulling Back Out

So what role do educators play in grappling with the contemporary media landscape? What kind of media literacy makes sense? To be honest, I don’t know. But it’s unfair to end a talk like this without offering some path forward so I’m going to make an educated guess.

I believe that we need to develop antibodies to help people not be deceived.

That’s really tricky because most people like to follow their gut more than than their mind. No one wants to hear that they’re being tricked. Still, I thinkthere might be some value in helping people understand their own psychology.

Consider the power of nightly news and talk radio personalities. If you bring Sean Hannity, Rachel Maddow, or any other host into your home every night,you start to appreciate how they think. You may not agree with them, but youbuild a cognitive model of their words such that they have a coherent logic to them. They become real to you, even if they don’t know who you are. This is what scholars call “parasocial interaction.” And the funny thing about humanpsychology is that we trust people who we invest our energies into understanding. That’s why bridging difference requires humanizing people across viewpoints.

Empathy is a powerful emotion, one that most educators want to encourage.But when you start to empathize with worldviews that are toxic, it’s very hard to stay grounded. It requires deep cognitive strength. Scholars who spend a lot of time trying to understand dangerous worldviews work hard to keep their emotional distance. One very basic tactic is to separate the different signals. Just read the text rather than consume the multimedia presentation of that. Narrow the scope. Actively taking things out of context can be helpful for analysis precisely because it creates a cognitive disconnect. This is the opposite of how most people encourage everyday analysis of media, where the goal is to appreciate the context first. Of course, the trick here is wanting to keep that emotional distance. Most people aren’t looking for that.

I also believe that it’s important to help students truly appreciate epistemological differences. In other words, why do people from different worldviews interpret the same piece of content differently? Rather than thinking about the intention behind the production, let’s analyze the contradictions in the interpretation. This requires developing a strong sense of how others think and where the differences in perspective lie. From an educational point of view, this means building the capacity to truly hear and embrace someone else’s perspective and teaching people to understand another’s view while also holding their view firm. It’s hard work, an extension of empathy into a practice that is common among ethnographers. It’s also a skill that is honed in many debate clubs. The goal is to understand the multiple ways of making sense of the world and use that to interpret media.Of course, appreciating the view of someone who is deeply toxic isn’t always psychologically stabilizing.

Another thing I recommend is to help students see how they fill in gaps when the information presented to them is sparse and how hard it is to overcome priors. Conversations about confirmation bias are important here because it’s important to understand what information we accept and what information we reject. Selective attention is another tool, most famously shown to students through the “gorilla experiment.” If you aren’t familiar with this experiment, it involves showing a basketball video and focusing on counting passes made by people in one color shirt and then asking if they saw the gorilla. Many people do not. Inverting these cognitive science exercises,asking students to consider different fan fiction that fills in the gaps of a story with divergent explanations is another way to train someone to recognize how their brain fills in gaps.

What’s common about the different approaches I’m suggesting is that they are designed to be cognitive strengthening exercises, to help students recognize their own fault lines, not the fault lines of the media landscape around them. I can imagine that this too could be called media literacy and if you want to bend your definition that way, I’ll accept it. But the key is to realize the humanity in ourselves and in others. We cannot and should not assert authority over epistemology, but we can encourage our students to be more aware of how interpretation is socially constructed. And to understand how that can be manipulated. Of course, just because you know you’re being manipulated doesn’t mean that you can resist it. And that’s where my proposal starts to get shaky.

Let’s be honest — our information landscape is going to get more and more complex. Educators have a critical role to play in helping individuals and societies navigate what we encounter. But the path forward isn’t about doubling down on what constitutes a fact or teaching people to assess sources.Rebuilding trust in institutions and information intermediaries is important, but we can’t assume the answer is teaching students to rely on those signals.The first wave of media literacy was responding to propaganda in a mass media context. We live in a world of networks now. We need to understand how those networks are intertwined and how information that spreads through dyadic — even if asymmetric — encounters is understood and experienced differently than that which is produced and disseminated through mass media.

Above all, we need to recognize that information can, is, and will be weaponized in new ways. Today’s propagandist messages are no longer simply created by Madison Avenue or Edward Bernays-style State campaigns. For the last 15 years, a cohort of young people has learned how to hack the attention economy in an effort to have power and status in this new information ecosystem. These aren’t just any youth. They are young people who are disenfranchised, who feel as though the information they’re getting isn’t fulfilling, who struggle to feel powerful. They are trying to make sense of an unstable world and trying to respond to it in a way that is personally fulfilling.Most youth are engaged in invigorating activities. Others are doing the same things youth have always done. But there are youth out there who feel alienated and disenfranchised, who distrust the system and want to see it all come down. Sometimes, this frustration leads to productive ends. Often it does not. But until we start understanding their response to our media society, we will not be able to produce responsible interventions. So I would argue that we need to start developing a networked response to this networked landscape. And it starts by understanding different ways of constructing knowledge.

---

Special thanks to Monica Bulger, Mimi Ito, Whitney Phillips, Cathy Davidson, Sam Hinds Garcia, Frank Shaw, and Alondra Nelson for feedback.

---

Update (March 16, 2018): I crafted some responses to the most common criticisms I’ve received to date about this work here. (Also, the original version of this blog post was published on Medium.)

(This was originally posted on NewCo Shift.)

A friend of mine worked for an online dating company whose audience was predominantly hetero 30-somethings. At some point, they realized that a large number of the “female” accounts were actually bait for porn sites and 1–900 numbers. I don’t remember if users complained or if they found it themselves, but they concluded that they needed to get rid of these fake profiles. So they did.

And then their numbers started dropping. And dropping. And dropping.

Trying to understand why, researchers were sent in. What they learned was that hot men were attracted to the site because there were women that they felt were out of their league. Most of these hot men didn’t really aim for these ultra-hot women, because they felt like they would be inaccessible, but they were happy to talk with women who they saw as being one rung down (as in actual hot women). These hot women, meanwhile, were excited to have these hot men (who they saw as equals) on the site. These also felt that, since there were women hotter than them, that this was a site for them. When they removed the fakes, the hot men felt the site was no longer for them. They disappeared. And then so did the hot women. Etc. The weirdest part? They reintroduced decoy profiles (not as redirects to porn but as fake women who just didn’t respond) and slowly folks came back.

Why am I telling you this story? Fake accounts and bots on social media are not new. Yet, in the last couple of weeks, there’s been newfound hysteria around Twitter bots and fake accounts. I find it deeply problematic that folks are saying that having fake followers is inauthentic. This is like saying that makeup is inauthentic. What is really going on here?

From Fakesters to Influencers

From the earliest days of Friendster and MySpace, people liked to show how cool they were by how many friends they had. As Alice Marwick eloquentlydocumented, self-branding and performing status were the name of the gamefor many in the early days of social media. This hasn’t changed. People made entire careers out of appearing to be influential, not just actually being influential. Of course a market emerged around this so that people could buy and sell followers, friends, likes, comments, etc. Indeed, standard practice, especially in the wink-nudge world of Instagram, where monetized content is the game and so-called organic “macroinfluencers” can easily double their follower size through bots are more than happily followed by bots, paid or not.

Some sites have tried to get rid of fake accounts. Indeed, Friendster played whack-a-mole with them, killing off “Fakesters” and any account that didn’t follow their strict requirements; this prompted a mass exodus. Facebook’s real-name policy also signaled that such shenanigans would not be allowed on their site, although shhh…. lots of folks figured out how to have multiple accounts and otherwise circumvent the policy.

And let’s be honest — fake accounts are all over most online dating profiles. Ashley Madison, anyone?

Bots, Bots, Bots

Bots have been an intrinsic part of Twitter since the early days. Following the Pope’s daily text messaging services, the Vatican set up numerous bots offering Catholics regular reflections. Most major news organizations have bots so that you can keep up with the headlines of their publications. Twitter’s almost-anything-goes policy meant that people have built bots for all sorts of purposes. There are bots that do poetry, ones that argue with anti-vaxxers about their beliefs, and ones that call out sexist comments people post. I’m a big fan of the @censusAmericans bot created by FiveThirtyEight to regularly send out data from the Census about Americans.

Over the last year, sentiment towards Twitter’s bots has become decidedly negative. Perhaps most people didn’t even realize that there were bots on the site. They probably don’t think of @NYTimes as a bot. When news coverage obsesses over bots, they primarily associate the phenomenon with nefarious activities meant to seed discord, create chaos, and do harm. It can all be boiled down to: Russian bots. As a result, Congress saw bots as inherently bad and journalists keep accusing Twitter of having a “bot problem” without accounting for how their stories appear on Twitter through bots.

Although we often hear about the millions and millions of bots on Twitter as though they’re all manipulative, the stark reality is that bots can be quite fun. I had my students build Twitter bots to teach them how these things worked — they had a field day, even if they didn’t get many followers.

Of course, there are definitely bots that you can buy to puff up your status. Some of them might even be Russian built. And here’s where we get to the crux of the current conversation.

Buying Status

People buy bots to increase their number of followers, retweets, and likes in order to appear cooler than they are. Think of this as mascara for your digital presence. While plenty of users are happy chatting away with their friends without their makeup on, there’s an entire class of professionals who feel the need to be dolled up and giving the best impression possible. It’s a competition for popularity and status, marked by numbers.

Number games are not new, especially not in the world of media. Take a well-established firm like Nielsen. Although journalists often uncritically quote Nielsen numbers as though they are “fact,” most people in the ad and media business know that they’re crap. But they’ve long been the best crap out there. And, more importantly, they’re uniform crap so businesses can make predictable decisions off of these numbers, fully aware that they might not be that accurate. The same has long been true of page views and clicks. No major news organization should take their page views literally. And yet, lots of news agencies rank their reporters based on this data.

What makes the purchasing of Twitter bots and status so nefarious? The NYTimes story suggests that doing so is especially deceptive. Their coverage shamed Twitter into deleting a bunch of Twitter accounts, outing all of the public figures who had bought bots. It almost felt like a discussion of who had gotten Botox.

Much of this recent flurry of coverage suggests that the so-called bot problem is a new thing that is “finally” known. It boggles my mind to think that any regular Twitter user hadn’t seen automated accounts in the past. And heck, there have been services like Twitter Audit to see how many fake followers you have since at least 2012. Gilad Lotan even detailed the ecosystem of buying fake followers in 2014. I think that what’s new is that the term “bot” is suddenly toxic. And it gives us an opportunity to engage in another round of social shaming targeted at insecure people’s vanity all under the false pretense of being about bad foreign actors.

I’ve never been one to feel the need to put on a lot of makeup in order to leave the house and I haven’t been someone who felt the need to buy bots to appear cool online. But I find it deeply hypocritical to listen to journalists and politicians wring their hands about fake followers and bots given that they’ve been playing at that game for a long time. Who among them is really innocent of trying to garner attention through any means possible?

At the end of the day, I don’t really blame Twitter for giving these deeply engaged users what they want and turning a blind eye towards their efforts to puff up their status online. After all, the cosmetic industry is $55 billion. Then again, even cosmetic companies sometimes change their formulas when their products receive bad press.

Note: I’m fully aware of hypotheses that bots have destroyed American democracy. That’s a different essay. But I think that the main impact that they have had, like spam, is to destabilize people’s trust in the media ecosystem. Still, we need to contend with the stark reality that they do serve a purpose and some people do want them.