AI is Not a Community Management Strategy
Let’s start from the beginning. Say you make a website that allows people to say something online. Most people use it like you intended, and everything’s fine.
If you’re exceptionally lucky, more people start to use your site. In fact, enough people use it that they begin to feel a kinship with each other.
Congratulations, you’ve got a community.
Because communities are made of people, and people are hopelessly, ridiculously complicated, eventually, inevitably, someone uses your site in a way you hadn’t expected, to say something terrible. So you remove that one thing.
Congratulations, you’re a community manager.
Again, you’re very lucky. The community grows. And grows. And soon there’s a lot of community management work to do. So you hire some people. And the first thing they ask is, but is this allowed? What about this other thing? And you realize that your community members have been asking those same questions.
So you write some things down. Rules. Community guidelines. Terms of service. And you think, I probably should have written these things down earlier. But it’s done now, so everything’s fine.
But then the community grows more and something truly frightening happens. Because you’ve been successful, because your community is so large, the community itself becomes a target. The bad actors arrive. The trolls, grifters, and criminals. They’re all coming because that’s where their victims, marks, and targets are. Because that’s where everybody is.
Congratulations, you’ve got a problem.
The community you started with optimism and naïveté is now a battleground. People get hurt feelings, but you can wave that away with idealistic platitudes about free speech. Then people get killed in SWATtings and broadcast their suicides, but you can just claim those are a few bad eggs.
And then you leak private data to Russians. And then you help elect Donald Trump. And then you get called in front of congress.
Congratulations, you’ve got a big fucking problem.
—
I’m old enough to remember when search engines were dumb. Alta Vista, HotBot, Lycos, Web Crawler. Names nobody remembers anymore. They were dumb because all they did was crawl the web and make indexes. Lists of which words appeared on which pages. So you’d type in a word and you’d get a list of pages that included that word. This was exciting at the time.
But it was dumb because knowing that a page included a word didn’t necessarily mean that page was about that word, or even necessarily good for that topic. And, worse, the people making the pages quickly learned that could just use a word a lot and get found that way. After all, a page that uses the word “dog” 400 times must really be about dogs, right?
The reason you “Google” things now and not “Alta Vista” them is because Google was the first company to really nail search results. And they did that with something they called PageRank. They made the usual index of which pages included which words, but then they made another list of which pages got linked to a lot, and used that linking behavior as a trust metric. So if a page got linked to a lot using the word “dog,” then it was a pretty good result for a search for “dog.”
This PageRank thing, they told us, was an “algorithm.” And, for a time, algorithms were all the rage. We were living in the age of the algorithm. And in all my client meetings and project plans, every time we had a decision to make, someone would say, “the algorithm will do it.”
The algorithm never did it.
—
When Facebook CEO Mark Zuckerberg testified in front of Congress on April 10, 2018, he was pressed repeatedly on what Facebook was doing to combat the rising tide of terribleness on his platform. And every time his answer was AI. He said it 25 times in one sitting.
So what’s AI? When you say “Artificial Intelligence” to a normal person, they probably think of a sentient robot. Star Trek’s Data. Star Wars’ C-3P0. It’s a romantic, futuristic notion. And totally wrong.
In the context of community moderation, all “AI” means is: teach a computer to do something so it can do it faster than you can. AI is the new algorithm – another way to avoid human responsibility.
If you want to see how meaningless the term “AI” is, just replace it with “recipe” when you see it.
-
SENATOR: How will you prevent abuse?
CEO: We will use a recipe.
SENATOR: A recipe? For what? What’s in the recipe? What does it do? Who’s making the recipe? How will it help?
CEO: That’s right! A RECIPE.
AI is just computers doing what they do. It’s not a solution to everything. And if we’re using it to avoid making hard decisions, then it’s part of the problem.
—
When technologists talk about AI, I think what they’re really talking about is machine learning, which is pretty cool and not nearly as new as people think it is. It’s been around since the 1960s. It just goes faster now because computers go faster.
Machine learning, at its simplest, it’s taking a pile of data, calling it a thing, and asking the computer to find more things like that. It’s pattern matching and computers are good at that.
Imagine you’re running an email system and you really need to help your users avoid spam. So you make a pile of spam messages and say, hey computer, this is spam. And the computer scans all that data and finds patterns in it. Then, when new messages come in, it can take a guess at how closely they match that pattern. This happens now, every second of every day, and every time you mark a message as spam, you’re adding to the pile and helping train the system.
What’s interesting about machine learning is that it requires you give the computer examples, but allows you to skip defining it. You can just let the computer find the similarities in the data. That works for something as simple as spam vs not-spam, or photos of faces vs photos of not-faces.
In his senate testimony, Zuckerberg claimed that internal “AI tools” at Facebook are already deployed against terrorist content. He said: “99 percent of the ISIS and Al Qaida content that we take down on Facebook, our AI systems flag before any human sees it.”
Even though this is unprovable (“trust us, we’re removing bad stuff before you see it 99% of the time!”), I don’t doubt it. Because terrorist content, like spam, is relatively easy to define, target, and remove. It’s identifiable because it includes telltale phrases, signifiers, and comes from known bad actors.
The trouble with taking this technique and applying it to general community management is that we are too messy, too inconsistent, too prone to human weirdness.
—
Anyone who’s ever managed a community knows how complicated people are. A reasonable community member can suddenly have a bad day. Sometimes things that look like bad contributions are honest mistakes. Other times things that look reasonable to a bystander are known to be abusive to the sender and recipient. (Nazis are using milk as a hate symbol. MILK.) When one person tells another to die on Twitter, it’s a threat. But when David Simon says it, he’s making a point. Abusers can use liking to remind their victims that they’re watching. And abuse isn’t limited to one system – just ask Kelly Marie Tran. Point is, we’re complicated critters.
Of course humans need tools to help manage community. I’ve built systems to do this. And, sure, machine learning can be part of that. But I fear the leaders of Twitter and Facebook are depending too much on technology (again), and overlooking the kinds of systems that are great at this kind of empathetic flexible pattern recognition: humans.
They’re also overlooking the reason they’re in this predicament in the first place: unfettered growth, design that encourages immediate engagement over thoughtfulness, and a general unwillingness to define and communicate who and what their platforms are for. Thus far, they’ve been for everyone and everything. It’s time to rethink that. While there’s a community for everyone, not everyone is welcome in every community, and that’s okay. That’s how communities work. And when the “everything” that your community is for includes destroying human lives and American democracy, it’s time to raise your standards.
You can’t create a system for everyone, where everything goes, not communicate the rules, not design for community, and then say it’s just too hard to protect everyone. This end state is the outcome of all of those decisions. And AI is not going to be the patch that fixes all the bugs.
—
AI is not a community management strategy because it’s skipping the hard part of community management: deciding what’s allowed and what’s not. You can’t skip the definition step in community management because that’s literally the very first thing you have to do, and the thing that only you can do. You can’t just give a pile of bad stuff to the computer and say “you figure it out.” That’s just outsourcing your responsibility.
You have to do the hard part. You have to decide what your platform is for and what it’s not for. And, yeah, that means deciding who it’s for and who it’s not for (hint: it’s not bots, nor nazis). That’s not a job you can outsource. The tech won’t do it for you. Not just because it’s your job, but because outsourcing it won’t work. It never does.
Call it “AI” or “machine learning” or “the algorithm” or whatever you like, but it’s really an abdication of your duty to care for the community that depends on you. And these days, that community is all of us, our fragile democracy, and possibly the stability of the world in general.