Guiding AI Safety


By Cody Fenwick

Photo courtesy of Global Panorama.

Elon Musk, CEO of SpaceX, Tesla Motors, and creator of PayPal, recently drew media attention for pledging $10 million to the Future of Life Institute (FLI) for research into the dangers of artificial intelligence (AI). Widely recognized as an important figure at the forefront of technological progress, Musk’s concerns about AI echo those raised by respected intellectuals such as Stephen Hawking and Nick Bostrom.

Some accuse Musk of acting out of self-interest, using the donation to shield criticism of his own work in AI technologies. Others respond to the warnings about AI with humor, eye-rolls, or mockery.

But for many experts and academics, Musk’s action addresses an issue that is far from trivial. FLI, a group promoting research into the mitigation of risks from AI, issued an open letter detailing their concerns and priorities. Thousands signed the letter, many of whom are leaders in the field.

BTR spoke with Viktoriya Krakovna, one of the co-founders of FLI. She indicated that public perception of the subject of AI safety is a barrier to robust research in this area.

“Particularly, because of the angle the media often takes on this, where they frame any concern about the safety of AI as parallel to [concerns about] The Terminator or Skynet, and all those science fiction references,” she says, “it makes it hard for people to take it seriously.”

As long as the public fails to take AI safety seriously, it will be more difficult to perform the research necessary to mitigate the risks. But if a cartoonish, “robot uprising” that many envision when they hear this kind of talk is unlikely, as Krakovna says, what are the dangers?

“We might have an AI system that is designed to have some kind of beneficial or maybe neutral goals, but it would be specified in some ways that make it dangerous. Even with today’s narrow AI algorithms, there are plenty of examples of the systems doing something [they were] programmed to do, but that we don’t actually want,” Krakovna warns.

Since a sufficiently advanced AI might be superior to human actors across any number of dimensions, its pursuit of a poorly chosen goal could be unstoppable.

People mistakenly believe the AI threat would look like a misanthropic robot revolution violently disobeying orders. As Krakovna explains, “The media keeps focusing on the image of this malevolent AI that has the intention to harm humans. But it doesn’t have to be malevolent, it just has to be mis-specified.”

One of the biggest dangers is that an AI will follow our commands, but that the result will be something we never could have predicted, and never would have desired.

Bostrom provides several examples of this potentiality, which he terms “perverse instantiations,” in his book Superintelligence. We might, for instance, program a general AI simply to “make us smile.” Finding us difficult to amuse, the AI may find it most efficient to simply paralyze our facial muscles in permanent grins. Alternatively, a paperclip manufacturer might ask an AI to maximize paperclip production. It could proceed by turning all the matter on Earth, including human beings, into paper clips.

The problem lies in the idea that once AI becomes smarter and more capable than humans, there is, in principle, no way to predict for sure what it will do. Whatever we ask it to do, it will do better and more efficiently than we ever could have done ourselves. If we later realize that we don’t want what we asked for, we can’t change our minds. The AI will have already been pursuing its goals, and the AI could in all likelihood predict and prevent any attempt to deviate from its goals.

There are, we can hope, methods and procedures to avert these difficulties. One suggestion Bostrom raises is that we could program AI to be ethically good, or to just do whatever is morally right. But specifying these instructions precisely is hardly a simple matter, requiring input from experts in computer science, math, philosophy, and law.

Addressing such concerns in AI development is where Musk and FLI come in. Musk’s donation to FLI will finance grants for promising research on the expansive subject of AI safety. FLI has provided a research priorities document detailing which areas it thinks offer fertile ground for investigation, and is accepting grant proposals from interested researchers.

So how worried should we be about AI? There’s no real consensus.

The path to human level AI (and beyond), and technological advancement in general, is uncertain and notoriously difficult to predict with any accuracy; some are skeptical that we are anywhere close to making it a reality. Further, AI is but one of several potential threats to human life.

Nevertheless, many are concerned about the lack of interest and urgency in understanding the problem seriously. Even if AI has a relatively low probability of posing a risk to our future, it would be worth taking very seriously. With so much uncertainty about the risk and general misperception of the problem, many in the field think the threat is disturbingly underappreciated compared to other global threats.

“Climate change is very important, but there’s already a lot of work and a lot of political debate going into that, while there’s not nearly so much work going into AI safety,” Krakovna says. “Until recently, there was less research funding going into AI safety as there was going into dung beetles.”

Initial proposals for FLI’s first round of research grants are due Mar 1, 2015.