(VIEW IN LANDSCAPE)

Why AI Suffering?

 

All technological advances risk increased biological suffering.

AI also brings a new risk:

It may lead to increased artificial suffering.

Metzinger's Moratorium

Recommendation 7
The EU should ban all research that risks or directly aims at the creation of synthetic phenomenology on its territory, and seek international agreements. This includes approaches that aim at a confluence of neuroscience and AI with the specific aim of fostering the development of machine consciousness (for recent examples see Dehaene, Lau & Kouider 2017, Graziano 2017 and Kanai 2017).

[Thomas Metzinger (2018): Alius Bulletin]

A ban may be going too far, and even counter-productive when the goal should be to understand machine consciousness as soon as possible, but it does serve to draw attention to the issue.
Synthetic Phenomenology ≈ Machine Consciousness.

AGI → Suffering


Is machine consciousness research the only way to AI suffering?

Perhaps all progress towards AGI leads towards the capacity for suffering?

Given current resource allocation, AGI research might even get there first.

AGI = Artificial General Intelligence which will be defined later.
The question then becomes: does AGI → the capacity for suffering? For this we need a working definition of suffering.

A theory of suffering...



  1. The system must be (self)-conscious.
    • There must be something it is like to be the system.
    • There must be something that experiences the suffering.

  2. The system must have preferences over its states.
    • It must have reasons to not want to be in certain states.
    • Its experiences must reflect these preferences.


These conditions are not perfect, but they'll do for the purposes of this initial exploration of the topic.
Crucially, we'll need to know whether or not the system is conscious.

...requires a theory of (machine) consciousness


  • Theory must be substrate independent

  • Based on information processing

  • Our experience is a controlled hallucination

  • We experience our model of the world

We need a theory of conciousness that is applicable to all possible future intelligences.
What we experience is the content of our model of the world and I think it may be possible to that a future theory of consciousness is at this level of explanation.

Experiencing our model

This demonstrates us experiencing our model.
Look at the image for a while and try to work out what it is.
Then go to the next slide to see the original.
When you come back you should now see what it actually is.
You don't just also know what it is, you can't help but see it.
You may even see the lines of the outline even in parts where it is completely black.

[Ludmer et al (2011) `Uncovering Camouflage: Amygdala Activation Predicts Long-Term Memory of Induced Perceptual Insight' Neuron 69:5]

Experiencing our model

This demonstrates us experiencing our model.
Look at the image for a while and try to work out what it is.
Then go to the next slide to see the original.
When you come back you should now see what it actually is.
You don't just also know what it is, you can't help but see it.
You may even see the lines of the outline even in parts where it is completely black.

[Ludmer et al (2011) `Uncovering Camouflage: Amygdala Activation Predicts Long-Term Memory of Induced Perceptual Insight' Neuron 69:5]

Machine Consciousness



"Conscious experience occurs if and only if an information-processing system has learned about its own representations of the world in such a way that these representations have acquired value for it."

[Axel Cleeremans (2011) The Radical Plasticity Thesis]

This is an example of a theory of consciousness that is pitched roughly at the level of explanation we are interested in.

Preliminary Requirements for Suffering



  • Model of the world.
  • Model of the self as a part of the world.
  • Metarepresentations (of the self).
  • Preferences over the states of these representations.


This is far from a complete list, or even a particularly good one.
However, part of the point is that we need more work to refine this and I think its enough to at least draw some parallels between AGI's and consciousness.

The Trivial Case: h-AGI

P1: h-AGI can do everything
a human can do.

P2: Humans can suffer.

C: h-AGI can suffer.

There are many ways that people use the term AGI.
h-AGI = humanlike-AGI. It can do everything a human can do.
This is the least useful definition.

What is AGI?

Incremental-AGI is the term I've used for when people claim the next big breakthrough will be general intelligence.
This is surprisingly common and we've seen the goalposts being pushed one step further back with each new breakthrough.
super-AGI is at the superintelligence end of the spectrum.
I think what is really meant by AGI takes something from both ends.
It is superintelligent in some respects, but doesn't have all human properties (e.g. suffering/consciousness/emotions/etc.)

The Harder Case: o-AGI.

  • An AI with a camera that can answer questions at least as well as a human.
    • It cannot act directly on the world.
    • It may use large lookup tables to solve many questions.







It is trivial to show hAGI → suffering.
What about the opposite case. That of the least likely suffering AGI?
Solving problems with a lookup table wouldn't match any of the requirements mentioned earlier.
If even this has the properties we mentioned earlier then we should at least consider it a possibility that AGI → suffering.

Winograd Schemas

The robot dog couldn't get through the door because a human kept closing it.

What did the human keep closing?
a) The door. ------------ About 1,370,000 results (0.82 seconds)
b) The robot dog. ------- About 392,000 results (0.75 seconds)

You can do a lot of intelligent things using lookup tables.
These sentences are ambiguous from understanding syntax alone. It was thought that you need true understanding to answer them.
However, even here lookup tables being surprisingly effective.
If you just google the two possibilities (the human kept closing the door/the dog), then the one with the most results is usually correct.

[Rahman, Ng (2012): Resolving complex cases of definite pronouns: the winograd schema challenge.]

Oracle-AGIs might meet the criteria

For an oracle-AGI to be a true AGI we would expect it to answer questions at least as well as a human.
In order for it to be able to answer general questions about the world it could just use methods like in the previous slide.
If we ask it about what it can see with its camera, then it would have to start modelling its inputs.
If we ask it about stuff currently out of sight of the camera it would have to start modelling the world.
If we want it to answer these as well as a child, it would ask to be moved.
This would require modelling what would happen if it was moved, and for example, where it is (i.e. a self).

Oracle-AGIs might meet the criteria

If we ask it how it can help us with something it will start to have to use the self component of its model.
This could lead to the necessary metarepresentations.
Finally, by setting resource bounded or time constrained problems, it will have to reason about its future states.
It should give different answers depending on the urgency of the required solution.
If not it wouldn't make generally intelligent decisions.
This would lead to preferences over its (future) states.

The aim here is not a proof, but to suggest a correlation between general intelligence and the capacity for suffering.

General Intelligence → Consciousness



It wasn't just a freak accident that evolution's most advanced general intelligence has the capacity to experience all the joys and wonders of existence.

Conclusion


We need to develop our understanding of consciousness alongside AGI research...
...ideally so that significant leaps in the former occur only after significant leaps in the latter.

It is probable that improving our understanding of consciousness also helps us build more useful AI.

Meaning


Although we've focused on the future of intelligence [suffering] ... the future of consciousness is even more important, since that's what enables meaning.
the very first goal on our wish list for the future should be retaining (and hopefully expanding) biological and/or artificial consciousness in our cosmos, rather than driving it extinct.

[Max Tegmark (2017) Life 3.0]

Why AI Suffering?

 

All technological advances have the potential to
allow for significantly increased meaning in our lives.

AI may allow for significantly increased meaning in the universe.
(by creating new entities with meaning).

Thanks to Matt Dixon (mattdixon.co.uk) for permission to use his artwork.