If a Lion could Speak

According to Wittgenstein, “if a lion could speak, we could not understand him.” (1) It is by no means clear what  Wittgenstein meant by this remark and I will suggest two possible explanations. Firstly, there might exist some languages which cannot be translated into any other. Secondly some minds might be so radically different from ours that the thoughts in those minds might be so radically different to our own that we couldn’t conceive them. It might appear that whilst the soundness of these propositions might be of academic interest it is of no practical importance. I would suggest that this appearance is mistaken. Suppose that some advanced AI, robots or even aliens could speak could we understand them? The answer to this question might help support or provide some evidence against Bostrom’s orthogonanlity thesis. Recently Facebook abandoned an experiment after two artificially intelligent programs appeared to be chatting to each other in a strange language only they understood, see the Independent . Stephen Hawking believes if we are ever contacted by aliens we should think very carefully before replying due to the dangers involved. I am extremely dubious about whether we will ever be contacted by aliens but the possibility exists as long as we are unsure of how life evolved in the universe. The first possible danger posed by our inability to communicate with aliens formed the subject matter of the 2016 film Arrival, might powerful minds with which we cannot communicate pose a threat to us? Besides the above possibility there also exists the possibility that alien minds might be so radically different from ours that they might consider us of no importance and even consider us as dangerous. This second possibility might also be posed by some advanced form of AI. On a more practical level if Wittgenstein is correct then Turing tests are pointless for we might be unable to converse with fully intelligent conscious entities.

Let us accept that language can be roughly defined as a system of communicating information. However, there is an important difference between language processors and language users. If language is simply a system of communication then ‘computer languages’ such as Java, C and Python are languages in much the same way as are English, Mandarin and Sign Languages used by the deaf. I would suggest that however fast a computer runs or however much information it can handle that if this is all it can do then it cannot be said to be a language user. What does it mean to be a language user? I would suggest that for some entity to be considered as a language user this entity must determine the use it puts language to.  At the present time computers, robots and AI don’t determine how the information they process is used and as a result aren’t language users. It follows that at the present time that any dangers posed by computers, robots or AI are due to our misuse or misunderstanding of them rather than some imagined purpose such entities might acquire. It might be objected by someone that accepting my suggestion means that because animals don’t determine the use the language they use that they also aren’t real language users. It would appear to follow that chimpanzees and clever crows which appear to communicate with us are really language processors in much the same way as computers rather than users. I would argue this objection is unsound. Animals might simply use language but the use of the language is put to, unlike the use of computers, is determined by the animals’ needs and wants. Accepting the above means accepting that certain animals are primitive language users. The rest of this posting will only be concerned with language used by language users as defined above.

 

Let us consider the possibility that we might be unable to understand the language of aliens or some advanced form of AI. It is possible that any AI, however advanced, must remain a language processor rather than a language user. Nonetheless because we are uncertain as to how we became language users the possibility of some advanced AI becoming a user cannot be completely ruled out. Let us now consider whether some language might be untranslatable into any other. By untranslatable I don’t mean some language which is difficult to translate but rather that some language is impossible to translate. Of course we may not capture all the nuances of some language in translation but is there any language that cannot translated at least to some degree? In order to answer this question, we must ask another what is meant by a language?  Let us accept that language is a system of communicating information among language users as defined above. Information about what? Information must include knowledge of things in the world shared by the language users. The world of language users must be a world of things. These things might include such things as, physical objects, descriptions of behaviour in the world and emotions among others. If any world was a completely undifferentiated one with no distinct things existing in it there could be no speech and no need for language users. Our original question might now be reframed. Is it possible for the users of one language to talk about a set of totally distinct things from the users of another language? This would only be possible if the world of one set of language users was totally separate from that of another set. This might be possible if language defines the world we live in, language could also help us make sense of the world we live in. Let us assume for the sake of argument that lions could talk. Would this talk define a lion’s world or help lions make sense of the world they live in? I would suggest language must touch the world rather than define it and that this world is shared by all of us to some degree. I don’t believe Wittgenstein would agree. It follows that if lions could talk they would talk about some things common to our world. For instance they might talk about being hot or cold, hunger or being content. It follows lions could speak that we should be able to understand them even if the translation proved to be difficult in practice and we couldn’t understand all the nuances of their language. However, would the same be true for some more fanciful language users such as advanced AI, robots or aliens? I would suggest the same argument can be applied and that all language users share the same universe to some degree and it is impossible for the users of one language to talk about a set of totally distinct things from the users of another language. Because language must touch the world any two sets of language users must talk about some of the same things. It follows we should be able to partly translate the language of any language users who share our world even if this might prove to be difficult in practice.

I have argued that we should be able to partly translate any language in our universe even if this might prove to be difficult in practice. This argument presumes that all language users share the same universe, share some common understandings. Lions and human beings all understand what is meant by trees, sleep and hunger but only humans understand what is meant by a galaxy. The above appears to suggest that there is a hierarchy of understanding and that some things can only be understood once a creature has understood some more basic things. The above also seems to suggest that there is a hierarchy of languages with simple ones only touching the more basic things in the world whilst more complex languages are able to touch a wider domain. In the light of the above it seems possible that aliens or some advanced AI might be able to talk about things we are unable to understand. Is it possible that our inability to fully understand the language of such entities might pose us with an existential threat?

Our failure to understand such entities means that we cannot completely discount the above possibility, however I will now suggest that we have some reasons to believe such a threat is unlikely to be posed to us by aliens. Language use is not simply a cognitive exercise. Any communication between entities that don’t have a will is not language use but language processing, language users must have a will. For something to have a will means it must care about something. If something cared about nothing, then it would have no basis on which to base decisions and all its decisions would be equally good meaning decisions could be made at random. The domain of our moral concern has expanded over time. Slavery is now unthinkable, women in the western world are considered of equal worth when compared to men and our moral concern extends to animals, all this is very different to ancient world. What has caused this increase in the domain of our moral concern? I would suggest this increase is due to an increase in our projective empathy. This increase is not simply driven by an increase in our ability to feel emotion. It is driven by our ability to see others as sharing with us some features of the world. Slaves can have a will even if the exercise of this will is restricted, animals can also feel restricted and pain. This ability is due our increase in our knowledge of the world rather than any increase in either cognitive ability or empathy. In the light of the above I would suggest that any aliens are unlikely to pose an existential threat to us. Language users must have a will. Having a will means caring about something. It seems probable that any aliens which might threaten us would have an advanced basis of knowledge, without such a basis it is difficult to see either how they would contact us or how they might threaten us. If some entity has an ability to care about and advanced knowledge basis, then it seems probable that it will have a wide domain of moral concern and that we would be included in that domain. I have argued above that if aliens ever contact we should be able to partly understand them. In the light of the above it seems that any failure on our part to fully understand possible aliens would not pose an existential threat to us.

Does the above apply to advanced AI or robots. If such entities don’t have a will then any threat posed by such entities would be due our failure to understand how such entities function or a failure to set them clear goals. The possibility exists that we might create some bio-hazard by failing to fully understand what we are doing. The threat posed by advanced AI or robots without a will is similar. However, provided we are extremely careful in how we set the goals of such entities this hazard can be minimised. I am extremely doubtful whether advanced AI or robots can acquire a will, nonetheless because we don’t fully understand how consciousness originated such a possibility cannot be completely ruled out. I have argued that it is unlikely that our inability to understand any possible aliens would pose an existential threat to us, however I would suggest any such failure to fully understand some advanced AI which is in the process of acquiring a will might pose such a threat. The threat might be due to an emerging primitive will being akin to that of a child. Perhaps the fact that some such emerging entity has a primitive will might mean it wouldn’t prefer the destruction of the whole world to the scratching of its metaphorical finger, but it might prefer the destruction of humanity rather than refraining from such scratching. It follows if the possibility exists that advanced AI or robots can acquire a will that we should take seriously the possibility that if this will starts happening that such emerging entities might well pose us with an existential threat. Any failure on our part to fully understand such entities would compound such a threat. Perhaps if such entities can fully acquire a will the threat will recede.

1. Ludwig Wittgenstein, Philosophical Investigations, 1953, Blackwell, page 223

Afterthoughts

Assuming that we can understand a lion AT/Alien to some degree the question arises what sort of things might we understand. Goals, intentions or reasons? Perhaps even if we understand the goals and intentions of some advanced AI we might be unable to understand its reasons. But we don't understand all our reasons, reasons run out according to Wittgenstein.  The question becomes how many reasons we need to understand and how many can we do.

Robots and Persons

In an interesting post John Danaher asks if someone can be friends with a robot, see philosophicaldisquisitions . He argues virtue friendship might be possible with a robot. Virtue friendship involves two entities sharing values and beliefs which benefit them. Let us accept that any entity which is capable of having values and beliefs can be regarded as a person. Perhaps one of the great apes might be regarded as a person but can the same be said of a robot? Does it make sense to say a robot might have rights or can be regarded as a person? In what follows I will limit my discussion to robots but my discussion could equally well be applied to some advanced system of AI or algorithms. At present he actions of some robot have some purpose but this purpose doesn’t have any meaning which is independent of human beings. At present the actions of a robot have no more meaning which is independent of us than the action of the wind in sculpting a sand dune. In the future it is conceivable that this situation might change but I am somewhat sceptical and believe at the present time there is no need to worry about granting rights to robots akin to human rights. In this posting I will argue the nature of belief means to worry about robot personhood is both premature and unnecessary.

How should we regard the actions of a robot if it has no beliefs? What are the differences between the wind sculpting a sand dune and the actions of a robot? One difference is that even if both the wind and a robot don’t have beliefs that nonetheless a robot’s actions are in accordance with someone’s beliefs, its designer or programmer. But does this difference matter? A refrigerator is acting in accordance with our belief that it will keep our food cold. If we don’t want to grant personhood to refrigerators, why should we do so for robots? Perhaps then we might implant some beliefs into robots and after some time such robots might acquire their own emergent beliefs. Perhaps such robots should be regarded as persons. Implanting such beliefs will not be easy and may well be impossible. However, I see no reason, even if such implantation is possible, why we should regard a such a robot as some sort of person. If a person has some belief, then this belief causes him to behave in certain ways. How do we implant a belief in a robot? We instruct the robot how to behave in certain circumstances. In this situation the of course the robot behaves in accordance with the implanted belief but the primary cause of this behaviour is not this implanted belief but rather a belief of those who carried out the implantation. A robot in this situation cannot be said to be behaving authentically. In this situation I can see no reason why we should attribute personhood to a robot which uses implanted beliefs as outlined above.

At this point it might be objected that even if a robot shouldn’t be considered as a person it might be of moral concern. According to Peter Singer what matters for something to matter morally is not that it can think but that it can feel. Animals can feel and should be of moral concern. Present day robots can’t and shouldn’t. Present day robots are made of inorganic materials such as steel and silicon. However it might be possible to construct a robot partly from biological material, see Mail Online. If such a robot could feel then it should be of moral concern but this doesn’t mean we should consider it as a person, frogs can feel and should be of moral concern but they aren't persons. Nonetheless I would suggest that the ability to feel is a necessary precursor for believing which is a precursor for personhood.

For the sake of argument let us assume that it is possible to create a robot which the primary cause of its behaviour is its implanted or emergent beliefs.  What can be said about this robot’s beliefs?  When such a robot decides to act the primary cause of the action is its internal beliefs, it is acting in a manner which might be regarded as authentic. How might such a robot’s beliefs and actions be connected? Perhaps they are linked by Kant’s hypothetical imperative.  The hypothetical imperative states,

“Whoever wills an end also wills (insofar as reason has decisive influence on his actions) the indispensably means to it that are within his power. (1)

Some might suggest that having a set of beliefs and accepting Kant’s hypothetical imperative are necessary conditions for personhood, some might even regard them as sufficient conditions. They might further suggest that any robot meeting these conditions should be regarded as a candidate for personhood. Of course it might be possible to design a robot which conforms to the hypothetical imperative, but conforming is not the same as accepting. Let us accept anyone or anything that can be regarded as person must have some beliefs and must accept rather than conform to the hypothetical imperative.

What does it mean for someone to accept the hypothetical imperative? Firstly, he must believe it is true, the hypothetical imperative is one of his beliefs. Someone might believe that he is made up of atoms but this belief doesn’t require any action when action is possible. The hypothetical imperative is different because it connects willed ends with action. Can the hypothetical imperative be used to explain why a robot should act on its beliefs, be they implanted by others or emergent? Kant seems to believe that the hypothetical imperative can be based on reason. I will argue reason can only give us reason to act in conjunction with our caring about something. I will now argue the hypothetical imperative only makes sense if an agent views beliefs in a particular way. What does it mean to will an end? I would suggest if someone wills an end that he must care about that end. If someone doesn’t care about or value some end, then he has no reason to pursue that end. What then does it mean to care about something? According to Frankfurt if someone cares about something he becomes vulnerable when that thing is diminished and is benefited when it is enhanced. (2) People by nature can suffer and feel joy robots can’t. It is worth noting animals can also suffer and feel joy making them like people with rights rather than like robots. The above raises an interesting question. Must any entity which is capable of being conscious, robot or animal, be able to suffer and feel joy? If we accept the above then the ends we will must be things we care about. Moreover, if we care about ends then we must value them. It follows if the hypothetical imperative is to give us cause to act on any belief that that belief must be of value to us. It follows the hypothetical imperative can only be used to explain why a robot should act on its beliefs provided such a robot values those beliefs which requires it becoming vulnerable. A right is of no use to any entity for which the implementation of the right doesn't matter, isn't vulnerable to the right not being implemented.

I have argued any belief which causes us to act must be of value to us and that if we find something valuable we are vulnerable to the fate of the thing we find valuable. What then does it mean to be vulnerable? To be vulnerable to something means that we can be harmed. Usually we are vulnerable to those thing we care about in a psychological sense. Frankfurt appears to believe that we don’t of necessity become vulnerable to the diminishment of the things we value by suffering negative affect. He might argue we can become dissatisfied and seek to alleviate our dissatisfaction without suffering any negative affect. I am reluctant to accept becoming vulnerable can be satisfactorily explained by becoming dissatisfied without any negative affect. It seems to me being dissatisfied must involve some desire to change things and that this desire must involve some negative affect. I would argue being vulnerable to those thing we value involves psychological harm and that this harm must involve negative affect.

Let us accept that in order to be a person at all someone or something must accept and act on the hypothetical imperative. Let us also accept that the hypothetical imperative only gives someone or something reason to act on some belief provided that someone or something must value that belief. Let us still further accept that to value something someone or something must care about what they value and that caring about of necessity must include some affect. People feel affect and so are candidates for personhood. It is hard to see how silicon based machines or algorithms can feel any affect, positive or negative. It follows it is hard to see why silicon based machines or algorithms should be considered as candidates for personhood. It appears the nature of belief means any worries concerning robot personhood when the robot intelligence are silicon based are unnecessary. Returning to my starting point it would appear that it is acceptable for young children to have imaginary friends but it that is delusional for adults to believe they have robotic friends. However I will end on a note of caution. We don’t fully understand consciousness so we don’t fully understand what sort of entity is capable of holding beliefs and values. It follows we cannot categorically rule out a silicon machine becoming conscious. Perhaps also it might become possible to build some machine not entirely based on silicon which does become conscious. 

1. Immanuel Kant, 1785, Groundwork of the Metaphysics of Morals,
2.  Harry Frankfurt, 1988, The Importance of What We Care about, Cambridge University Press, page 83.

Outsourcing Ethical Decision Making and Authenticity

In a previous posting I questioned whether algorithmic assisted moral decision making is possible. Let us assume for the sake of argument that AAMD is possible. Using such a system might be considered as an example of algorithmic outsourcing of our moral decision making. Such outsourcing according to John Danaher means taking away the cognitive and emotional burden associated with certain activities, see Danaher . Intuitively outsourced moral decisions are inauthentic decisions. In this posting I will argue that under certain conditions outsourced ethical decisions using AAMD could be authentic ones.

Before proceeding I must make it clear what I mean by algorithmic assisted moral decision making, outsourcing and authenticity. Any moral decision simply made by an algorithm is not an authentic decision. In my previous posting I suggested when initialising an AAMD system we should first use a top down approach and install simple human values such as avoiding harm. However once initialised such a system should be fine-tuned by the user from the bottom up by adding his personal weights to the installed values. This primitive system might then be further modified from the bottom up using of two feedback loops. Firstly, the user of a system must inform the system whether she accepts any proposed decision. If the user accepts the proposed decision, then this decision can form a basis for similar future decisions in much the same way as in the legal judgements set precedents for further judgements. If the user doesn’t accept a particular decision, then the system must make it clear to the user the weights which are attached to the values it used in making this decision and any previous decisions employed. The user might then further refine the system either by altering these weights or highlighting differences between the current decision and any previous decisions the system employed. According to Danaher outsourcing can take two forms. Cognitive outsourcing means someone using a device to perform cognitive tasks that she would otherwise have to perform himself. Affective outsourcing means someone using a device to perform an affective task that she would otherwise have to perform himself. I will assume here that an authentic decision is a decision that the decision maker identifies herself with or cares about.

According to Danaher taking responsibility for certain outcomes is an important social and personal virtue. Further, someone only takes responsibility for certain decisions if he voluntary wills his chosen outcomes of these decisions. Authenticity is an important social and personal virtue. Getting an app to automatically send flowers to someone’s partner on her birthday doesn’t seem to be an authentic action because the sender doesn’t cause the action. However, here I am only interested in outsourcing our ethical decisions, does outsourcing such decisions damage their authenticity?

I will now argue the answer to the above question depends not on outsourcing, per se, but on the manner of the outsourcing. Let us assume that in the future there exists a computer which makes decisions based on a set of values built into it by a committee of philosophers. Let us consider someone who outsources his moral decisions to this computer. I would suggest that if she implements a moral decision made in this way that his decision is an inauthentic one. It is hard to see how someone in this situation could either identify with the decision or consider herself to be responsible for the outcome. Let us now consider someone who outsources her moral decision making to a AAMD system which is finely tuned by the user as outlined above, are her decisions also inauthentic? I would suggest someone who makes a moral decision in this way is acting authentically because she can identify with his decision. She is able to identify with the systems decisions because, once initialised, the system is built from the bottom up. Her weights are attached to the incorporated values and her past decisions are built into its database.

I suggested that some who uses such a system must accept or reject its decisions. Someone might object that someone who simply accepts the systems decisions without reflection is not acting authentically. In response I would point in virtue ethics someone can simply act and still be regarded as acting authentically. My objector might respond by pointing out Christine Korsgaard pictures the simply virtuous human as a sort of Good Dog (1). Perhaps someone who simply accepts the results of an AADM system might also be pictured as behaving as a good dog with the system replacing the dog’s owner. Surely such a person cannot be regarded as acting authentically. In response I would suggest what matters is that the agent identified himself with the system’s decision. To identify with a decision someone has to be satisfied with that decision. What does it mean to be satisfied with a decision? According to Frankfurt satisfaction entails,

“an absence of restlessness or resistance. A satisfied person may be willing to accept a change in his condition, but he has no active interest in bringing about a change.” (2)

I’m not sure that an absence of restlessness or resistance with a decision is sufficient to guarantee its authenticity. I would suggest authentic decisions are ones that flow from our true self. I have argued our true self is defined by what we are proud or ashamed of, see  true selves do they exist . Let consider someone who accepts the recommendation of an AAMD system without feeling any shame, is her acceptance an authentic one or simply not an inauthentic one? I have argued that there are two types of shame . Type one shame is anxiety about social disqualification. Type two shame is someone’s anxiety about harming the things she cares about, loves and identifies with. Let us accept someone must feel type two shame when she acts in a way which harms the things she cares about, loves and identifies with. In the above situation if someone simply accepts the recommendation of an AAMD system without feeling any type two shame then he is acting in accordance with what he loves and identifies with and is acting authentically.

What conclusions can be drawn from the above. If someone outsources some of his moral decision making to a computer, she may not be acting authentically. However, if she outsources such decision making to an AAMD system designed using a bottom up approach as outlined above it is at least conceivable that she is acting authentically.

1. Christine Korsgaard, 2009, Self-Constitution, Oxford University Press, page 3. 
2. Frankfurt, 1999, Necessity, Volition, and Love. Cambridge University Press, page 103.

Emerging AI and Existential Threats

AI is much in the news recently. Google’s chairman Eric Schmidt believes AI is starting to make real progress whilst others such as Nick Bostrom believe AI might pose an existential danger to humanity (1). In this posting I want first to question whether any real progress is in fact being made and secondly examine the potential dangers involved. Before proceeding I must make it clear I don’t deny real AI is feasible for after all human beings have evolved intelligence. If intelligence can evolve due to natural selection then it seems feasible that it can be created by artificial means however I believe this will be harder to achieve than many people seem to believe.

At present computing power is rising fast and algorithms are increasing in complexity leading to optimism about the emergence of real AI. However it seems to me that larger faster computers and more complex algorithms alone are unlikely to lead to real AI. I will argue genuine intelligence requires a will and as yet no progress has been made to creating for or endowing AI with a will. Famously Hume argued that reason are the slave of the passions. Reason according to Hume is purely instrumental. It might be thought that better computers and better algorithms ought to be better at reasoning. I would question whether they can reason at all because I would suggest that reason cannot be separated from the will. Unlike Hume I would suggest that reason is not the slave of the passions. Reason and the will, the passions, are of necessity bound together. In the present situation seems to me that better computers and better algorithms only mean they are better instruments to serve our will, they don’t reason at all. The output of some computer program may indeed have some form but this form doesn’t have any meaning which is independent of us. The form of its output alone has no more meaning than that of a sand dune sculpted by the wind. However sophisticated computers or algorithms become if the interpretation of their output depends on human beings then they don’t have any genuine intelligence and as a result I believe it is misleading to attribute AI to such computers or algorithms. Real AI in this posting will mean computers, algorithms or robots which have genuine intelligence. Genuine intelligence requires reasoning independently of human beings and this reasoning involves having a will.

Let us accept that if some supposed AI doesn’t have a will that it doesn’t have any genuine intelligence. What then does it mean to have a will? According to Harry Frankfurt,

“The formation of a person’s will is most fundamentally a matter of his coming to care about certain things, and of his coming to care about some of them more than others.” (2)

For something to have a will it must be capable of ‘caring about’ or loving something. If computers, algorithms or robots are mere instruments or tools, in much the same way as a hammer is, then they don’t have any will and real AI is no more than a dream. How might we give a potential AI a will or create the conditions from which a potential AI will acquire an emergent will? Before trying to answer this question I want to consider one further question. If something has a will must we regard it as a person? Let us assume Frankfurt is correct in believing that for something to have a will it must be capable of ‘caring about’ something. Frankfurt argues that something

“to whom its own condition and activities do not matter in the slightest properly be regarded as a person at all. Perhaps nothing that is entirely indifferent to itself is really a person, regardless of how intelligent or emotional or in other respects similar to persons it may be. There could not be a person of no importance to himself.” (3)

Accepting the above means that to have a will is essential to being a person. It also suggests that if something has a will it might be regarded as a person. This suggestion has moral implications for AI. Clearly when we switch off our computers we are not committing murder however if we switched off a computer or terminated an algorithm which had acquired a will we would. I will not follow this implication further here.

Let us return to the question as to whether it is possible to seed a potential AI with a will or create the conditions in which it might acquire one. If we accept Frankfurt’s position then for something to have a will it must satisfy three conditions.

It must be able to ‘care about’ some things and care about some of them more than others.

It must ‘care about itself.

In order to ‘care about’ it must be aware of itself and other things.

Before being able to satisfy conditions 1 and 2 a potential AI must firstly satisfy condition 3. If we program a potential AI to be aware of itself and other things it seems possible we are only programming the AI to mimic awareness. For this reason it might be preferable to try and create the conditions from which a potential AI might acquire an emergent awareness of itself and other things. How might we set about achieving this? The first step must be to give a potential AI a map of the world it will operate in. Initially it need not understand this map and only be able to use it to react to the world. Secondly it must be able to use its reactions with the world to refine this map. If intelligence is to be real then the world it operates in must be our world and the map it creates by refinement must resemble our world. Robots react more meaningfully with our world than computers so perhaps real AI will emerge from robots or robot swarms connected to computers. However it seems to me that creating a map of the things in our world will not be enough for a potential AI to acquire emergent awareness. For any awareness to emerge it must learn to differentiate how different things in that world react to its actions. Firstly it must learn what it can and cannot change by physical action. Secondly and more importantly it must learn to pick from amongst those things it cannot change by physical action the things it can sometimes change by change by simply changing its own state. A potential AI must learn which things are aware of the potential AI’s states and perhaps by doing so become aware of itself satisfying the third of the conditions above. Meeting this condition might facilitate the meeting of the first two conditions.

For the sake of argument let us assume a potential AI can acquire a will and in the process become a real AI. This might be done by the rather speculative process I sketched above. Bostrom believes AI might be an existential threat to humanity. I am somewhat doubtful whether a real AI would pose such a threat. Any so called intelligent machine which doesn’t have a will is an instrument and does not in itself pose an existential threat to us. Of course the way we use it may threaten us but the cause of the threat lies in ourselves in much the same way as nuclear weapons do. However I do believe the change from a potential AI to a real AI by acquiring a will does pose such a theat. Hume argued it wasn’t “contrary to reason to prefer the destruction of the whole world to scratching of my finger.” It certainly seems possible that a potential AI with an emerging will might behave in this way. It might have the will equivalent to that of a very young child whilst at the same time possessing immense powers, possibly the power to destroy humanity. Any parent with a young child who throws a tantrum because he can’t get his own way will appreciate how an emerging AI with immense powers and an emergent will potentially might poses an existential threat.

How might we address such a threat? Alan Turing proposed his Turing test for intelligence. Perhaps we need a refinement of his test to test for good will, such a refinement might called the Humean test. Firstly such a test must test for a good will and secondly, but much more importantly, it must test whether any emergent AI might in any possible circumstances consider the destruction of humanity. Creating such a test will not be easy and it will be difficult to deal with the problem of deceit. Moreover it is worth noting some people, such as Hitler and Pol Pot, might have passed such a test. Nonetheless if an emerging AI is not to pose a threat to humanity the development of such is vital and any potential AI which is not purely an instrument and cannot pass the test should be destroyed even if this involves killing a proto person.

1.  Nick Bostrom, 2004, Superintelligence, Oxford University Press
2. Harry Frankfurt, 1988, The Importance of What We Care About. Cambridge University Press, page 91
3. Frankfurt, 1999 Necessity, Volition, and Love. Cambridge University Press. Page 90.