AI News, BOOK REVIEW: Ethical design in the age of AI artificial intelligence

Nick Bostrom

Bostrom argues that a computer with near human-level general intellectual ability could initiate an intelligence explosion on a digital time scale with the resultant rapid creation of something so powerful that it might deliberately or accidentally destroy human kind.[22]

Bostrom contends the power of a superintelligence would be so great that a task given to it by humans might be taken to open ended extremes, for example a goal of calculating pi might collaterally cause nanotechnology manufactured facilities to sprout over the entire Earth's surface and cover it within days.[23]

Warning that a human-friendly prime directive for AI would rely on the absolute correctness of the human knowledge it was based on, Bostrom points to the lack of agreement among most philosophers as an indication that most philosophers are wrong, with the attendant possibility that a fundamental concept of current science may be incorrect.

Bostrom says a common assumption is that high intelligence would entail a 'nerdy' unaggressive personality, but notes that both John von Neumann and Bertrand Russell advocated a nuclear strike, or the threat of one, to prevent the Soviets acquiring the atomic bomb.

Given that there are few precedents to guide an understanding what, pure, non-anthropocentric rationality, would dictate for a potential singleton AI being held in quarantine, the relatively unlimited means of superintelligence might make for its analysis moving along different lines to the evolved 'diminishing returns' assessments that in humans confer a basic aversion to risk.[24]

Group selection in predators working by means of cannibalism shows the counter-intuitive nature of non-anthropocentric 'evolutionary search' reasoning, and thus humans are ill-equipped to perceive what an artificial intelligence's intentions might be.[25]

Other safety measures include the AI being 'boxed', (run in a virtual reality simulation), and being used only as an 'oracle' to answer carefully defined questions in a limited reply (to prevent it manipulating humans).[22]

Although he canvasses disruption of international economic, political and military stability including hacked nuclear missile launches, Bostrom thinks the most effective and likely means for the superintelligence to use, would be a coup de main with weapons several generations more advanced than current state-of-the-art.

Once a superintelligence has achieved world domination, humankind would be relevant only as resources for the achievement of the AI's objectives ('Human brains, if they contain information relevant to the AI’s goals, could be disassembled and scanned, and the extracted data transferred to some more efficient and secure storage format').[32]

Hassabis suggested the main safety measure would be an agreement for whichever AI research team began to make strides toward an artificial general intelligence to halt their project for a complete solution to the control problem prior to proceeding.[36]

Bostrom had pointed out that even if the crucial advances require the resources of a state, such a halt by a lead project might be likely to motivate a lagging country to a catch-up crash program or even physical destruction of the project suspected of being on the verge of success.[37]

Dennett modified his views somewhat after reading The Master Algorithm, and now acknowledges that it is 'possible in principle' to create 'strong AI' with human-like comprehension and agency, but maintains that the difficulties of any such 'strong AI' project as predicated by Bostrom's 'alarming' work would be orders of magnitude greater than those raising concerns have realized, and at least 50 years away.[39]

Since a 2014 book in which he expressed the opinion that artificial intelligence developments would never challenge humans' supremacy, environmentalist James Lovelock has moved far closer to Bostrom's position, and in 2018 Lovelock said that he thought the overthrow of humankind will happen within the foreseeable future.[41][42]

In later work, he has described the phenomenon of anthropic shadow, an observation selection effect that prevents observers from observing certain kinds of catastrophes in their recent geological and evolutionary past.[44]

This principle states that we ought to retard the development of dangerous technologies, particularly ones that raise the level of existential risk, and accelerate the development of beneficial technologies, particularly those that protect against the existential risks posed by nature or by other technologies.[53][54]