Jump to ContentJump to Main Navigation
Robot Ethics 2.0From Autonomous Cars to Artificial Intelligence$
Users without a subscription are not able to see the full content.

Patrick Lin, Keith Abney, and Ryan Jenkins

Print publication date: 2017

Print ISBN-13: 9780190652951

Published to Oxford Scholarship Online: October 2017

DOI: 10.1093/oso/9780190652951.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2019. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see www.oxfordscholarship.com/page/privacy-policy).date: 18 March 2019

Superintelligence as Superethical

Superintelligence as Superethical

Chapter:
(p.322) 21 Superintelligence as Superethical
Source:
Robot Ethics 2.0
Author(s):

Steve Petersen

Publisher:
Oxford University Press
DOI:10.1093/oso/9780190652951.003.0021

Nick Bostrom’s book Superintelligence outlines a frightening but realistic scenario for human extinction: true artificial intelligence is likely to bootstrap itself into superintelligence, and thereby become ideally effective at achieving its goals. Human-friendly goals seem too abstract to be preprogrammed with any confidence; and if those goals are not explicitly favorable to humans, the superintelligence will extinguish us—not through any malice, but simply because it will want our resources for its own purposes. In response, I argue that things might not be as bad as Bostrom suggests. If the superintelligence must learn complex final goals, then this means such a superintelligence must in effect reason about its own goals. And because it will be especially clear to a superintelligence that there are no sharp lines between one agent’s goals and another’s, that reasoning could therefore automatically be ethical in nature.

Keywords:   robots, ethics, superintelligence, singularity, practical reasoning, value alignment, specificationism, coherence reasoning, personal identity

Oxford Scholarship Online requires a subscription or purchase to access the full text of books within the service. Public users can however freely search the site and view the abstracts and keywords for each book and chapter.

Please, subscribe or login to access full text content.

If you think you should have access to this title, please contact your librarian.

To troubleshoot, please check our FAQs , and if you can't find the answer there, please contact us .