复旦智库报告 FUDAN REPORT SERIES | ASI: Minority Report

复旦智库报告（总第119期）

超级智能：少数派报告

ASI: Minority Report

上海论坛2025分论坛“超级智能机器、技术想象与人类未来”于4月26日在复旦大学举办，复旦智库报告·未来系列《超级智能：少数派报告》（ASI: Minority Report）在本次分论坛上发布。 报告由复旦大学杨庆峰教授领衔，来自中山大学、苏州大学、上海社会科学院、复旦大学等高校教师参与了撰写。该报告提出，应将超级智能视为一个日益迫近的现实问题，而非仅仅停留在科幻想象或生存焦虑的建构层面。报告认为，承认超级智能现实性的观点虽仍属“少数派”，但这部分意见却日益显现出其真理性和紧迫性。报告内容分为超级智能的历史想象、现实分析与未来应对。报告为中英双语，目前发布英文简版。

Co-Publisher

杨庆峰

复旦大学科技伦理与人类未来研究院教授、研究员

Yang Qingfeng, Professor and research fellow at the Institute of Technology Ethics for Human Future, Fudan University

刘永谋

中国人民大学吴玉章讲席教授，哲学院博士生导师

Liu Yongmou, Wu Yuzhang Chair Professor at Renmin University of China (RUC), a doctoral supervisor at the School of Philosophy

闫宏秀

上海交通大学教授，博士生导师

Yan Hongxiu,Professor and doctoral supervisor at Shanghai Jiao Tong University.

Executive Summary

In the debate over the feasibility of ASI, supporting view remain in the minority. The concept of ultraintelligent machine and the intelligence explosion, proposed by Irving John Good in 1964, did not achieve much attention. In 1998, Hans Moravec raised concerns about the threat ASI poses to humanity. In 2014, Superintelligence: Paths, Dangers, Strategies by Nick Bostrom sparked public interest, but these issues remained confined to sci-fictional narratives and poetic narratives. Ilya Sutskever proposed at NeurIPS 2024 conference that ASI with reasoning power would lead to “incredibly unpredictable” behavior. OpenAI’s Sam Altman suggested that ASI could massively accelerate scientific discovery and innovation, enhancing wealth and prosperity. This has created an opportunity for a shift in perspectives on ASI, transforming it into a pressing issue that demands serious consideration. In 2025, Yoshua Bengio has used ‘Scientist AI’ as a potential solution to address the catastrophic risks posed by superintelligent agents. However, people remain unaware of this change, continuing to view ASI as a mere product of imagination. This report endeavors to change such perceptions.

This report is mainly divided into three parts: The first part, “Historical Imagination,” examines the discourse surrounding ASI in the intellectual histories of both China and the West. It traces the evolution of AI from Weak AI and AGI to ASI, with its reality becoming increasingly evident. The scientific, psychological, and philosophical foundations have collectively fostered the emergence of ASI. This section also analyzes the perspectives of four key narrative subjects—scientists, science fiction writers, philosophers, and the media scholars—regarding ASI, categorizing their views into two primary stances: realism and anti-realism. The second part, “Present Analysis,” explores the scientific pathways through which ASI can be realized: AGI, Embodied AI, and Interactive Intelligence. It discusses the opportunities ASI presents, including the acceleration of scientific discovery and innovation, the enhancement of wealth, and the potential for intelligence augmentation. However, it also examines the risks involved, such as ASI’s potential for self-replication or self-preservation in response to shutdown problem, the consequences of autonomous logic exceeding human-imposed boundaries, and the threat of self-awareness leading to the collapse of human civilization. The third part, “Futuristic Responses,” examines potential response strategies, with a focus on the integration of technology and culture. It highlights the significance of philosophical and ethical approaches in addressing the challenges posed by ASI. Technical responses can address technological risks through methods such as aligning ASI with human values, mitigating the collapse caused by synthetic data, and addressing the dilution of capabilities caused by data distillation, with the principle of “governing technology with technology” being overly evident. Philosophically, reflections can be made through strategies such as designing Gödel bombs and responding to risks. Ethically, the adoption of virtue ethics focused on dedication and the promotion of contractual ethics as a new ethical approach are advocated.

Contents

您可通过访问以下链接阅读简版报告全文 Read the full text via read more or the following link：

https://fddi.fudan.edu.cn/fddien/1f/36/c19514a728886/page.htm

排版 | 毕瑛璞