arXiv subjects

Nick Bostrom

Publications and source records attributed to Nick Bostrom.

4 recordsLinked to original sources

arXiv2025

Future progress in artificial intelligence: A survey of expert opinion

There is, in some quarters, concern about high-level machine intelligence and superintelligent AI coming up in a few decades, bringing with it significant risks for humanity. In other quarters, these issues are ignored or considered science fiction. We wanted to clarify what the distribution of opinions actually is, what probability the best experts currently assign to high-level machine intelligence coming up within a particular time-frame, which risks they see with that development, and how fast they see these developing. We thus designed a brief questionnaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be 'bad' or 'extremely bad' for humanity.

cs.CY↗

arXiv2025

How malicious AI swarms can threaten democracy: The fusion of agentic AI and LLMs marks a new frontier in information warfare

Advances in AI offer the prospect of manipulating beliefs and behaviors on a population-wide level. Large language models and autonomous agents now let influence campaigns reach unprecedented scale and precision. Generative tools can expand propaganda output without sacrificing credibility and inexpensively create falsehoods that are rated as more human-like than those written by humans. Techniques meant to refine AI reasoning, such as chain-of-thought prompting, can just as effectively be used to generate more convincing falsehoods. Enabled by these capabilities, a disruptive threat is emerging: swarms of collaborative, malicious AI agents. Fusing LLM reasoning with multi-agent architectures, these systems are capable of coordinating autonomously, infiltrating communities, and fabricating consensus efficiently. By adaptively mimicking human social dynamics, they threaten democracy. Because the resulting harms stem from design, commercial incentives, and governance, we prioritize interventions at multiple leverage points, focusing on pragmatic mechanisms over voluntary compliance.

cs.CY↗

arXiv2005

How unlikely is a doomsday catastrophe?

Numerous Earth-destroying doomsday scenarios have recently been analyzed, including breakdown of a metastable vacuum state and planetary destruction triggered by a "strangelet'' or microscopic black hole. We point out that many previous bounds on their frequency give a false sense of security: one cannot infer that such events are rare from the the fact that Earth has survived for so long, because observers are by definition in places lucky enough to have avoided destruction. We derive a new upper bound of one per 10^9 years (99.9% c.l.) on the exogenous terminal catastrophe rate that is free of such selection bias, using planetary age distributions and the relatively late formation time of Earth.

astro-ph↗

arXiv1999

Cosmological Constant and the Final Anthropic Hypothesis

The influence of recent detections of a finite vacuum energy ("cosmological constant") on our formulation of anthropic conjectures, particularly the so-called Final Anthropic Principle is investigated. It is shown that non-zero vacuum energy implies the onset of a quasi-exponential expansion of our causally connected domain ("the universe") at some point in the future, a stage similar to the inflationary expansion at the very beginning of time. The transition to this future inflationary phase of cosmological expansion will preclude indefinite survival of intelligent species in our domain, because of the rapid shrinking of particle horizons and subsequent depletion of energy necessary for information processes within the horizon of any observer. Therefore, to satisfy the Final Anthropic Hypothesis (reformulated to apply to the entire ensemble of universes), it is necessary to show that (i) chaotic inflation of Linde (or some similar model) provides a satisfactory description of reality, (ii) migration between causally connected domains within the multiverse is physically permitted, and (iii) the time interval left to the onset of the future inflationary phase is sufficient for development of the technology necessary for such inter-domain travel. These stringent requirements diminish the probability of the Final Anthropic Hypothesis being true.

gr-qc↗