Academic Jobs - Home of Higher Ed Logo

technological singularity: top research papers revealed

10 min read0 Ansichten

At a glance

The singularity debate has split into rival research programmes. From Science's 'society of thought' paper to Zenil's collapse theorems and METR's task-horizon data, here is what the strongest work establishes — and what it doesn't.

a computer circuit board with a brain on it
Photo by Steve A Johnson on Unsplash

The technological singularity has stopped being a prophecy and become a literature. It has factions now, replication problems, and papers willing to lose arguments in public.

The shape of the argument changed in March 2026, when Science carried a perspective piece by James Evans of the University of Chicago, Benjamin Bratton of the University of California, San Diego, and Google's Blaise Agüera y Arcas. Their claim was not that the singularity is near or far. It was that the thing people keep waiting for is already happening, and looks nothing like the solitary super-brain of the folklore. A society is forming, they argue, not a god.

Set that against the strongest work on the other side and the field stops looking like a dispute about dates. It becomes a dispute about what an explosion would even be made of.

Inside the reasoning model, an argument breaks out

The empirical floor of the Science essay is a finding about how reasoning models work. Systems such as DeepSeek-R1 and QwQ-32B outperform ordinary models of the same size not simply because they think longer. Analysing outputs from more than 8,000 reasoning questions, the researchers found that the models spawn internal perspectives that question, contradict and reconcile with one another — a "society of thought" — and that the effect is strongest on the hardest items, graduate-level science and competition mathematics, and nearly absent on routine procedure. In one experiment on DeepSeek-R1-Llama-8B, turning up a single internal feature associated with surprise or insight lifted accuracy on a mathematics task from 27.1% to 54.8%.

Nobody trained them to argue with themselves. Reinforcement learning rewarded accuracy, and debate emerged as the cheapest route to it.

From there the authors make a historical move, and it is the most interesting thing in the paper. Primate intelligence tracks group size more closely than habitat difficulty. Human language gave us what Michael Tomasello called the cultural ratchet, so knowledge accumulates across generations instead of being rebuilt by each one. Their example is a Sumerian scribe maintaining a grain accounting system whose macroeconomic function he could not have explained. The system was smarter than the man inside it.

The prescription follows: institutional alignment. Courtrooms work because judge, attorney and jury are defined slots, whoever fills them. AI ecosystems, they argue, will need digital equivalents, including the constitutional oddity of agencies auditing each other's algorithms. "No single concentration of intelligence, human or artificial, should regulate itself."

It is an elegant reframing, and it is a reframing rather than a measurement. The empirical core is interpretability work; the institutional half is a design proposal. The objection that has followed the paper is fair: institutional alignment names a destination without specifying the mechanism by which institutions avoid fossilising, which is the one thing history says about institutions.

The largest survey ever taken, and the number nobody quotes

If you want a forecast with a sample behind it, the paper is Thousands of AI Authors on the Future of AI, by Katja Grace and colleagues, published in the Journal of Artificial Intelligence Research in January 2026. In October 2023 they surveyed 2,778 researchers who had published at top-tier AI venues, four times the sample of their 2022 round, and split respondents between two question framings to test how much the answers were an artefact of phrasing.

The headline figure: a 10% chance by 2027, and a 50% chance by 2047, that unaided machines outperform humans at every task. The 2047 median arrived thirteen years earlier than the same question had a year before. Forecasts that move thirteen years in twelve months are not measurements of the world. They are measurements of a mood.

The number worth carrying around is a different one. Asked when all human occupations could be fully automated, the same researchers put 10% probability at 2037 and 50% at 2116. Between those two estimates — superhuman at everything, and everything actually automated — sits most of the uncertainty people pretend doesn't exist.

On risk, the respondents were less divided than the commentary suggests. Depending on framing, 38% to 51% gave at least a 10% chance of outcomes as bad as human extinction. Of the self-identified optimists, nearly half still assigned at least a 5% chance to catastrophe. Asked which scenarios deserved substantial or extreme concern, 86% named deepfakes and the spread of false information, and 79% named manipulation of large-scale public opinion.

The arithmetic objection: a photocopier with opinions

The most concrete challenge to the whole edifice comes from Hector Zenil at King's College London, in On the Limits of Self-Improving in Large Language Models. Zenil models recursive self-training as a discrete-time dynamical system and asks what happens as the proportion of externally grounded signal goes to zero — which is precisely the autonomy the singularity story requires.

Two failure modes follow. Entropy decay: finite sampling makes the model over-represent common cases and forget the tails, so diversity falls monotonically with each round of self-training. Variance amplification: without a persistent anchor to the world, small errors compound like a random walk and the model's sense of the centre drifts. The industry term is model collapse. Zenil's contribution is to show it isn't a bug to be patched but a property of the architecture.

The later versions of the paper, written with Abicumaran Uthamacumaran and Luan Ozelim, put numbers on it. Across 26 served model checkpoints, spanning five pre-training families and 0.8 to 35 billion parameters, every one violated the dominance guarantee that defines a universal mixture. The best trailed a classical compression baseline by 4.5 bits; the median by 36; the worst by 128. More scale did not close the gap, and neither did post-training.

The proposed exit is neurosymbolic: teach the system to find the generating rule rather than the correlation, so that a sequence reads as add two each time and not as a list of numbers that tend to appear together. That is a research programme, not a result. What Zenil establishes is narrower than the headlines claimed and more useful: a closed loop with no external check is a machine for becoming confidently wrong.

The growth assumption nobody quite wants to defend

Philosophers got to the singularity before engineers did, and one of them has now done the ungrateful work of reading the maths. David Thorstad's Against the Singularity Hypothesis, in Philosophical Studies, grants the field its ambitions and attacks its arithmetic. Hyperbolic growth is not a metaphor. In Nick Bostrom's model, intelligence doubles every seven and a half months, grows a thousandfold within eighteen, and approaches infinity at eighteen. An argument that strong needs evidence to match, and Thorstad finds the philosophical defences relying on growth assumptions borrowed rather than shown.

His historical counterexample is a small masterpiece of deflation. A famous 1960 model extrapolated then-current population trends and predicted infinite human population by 2026. The curve was real. The projection was not.

Economics has run a parallel audit. Ege Erdil and Tamay Besiroglu's review, Explosive growth from AI automation, works through nine counterarguments — regulation, physical bottlenecks, alignment, the pace of automation itself — and concludes that explosive growth is plausible if AI broadly substitutes for labour, while high confidence in that claim is unwarranted. Their reason is almost mundane: growth requires accumulable inputs, and labour has never been one of them. Make labour accumulable and the models do go vertical. Whether the world cooperates is another question.

What can be measured, and how fast it moves

The most quoted chart in AI policy comes from METR, the evaluation nonprofit, and it measures something narrower than intelligence. In Measuring AI Ability to Complete Long Tasks, Thomas Kwa and colleagues scored tasks by how long a human expert took to finish them, then asked how long a task an AI agent could complete with 50% reliability. Frontier agents sat at roughly 50 minutes in early 2025 and around 110 minutes by the o3 generation. The horizon had been doubling about every seven months since 2019. Naive extrapolation put a one-month horizon somewhere between late 2028 and early 2031.

The same organisation has supplied the most useful corrective to its own metric. METR's earlier randomised trial gave 16 experienced open-source developers, averaging five years on their repositories, access to AI tools across 246 real issues. They took 19% longer. They had expected to be 24% faster.

In November 2026 METR published a revised suite with longer and more open-ended tasks, including one where models design AI evaluations for METR's own staff to judge. Not one of the AI-designed evaluations was considered usable. On scoring philosophical arguments, models fell short of human performance too. The lesson is not that the trend has broken. It's that the trend is steepest exactly where success can be checked automatically, and the automation of research depends on the places where it can't.

The loop that reportedly closed, by a factor of two

Then came the report that moved the forecasts. On 5 November 2026, after eleven weeks examining internal research materials, model scores and staff interviews at Anthropic, METR concluded that the company was likely improving its models at roughly twice the rate it had managed between March 2024 and October 2025 — with a stated range of 1.3 to 2.8 times — and that perhaps 80% of the acceleration came from Anthropic's own engineers using unreleased models. Anthropic redacted most of the informative evidence, citing trade secrets ahead of a public listing. Its amended filing, three days earlier, conceded the company "may imminently meet" its internal threshold for dramatically accelerated progress attributable to AI.

The AI Futures Project, the group behind the AI 2027 scenario, recalculated and found an implied 1.7x speed-up in overall AI research, a level to which its most aggressive prior model had assigned under 4% probability. On 10 November 2026 its six researchers jointly moved their median for a Superhuman AI Researcher to between May and November 2027. Earlier in the same year, several of them had pushed their medians into the 2030s. Daniel Kokotajlo's public reaction was not triumphal. "I am scared, sad, and angry," he wrote.

The sceptics' reply is straightforward and not yet answered: a one-time productivity gain from handing good engineers better tools is not the same as a system that invents science. A bottleneck that moves from writing code to reviewing it has moved, not vanished. And a claim resting on redacted evidence is a claim on credit.

Put the papers side by side and one pattern holds across every faction. The constraint is not generation. It is verification.

Zenil's collapse theorems, METR's useless AI-designed evaluations, Thorstad's borrowed growth rates and Evans's institutional protocols are all describing the same wall from different sides: producing candidates is now cheap, and deciding which candidates survive contact with the world is not. That is what a scientific institution is for. It is also, awkwardly for the futurists, the slowest thing we have ever built.

The singularity was always sold as a machine that could outthink us. The research suggests the harder threshold is a machine we can trust to tell us when it is wrong. One of those is an event. The other is an institution — and institutions are duller, slower, and the only mechanism in recorded history that has ever made intelligence cumulative rather than merely bright.

  • ai-safety
  • ai-safety
  • ai-research
  • ai-research
  • ai-forecasting
Porträt von Prof. Clara Voss
Über den Autor

Prof. Clara VossAutor ansehen

Academic Jobs In House Author

Prof. Clara Voss is a Contributing Writer at AcademicJobs, specializing in humanities and social sciences research. As a Professor of History at Humboldt University of Berlin with more than 20 years in academia, she exa…

Diskussionen

Sort von:

Seien Sie der Erste, der diesen Artikel kommentiert!

Du bist

Sie werden gebeten, sich anzumelden, bevor Ihr Kommentar veröffentlicht wird.

Browse nach Fakultät

Browse nach Thema

Frequently Asked Questions

🌀What is the technological singularity?

The technological singularity is the hypothetical point at which self-improving artificial intelligence accelerates so quickly that ordinary forecasting fails. The idea descends from I. J. Good's 1965 argument that an ultraintelligent machine could design better machines, and from Vernor Vinge's use of the physics term in 1993. Its defining claim is not that machines become smart, but that improvement feeds on itself fast enough to outrun human comprehension and oversight.

📄Which research paper changed the singularity debate most recently?

James Evans, Benjamin Bratton and Blaise Agüera y Arcas published "Agentic AI and the next intelligence explosion" in Science in March 2026. They argue that the explosion is already under way as a social phenomenon: reasoning models develop internal "societies of thought," and human-AI centaur workflows are reshaping knowledge work. Their prescription is institutional alignment rather than the alignment of a single monolithic agent.

📊What did the survey of 2,778 AI researchers find?

Published in the Journal of Artificial Intelligence Research, the survey found researchers gave a 10% chance by 2027 and a 50% chance by 2047 that unaided machines would outperform humans at every task — thirteen years earlier than the same question asked a year before. The same respondents put only a 10% chance on full automation of all occupations by 2037, and 50% as late as 2116.

🔁What is model collapse and why does it matter for the singularity?

Model collapse is the degradation that occurs when a generative model is trained repeatedly on its own output. Hector Zenil's work formalises two mechanisms: entropy decay, where rare cases disappear and diversity falls, and variance amplification, where small errors compound into drift away from reality. It matters because the singularity story requires a system that improves without human input — precisely the regime in which externally grounded signal vanishes.

⏱️What is a 50% task-completion time horizon?

METR's metric asks how long a task would take a human expert, then measures the length of task an AI agent can complete with 50% reliability. Frontier agents reached roughly 50 minutes in early 2025 and about 110 minutes by the o3 generation, with the horizon doubling roughly every seven months since 2019. It is a measure of sustained autonomous work, not of judgement or research taste.

⚙️Does the evidence show AI is accelerating its own development?

Partly, and under heavy caveats. A November 2026 METR review of Anthropic concluded the company was likely improving models at about twice its previous rate, with perhaps 80% of that acceleration attributable to internal AI use, within a range of 1.3 to 2.8 times. Most of the underlying evidence was redacted. Critics argue a one-time productivity gain from better tools is not the same as a system that independently invents new science.

📐What is the strongest philosophical objection to the singularity hypothesis?

David Thorstad's Against the Singularity Hypothesis, published in Philosophical Studies, argues that the case rests on undersupported growth assumptions. Hyperbolic models imply intelligence doubling every few months and approaching infinity in under two years. Thorstad notes that a celebrated 1960 model extrapolated population trends to predict infinite human population by 2026 — the trend was real, the projection was not.

📈Could AI automation produce explosive economic growth?

Ege Erdil and Tamay Besiroglu reviewed the arguments and concluded that explosive growth is plausible if AI broadly substitutes for human labour, because labour becomes an accumulable input for the first time. They also concluded that high confidence is unwarranted, since regulation, physical bottlenecks, alignment difficulty and the pace of automation remain unresolved.

🔄What is recursive self-improvement?

Recursive self-improvement (RSI) describes a system that proposes changes to itself, evaluates them, and retains the improvements, then repeats. The subtlety is the second half: a generated modification only counts as improvement relative to a criterion that stays meaningful as the system changes. If the same model supplies the proposal, the target and the judgement, a rising score can mean growing competence or growing self-agreement.

🔍Why does verification matter more than generation?

Because producing candidates has become cheap and deciding which ones survive contact with reality has not. Zenil's collapse results, METR's finding that no AI-designed evaluation was usable by its own researchers, and the institutional-alignment argument in Science all point at the same bottleneck. Science advances by rejecting bad ideas reliably, not by producing many of them.

🧩What is neurosymbolic AI and why is it proposed as a fix?

Neurosymbolic systems combine statistical learning with explicit symbolic structures such as programs, equations or formal rules. Zenil proposes program synthesis and algorithmic probability methods so a model identifies the generative mechanism behind data rather than the correlations in it. Frontier developers have moved partly in this direction, but it remains a research programme rather than a demonstrated route to unlimited self-improvement.

🗺️Should the singularity be treated as a forecast or a scenario?

A scenario. The AI 2027 project was widely read as a prediction, and its authors have since moved their medians more than once — into the 2030s in early 2026, then back to 2027 after new evaluation data. Scenarios are useful for mapping chains of preconditions. They are poor instruments for assigning dates, and the difference between the two uses is where most public misunderstanding begins.