Phase Transitions and Emergent Behaviour in Large Language Models

About the Project

These projects are open to students worldwide, but have no funding attached. Therefore, the successful applicant will be expected to fund tuition fees at the relevant level (home or international) and any applicable additional research costs. Please consider this before applying.

Large Language Models (LLMs) such as GPT and Claude represent some of the most complex computational systems ever built—yet, despite their ubiquity, their dynamical behaviour remains poorly understood. This PhD project will explore LLMs as nonlinear dynamical systems, investigating whether they exhibit phase transitions—sharp, qualitative changes in behaviour as a function of control parameters such as temperature, context length, reward structure, or fine-tuning regime.

Analogous to critical phenomena in physics, small perturbations in these parameters can sometimes cause abrupt shifts in model responses: from cooperative to adversarial dialogue, from truthful to hallucinatory behaviour, or from exploration to exploitation. Understanding these thresholds could help us characterise “order parameters” for LLMs—quantities that summarise their macroscopic behaviour—and identify critical points where qualitative transitions occur.

The project will combine nonlinear dynamics, information theory, and computational experimentation. Using both open-source and proprietary LLMs, the student will perform controlled experiments that vary temperature, randomness, reward incentives, and context complexity to quantify behavioural shifts. Techniques from statistical physics—such as bifurcation analysis, entropy measures, correlation length, and susceptibility—will be adapted to high-dimensional model responses. We aim to construct phase diagrams describing regions of stability, chaos, and transition in the model’s behaviour.

Complementary to empirical exploration, the project will develop theoretical models inspired by mean-field and agent-based approaches to describe how internal representations or token-level dynamics give rise to emergent, macroscopic patterns. This will link modern machine learning to classical ideas from dynamical systems and critical phenomena, providing a bridge between physics-inspired theory and AI practice.

Potential extensions include:

• studying collective behaviour in ensembles of LLMs interacting as agents (e.g. consensus, polarisation, and cooperation transitions);
• analysing the sensitivity of model decisions to small prompt perturbations or reinforcement signals;
• exploring analogues of hysteresis and metastability in model output distributions.

The project suits students with backgrounds in mathematics, physics, computer science, or data science, who are interested in combining analytical reasoning with computational experiments. Experience in programming (Python or Wolfram Language) and familiarity with nonlinear systems, information theory, or AI models will be advantageous.

Ultimately, this work seeks to establish a science of LLM behaviour, revealing whether phenomena familiar from condensed-matter physics—criticality, universality, and bifurcation—also govern the emergent dynamics of intelligent systems.

Informal enquiries can be made by contacting Prof M Thiel (m.thiel@abdn.ac.uk)

Decisions will be based on academic merit. The successful applicant should have, or expect to obtain, a UK Honours Degree at 2.1 (or equivalent) in Physics .

We encourage applications from all backgrounds and communities, and are committed to having a diverse, inclusive team.

Application Procedure:

Formal applications can be completed online: https://www.abdn.ac.uk/pgap/login.php.

You should apply for Degree of Doctor of Philosophy in Physics to ensure your application is passed to the correct team for processing.

Please clearly note the name of the lead supervisor and project titleon the application form. If you do not include these details, it may not be considered for the project.

Your application must include: A personal statement, an up-to-date copy of your academic CV, and clear copies of your educational certificates and transcripts .

Please note: you do not need to provide a research proposal with this application.

If you require any additional assistance in submitting your application or have any queries about the application process, please don't hesitate to contact us at researchadmissions@abdn.ac.uk

Funding Notes

This is a self-funding project open to students worldwide. Our typical start dates for this programme are February or October.

Fees for this programme can be found here Finance and Funding | Study Here | The University of Aberdeen.

Additional Research costs of £500 will be required for this project, in addition to tuition fees.

References

1. Wei, J. et al. (2022). “Emergent Abilities of Large Language Models.” Transactions on Machine Learning Research (TMLR).
2. Arnold, J., Holtorf, F., Schäfer, F., Lörch, N. (2024). “Phase Transitions in the Output Distribution of Large Language Models.” arXiv:2405.17088.