About the Project
A fully funded PhD opportunity to participate in the world-leading research undertaken by the EPSRC Doctoral Landscape Award at the University of Sheffield.
Supervised by Dr Victor-Alexandru Darvariu and Prof George Panoutsos.
The University of Sheffield is one of the leading Russell Group universities in the UK. We carry out cutting-edge research with strong links to industry. When you enrol to do a PhD with us, you will be working with world-leading academics and have access to top of the range facilities. As a PhD student you will have the opportunity to gain skills not only to conduct research, but also to take your career to the next level, whether you want to stay in academia, go into industry or the public sector, or set up your own company. You will have access to a range of training and support services to help you excel in your studies and beyond.
Project Description
Artificial intelligence (AI) is increasingly being used to solve complex decision-making problems. One of its most powerful approaches is reinforcement learning, where a computer learns what actions to take by interacting with a system and receiving feedback (called rewards) based on how well it performs. Reinforcement learning has been used in areas ranging from robotics and games to the tuning of large language models. It also has growing potential in engineering and science since, under the hood, many problems involve making a sequence of decisions under uncertainty.
This project focuses on problems that have a network (“graph”) structure. A graph represents a system as a collection of objects and the relationships between them: for example, machines connected through a manufacturing process, stations connected through links in a transportation network, or components interacting in a scientific system. Many real-world engineering problems naturally take this form.
At present, applying reinforcement learning to such problems usually requires a human expert to translate the real-world system into a precise mathematical description. This creates a major barrier. Domain experts may have large amounts of data about a system but may not have the specialist expertise in AI needed to formulate the system in the way reinforcement learning algorithms require. Furthermore, hand-designed mathematical descriptions may fail to capture complex system dynamics and reflect important patterns that are present in the data.
A promising alternative is to build a “world model”: an AI model that learns the behaviour of a system directly from observations of it. Rather than relying entirely on human-designed formulations, the world model learns an internal representation of the system and how it changes over time. World models are attracting considerable attention in recent AI research, particularly for visual environments such as robotics and game-playing. However, existing approaches are largely designed for image inputs and do not naturally capture the complex relationships found in graph-structured systems.
The first aim of this PhD will therefore be to develop new world model architectures specifically for graph-structured data. The central challenge is to learn representations that simultaneously preserve relational structure, model how graph structure and features evolve, and retain the relevant information required for downstream decision-making. The second will be to investigate how these learned representations can best be used for decision-making. For example, should the learned model translate observations into a conventional mathematical description, or should reinforcement learning take place directly inside the learned representation? The project will develop and systematically compare these different approaches in terms of performance, data efficiency, generalisation, and computational costs.
The final stage of the project will demonstrate the methods on a real engineering challenge. The initial application will be the scheduling of batch manufacturing processes, where decisions must be made about which tasks should run on which pieces of equipment and when. Better decision-making in such systems can improve the use of equipment, energy, time and other resources.
The project sits at the intersection of AI, machine learning, and engineering. It would suit a student interested in developing fundamental AI methods while also testing them on challenging real-world problems.
How to apply
Interested candidates are strongly encouraged to contact the project supervisors to discuss your interest in and suitability for the project prior to submitting your application.
Please refer to the EPSRC DLA webpage for detailed information about the EPSRC DLA and how to apply.
Applicants must ensure that they use the correct application form linked to the EPSRC DLA webpage and must submit an application for October 2027 entry. The form is unique to the EPSRC DLA. It contains the list of advertised projects from which to select, and three questions specific to the DLA.
The application deadline is 17:00 GMT 13 January 2027.
The majority of our DLA studentships will be allocated to candidates who qualify for Home fee status, in accordance with EPSRC funding regulations which place strict limits on international recruitment.
Funding Notes
The award will fund the full tuition fee and a maintenance stipend at the UKRI rate (currently £21,805 per annum) for 3.5 years, as well as a research grant to support costs associated with the project.
