Surgical AI Copilot: Multimodal LLM Agent in Minimally Invasive Surgery

About the Project

In modern minimally invasive surgery, clinicians face overwhelming streams of real-time data. Integrating preoperative plans with live video, intraoperative imaging, and patient vitals places immense cognitive strain on the surgical team, impacting procedural precision. Anticipating surgical events and potential complications by holistically processing this multifaceted data is crucial for enhancing patient safety and optimising outcomes. However, current static AI models struggle with the dynamic nature of the operating room and lack the capacity for adaptive, interactive guidance.

This project will develop a Surgical AI Copilot, powered by a novel multimodal LLM agent, designed for seamless collaboration with the surgical team. The copilot will act as an intelligent orchestrator, fusing data streams from preoperative scans, live video, and physiological monitors to build a comprehensive, predictive model of the procedure. Beyond simple visualisation, it actively anticipates upcoming surgical steps, forecasts complications like bleeding or proximity to critical nerves, and synthesises this predictive insight into clear, actionable guidance.

A key innovation is its ability to deliver personalised, context-aware recommendations, safeguarded by robust internal checks. This is enabled through an interactive dialogue system, allowing surgeons to query the agent for adaptive support or request on-demand visual support. Development will leverage diverse surgical datasets, with performance rigorously evaluated in high-fidelity simulations. A safety-first design, featuring confidence monitoring and mandatory human verification for critical alerts, is paramount to ensure clinical reliability.

While engineered for broad applicability, its capabilities will be demonstrated in complex image-guided surgery, focusing on high-stakes oncological procedures for brain, kidney, and prostate tumours. The student will gain unique expertise in real-time computer vision, multimodal LLM orchestration, AI explainability, and clinical AI safety. This work seeks to transform surgical guidance from reactive to proactive, enhancing patient safety and pioneering the future of human-AI collaboration in the operating room.

Eligibility

Applicants must have obtained or be about to obtain a minimum Upper Second class UK honours degree, or the equivalent qualifications gained outside the UK, in a relevant discipline.

Before you Apply

Applicants must make direct contact with preferred supervisors before applying. It is your responsibility to make arrangements to meet with potential supervisors, prior to submitting a formal online application.

How to Apply

To be considered for this project you MUST submit a formal online application form – on the application form select PhD Biomedical Imaging Sciences Programme. Full details on how to apply can be found on the Website: How to apply for postgraduate research at The University of Manchester

If you have any queries regarding making an application please contact our admissions team FBMH.doctoralacademy.admissions@manchester.ac.uk

Equality, Diversity and Inclusion

Equality, diversity and inclusion is fundamental to the success of The University of Manchester, and is at the heart of all of our activities. The full Equality, diversity and inclusion statement can be found on the website: Equality, diversity and inclusion (EDI | Postgraduate Research | Biology, Medicine and Health | University of Manchester)

Funding Notes

Applications are invited from self-funded students. This project has a Band Standard fee. Details of our different fee bands can be found on our website View Website