Multimodal Large Language Model for Human-Centered Robots

About the Project

Supervisory Team: Dr. Tan Viet Tuyen Nguyen

This project explores the development of a Multimodal Large Language Model that empowers robots to understand and respond to humans through vision, language, and other sensory data. By enabling natural, adaptive, and context-aware communication, the research advances the next generation of intelligent, human-centered robotic systems.

Robots are increasingly being integrated into various aspects of daily life, including education, healthcare, workplaces, and personal settings. These practical applications require that robots collaborate effectively with humans in shared environments to enhance human capabilities, prioritize human needs, and seamlessly fit into everyday contexts This project aims to develop a

Multimodal Large Language Model framework that enables robots to interpret interaction contexts from multiple modalities, such as vision, language, audio, and other sensory data, and to engage users through diverse communication channels, including actions, speech, and visual outputs. The framework will allow robots to perceive multimodal signals collected from both the environment and human users during everyday interactions, build a deep contextual understanding, and respond appropriately across different modalities to achieve natural, adaptive, and human-centered interaction.

You will have the opportunity to collaborate with other researchers in the following areas:

• Agents, Interaction, and Complexity research group
• Centre for Robotics
• Responsible AI project team

You'll contribute to high-impact journals and conferences. You will have access to high-performance computers and a range of robot platforms to support your research.

Entry requirements

A UK 2:1 honours degree, or its international equivalent.

You must have solid programming skills in Python and machine learning, and a strong interest in the intersection of machine learning and robotics. Prior experience with large language models or multimodal learning would be advantageous.

Fees and funding

UK, EU and Horizon Europe students are eligible for scholarships. CSC students are eligible for fee waivers. Funding for other international applicants is very limited and highly competitive. Overseas students who have secured or are seeking external funding are welcome to apply.

How to apply

Apply now

• programme type: research
• academic year: 2026/27
• if you will be full time or part time
• faculty: Engineering and Physical Sciences
• search for programme: PhD Computer Science (7089)
• please add the name of the supervisor in section 2 of the application.

Applications should include:

• research proposal
• your CV (resumé)
• 2 academic references
• degree transcripts/ certificates to date
• English language qualification (if applicable)

Contact us

Faculty of Engineering and Physical Sciences

If you have a general question, email: feps-pgr-apply@soton.ac.uk

Project leader

For an initial conversation, email Dr. Tan Viet Tuyen Nguyen - tuyen.nguyen@soton.ac.uk