Real-Time Interaction Quality Evaluation and Adaptive Strategies for Socially Interactive Agents

About the Project

Socially Interactive Agents, such as digital humans, embodied conversational agents, and virtual avatars, are increasingly used in education, healthcare, entertainment, and service contexts. However, current evaluation methods for human-agent interaction still rely heavily on post-interaction questionnaires and subjective feedback. These methods are useful but limited: they cannot capture the dynamic changes that occur during live interaction, nor can they provide immediate feedback for agents to adjust their behaviour.

This PhD project aims to develop a real-time, multi-dimensional framework for evaluating interaction quality between humans and Socially Interactive Agents. The project will investigate how users perceive agent behaviours, how interaction quality can be operationalised and quantified, and how multimodal behavioural signals can be used to support real-time evaluation. Relevant signals may include facial expressions, gaze, gestures, speech, turn-taking patterns, response timing, and physiological or sensor-based signals.

The project will combine Human-Computer Interaction, Human-Agent Interaction, multimodal machine learning, behavioural data analysis, and interactive system development. The successful candidate will contribute to designing user studies, collecting and analysing multimodal interaction data, developing computational models for interaction quality assessment, and exploring how real-time evaluation outputs can support adaptive agent behaviours. Potential adaptive strategies may include adjusting gaze patterns, gesture frequency, speech tone, response timing, interruption strategies, and dialogue pacing.

The expected outcome of this project is a research framework and prototype system that enables Socially Interactive Agents to monitor interaction quality during live conversations and adapt their behaviours according to user responses and interaction context. This work has the potential to support the development of more natural, trustworthy, and user-centred interactive agents in real-world applications. The project is based on a proposal focusing on real-time interaction quality evaluation, multimodal behavioural signals, and adaptive SIA strategies .

Candidate Profile:

This project is technically demanding and interdisciplinary. We are looking for a candidate with strong evidence of technical competence, research potential, and willingness to work across AI and HCI.

Applicants should have a background in computer science, artificial intelligence, data science, human-computer interaction, cognitive science, psychology, or a closely related discipline. Essential skills include strong programming ability, preferably in Python, and prior training or project experience in machine learning, deep learning, data mining, or data-driven modelling.

Applicants should also have experience processing at least one type of complex data, such as video, audio, behavioural logs, sensor data, physiological signals, time-series data, or questionnaire data. Basic knowledge of quantitative data analysis, statistical testing, model evaluation, and academic writing in English is expected.

Experience in one or more of the following areas would be highly desirable: multimodal learning, affective computing, human-agent interaction, embodied conversational agents, digital humans, real-time sensing systems, Unity or Unreal Engine, MediaPipe, OpenFace, speech/audio analysis, motion capture, physiological signal processing, experimental design, or mixedmethods HCI research.

Applicants should be prepared to conduct human-subject studies, including participant recruitment, experimental procedure design, questionnaire or interview design, behavioural data analysis, and research ethics preparation.

Interested applicants are welcome to contact liu.yang@xjtlu.edu.cn for an initial enquiry with a CV, academic transcript, and evidence of previous work, such as a portfolio, GitHub repository, technical report, research paper, or thesis.

For more information about doctoral scholarship and PhD programme at Xi’an Jiaotong Liverpool University (XJTLU), please visit:

• https://www.xjtlu.edu.cn/en/admissions/doctoral/entry-requirement-phd
• https://www.xjtlu.edu.cn/en/admissions/doctoral/postgraduate-research-scholarships

Supervisors:

• Principal supervisor: Professor/Dr Liu YANG (XJTLU)
• Co-supervisor: Professor/Dr Jionglong Su (XJTLU)
• Co-supervisor: Professor/Dr Anh Nguyen (UoL)

Requirements:

A Master's degree with Merit and a Bachelor's degree with first-class or upper second-class honors are required for PhD admissions. Exceptional candidates holding only a Bachelor's degree may be considered on an individual basis in certain disciplines.

Evidence of good spoken and written English is essential. The candidate should have an IELTS (or equivalent) score of 6.5 or above, if the first language is not English. This position is open to all qualified candidates irrespective of nationality.

Degree:

The student will be awarded a PhD degree from the University of Liverpool (UK) upon successful completion of the program.

Funding:

The PhD studentship is available for three years subject to satisfactory progress by the student. The award covers tuition fees for three years (currently equivalent to RMB 99,000 per annum). It also provides up to RMB 16,500 to allow participation at international conferences during the period of the award. The scholarship holders are expected to conduct the majority of their research at XJTLU in Suzhou, China. However, they may apply for a short-term research visit to the University of Liverpool if the project requires it.

How to Apply:

Interested applicants are advised to email Liu.yang@xjtlu.edu.cn the following documents for initial review and assessment (please put the project title in the subject line):

• CV
• Two formal reference letters
• Personal statement outlining your interest in the position
• Certificates of English language qualifications (IELTS or equivalent)
• Full academic transcripts in both Chinese and English (for international students, only the English version is required)
• Verified certificates of education qualifications in both Chinese and English (for international students, only the English version is required)
• PDF copy of Master Degree dissertation (or an equivalent writing sample) and examiners reports available

Contact:

Please email liu.yang@xjtlu.edu.cn with a subject line of the PhD project title.

The principal supervisor’s profile is linked here: https://scholar.xjtlu.edu.cn/en/persons/LiuYang/