Small Language Models

These projects are open to students worldwide, but have no funding attached. Therefore, the successful applicant will be expected to fund tuition fees at the relevant level (home or international) and any applicable additional research costs. Please consider this before applying.

This project aims to build compact models that achieve LLM-level accuracy while being efficient enough for everyday devices. Instead of running all layers uniformly, the models will learn to adaptively skip or shorten computation, using full depth only when needed. Teacher–student distillation and lightweight compression will help preserve accuracy while reducing size and cost.

We will design these models with edge hardware in mind, focusing on balancing accuracy, speed, and energy efficiency. The outcome will be practical small language models that retain the power of large ones but can run locally on phones, wearables, and embedded systems, enabling private, low-latency, and reliable AI on the edge.

Informal enquiries can be made by contacting Dr Y Huang (yongchao.huang@abdn.ac.uk)

Decisions will be based on academic merit. The successful applicant should have, or expect to obtain, a UK Honours Degree at 2.1 (or equivalent) in Computing Science. Candidates should possess a solid foundation in machine learning and be proficient in Python and deep learning frameworks (e.g. PyTorch, TensorFlow). An interest in model compression and information theory is highly beneficial.

We encourage applications from all backgrounds and communities, and are committed to having a diverse, inclusive team.

Application Procedure:

Formal applications can be completed online: https://www.abdn.ac.uk/pgap/login.php.

You should apply for Degree of Doctor of Philosophy in Computing Science to ensure your application is passed to the correct team for processing.

Please clearly note the name of the lead supervisor and project titleon the application form. If you do not include these details, it may not be considered for the project.

Your application must include: A personal statement, an up-to-date copy of your academic CV, and clear copies of your educational certificates and transcripts.

Please note: you do not need to provide a research proposal with this application.

If you require any additional assistance in submitting your application or have any queries about the application process, please don't hesitate to contact us at researchadmissions@abdn.ac.uk

Funding Notes

This is a self-funding project open to students worldwide. Our typical start dates for this programme are February or October.

Fees for this programme can be found here Finance and Funding | Study Here | The University of Aberdeen