Machine Learning for Semi-Automated Evaluation of Computer Programming Assignments

About the Project

Evaluating student assignments puts a tremendous burden to teachers. Besides investigating if, and in what extent, a student has used AI, other aspects have to be evaluated as well, such as creativity and/or originality, just to name a few. Another issue, especially in case of evaluating many assignments, is the inconsistencies in evaluation when the grades for two very similar solutions are too much different. While the use of Large Language Models (LLMs) seems a straightforward solution, one might consider several aspects: First, LLMs are not always precise and, even more concerning, their results is not easy to explain. Second, assignments should be treated confidentially and, in some cases, might even contain sensitive information (e.g. an essay about the student's family), thus, offline models runnable on the teacher's PC would be preferred. Finally, the environmental footprint of such models is also not negligible.

The main objective of the project are the following:

1. Develop novel Machine Learning (ML) models for Semi-Automated Student Assignment Evaluation, particularly focused on computer programming assignment, and integrate them into a software application.
2. Evaluate and test the developed framework in real use-case scenarios from university programming courses at Edinburgh Napier University.

Academic qualifications

Have, or expect to achieve by the time of start of the studentship a first-class honours degree, or a distinction at master level, ideally in Computer Science with a good functional knowledge of Artificial Intelligence, Data Science.

English language requirement

IELTS score must be at least 6.5 (with not less than 6.0 in each of the four components). Other, equivalent qualifications will be accepted. Full details of the University’s policy are available online.

Essential attributes:

• Deep Theoretical background on Machine Learning
• Good Programming skills (not just using libraries)

Desirable attributes:

• Knowledge in software development

APPLICATION CHECKLIST

• Completed application form
• CV
• 2 academic references, using the Postgraduate Educational Reference Form (download)
• Research project outline of 2 pages (list of references excluded). The outline may provide details about:
  1. Background and motivation of the project. The motivation, explaining the importance of the project, should be supported also by relevant literature. You can also discuss the applications you expect for the project results.
  2. Research questions or objectives.
  3. Methodology: types of data to be used, approach to data collection, and data analysis methods.
  4. List of references.

The outline must be created solely by the applicant. Supervisors can only offer general discussions about the project idea without providing any additional support.

• Statement no longer than 1 page describing your motivations and fit with the project.
• Evidence of proficiency in English (if appropriate)

To be considered, the application must use

• the advertised title as project title

For informal enquiries about this PhD project, please contact Prof Tomás Horváth - t.horvath@napier.ac.uk