About the Project
AI_CDT_DecisionMaking
Details
As Deep Neural Networks (DNNs) continue to drive the success of AI, there has been a growing concern regarding the ethical and legal use of DNNs in sensitive tasks (e.g., healthcare). These concerns have inspired the development of interpretable-by-construction neural architectures that explain their predictions using “high-level concepts” (e.g., “has fever”, “has hypoxia”). Concept learning models (e.g., CBMs [1] and CEMs [2]) are a recent example of such neural architectures where a DNN is…
trained first to learn a set of concepts that represent the activation or inactivation of known concepts and then use those to predict a task of interest. The utility of these models arises from the fact that at inference time they first predict a set of concepts whose semantical alignment can be used to explain the concept model’s output prediction (e.g., if the model predicted that a knee’s x-ray shows severe osteoarthritis, a possible concept explanation is that the concepts “bone spurs” and “osteophytes” are present in the input image). Moreover, these types of explanations enable very powerful concept interventions, where a human-in-the-loop is able to correct mispredicted concepts influencing the prediction and accuracy of the model.
This project is part of this emerging field of eXplainable AI (XAI), which seeks to improve the understanding of complex-yet-powerful machine learning models, like DNNs. While XAI has made progress by introducing techniques that shed some light on model predictions [3] (e.g., highlighting the pixels on a knee X-ray that influenced a model to predict a grade of arthritis), these methods still suffer from significant limitations. They are often inconsistent [4], providing different explanations with each iteration, or incorrect, with models fabricating answers [5]. To advance XAI, in this project we address the following key question:
* How can we create interactive AI decision-making systems that not only incorporate human-in-the-loop feedback but also dynamically adapt to evolving user inputs and queries?
The goal is to devise XAI systems that empower real-time human-AI collaboration in realistic setups, while providing explanations tailored to user needs. The challenge lies in balancing the need for rapid, interpretable model updates with the complexities of integrating potentially faulty human feedback effectively, ensuring the explanations are both meaningful and actionable without causing cognitive overload for users. Building on our novel Intervention-aware CEM (IntCEM) [6] mechanisms in concept-learning, the proposed innovations include creating methods that better enable users to provide real-time feedback to a model by correcting missteps in its reasoning process while considering some of the limitations and challenges of real-world feedback (e.g., human mistakes, uncertainty, adversarial inputs, etc). Additionally, the aim is to develop models that incorporate causal reasoning, producing verifiable explanations that clarify not only associations, but also the underlying reasons for decisions made by the model. The envisioned deliverable is a system that produces interactive and dynamic explanations that enable users to query, challenge, and refine model outputs in real-time
Student Desirable Background:
machine learning, DNNs, LLMs, strong maths
By submitting an application for a University of Cambridge project, you consent to your documents being shared with the relevant supervisory team at the partner institution. If you have questions about this process aidecisionscdt@manchester.ac.uk before applying.
How to apply:
Please apply under the University of Manchester application portal and select 'PhD in Artificial Intelligence' (OAA Application form)
When applying, you must include the project’s full name, your supervisor’s name, funding status, academic history, and two referees’ contact details. Applications with missing documents will not be processed. Late or incomplete submissions will not be considered, and the university is not responsible for missed deadlines.
After you have applied you will be asked to upload the following supporting documents:
- Final Transcript and certificates of all awarded university level qualifications
- Interim Transcript of any university level qualifications in progress
- CV
- Supporting statement: A mandatory one- to two-page statement explaining your motivation for postgraduate research, why you chose Manchester, relevant academic or work experience, key research findings, and the techniques or skills you've gained. Applications will be put on hold if this is not provided.
- Contact details for two referees (Please ensure referee email addresses are official university or workplace accounts, as we may need to verify the references.)
- English Language Certificate (if applicable)
Equality, diversity and inclusion is fundamental to the success of The University of Manchester, and is at the heart of all of our activities. We know that diversity strengthens our research community, leading to enhanced research creativity, productivity and quality, and societal and economic impact. (Equality, diversity and inclusion | The University of Manchester)
We actively encourage applicants from diverse career paths and backgrounds and from all sections of the community, regardless of age, disability, ethnicity, gender, gender expression, sexual orientation and transgender status.
We also support applications from those returning from a career break or other roles. We are dedicated to supporting work-life balance and offer flexible working arrangements to accommodate individual needs. Our selection process is free from bias, and we are committed to ensuring fair and equal opportunities for all applicants.
We are dedicated to supporting work-life balance and offer flexible working arrangements to accommodate individual needs, including a part-time study option.
Join us and help grow the UK’s national capability in artificial intelligence, part of the UK’s modern industrial strategy. Everyone is welcome if you are passionate about uncertainty quantification, decision making with humans in the loop and decision making for machine learning systems and their applications in Science and Engineering, and looking to develop your research and innovation skills at the doctoral level to further your career in the AI sector.
Funding Notes
The UKRI AI CDT in Decision Making for Complex Systems is part of the TechFirst offering stipends of £30,000 for students eligible for home fee status. The start date is September 2027.
The Project will be based in University of Cambridge.
References
- Koh, Pang Wei, et al. "Concept bottleneck models." International Conference on Machine Learning. PMLR, 2020.
- Espinosa Zarlenga, Mateo, et al. "Concept embedding models: Beyond the accuracy-explainability trade-off." Advances in Neural Information Processing Systems 35 (2022): 21400-21413.
- M. T. Ribeiro, S. Singh, and C. Guestrin. Why Should I Trust You?: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 1135–1144. ACM, 2016.
- J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim. (2018). Sanity checks for saliency maps. Advances in neural information processing systems, 31.
- R. Azamfirei, S. R. Kudchadkar, and J. Fackler. Large language models and the perils of their hallucinations. Critical Care, 27(1):1–2, 2023.
- M. Espinosa Zarlenga, K. Collins, K. Dvijotham, A. Weller, Z. Shams, and M. Jamnik. Learning to receive help: Intervention-aware concept embedding models. In A. Oh, et al., editors, Advances in Neural Information Processing Systems, NeurIPS, 2023.