About the Project
Background
The rapid rise of LLMs and implementation of chatbots mean that applications such as ChatGPT, Claude and Gemini are accessible to businesses, organisations and the broader public.
In the context of weather services, the use of AI chatbots have been discussed as a potential way to provide personalised and tailored information to users. In the context of climate information LLMs have been explored as a way to summarise climate information for a non-technical audience.
However, this raises questions about the extent to which the outputs of prompts and queries capture uncertainties and how this influences how these outputs are interpreted and used to inform both general understanding of climate change and adaptation planning.
This project will use insights from data and behavioural sciences to examine how LLM output aligns with current expert understanding of climate and the uncertainties associated with climate information, as well as how it might be used to inform decision making.
PhD project
People are increasingly using AI in the workplace, including in senior decision-making roles. However, people may be using AI as an input in contexts where they are not a domain expert, and users may have limited understanding of how products like Large Language Models (LLMs) actually work, or their limitations. While the potential to use specialist chatbots to provide user-relevant climate information based on credible sources such as IPCC AR6 has been highlighted (Vaghefi et al., 2023), LLM output may be inconsistent, inaccurate, contain irrelevant information and vary depending on user history.
This project aims to identify the practical usefulness and limitations of LLMs in the context of climate information and communication about climate, particularly in relation to the communication or representation of uncertainty. A second aim is to identify the training or programmes that can help overcome the limitations to the greatest possible extent. The project will draw on data science and behavioural science themes, investigating the human interaction and exploring options to improve outcomes and avoid harms from misinformation.
As a starting point the project could explore these issues:
- what does AI generate when asked about climate forecasts and/or projections for specific regions in general terms?
- what sources does it prioritise?
- how do commonly used AI application responses characterise uncertainty?
- Is the information provided accurate?
- How does the information generated change with varying prompt engineering and specificity of user context?
An important issue will be establishing the extent of inaccurate information generated and the probability of errors that could adversely affect the decision being taken. Another issue is how users respond to different LLM outputs (e.g. do they check references provided, attribution of credibility?).
The findings will be useful for decision makers using LLMs in their work or whose teams use LLMs.
Applicant Profile
Students with an interest in the application of AI for communication in real-world contexts. Some experience of data science or a related field would be an advantage.
Funding Notes
This project is part of the UNRISK CDT, which offers 15-18 fully-funded NERC studentships, covering full university tuition fees; a personal stipend at standard UKRI rates; £6000 individual research and training costs; £5000 (per student) of cohort-level training; and a ‘Flexible Fund’ for special projects.
International applicants will need to cover costs related to applying for a student visa and the international health surcharge (IHS)
Applications are open to UK and international applicants. The number of awards for international applicants is limited by UKRI rules.
More information is available on the UNRISK website: View Website
References
- https://business.leeds.ac.uk/research-cdr/staff/357/professor-barbara-summers
- https://business.leeds.ac.uk/departments-analytics-technology-operations/staff/358/dr-andrea-taylor
- https://www.nature.com/articles/s43247-023-01084-x
- https://business.leeds.ac.uk/faculty/dir-record/research-projects/2296/an-integrative-approach-to-the-world-risk-poll-analysing-risk-perception-and-preparedness-to-develop-effective-disaster-risk-communication-strategies
- https://cdr.leeds.ac.uk/
Project supervisors
Professor Barbara Summers
Career overview
Professor Barbara Summers is a Professor of Human Judgment and Decision Making at Leeds University Business School. She holds a PhD and an MBA from the University of Bradford, as well as a BSc (Hons.) in Genetics from the University of Leeds. Her expertise lies in individual decision making, and her research employs both cognitive and emotional perspectives, utilising psychological and analytical methods. Professor Summers has applied her research across various domains, including finance, health, and public perceptions regarding severe weather. She has collaborated with experts and organisations in these fields both in the UK and internationally. Before her academic career, Professor Summers gained commercial experience as Head of Systems Development at Equifax Europe UK. She is an invited member of the Institute for Credit Management, the UK professional body for credit management. Her research has been published in peer-reviewed journals and has had a significant impact on policy, notably informing the UK Department for Work and Pensions and NEST, the pension provider established under the Pensions Act 2008. Additionally, her Knowledge Transfer project with Clydesdale Bank received an award from the UK Economic and Social Research Council. Recent research projects led by Professor Summers include analysing risk perception and preparedness to enhance disaster risk communication strategies, identifying musical talent using AI, and exploring financial decision making related to consumer debt and gambling. She has also investigated moral reasoning and motivations within the context of taxation. Professor Summers has served as the President of the European Association for Decision Making (EADM) and has contributed to the development of various academic programmes, including serving as Programme Director for MSc Management and BSc Business Analytics.
Research interests
Professor Summers' research focuses on individual decision making from both cognitive and emotional perspectives, employing psychological and analytical methods. Her work has applications in various domains, including finance, health, and public perceptions and responses to severe weather. She collaborates with experts and organisations in these fields both in the UK and internationally. Professor Summers' research has informed significant initiatives, such as the UK Department for Work and Pensions and NEST, the pension provider established under the Pensions Act 2008. Her recent projects include analysing risk perception and preparedness to develop effective disaster risk communication strategies, identifying musical talent using AI, financial decision making related to consumer debt and gambling, and exploring moral reasoning and motivations in the context of taxation. She has also received recognition for her Knowledge Transfer project with Clydesdale Bank, which won an award from the UK Economic and Social Research Council.
Dr Andrea Taylor
Career overview
Dr Andrea Taylor holds joint posts at Leeds University Business School and the School of Earth and Environment. She has a background in cognitive psychology and is a behavioural decision researcher with over a decade of experience in teaching in higher education. Dr Taylor's research interests encompass risk perception and communication, climate and weather risk preparedness, and expert judgement. Her work primarily focuses on applying insights and methodologies from the behavioural sciences to tackle communication challenges in providing climate and weather information services, as well as supporting decision-making under uncertainty. To address these challenges, she collaborates with experts across various disciplines, including climatology, meteorology, environmental science, health economics, and risk management. Current research projects include examining public responses to severe weather warnings in the UK, developing weather forecasting science and services in West and East Africa, creating climate information tools for water resource management, and enhancing the communication of uncertainty in warning systems. Dr Taylor has also contributed to various leadership roles, including being a joint lead of the Business and Organisations for Sustainable Societies (BOSS) Research Group and the World Meteorological Organisation’s High Impact Weather (HIWeather) Task Team on Communication. She is also a joint lead of the University of Leeds Met Office Partnership Theme on Social Science. Dr Taylor's academic qualifications include a PhD in Psychology from the University of Bolton, an MSc in Psychology (distinction) from the University of Bolton, and a BSc (Hons) in Psychology (2(i)) from the University of Lancaster. She is a Fellow of the Higher Education Academy and an Associate Fellow of the Higher Education Academy. Additionally, she is a member of the Priestley Centre for Climate and the European Association for Decision Making. Dr Taylor is interested in supervising PhD projects in areas such as risk communication, weather and climate information services, decision-making in the context of climate change adaptation, and the role of emotion in decision-making under risk.
Research interests
Dr Andrea Taylor's research focuses on behavioural decision making, risk perception and communication, decision making under uncertainty, and weather and climate risk. She applies insights and methodologies from the behavioural sciences to tackle communication challenges in the provision of climate and weather information services, thereby supporting decision making under uncertainty. Dr Taylor collaborates with experts across various disciplines, including climatology, meteorology, environmental science, health economics, and risk management. Current research projects include examining public responses to severe weather warnings in the UK, developing weather forecasting science and services in West and East Africa, creating climate information tools for water resource management, and enhancing the communication of uncertainty in warning systems.