Decision-Making under Uncertainty: The Priority Elicitation Problem Ref: 5523

Supervisors

Primary supervisor: Dr Huma Samin  Exeter University Computer Science

Secondary supervisors: Prof Andrew Howes Exeter University Computer Science

The University of Exeter’s Department of Computer Science is inviting applications for a PhD studentship funded by Faculty of Environmental Science and Economy  to commence on  September 2025 or as soon as possible thereafter.  For eligible students, the studentship will cover Home tuition fees plus an annual tax-free stipend of at least £19,237 for 3.5 years full-time, or pro rata for part-time study.  The student would be based in the Department of Computer Science in the Faculty of Environment, Science and Economy at the Streatham Campus in Exeter.

Computing systems increasingly operate in dynamic and uncertain environments [1]. A key application of such systems is in the Internet of Things (IoT), where networked sensors and actuators enable real-time adaptation to environmental changes. Consider a self-adaptive IoT network such as a smart home that autonomously manages energy consumption while balancing multiple, often conflicting requirements, such as comfort, cost efficiency, and sustainability. Effective decision-making in such a system requires continuously evaluating trade-offs under different environmental conditions, such as changing weather patterns, fluctuating energy costs, or variations in occupants’ schedules. 

Typically, the initial priority settings for requirements are made by domain experts during the design phase based on expected system behaviour. However, unforeseen situations may arise that challenge these initial assignments, potentially compromising the system’s ability to adapt effectively. For example, during a sudden heatwave, a smart home system must decide whether to prioritise maintaining a comfortable indoor temperature or minimising electricity costs. While the initial priority settings may emphasize energy efficiency or cost reduction, extreme environmental conditions may require a change in priorities to ensure occupant’s comfort. 

The research project aims to explore techniques that could facilitate experts in the elicitation of priorities. One possible direction could be to use the technique of Inverse Reinforcement Learning (IRL) [2], [3]. IRL is an AI-based technique that supports imitation of the preferred system behaviour by using its behavioural history. It helps in the inference of the reward values by taking the observed history of policies as input. As the priorities are represented as utilities based on the multiple rewards [4].[5]; techniques like IRL could infer these reward values and support the experts with the elicitation process. 

 References

[1]          L. Garcia, H. Samin, and N. Bencomo, ‘Decision Making for Self-Adaptation Based on Partially Observable Satisfaction of Non-Functional Requirements’, ACM Trans Auton Adapt Syst, vol. 19, no. 2, p. 11:1-11:44, Apr. 2024, doi: 10.1145/3643889.

[2]          S. Arora and P. Doshi, ‘A survey of inverse reinforcement learning: Challenges, methods and progress’, Artif. Intell., vol. 297, p. 103500, Aug. 2021, doi: 10.1016/j.artint.2021.103500.

[3]          C. A. Rothkopf and C. Dimitrakakis, ‘Preference Elicitation and Inverse Reinforcement Learning’, in Machine Learning and Knowledge Discovery in Databases, D. Gunopulos, T. Hofmann, D. Malerba, and M. Vazirgiannis, Eds., Berlin, Heidelberg: Springer, 2011, pp. 34–48. doi: 10.1007/978-3-642-23808-6_3.

[4]          C. F. Hayes et al., ‘A practical guide to multi-objective reinforcement learning and planning’, Auton. Agents Multi-Agent Syst., vol. 36, no. 1, p. 26, Apr. 2022, doi: 10.1007/s10458-022-09552-y.

[5]          H. Samin, N. Bencomo, and P. Sawyer, ‘Decision-making under uncertainty: be aware of your priorities’, Softw. Syst. Model., vol. 21, no. 6, pp. 2213–2242, Dec. 2022, doi: 10.1007/s10270-021-00956-0.

The studentship will be awarded on the basis of merit. Students who pay international tuition fees are eligible to apply, but should note that the award will only provide payment for part of the international tuition fee (~£24k) and no stipend.   

International applicants need to be aware that they will have to cover the cost of their student visa, healthcare surcharge and other costs of moving to the UK to do a PhD.

The conditions for eligibility of home fees status are complex and you will need to seek advice if you have moved to or from the UK (or Republic of Ireland) within the past 3 years or have applied for settled status under the EU Settlement Scheme.  

Entry requirements

Applicants for this studentship must have obtained, or be about to obtain, a First or Upper Second Class UK Honours degree, or the equivalent qualifications gained outside the UK, in  Computer Science.

If English is not your first language you will need to meet the English language requirements and provide proof of proficiency. Click here for more information.

How to apply

Apply now

To apply, please click the ‘Apply Now’ button above. In the application process you will be asked to upload several documents 

•             CV

•             Letter of application (outlining your academic interests, prior research experience and reasons for wishing to undertake the project).

•             Research proposal

•             Transcript(s) giving full details of subjects studied and grades/marks obtained (this should be an interim transcript if you are still studying)

•             Two references from referees familiar with your academic work. If your referees prefer, they can email the reference direct to PGRApplicants@exeter.ac.uk quoting the studentship reference number.

•             Names of two referees familiar with your academic work. You are not required to obtain references yourself. We will request references directly from your referees if you are shortlisted. 

•             If you are not a national of a majority English-speaking country you will need to submit evidence of your proficiency in English.

The closing date for applications is midnight on 21st May 2025.  Interview date TBC

All application documents must be submitted in English. Certified translated copies of academic qualifications must also be provided.

Please quote reference 5523 on your application and in any correspondence about this studentship.

Summary