Towards applications of reinforcement learning in continuous environments with Professor Adam Sobey

Increasingly, autonomous agents will be required to operate on long-term missions. This will create a demand for general intelligence because feedback from a human operator may be sparse and delayed, and because not all behaviours can be prescribed. A long-standing challenge in artificial intelligence is lifelong reinforcement learning, where learners are given many tasks in sequence and must transfer knowledge between tasks while avoiding catastrophic forgetting. We see problems with interference between similar tasks, unlearning actions we would like to see and difficulty in learning due to inductive biases. An example from Supply Chain Optimisation will be introduced, highlighting some of the issues and two approaches introduced, Active Adaptive Perception and Task Capacity which aim to improve the performance of learners in these scenarios with limited compute and reduced training data.

Biography: Adam Sobey is Programme Director for Data-Centric Engineering at The Alan Turing Institute and Chair of Data-Centric Engineering in the Maritime Engineering Group at The University of Southampton. He completed his MEng in Aerospace Engineering with Astronautics in 2006 before moving to the Maritime Engineering group to do a PhD focusing on implementing AI approaches into Leisure Boatbuilding in 2010. He joined the group as a Research Fellow in 2009 to lead the LR/MoD Centre of Excellence in Marine Structures before moving to LRET funding to explore the use of Multi-Level Selection mechanisms within Evolutionary Computation. Between 2014-2017 he was a visiting Scientist at the Institute of High Performance Computing before returning to take up a lectureship at the University of Southampton 2018, Associate Professor 2020 and Professorship 2022. In 2019 he was asked to start the Marine and Maritime group within the Data-Centric Engineering programme at The Alan Turing Institute, before becoming director in 2023. He received the Royal Institute of Naval Architects Jeom Paik award for best Paper on Structures under 30 in 2015 for work related to improving Structural Safety of Composite Ships. He is currently an Editor of Data-Centric Engineering and Associate Editor of Ship and Offshore Structures. He is also the Director of the Centre of Excellence of Data-centric Engineering for Clean Oceans at the University of Southampton, bringing together experts from across the University to provide a real world difference in decarbonisation.