| Unit name | Learning in Autonomous Systems |
|---|---|
| Unit code | COMSM0305 |
| Credit points | 10 |
| Level of study | M/7 |
| Teaching block(s) |
Teaching Block 1 (weeks 1 - 12) |
| Unit director | Dr. Tim Kovacs |
| Open unit status | Not open |
| Pre-requisites |
None |
| Co-requisites |
COMSM0300 |
| School/department | Department of Computer Science |
| Faculty | Faculty of Engineering |
This unit introduces the reinforcement learning paradigm, in which an autonomous agent interacts with a task environment and learns only from rewards and punishments. Training a dog is an appropriate analogy, and applications range from learning control systems for autonomous agents, to optimisation of industrial processes, to modelling human and animal cognition. We are concerned with computational methods for solving such tasks, and cover both their theoretical foundations and implementation. Three fundamental approaches - dynamic programming, temporal difference and monte carlo methods - are covered, and their relationship developed.
There is no text for the EC part of the unit. The handouts are based on different sources and usually indicate what they are. Some are based in part on chapter 2 of:
C.R. Reeves and J.E. Rowe Genetic Algorithms - Principles and Perspectives: A Guide to GA Theory, 2003. [library: https://mirakc.lib.bris.ac.uk/F?func=find-c&ccl_term=ISBN=1402072406] [google preview: http://books.google.co.uk/books?id=7Gy3yevFd0kC&printsec=frontcover&source=gbs_ViewAPI&redir_esc=y] [errata: https://www.cs.bris.ac.uk/Teaching/Resources/COMSM0305/GaTheoryErrata.pdf]
There are a number of books suitable for background reading, including:
Chapters 1-3 of: A.E. Eiben and J.E. Smith. Introduction to Evolutionary Computing, 2003. [library: https://mirakc.lib.bris.ac.uk/F?func=find-c&ccl_term=ISBN=0262631857] [google preview: http://books.google.co.uk/books?id=RRKo9xVFW_QC&printsec=frontcover&source=gbs_ViewAPI&redir_esc=y]
The introductory chapters of: Melanie Mitchell. An introduction to genetic algorithms, 1998. [library: https://mirakc.lib.bris.ac.uk/F?func=find-c&ccl_term=ISBN=0262631857] (Wider coverage than most.)
The introductory chapters of: David E. Goldberg Genetic algorithms in search, optimization and machine learning, 1989. [library: https://mirakc.lib.bris.ac.uk/F?func=find-c&ccl_term=ISBN=0201157675] (Old, but a gentle introduction.)
The textbook for the RL part of this unit is:
R. Sutton and A. Barto. Reinforcement Learning. An Introduction. The MIT Press, 1998. ISBN: 0-262-19398-1. [library: https://mirakc.lib.bris.ac.uk/F?func=find-c&ccl_term=ISBN=0262193981] [HTML and scanned versions: http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html] [errata: http://webdocs.cs.ualberta.ca/~sutton/book/errata.html]
