In this course, you will learn about several algorithms that can learn near optimal policies based on trial and error interaction with the environment---learning from the agent’s own experience. Learning from actual experience is striking because it requires no prior knowledge of the environment’s dynamics, yet can still attain optimal behavior. We will cover intuitively simple but powerful Monte Carlo methods, and temporal difference learning methods including Q-learning. We will wrap up this course investigating how we can get the best of both worlds: algorithms that can combine model-based planning (similar to dynamic programming) and temporal difference updates to radically accelerate learning.
Ce cours fait partie de la Spécialisation Apprentissage par renforcement
Sample-based Learning Methods
Offert par
Université de l'Alberta
Alberta Machine Intelligence Institute
À propos de ce cours
76,160 consultations récentes
Compétences que vous acquerrez
Artificial Intelligence (AI)Machine LearningReinforcement LearningFunction ApproximationIntelligent Systems
100 % en ligne
Commencez dès maintenant et apprenez aux horaires qui vous conviennent.
Cours 2 sur 4 dans le
Dates limites flexibles
Réinitialisez les dates limites selon votre disponibilité.
Niveau intermédiaire
Probabilities & Expectations, basic linear algebra, basic calculus, Python 3.0 (at least 1 year), implementing algorithms from pseudocode
Approx. 18 heures pour terminer
Recommandé : 4-6 hours/week
Anglais
Sous-titres : Anglais
Programme du cours : ce que vous apprendrez dans ce cours
1 heure pour terminer
Welcome to the Course!
Welcome to the second course in the Reinforcement Learning Specialization: Sample-Based Learning Methods, brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, and get a flavour of what the course has in store for you. Make sure to introduce yourself to your classmates in the "Meet and Greet" section!
1 heure pour terminer
2 vidéos (Total 10 min), 2 lectures
2 lectures
Reinforcement Learning Textbook10 min
Read Me: Pre-requisites and Learning Objectives10 min
3 heures pour terminer
Monte Carlo Methods for Prediction & Control
This week you will learn how to estimate value functions and optimal policies, using only sampled experience from the environment. This module represents our first step toward incremental learning methods that learn from the agent’s own interaction with the world, rather than a model of the world. You will learn about on-policy and off-policy methods for prediction and control, using Monte Carlo methods---methods that use sampled returns. You will also be reintroduced to the exploration problem, but more generally in RL, beyond bandits.
3 heures pour terminer
11 vidéos (Total 58 min), 2 lectures, 1 quiz
11 vidéos
What is Monte Carlo?6 min
Using Monte Carlo for Prediction6 min
Using Monte Carlo for Action Values2 min
Using Monte Carlo methods for generalized policy iteration2 min
Solving the Blackjack Example3 min
Epsilon-soft policies5 min
Why does off-policy learning matter?4 min
Importance Sampling4 min
Off-Policy Monte Carlo Prediction5 min
Emma Brunskill: Batch Reinforcement Learning12 min
Week 1 Summary3 min
2 lectures
Weekly Reading40 min
Chapter Summary40 min
1 exercice pour s'entraîner
Graded Quiz
6 heures pour terminer
Temporal Difference Learning Methods for Prediction
This week, you will learn about one of the most fundamental concepts in reinforcement learning: temporal difference (TD) learning. TD learning combines some of the features of both Monte Carlo and Dynamic Programming (DP) methods. TD methods are similar to Monte Carlo methods in that they can learn from the agent’s interaction with the world, and do not require knowledge of the model. TD methods are similar to DP methods in that they bootstrap, and thus can learn online---no waiting until the end of an episode. You will see how TD can learn more efficiently than Monte Carlo, due to bootstrapping. For this module, we first focus on TD for prediction, and discuss TD for control in the next module. This week, you will implement TD to estimate the value function for a fixed policy, in a simulated domain.
6 heures pour terminer
6 vidéos (Total 37 min), 1 lecture, 2 quiz
6 vidéos
Rich Sutton: The Importance of TD Learning6 min
The advantages of temporal difference learning5 min
Comparing TD and Monte Carlo5 min
Andy Barto and Rich Sutton: More on the History of RL12 min
Week 2 Summary2 min
1 lecture
Weekly Reading40 min
1 exercice pour s'entraîner
Practice Quiz30 min
8 heures pour terminer
Temporal Difference Learning Methods for Control
This week, you will learn about using temporal difference learning for control, as a generalized policy iteration strategy. You will see three different algorithms based on bootstrapping and Bellman equations for control: Sarsa, Q-learning and Expected Sarsa. You will see some of the differences between the methods for on-policy and off-policy control, and that Expected Sarsa is a unified algorithm for both. You will implement Expected Sarsa and Q-learning, on Cliff World.
8 heures pour terminer
9 vidéos (Total 30 min), 2 lectures, 2 quiz
9 vidéos
Sarsa: GPI with TD4 min
Sarsa in the Windy Grid World3 min
What is Q-learning?3 min
Q-learning in the Windy Grid World3 min
How is Q-learning off-policy?4 min
Expected Sarsa3 min
Expected Sarsa in the Cliff World3 min
Generality of Expected Sarsa1 min
Week 3 Summary2 min
2 lectures
Weekly Reading40 min
Chapter summary40 min
1 exercice pour s'entraîner
Practice Quiz30 min
Avis
4.8
Meilleurs avis pour SAMPLE-BASED LEARNING METHODS
par KMJan 9, 2020
Really great resource to follow along the RL Book. IMP Suggestion: Do not skip the reading assignments, they are really helpful and following the videos and assignments becomes easy.
par KNOct 2, 2019
Great course! The notebooks are a perfect level of difficulty for someone learning RL for the first time. Thanks Martha and Adam for all your work on this!! Great content!!
par UZNov 22, 2019
Good balance of theory and programming assignments. I really like the weekly bonus videos with professors and developers. Recommend to everyone.
par NHOct 15, 2019
definitely interesting subjects, but I do not like the teaching method. Very mechanic and dull, with not enough connection to the real world
par IFSep 28, 2019
Great course. Clear, concise, practical. Right amount of programming. Right amount of tests of conceptual knowledge. Almost perfect course.
par TGMar 19, 2020
Pretty clear explanations! Nice starting point if you want to deep dive into RL. It gives clear picture over some confusing terms in RL.
par KOJan 6, 2020
This course is well explained, easy to follow and made me understand much better the tabular RL methods. I liked it very much.
par NGNov 24, 2019
Excellent course companion to the textbook, clarifies many of the vague topics and gives good tests to ensure understanding
par NSFeb 1, 2020
I like the teaching style the emphasis on understanding and the fruitful combination with the textbook. Highly recommended!
par MCOct 23, 2019
Some of the programming homeworks were difficult to debug due to the feedback from autograder being unhelpful.
par SASep 2, 2019
Great course! Lots of hands-on RL algorithms. I'm looking forward to the next course in the specialization.
par OEMar 21, 2020
Amazing course, truly academy-grade. And RL is such a fascinating topic!
À propos de Université de l'Alberta
UAlberta is considered among the world’s leading public research- and teaching-intensive universities. As one of Canada’s top universities, we’re known for excellence across the humanities, sciences, creative arts, business, engineering and health sciences.
À propos de Alberta Machine Intelligence Institute
The Alberta Machine Intelligence Institute (Amii) is home to some of the world’s top talent in machine intelligence. We’re an Alberta-based
research institute that pushes the bounds of academic knowledge and guides business understanding of artificial intelligence and machine learning.
À propos du Spécialisation Apprentissage par renforcement
The Reinforcement Learning Specialization consists of 4 courses exploring the power of adaptive learning systems and artificial intelligence (AI).
Harnessing the full potential of artificial intelligence requires adaptive learning systems. Learn how Reinforcement Learning (RL) solutions help solve real-world problems through trial-and-error interaction by implementing a complete RL solution from beginning to end.
By the end of this Specialization, learners will understand the foundations of much of modern probabilistic artificial intelligence (AI) and be prepared to take more advanced courses or to apply AI tools and ideas to real-world problems. This content will focus on “small-scale” problems in order to understand the foundations of Reinforcement Learning, as taught by world-renowned experts at the University of Alberta, Faculty of Science.
The tools learned in this Specialization can be applied to game development (AI), customer interaction (how a website interacts with customers), smart assistants, recommender systems, supply chain, industrial control, finance, oil & gas pipelines, industrial control systems, and more.
Foire Aux Questions
Quand aurai-je accès aux vidéos de cours et aux devoirs ?
Une fois que vous êtes inscrit(e) pour un Certificat, vous pouvez accéder à toutes les vidéos de cours, et à tous les quiz et exercices de programmation (le cas échéant). Vous pouvez soumettre des devoirs à examiner par vos pairs et en examiner vous-même uniquement après le début de votre session. Si vous préférez explorer le cours sans l'acheter, vous ne serez peut-être pas en mesure d'accéder à certains devoirs.
À quoi ai-je droit si je m'abonne à cette Spécialisation ?
Lorsque vous vous inscrivez au cours, vous bénéficiez d'un accès à tous les cours de la Spécialisation, et vous obtenez un Certificat lorsque vous avez réussi. Votre Certificat électronique est alors ajouté à votre page Accomplissements. À partir de cette page, vous pouvez imprimer votre Certificat ou l'ajouter à votre profil LinkedIn. Si vous souhaitez seulement lire et visualiser le contenu du cours, vous pouvez accéder gratuitement au cours en tant qu'auditeur libre.
Quelle est la politique de remboursement ?
Une aide financière est-elle possible ?
D'autres questions ? Visitez le Centre d'Aide pour les Etudiants.
