À propos de ce cours : Bayesian methods are used in lots of fields: from game development to drug discovery. They give superpowers to many machine learning algorithms: handling missing data, extracting much more information from small datasets. Bayesian methods also allow us to estimate uncertainty in predictions, which is a really desirable feature for fields like medicine. When Bayesian methods are applied to deep learning, it turns out that they allow you to compress your models 100 folds, and automatically tune hyperparametrs, saving your time and money. In six weeks we will discuss the basics of Bayesian methods: from how to define a probabilistic model to how to make predictions from it. We will see how one can fully automate this workflow and how to speed it up using some advanced techniques. We will also see applications of Bayesian methods to deep learning and how to generate new images with it. We will see how new drugs that cure severe diseases be found with Bayesian methods.
Les destinataires de ce cours : This course was designed for students with strong mathematical and machine learning background who want to get a different perspective of ML algorithms. Note that this is a very advanced course! You must have strong background in statistics, calculus and linear algebra.
Créé par : National Research University Higher School of Economics
Enseigné par : Daniil Polykovskiy, Researcher
HSE Faculty of Computer Science
Enseigné par : Alexander Novikov, Researcher
HSE Faculty of Computer Science
| Informations de base | Cours 3 de 7 dans Advanced Machine Learning Specialization |
| Niveau | Advanced |
| Engagement | 6 weeks of study, 6 hours/week |
| Langue | English |
| Matériel requis | Access to GPUs will be a plus, but you can complete the course without them. |
| Comment réussir | Réussissez tous les devoirs notés pour terminer le cours. |
| Notes des utilisateurs |
Programme de cours
SEMAINE 1
Introduction to Bayesian methods & Conjugate priors
Welcome to first week of our course! Today we will discuss what bayesian methods are and what are probabilistic models. We will see how they can be used to model real-life situations and how to make conclusions from them. We will also learn about conjugate priors — a class of models where all math becomes really simple.
9 vidéos, 1 lecture
- Vidéo: Think bayesian & Statistics review
- Vidéo: Bayesian approach to statistics
- Vidéo: How to define a model
- Vidéo: Example: thief & alarm
- Vidéo: Linear regression
- Reading: MLE estimation of Gaussian mean
- Vidéo: Analytical inference
- Vidéo: Conjugate distributions
- Vidéo: Example: Normal, precision
- Vidéo: Example: Bernoulli
Noté: Introduction to Bayesian methods
Noté: Conjugate priors
SEMAINE 2
Expectation-Maximization algorithm
This week we will about the central topic in probabilistic modeling: the Latent Variable Models and how to train them, namely the Expectation Maximization algorithm. We will see models for clustering and dimensionality reduction where Expectation Maximization algorithm can be applied as is. In the following weeks, we will spend weeks 3, 4, and 5 discussing numerous extensions to this algorithm to make it work for more complicated models and scale to large datasets.
17 vidéos
- Vidéo: Latent Variable Models
- Vidéo: Probabilistic clustering
- Vidéo: Gaussian Mixture Model
- Vidéo: Training GMM
- Vidéo: Example of GMM training
- Vidéo: Jensen's inequality & Kullback Leibler divergence
- Vidéo: Expectation-Maximization algorithm
- Vidéo: E-step details
- Vidéo: M-step details
- Vidéo: Example: EM for discrete mixture, E-step
- Vidéo: Example: EM for discrete mixture, M-step
- Vidéo: Summary of Expectation Maximization
- Vidéo: General EM for GMM
- Vidéo: K-means from probabilistic perspective
- Vidéo: K-means, M-step
- Vidéo: Probabilistic PCA
- Vidéo: EM for Probabilistic PCA
- Notebook: EM algorithm for GMM
Noté: EM algorithm
Noté: Latent Variable Models and EM algorithm
Noté: EM algorithm for GMM
SEMAINE 3
Variational Inference & Latent Dirichlet Allocation
This week we will move on to approximate inference methods. We will see why we care about approximating distributions and see variational inference — one of the most powerful methods for this task. We will also see mean-field approximation in details. And apply it to text-mining algorithm called Latent Dirichlet Allocation
11 vidéos
- Vidéo: Why approximate inference?
- Vidéo: Mean field approximation
- Vidéo: Example: Ising model
- Vidéo: Variational EM & Review
- Vidéo: Topic modeling
- Vidéo: Dirichlet distribution
- Vidéo: Latent Dirichlet Allocation
- Vidéo: LDA: E-step, theta
- Vidéo: LDA: E-step, z
- Vidéo: LDA: M-step & prediction
- Vidéo: Extensions of LDA
Noté: Variational inference
Noté: Latent Dirichlet Allocation
SEMAINE 4
Markov chain Monte Carlo
This week we will learn how to approximate training and inference with sampling and how to sample from complicated distributions. This will allow us to build simple method to deal with LDA and with Bayesian Neural Networks — Neural Networks which weights are random variables themselves and instead of training (finding the best value for the weights) we will sample from the posterior distributions on weights.
11 vidéos
- Vidéo: Monte Carlo estimation
- Vidéo: Sampling from 1-d distributions
- Vidéo: Markov Chains
- Vidéo: Gibbs sampling
- Vidéo: Example of Gibbs sampling
- Vidéo: Metropolis-Hastings
- Vidéo: Metropolis-Hastings: choosing the critic
- Vidéo: Example of Metropolis-Hastings
- Vidéo: Markov Chain Monte Carlo summary
- Vidéo: MCMC for LDA
- Vidéo: Bayesian Neural Networks
- Notebook: PyMC
Noté: Markov Chain Monte Carlo
Noté: PyMC
SEMAINE 5
Variational Autoencoder
Welcome to the fifth week of the course! This week we will combine many ideas from the previous weeks and add some new to build Variational Autoencoder -- a model that can learn a distribution over structured data (like photographs or molecules) and then sample new data points from the learned distribution, hallucinating new photographs of non-existing people. We will also the same techniques to Bayesian Neural Networks and will see how this can greatly compress the weights of the network without reducing the accuracy.
10 vidéos, 3 lectures
- Vidéo: Scaling Variational Inference & Unbiased estimates
- Vidéo: Modeling a distribution of images
- Vidéo: Using CNNs with a mixture of Gaussians
- Vidéo: Scaling variational EM
- Vidéo: Gradient of decoder
- Vidéo: Log derivative trick
- Vidéo: Reparameterization trick
- Reading: VAE paper
- Notebook: Variational Autoencoder
- Vidéo: Learning with priors
- Vidéo: Dropout as Bayesian procedure
- Vidéo: Sparse variational dropout
- Reading: Relevant papers
- Reading: Categorical Reparametrization with Gumbel-Softmax
Noté: Variational autoencoders
Noté: Variational Autoencoder
Noté: Categorical Reparametrization with Gumbel-Softmax
SEMAINE 6
Gaussian processes & Bayesian optimization
Welcome to the final week of our course! This time we will see nonparametric Bayesian methods. Specifically, we will learn about Gaussian processes and their application to Bayesian optimization that allows one to perform optimization for scenarios in which each function evaluation is very expensive: oil probe, drug discovery and neural network architecture tuning.
7 vidéos
- Vidéo: Nonparametric methods
- Vidéo: Gaussian processes
- Vidéo: GP for machine learning
- Vidéo: Derivation of main formula
- Vidéo: Nuances of GP
- Vidéo: Bayesian optimization
- Vidéo: Applications of Bayesian optimization
- Notebook: GPy and GPyOpt
Noté: Gaussian Processes and Bayesian Optimization
Noté: GPy and GPyOpt
Final project
In this module you will apply methods that you learned in this course to this final project
Noté: Final project: Finding the suspect
FAQ
Comment cela fonctionne
Coursework
Each course is like an interactive textbook, featuring pre-recorded videos, quizzes and projects.
Help from Your Peers
Connect with thousands of other learners and debate ideas, discuss course material, and get help mastering concepts.
Certificates
Earn official recognition for your work, and share your success with friends, colleagues, and employers.
Créateurs
National Research University Higher School of Economics
National Research University - Higher School of Economics (HSE) is one of the top research universities in Russia. Established in 1992 to promote new research and teaching in economics and related disciplines, it now offers programs at all levels of university education across an extraordinary range of fields of study including business, sociology, cultural studies, philosophy, political science, international relations, law, Asian studies, media and communications, IT, mathematics, engineering, and more.
Learn more on www.hse.ru
Tarification
| Acheter le cours | |
|---|---|
| Accéder au contenu du cours | Disponible |
| Accéder au contenu noté | Disponible |
| Recevoir une Note Finale | Disponible |
| Obtenir un Certificat de Cours partageable | Disponible |
Notation et examens
Tough but useful!
clear instruction and great insights to algorithm, I love it. Really regret for lacking the time to finish all the programming assignments.
Many more theoretical formulas and derivations than previous courses of the specialization, which might require quite a bit of probability theory knowledge. But it is really helpful to understand EM and VAE in depth as well as to use GPy/GPyOpt tools in practice. It would be better to have detail explanation for some quizzes.
Coursera propose un accès universel à la meilleure formation au monde,
en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.
© 2018 Coursera Inc. Tous droits réservés.
very professional