COURSE 3

Unsupervised Learning, Recommenders, Reinforcement Learning

Project Details

  • Date: Fall 2023
  • Subject: Machine Learning
  • Lecturer: Théo Gachet, MSc student
  • Ressources: GitHub available soon

In this course, students delve into unsupervised learning, covering clustering, anomaly detection, and hands-on labs in the first week, followed by an exploration of recommender systems through collaborative and content-based filtering, alongside TensorFlow implementations in week two. The course concludes with an in-depth study of reinforcement learning, its applications, and algorithm refinements, consolidating the knowledge through quizzes and a practical lab. This streamlined curriculum ensures a robust understanding and application skills in unsupervised learning and recommender systems.



WEEK 1

Unsupervised Learning





Clustering

  • What is clustering?
  • K-means intuition
  • K-means algorithm
  • Optimization objective
  • Initializing K-means
  • Choosing the number of clusters
  • Quizz
  • Lab : K-means

  • Anomaly detection

  • Finding unusual events
  • Gaussian (normal) distribution
  • Anomaly detection algorithm
  • Developing and evaluating an anomaly detection system
  • Anomaly detection vs. supervised learning
  • Choosing what features to use
  • Quizz
  • Lab : Anomaly detection



  • WEEK 2

    Recommander Systems





    Collaborative filtering

  • Making recommendations
  • Using per-item features
  • Collaborative filtering algorithm
  • Binary levels : favs, likes and clicks
  • Quizz

  • Recommender systems implementation

  • Mean normalization
  • TensorFlow implementation of collaborative filtering
  • Finding related items
  • Lab : Collaborative Filtering Recommender Systems
  • Quizz

  • Content-based filtering

  • Collaborative filtering vs. Content-based filtering
  • Deep learning for content-based filtering
  • Recommending from a large catalogue
  • Ethical use of recommender systems
  • TensorFlow implementation of content-based filtering
  • Quizz
  • Lab : Deep Learning for Content-Based filtering

  • Principal Component Analysis

  • Reducing the number of features
  • PCA algorithm
  • PCA in code



  • WEEK 3

    Reinforcement Learning





    Reinforcement learning introduction

  • What is Reinforcement Learning?
  • Mars rover example
  • The Return in reinforcement leaning
  • Making decisions: Policies in reinforcement learning
  • Review of key concepts
  • Quizz

  • State-action value function

  • State-action value function definition
  • State-action value function example
  • Bellman Equation
  • Random (stochastic) environment
  • Quizz

  • Continuous state spaces

  • Example of continuous state space applications
  • Lunar lander
  • Learning the stat-value function
  • Algorithm refinement: Improved neural network architecture
  • Algorithm refinement: ε-greedy policy
  • Algorithm refinement: Mini-batch and soft updates
  • The state of reinforcement learning
  • Quizz
  • Lab : Reinforcement learning