COURSE 3

Unsupervised Learning, Recommenders, Reinforcement Learning

Project Details

  • Date: Fall 2023
  • Subject: Machine Learning
  • Lecturer: Théo Gachet, MSc student
  • Ressources: GitHub available soon

In this course, students delve into unsupervised learning, covering clustering, anomaly detection, and hands-on labs in the first week, followed by an exploration of recommender systems through collaborative and content-based filtering, alongside TensorFlow implementations in week two. The course concludes with an in-depth study of reinforcement learning, its applications, and algorithm refinements, consolidating the knowledge through quizzes and a practical lab. This streamlined curriculum ensures a robust understanding and application skills in unsupervised learning and recommender systems.


WEEK 1

Unsupervised Learning




Clustering

  • What is clustering?
  • K-means intuition
  • K-means algorithm
  • Optimization objective
  • Initializing K-means
  • Choosing the number of clusters
  • Quizz
  • Lab : K-means

    • Anomaly detection

  • Finding unusual events
  • Gaussian (normal) distribution
  • Anomaly detection algorithm
  • Developing and evaluating an anomaly detection system
  • Anomaly detection vs. supervised learning
  • Choosing what features to use
  • Quizz
  • Lab : Anomaly detection


    • WEEK 2

    Recommander Systems




    Collaborative filtering

  • Making recommendations
  • Using per-item features
  • Collaborative filtering algorithm
  • Binary levels : favs, likes and clicks
  • Quizz

    • Recommender systems implementation

  • Mean normalization
  • TensorFlow implementation of collaborative filtering
  • Finding related items
  • Lab : Collaborative Filtering Recommender Systems
  • Quizz

    • Content-based filtering

  • Collaborative filtering vs. Content-based filtering
  • Deep learning for content-based filtering
  • Recommending from a large catalogue
  • Ethical use of recommender systems
  • TensorFlow implementation of content-based filtering
  • Quizz
  • Lab : Deep Learning for Content-Based filtering

    • Principal Component Analysis

  • Reducing the number of features
  • PCA algorithm
  • PCA in code


    • WEEK 3

    Reinforcement Learning




    Reinforcement learning introduction

  • What is Reinforcement Learning?
  • Mars rover example
  • The Return in reinforcement leaning
  • Making decisions: Policies in reinforcement learning
  • Review of key concepts
  • Quizz

    • State-action value function

  • State-action value function definition
  • State-action value function example
  • Bellman Equation
  • Random (stochastic) environment
  • Quizz

    • Continuous state spaces

  • Example of continuous state space applications
  • Lunar lander
  • Learning the stat-value function
  • Algorithm refinement: Improved neural network architecture
  • Algorithm refinement: ε-greedy policy
  • Algorithm refinement: Mini-batch and soft updates
  • The state of reinforcement learning
  • Quizz
  • Lab : Reinforcement learning

    • © 2023 Théo Gachet