COURSE 2

Advanced Learning Algorithms

Project Details

  • Date: Fall 2023
  • Subject: Machine Learning
  • Lecturer: Théo Gachet, MSc student
  • Ressources: Course Repository

This course covers neural networks from basics to advanced applications, diving into training techniques, activation functions, and multiclass classification, complemented by hands-on labs and quizzes. Week three offers strategic advice on applying machine learning, tackling bias and variance, and the machine learning development process, while the final week focuses on decision trees, learning algorithms, and tree ensembles, ensuring participants have both theoretical knowledge and practical experience.


WEEK 1

Neural Networks



Neural networks intuition

  • Neurons and the brain
  • Demand Prediction
  • Example : Recognizing Images
  • Quizz

    • Neural network model

  • Neural network layer
  • More complex neural networks
  • Inference : making predictions (forward propagation)
  • Lab : Neurons and Layers
  • Quizz

    • TensorFlow implementation

  • Inference ine code
  • Data in TensorFlow
  • Building a neural network
  • Lab : Coffee Roasting in TensorFlow
  • Quizz

    • Neural network implementation in Python

  • Forward propagation in a single layer
  • General implementation of forward propagation
  • Lab : Coffee Roasting with NumPy
  • Quizz

    • Vectorization

  • How neural networks are implemented efficiently
  • Matric multiplication
  • Lab : Neural Networks for Binary Classification


    • WEEK 2

    Neural Networks training



    Neural Network Training

  • TensorFlow implementation
  • Training Details
  • Quizz

    • Activation functions

  • Alternatives to the sigmoid activation
  • Choosing activation functions
  • Why do we need activation functions ?
  • Lab : ReLU activation
  • Quizz

    • Multiclass Classification

  • Multiclass
  • Softmax
  • Neural Network with Softmax output
  • Improved implementation of softmax
  • Classification with multiple outputs
  • Lab : Softmax
  • Lab : Multiclass
  • Quizz

    • Additional Neural Network Concepts

  • Advanced Optimization
  • Additional Layer Types
  • Quizz

    • Back Propagation

  • What is a derivative ?
  • Computation graph
  • Larger neural network example
  • Lab : Derivatives
  • Lab : Back propagation
  • Lab : Neural Networks for Multiclass classification


    • WEEK 3

    Advice for applying Machine Learning



    Advice for applying Machine Learning

  • Deciding what to try next
  • Evaluating a model
  • Model selection and training/cross validation/test
  • Lab : Model Evaluation and Selection
  • Quizz

    • Bias and variance

  • Diagnosing bias and variance
  • Regularization and bias/variance
  • Establishing a baseline level of performance
  • Learning curves
  • Deciding what to try next revisited
  • Bias/variance and neural networks
  • Lab : Diagnosing bias and variance
  • Quizz

    • Machine learning development process

  • Iterative loop of Machine Learning development
  • Error analysis
  • Adding data
  • Transfer learning : using data from a different task
  • Full cycle of a machine learning project
  • Fairness, bias, and ethics
  • Quizz

    • Skewed datasets

  • Error metrics for skewed datasets
  • Trading off precision and recall
  • Lab : Advice for applying Machine Learning


    • WEEK 4

    Decision Trees



    Decision trees

  • Decision tree model
  • Learning Process
  • Quizz

    • Decision tree learning

  • Measuring purity
  • Choosing a split : Information Gain
  • Putting it together
  • Using one-hot encoding of categorital features
  • Continuous valued features
  • Regression Trees
  • Lab : Decision Trees
  • Quizz

    • Tree ensembles

  • Using multiple decision trees
  • Sampling with replacement
  • Random forest algorithm
  • XGBoost
  • When to use decision trees
  • Lab : Tree Ensembles
  • Quizz
  • Lab : Decision Trees


    • © 2023 Théo Gachet