Summary

Over the past few years we have seen a convergence of two large scale trends: Big Data and Big Compute. The resulting combination of large amounts of data and abundant CPU (and GPU) cycles has brought to the forefront and highlighted the power of neural network techniques and approaches that were once thought to be too impractical.

Deep Learning, as this new wave of interest has come to be known, has made impressive and unprecedented progress on applications as diverse as Natural Language Processing, Machine Translation, Computer Vision, Robotics, etc. In this lecture, students will learn, in a hands-on way, the theoretical foundations and principal ideas underlying this burgeoning field. The code structure of the implementations provided is meant to closely resemble the way the state of the art deep learning libraries Keras is structured so that by the end of the course, students will be prepared to dive deeper into the deep learning applications of their choice.

Program

Regression
- Understand the different types of Machine Learning
- Define Supervised Learning and Regression
- Visualize Linear Regression
- Implement Gradient Descent
- Use Linear Regression
Classification
- Define Classification
- Visualize Logistic Regression
- Learning Procedure
- Implement Logistic Regression
- Understand Classification evaluation
Data Preparation
- Visualize decision boundaries
- Use Data Normalization
- Understand Overfitting
- Define the Bias Variance Tradeoff
Neural Networks
- Understand how the brain works
- Understand Perceptrons and Forward Propagation
- Define Activation Functions
- Understand Back Propagation
Recognizing Numbers
- Understand the MNIST Dataset
- Understand Data Preparation
- Implement Back Propagation
- Understand the effect of the Learning Rate
- Generalize the Code
Advanced network applications
- Understand unsupervised Learning
- Implement an Auto-Encoder
- Understand Recurrent Neural Networks
- Understand Convolutional Neural Networks
- Explore the Learning vs Memorization Tradeoff