🫙 Archive

This was created at a time when I was TA for machine learning, aggregating students' questions and made them into a book. I don't see an update in the near future.

Nowadays I'm mainly working on aioway, a deep learning algorithm compiler! Check it out.

Learning Machine

Straight forward machine learning based on answers and intuition for machine learners

📚 This handbook accompanies the course: Machine Learning with Hung-Yi Lee

🤓 Whoever fights monsters should see to it that in the process he does not become a monster. And if you gaze long enough into an abyss, the abyss will gaze back into you. And in order to tame machine learning, one must first know how to learn machine. --- Me, 2021

✍️ Where does the logo come from?

The logo is made with Inkscape and the following meme.

🤔 Why this book?

There are many resources for machine learning on the internet. However, most of them are either

😒 Too long. It takes half an hour just to read through.

📐 Too math heavy. It takes you forever to understand.

🤪 Too confusing. The concepts are not straight-forward.

This book aims to solve all of that. It tries to be as concise but easy to grasp as possible.

🧍 Who is this book for?

This book is for learners who want to quickly grasp an idea, without diving deep into a topic (it takes way too long!). The book is a handbook for people who want to preserve their time.

If you find this book helpful, please consider starring (★) this repository!

❗ Disclaimer

This book assumes that you have at least some basic understanding of programming.

💁 Contributing

We take openness and inclusiveness very seriously. We have adopted the following code of conduct.

Contributor code of conduct

🔖 Index

• Getting Started

  • Data
  • Model
  • Loss Function
  • Approximation
  • Gradients
    • Loss Function Derivative
    • Back Propagation

• Common Tasks

  • Regression
  • Auto Regression
  • Classification

• Common Building Blocks

  • Linear
    • Linear Layers' Gradients
  • Convolution
  • Recurrent
    • LSTM
    • GRU
  • Embedding
  • Dropout
  • Normalization
  • Padding
  • Pooling
  • Transformer
    • Attention
    • Self Attention
    • Versus RNN
    • Training
    • Teacher Forcing
    • Tokenization
    • Without Training
  • Activation
    • ReLU
    • Sigmoid
    • Softmax
    • Tanh

• Other Things To Notice

  • Batch Size
  • Gradient Norm
  • Saddle Point
  • Learning Rate
  • Optimizer
  • Overfit
  • Underfit

• Generative Models

  • Auto Encoder
    • Architecture
    • Semi Supervised
    • Variational
  • Generative Adversarial Networks
  • Gaussian Mixture Models

• Improving Models

  • Explainable
    • Saliency Maps
  • Meta Learning
  • Life Long Learning
  • Compression

• Reuse Existing Models

  • Transfer Learning and Domain Adaptation
    • Transfer Learning vs Domain Adaptation
  • Knowledge Distillation

• Beyond Supervised Training

  • Clustering
  • Decision Tree
  • Self Supervised
  • Semi Supervised

• Reinforcement Learning

  • State
  • Agent
  • Action
  • Reward
  • Online vs Offline
  • Value
    • Q Learning
  • Policy
    • Policy Gradient
  • Actor Critic