Awesome Deep Neural Network Compression

Paper collection, Summary, Code for Deep Neural Network Compression, including:

• Quantization,
• Pruning (Unstructure, structure)
• Distillation

and so on.

Paper:

• By Topic:

  • Quantization
  • Pruning
  • Efficient Model Design
  • Network Architecture Search (NAS) for Model Compression
  • Compression Meets Robustness (Adversarial)
  • NLP Compression
  • Differentiable Compression
  • Large Pretraining Models: Including language, vision

• By Conference:

  • 2024
  • 2023
  • 2022
  • 2021
  • 2020
  • 2019
  • 2018

• Survey

• Related Topic:

  • Optimization
  • Meta Learning

Compression System:

• DeepSpeed
• ColossalAI
• Distiller
• PocketFlow

Codes / Tools:

• My Implementation: My re-implementation of state-of-the-art compression methods.

Summary:

My summary (slides) for network compression. Some papers are chosen to be represented.

• Quantization Summary
• Pruning Summary
• Theory: From basic convex optimization to quantization