how-to-learn-deep-learning-framework
Learning notes for deep learning framework internals.
This repository collects resources and notes about PyTorch, OneFlow, TorchScript, distributed training, autograd, memory management, operator development, and framework-level performance optimization.
Focus Areas
- PyTorch internals: autograd, CUDA extension, data loading, memory management, AMP, TorchScript, Dynamo, AOTAutograd, and performance tuning.
- OneFlow internals: execution model, operators, distributed tensors, runtime, VM, and CUDA kernels.
- ML systems engineering: framework architecture, operator implementation, and training/runtime optimization.
Related Repositories
- CUDA and GPU optimization: https://github.com/BBuf/how-to-optim-algorithm-in-cuda
- Deep learning compiler notes: https://github.com/BBuf/tvm_mlir_learn
Status
Legacy learning archive. The repository remains public for reference, with English public-facing documentation going forward.