A simple Pytorch reimplementation.

reference

https://dlsyscourse.org/

install/develop

pip install -r requirements.txt
python setup.py develop

example

python example/mnist.py

test

python -m unittest

PS: pytorch is required to check the correctness of implementation.

todo

Operator

• mm [done]
• relu [done]
• max_pool2d [done]
• conv2d []
  • efficient im2col with stride
• log_softmax [done]
• view [done]
• nll_loss [done]
• in-place add/sub/mul/div [done]

Autograd

• gradient accum in backward [done]
• gradient_check_tool [done]
• no_grad [done]
• grad as tensor [done]
• inplace gradient calculation []

Module

• module load/store [done]

Test

• test []

Example

• mnist example [done]

Memory Optimization

• ref count []
  • use topological sort and fix inplace operation
• in-place[]
• normal sharing []

Other

• dataloader [done]
• hide numpy from user []
• dot graph []

Bug

• Conv grad with padding