the yaml thing is not a switch statement though. torchgen takes native_functions.yaml and generates the bindings, autograd nodes and dispatch entries from that one file.
there are ~~3700 ops. try keeping that in sync by hand.
also dynamo never calls gcc. it just captures the graph from python bytecode. inductor is what writes triton/cpp and sends it to real compilers. they built their own compiler stack once (torchscript), it's dead now.
so this time they reused what exists.
spaghetti yes. 10 years of layers. you can see every era in the repo.
The "compiler" dynamo part is also a hack that calls real compilers like gcc and Triton.
It is a big bowl of spaghetti.
also dynamo never calls gcc. it just captures the graph from python bytecode. inductor is what writes triton/cpp and sends it to real compilers. they built their own compiler stack once (torchscript), it's dead now. so this time they reused what exists.
spaghetti yes. 10 years of layers. you can see every era in the repo.
but the ones I learned from: - Edward Yang's "pytorch internals" post: http://blog.ezyang.com/2019/05/pytorch-internals/ it's the classic, goes deep on the c++ side.
- the pytorch developer podcast, same author. short episodes, one internal topic each
- the pytorch repo's own CONTRIBUTING.md, the folder layout explained by the maintainers
- and if you want to build autograd instead of reading about it, Karpathy's micrograd / zero to hero
mine is planned as 12 parts, the list is at the end of the post. part 1 (chapter Tensor: storage, strides, views) is in progress now.