Tech
Show HN: Sparse kernel delivering 36% higher training throughput
I developed a custom GPU kernel in Triton to accelerate training with a sparse feed-forward network (FFN).With K=256, activating 256 of 4,096 FFN neurons per token…
Discussion (0)
No comments yet. Start the conversation!