Monthly Archives: June 2023

Gumbel Softmax is basically TopK with some noise

I have been mystified by the notion of Gumbel Softmax. Even though the idea is very simple (we add Gumbel noise to make top k equivalent to sampling without replacement), papers and blogs make it appear very theoretical, and it’s … Continue reading

Posted in Uncategorized | Leave a comment