https://rithinnagaraj.github.io/Recent content on Rithin NagarajHugoen-usTue, 06 Jan 2026 00:00:00 +0000https://rithinnagaraj.github.io/posts/slot-attention-jax/Tue, 06 Jan 2026 00:00:00 +0000https://rithinnagaraj.github.io/posts/slot-attention-jax/<p>The mechanism of Attention, widely popularized by the advent of ChatGPT and subsequent Large Language Models is one of the most beautiful and elegant mechanisms that have revolutionized the world. Attention finds an effective way to find out the contextual representation of any token with respect to all tokens in that input sequence using the dot products of <em>Query, Key and Value</em> pairs. But wait, what if we take this mechanism, tweak it slightly and apply it to images? Well, that&rsquo;s how Slot Attention is born!</p>https://rithinnagaraj.github.io/posts/grokking-neural-fourier-circuits/Wed, 10 Dec 2025 00:00:00 +0000https://rithinnagaraj.github.io/posts/grokking-neural-fourier-circuits/<p>Most of the time when we train a model, we usually expect the model to learn the task at hand and generalize well on it. However, there are chances of overfitting where either the model complexity is too high for the amount of data at hand or a variety of other factors that prevent the model from generalizing well.</p> <p>We assume that once a model starts overfitting, it will only continue to overfit even more and not return to generalizing. However, this is not always the case. <a href="https://arxiv.org/abs/2201.02177">Power et al. (2022)</a> shows that a model can in fact, start to generalize very quickly after a period of overfitting.</p>https://rithinnagaraj.github.io/posts/intro-to-rl-blog/Wed, 17 Sep 2025 00:00:00 +0000https://rithinnagaraj.github.io/posts/intro-to-rl-blog/<p>A lot of exciting stuff has happened in the world of Artificial Intelligence recently, and a lot of brilliant new ideas and technologies have emerged as a result of that with many core technologies being explored extensively. However, the art of fine-tuning these models to cater to human needs and how AI can be used to learn about strategies and patterns without any human data is a very beautiful area of its own. In this post, I will be briefly covering this exciting and wonderful field.</p>