Hugging Face's logo
Sunny Sanyal's picture
5 1 3

Sunny Sanyal

Sunny111
thomwolf's profile picture abao's profile picture 21world's profile picture
·
https://sites.google.com/view/sunnysanyal/home

AI & ML interests

Efficient Training Recipes of Large Models (mostly LLMs)

Recent Activity

replied to their post about 1 month ago
Are you familiar with reverse residual connections or looping in language models? Excited to share my Looped-GPT blog post and codebase 🚀 https://github.com/sanyalsunny111/Looped-GPT TL;DR: looping during pre-training improves generalization. Plot shows GPT2 LMs pre-trained with 15.73B OWT tokens P.S. This is my first post here — I have ~4 followers and zero expectations for reach 😄
posted an update about 1 month ago
Are you familiar with reverse residual connections or looping in language models? Excited to share my Looped-GPT blog post and codebase 🚀 https://github.com/sanyalsunny111/Looped-GPT TL;DR: looping during pre-training improves generalization. Plot shows GPT2 LMs pre-trained with 15.73B OWT tokens P.S. This is my first post here — I have ~4 followers and zero expectations for reach 😄
upvoted a paper 2 months ago
Pre-training Small Base LMs with Fewer Tokens
View all activity

Organizations

University of Texas at Austin's profile picture ML Foundations's profile picture Institute for Foundations of Machine Learning's profile picture