Chinchilla paper actually shows that for a fixed compute budget, it is better to train a smaller model on more data rather than training a larger model for fewer steps.
tm23
tm23hgf
AI & ML interests
None yet
Recent Activity
new activity
about 1 month ago
BibbyResearch/3blue1brown-manim:Not a good dataset
commented on
an
article
about 1 month ago
Mixture of Experts Explained
upvoted
an
article
about 2 months ago
Mixture of Experts Explained
Organizations
None yet
Not a good dataset
2
#2 opened about 1 month ago
by
tm23hgf
commented on
Mixture of Experts Explained
about 1 month ago
upvoted
an
article
about 2 months ago
Article
Mixture of Experts Explained
- +4
•
1.03k