Jump to content

Boris-1.7-D60M-n30M

From Digital Millennium Wiki

Boris-1.7-D60M-n30M is an experimental base model: a 60M-parameter dense transformer with a 30M-parameter n-gram lookup table spliced into the residual stream, testing whether 30M of cheap lookup buys what 30M of dense weight would at a fixed 2.5B-token budget.

External links

Image by Jakub Steiner / Wikimedia Commons / Licensed under CC BY-SA 2.0
This page or section is a stub. You can help the wiki by accurately contributing to it.