<?xml version="1.0"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
	<id>https://wiki.kunix.org/index.php?action=history&amp;feed=atom&amp;title=Boris-1.7-D60M-n30M</id>
	<title>Boris-1.7-D60M-n30M - Revision history</title>
	<link rel="self" type="application/atom+xml" href="https://wiki.kunix.org/index.php?action=history&amp;feed=atom&amp;title=Boris-1.7-D60M-n30M"/>
	<link rel="alternate" type="text/html" href="https://wiki.kunix.org/index.php?title=Boris-1.7-D60M-n30M&amp;action=history"/>
	<updated>2026-11-22T13:53:25Z</updated>
	<subtitle>Revision history for this page on the wiki</subtitle>
	<generator>MediaWiki 1.43.9</generator>
	<entry>
		<id>https://wiki.kunix.org/index.php?title=Boris-1.7-D60M-n30M&amp;diff=124&amp;oldid=prev</id>
		<title>Jdj: New article with Infobox model (data from the Hugging Face model card)</title>
		<link rel="alternate" type="text/html" href="https://wiki.kunix.org/index.php?title=Boris-1.7-D60M-n30M&amp;diff=124&amp;oldid=prev"/>
		<updated>2026-11-16T00:43:48Z</updated>

		<summary type="html">&lt;p&gt;New article with Infobox model (data from the Hugging Face model card)&lt;/p&gt;
&lt;p&gt;&lt;b&gt;New page&lt;/b&gt;&lt;/p&gt;&lt;div&gt;{{Infobox model&lt;br /&gt;
| developer       = [[OpenCerebral]]&lt;br /&gt;
| family          = Boris-1.7&lt;br /&gt;
| model_type      = Base (pretrained, experimental)&lt;br /&gt;
| license         = Apache 2.0&lt;br /&gt;
| status          = Released&lt;br /&gt;
| architecture    = Decoder-only transformer: 12 heads / 4 KV heads (GQA), SwiGLU 1920, RoPE, RMSNorm, d&amp;lt;sub&amp;gt;model&amp;lt;/sub&amp;gt; 768, 30M n-gram table after block 2&lt;br /&gt;
| parameters      = 116.1M (60.0M dense + 25.2M embedding + 30.0M n-gram table)&lt;br /&gt;
| layers          = 10&lt;br /&gt;
| context         = 1,024 tokens&lt;br /&gt;
| tokenizer       = Own byte-level BPE, 32,768&lt;br /&gt;
| training_tokens = 2.5B&lt;br /&gt;
| dataset         = Ultra-FineWeb bulk; Cosmopedia v2 + FineWeb-Edu anneal over the final 15%&lt;br /&gt;
| hardware        = 1 × RTX 3060 12GB&lt;br /&gt;
| training_time   = 22 h 37 m (10,172 steps)&lt;br /&gt;
| compute         = ≈ 1.3 × 10&amp;lt;sup&amp;gt;18&amp;lt;/sup&amp;gt; FLOPs (6ND estimate), n-gram table excluded&lt;br /&gt;
| precision       = bf16, torch.compile&lt;br /&gt;
| hellaswag       = 31.22&lt;br /&gt;
| piqa            = 62.57&lt;br /&gt;
| winogrande      = 51.07&lt;br /&gt;
| arc_easy        = 45.88&lt;br /&gt;
| arc_challenge   = 26.02&lt;br /&gt;
| lambada         = 26.66&lt;br /&gt;
| bench1          = ArithMark-3&lt;br /&gt;
| bench1_score    = 36.1&lt;br /&gt;
| bench1_mean     = no&lt;br /&gt;
| bench2          = BananaMind Base Bench 1.1&lt;br /&gt;
| bench2_score    = 1022 Elo&lt;br /&gt;
| bench_note      = Zero-shot; acc_norm for HellaSwag, PIQA and ARC, acc for WinoGrande and LAMBADA. ArithMark-3 is left out of the mean&lt;br /&gt;
| bench_source    = [https://huggingface.co/opencerebral/Boris-1.7-D60M-n30M model card]&lt;br /&gt;
| weights         = [https://huggingface.co/opencerebral/Boris-1.7-D60M-n30M opencerebral/Boris-1.7-D60M-n30M]&lt;br /&gt;
| formats         = safetensors (custom loader, not transformers)&lt;br /&gt;
| website         = [https://oc.kunix.org oc.kunix.org]&lt;br /&gt;
| released        = 2026-08-31&lt;br /&gt;
| series          = Boris base models&lt;br /&gt;
| preceded_by     = [[Boris-1.3-125M]]&lt;br /&gt;
| succeeded_by    = [[Boris-2-Preview-1]]&lt;br /&gt;
}}&lt;br /&gt;
&amp;#039;&amp;#039;&amp;#039;Boris-1.7-D60M-n30M&amp;#039;&amp;#039;&amp;#039; is an experimental base model: a 60M-parameter dense transformer with a 30M-parameter n-gram lookup table spliced into the residual stream, testing whether 30M of cheap lookup buys what 30M of dense weight would at a fixed 2.5B-token budget.&lt;br /&gt;
&lt;br /&gt;
== External links ==&lt;br /&gt;
* [https://huggingface.co/opencerebral/Boris-1.7-D60M-n30M Boris-1.7-D60M-n30M on Hugging Face]&lt;br /&gt;
&lt;br /&gt;
{{Stub}}&lt;/div&gt;</summary>
		<author><name>Jdj</name></author>
	</entry>
</feed>