Boris-2
Appearance
Boris-2
Overview
Developer
Family
Boris-2
Type
Base (pretrained)
License
Apache 2.0
Status
Released
Architecture
Architecture
Decoder-only transformer: 8-head MHA, SwiGLU 1664, RoPE, Canon layers, 15M n-gram table
Parameters
143.7M
Layers
31 (dmodel 512)
Context length
2,048 tokens
Tokenizer
Own byte-level BPE, 32,768
Training
Tokens
200.04B (200B main run + 40M final stages)
Tokens / param
1,392 Chinchilla-optimal ≈ 20
Data
Main: ClimbMix ~90%, UltraData-Math L3 10%, Orca-Math (FineMath before 55B). Decay from 174B adds Cosmopedia tutorials, DCLM, MetaMathQA, Calc-ape210k/mawps; final stage LibreMath v1
Hardware
1 × NVIDIA V100 32GB (~59,500 tok/s)
Compute
≈ 1.5 × 1020 FLOPs (6ND estimate), n-gram table excluded
Precision
fp16 mixed, fp32 residual stream
Benchmarks
Benchmark
Score
Preview-2
HellaSwag
50.27
46.60
ARC-Easy
59.76
59.13
ARC-Challenge
32.00
30.20
PIQA
72.85
70.84
WinoGrande
52.57
–
OpenBookQA
36.60
–
SciQ
85.10
–
MMLU (continuation)
28.96
–
CommonsenseQA
20.88
–
ArithMark-3
45.10
42.20
Mean 9 tasks
48.8
–
Zero-shot, lm-evaluation-harness 0.4.12, acc_norm (WinoGrande: acc). Nine-task mean; ArithMark-3 is left out · Source: benchmarks_comparison.csv
Links
Weights
Formats
safetensors (transformers, trust_remote_code)
Website
- Released11 days ago
Boris base models
← Preceded by
Boris-2-Preview-2Boris-2 is a 143.7M-parameter base language model from OpenCerebral, trained from scratch for 200B tokens on a single NVIDIA V100, followed by two short final stages. It follows Boris-2-Preview-1 and Boris-2-Preview-2.
External links
Image by Jakub Steiner / Wikimedia Commons / Licensed under CC BY-SA 2.0
This page or section is a stub. You can help the wiki by accurately contributing to it.