Boris-2-Preview-1
Appearance
Boris-2-Preview-1
Overview
Developer
Family
Boris-2
Type
Base (pretrained, preview)
License
Apache 2.0
Status
Released; superseded by Boris-2-Preview-2
Architecture
Architecture
Decoder-only transformer: 8-head MHA, SwiGLU 1664, RoPE, Canon layers, 15M n-gram table
Parameters
143.7M
Layers
31 (dmodel 512)
Context length
2,048 tokens
Tokenizer
Own byte-level BPE, 32,768
Training
Tokens
52.5B (50B stable + 2.5B decay)
Tokens / param
365 Chinchilla-optimal ≈ 20
Data
Main: ClimbMix ~89%, FineMath ~11%. Decay: ClimbMix, Cosmopedia tutorials, OpenMathInstruct-2, FineMath, Cosmopedia math, DCLM
Hardware
1 × NVIDIA V100 32GB (~59,500 tok/s)
Compute
≈ 4.1 × 1019 FLOPs (6ND estimate), n-gram table excluded
Precision
fp16 mixed, fp32 residual stream
Benchmarks
HellaSwag
45.23
ARC-Easy
58.38
ARC-Challenge
29.95
PIQA
69.21
ArithMark-3
37.60
ArithMark-2
37.68
Mean 4 tasks
50.7
Zero-shot, lm-evaluation-harness 0.4.12, acc_norm (ArithMark-2: acc). ArithMark is left out of the mean · Source: model card
Links
Formats
safetensors (transformers, trust_remote_code)
Website
- Released1 month, 13 days ago
Boris base models
← Preceded by
Boris-1.7-D60M-n30MSucceeded by →
Boris-2-Preview-2Boris-2-Preview-1 is the first preview of Boris-2: the main run's 50B-token checkpoint with a short 2.5B-token learning-rate decay, trained from scratch on a single NVIDIA V100.
External links
Image by Jakub Steiner / Wikimedia Commons / Licensed under CC BY-SA 2.0
This page or section is a stub. You can help the wiki by accurately contributing to it.