Sansar

Passes MuonIO b

Sanskrit DoneNeutral

passes_p2b_muonio_d60m

d60m (68.9M parameters) on plus_all v2 subset (155M tokens), trained on RTX 3060 12GB (Home GPU). Started 11 Oct 2026, ended 11 Oct 2026.

Hypothesis

MuonIO with its step sizes scaled down to this model's table sizes gives a better four-pass model.

What we learned

Tie: ex-Gita 0.6598, 0.21% better than Passes 4 (clean_v1 -0.10%), inside the 0.52% noise bar. Step sizes were the paper's divided by 8, the best of three short trials. Not adopted.

Scores

Bits per byte, lower is better; change against the parent run, Passes 4.

ex-Gītā (headline)
0.6598
bits per byte
−0.2%
ex-Gītā, clean_v1
0.6824
bits per byte
−0.1%
Pooled, all five sets
0.6516
bits per byte
−0.3%
Validation split
0.6508
bits per byte
−0.2%
Bits per byte on each test set
Test setPasses MuonIO bPasses 4 (parent)Change
DCS gold (classical)0.66930.6698−0.1%
Bhagavad-gītā (memorisation)0.43620.4451−2.0%
Out of domain0.66370.6655−0.3%
Prose0.62680.628−0.2%
Vedic (Ṛgveda)0.89930.8921+0.8%

Curves

Training loss

Cross-entropy per token, by step. The first few percent of the run, far higher, run off the top; hover or the table has every value.

Held-out bits per byte

The validation split, evaluated during training, by step. Lower is better.

Throughput

Tokens per second, by step.

Model

Preset
d60m
Parameters
68,924,160
Outside embeddings
56,636,160
Layers · heads · width
8 · 12 · 768
Tokenizer
SLP1 unigram 8k

Muon + AdamW, rope, qk_norm, relu^2, untied head, weight decay (F10 recipe), MuonIO for embedding and head (rescaled)

Data

Slice
plus_all v2 subset (155M tokens)
Words
—
Training tokens
155,261,295
Passes
4 passes
Tokens seen
621,084,672

Compute

GPU
RTX 3060 12GB
Where
Home GPU
Steps
12,636 / 12,636
GPU hours
3.84
Cost
—
Spot restarts
—

Lineage

← Parent: Passes 4

homed60m