passes_n_ocr_d60m
d60m (68.9M parameters) on plus_all v2 subset + pilot OCR text (39M tokens), trained on RTX 3060 12GB (Home GPU). Started 10 Oct 2026, ended 10 Oct 2026.
Hypothesis
Fresh text from our OCR of 127 scanned books (39M tokens, ~20% of the diet) beats a fourth pass by more than the noise bar and ties fresh clean text. If so, more OCR is worth paying for.
What we learned
No: ex-Gita 0.6623, a tie with a fourth pass (+0.16%; clean_v1 +0.17%) and 1.2% behind the same amount of fresh clean text. At this dose the OCR text is worth no more than repeating clean text. Internal test (first OCR model).
Scores
Bits per byte, lower is better; change against the parent run, Passes 4.
| Test set | Passes N-ocr | Passes 4 (parent) | Change |
|---|---|---|---|
| DCS gold (classical) | 0.6711 | 0.6698 | +0.2% |
| Bhagavad-gītā (memorisation) | 0.46 | 0.4451 | +3.3% |
| Out of domain | 0.6658 | 0.6655 | +0.0% |
| Prose | 0.6312 | 0.628 | +0.5% |
| Vedic (Ṛgveda) | 0.8937 | 0.8921 | +0.2% |
Curves
Training loss
Cross-entropy per token, by step. The first few percent of the run, far higher, run off the top; hover or the table has every value.
Held-out bits per byte
The validation split, evaluated during training, by step. Lower is better.
Throughput
Tokens per second, by step.
Model
- Preset
- d60m
- Parameters
- 68,924,160
- Outside embeddings
- 56,636,160
- Layers · heads · width
- 8 · 12 · 768
- Tokenizer
- SLP1 unigram 8k
Data
- Slice
- plus_all v2 subset + pilot OCR text (39M tokens)
- Words
- —
- Training tokens
- 194,623,344
- Passes
- 3.19 passes
- Tokens seen
- 621,084,672
Compute
- GPU
- RTX 3060 12GB
- Where
- Home GPU
- Steps
- 12,636 / 12,636
- GPU hours
- 3.84
- Cost
- —
- Spot restarts
- —
Lineage
homed60m