An English spell checker leans on two things Tibetan does not give you: spaces to find word boundaries, and a dictionary to check the words against.
Tibetan separates syllables with a tsek (་), not words. And a syllable like བསྒྲུབས is a stack of several Unicode codepoints rendered as one glyph. Edit those codepoints carelessly and you get something that looks Tibetan but is not a valid syllable.
Errors reach us from OCR, from annotators, and from EPUBs typed or edited by hand. We wanted one corrector for all of them.
We started from TiSpell, a published Tibetan corrector that works at both character and syllable level. No pretrained weights are released, so we trained it ourselves.
The Data
92,266 pairs, 82,037 train, 5,146 val, 5,083 test. The test set is 3,839 differing pairs plus 1,244 where source already equals target.
Two sources:
Real annotation pairs. Manuscript pages transcribed by annotators, checked by reviewers. Genuine pipeline errors, but reviewers transcribe faithfully, including the original scribe’s misspellings. So the target is an accurate copy of the page, not guaranteed-correct Tibetan. Useful and noisy.
Our own corruptions. Clean text from BoCorpus and a modern news collection, damaged deliberately, so the right answer is known by construction.
Preprocessing
Every sentence passes the same filters:
- hunspell (hunspell-bo): synthetic targets must be in the Tibetan dictionary
- StructureCheck: syllables must be structurally possible (prefix / root / superscript / subscript / suffix)
- Drop non-Tibetan characters, stray Latin, annotator brackets
- Drop Tibetan digits, since one number is not a misspelling of another
- Collapse repeated-tsek and repeated-digit artifacts
bophono supplies grapheme-to-pronunciation for the sound-alike corruptions; a verb/lemma list supplies tense forms.
We also ran a Gemini zero-shot pass over the annotation targets: does this contain a genuine orthographic error, yes or no? The prompt took several revisions. A naive checker flags a lot of valid Tibetan: Old Tibetan spellings, ritual jargon, archaic particles, rare legal vocabulary. The final version treats all of that as correct and flags only blatant typos. Flagged rows were dropped from new training data. Val and test were left frozen.
Generating Errors
TiSpell ships syllable- and character-level corruptions. We dropped the ones that produced unrealistic output (syllable_random_delete, syllable_random_exchange, char_tall_short_replace), kept the rest, and added three: sound-alike substitution, verb-tense confusion, and a whole-syllable swap drawn from measured real confusions. Nine types total.
Syllable merge. A lost tsek fuses two syllables. Characters unchanged, boundary gone.
| Text | |
|---|---|
| Target | ཞེས་ཡོངས་ཁྱབ་ཏུ་ངོས་འཛིན་བྱེད་ཀྱི་ཡོད་པ་རེད |
| Source | ཞེསཡོངས་ཁྱབ་ཏུ་ངོས་འཛིན་བྱེད་ཀྱི་ཡོད་པ་རེད |
Sound-alike. Swap a syllable for a homophone. Both valid Tibetan; the error is phonetic, not visual.
| Text | |
|---|---|
| Target | འདིར་ནི་ལྷག་པ་ཤེས་རབ་ཀྱི་བསླབ་པ་… |
| Source | རྡིར་ནི་ལྷག་པ་ཤེས་རབ་ཀྱི་བསླབ་པ་… |
| Change | འདིར → རྡིར |
The rest cover lookalike characters, verb tense, insertion, deletion.
Two constraints keep it realistic. Corruption sites stay apart. Real errors are sparse, and stacking three of them on one short sentence gives you text nothing can learn from. And most types must leave a structurally valid syllable behind. The two built from real confusion tables skip that check, because real OCR routinely produces stacks the checker rejects.
if too_close(changed_pos, used_positions):
continue
if changed and not all(check.ok(x) for x in changed):
continue
cur = out
used_positions.update(changed_pos)
Some sentences come out untouched. That is deliberate. A spell checker has to learn when not to edit.
The Metric
Most of a sentence is already correct, so positional F1 flatters a model that changes nothing. We use edit-level CSC metrics instead: detection (did it find the error?) and correction (did it fix it?), scored only on positions that changed. We also report character and syllable error rates, and compare everything against a copy baseline.
Source and target have to be aligned first. Otherwise one deleted syllable shifts the rest and the whole tail looks wrong.
TiSpell
RoBERTa encoder, two correction heads (character and syllable) summed, plus a copy gate that decides per position whether to keep the input or trust the heads.
input_ids
|
v
+-----------------+
| Tibetan RoBERTa |
| encoder |
+--------+--------+
|
hidden states
|
+------------+------------+
v v v
character syllable copy gate
corrector corrector |
| | |
| | v
| | alpha
+------+-----+ |
v |
logit_c + logit_s |
| |
+---------+--------+
v
final prediction
Low alpha keeps the input; high alpha lets the heads win. We also trained a semi-mask variant with no gate.
| Model | Detection F1 | Correction F1 | Exact Match |
|---|---|---|---|
| Copy-gate | 0.649 | 0.485 | 0.307 |
| Semi-mask | 0.678 | 0.521 | 0.319 |
Closer than expected, a few points apart. Architecture was not where the gains were. So we tried a different shape entirely.
ByT5
TiSpell asks: for this position, keep or replace? ByT5 asks: given this noisy sentence, what should it say?
ByT5 is byte-level, so there is no tokenizer to split a glyph stack, and multi-syllable rewrites come naturally.
We fine-tuned two checkpoints: google/byt5-base and buddhist-nlp/byt5-mitra-bo, the latter already adapted to Tibetan. The Tibetan one converged much faster and scored the same.
We found out why afterwards: mitra was trained on Wylie transliteration, our data is Unicode Tibetan. To a byte-level model those are unrelated inputs. Wylie is Latin bytes, Unicode Tibetan is three-byte sequences. So almost none of that pretraining transferred.
Gemini as a Baseline
Separately from the target check, we ran Gemini as a corrector: noisy sentence in, corrected sentence out. Minimal-edit prompt, temperature 0, batched, scored with the same CSC script as everything else. Two tiers: 3.6-flash and the cheaper 3.1-flash-lite. Prompt is in the repo.
Results
3,839 differing test pairs, one scorer, all models.
| Model | Correction F1 | Exact Match |
|---|---|---|
| TiSpell (semi-mask) | 0.521 | 0.319 |
| ByT5 base (v11) | 0.626 | 0.541 |
| ByT5 mitra (v11) | 0.623 | 0.532 |
| ByT5 mitra (v12b) | 0.654 | 0.587 |
| Gemini 3.6-flash | 0.692 | 0.519 |
| Gemini 3.1-flash-lite | 0.628 | 0.409 |
Gemini 3.6 wins on correction F1. ByT5 v12b wins on exact match, beating both Gemini tiers. On this test set a do-nothing baseline scores 0 exact match, so that number means whole sentences fully corrected.
Split by error source and the picture changes:
| Slice | ByT5 v12b | Gemini 3.6 |
|---|---|---|
| Synthetic (n=2,944) | 0.707 | 0.788 |
| Real OCR (n=895) | 0.384 | 0.354 |
Gemini’s overall lead is almost entirely on errors we generated ourselves. On real OCR, which is what the pipeline will actually see, ByT5 is ahead.
Same story on sentence-level F1, where a sentence only counts if every error in it is fixed: ByT5 0.632, Gemini 0.534. Gemini finds more individual errors; ByT5 finishes more sentences.
ByT5 v12b is the pipeline model. Pushed to BDRC/tibetan-byt5-v12b (private).
Inference Cost
One pass over the 3,839-row test set:
| Model | Runs on | Time | Cost |
|---|---|---|---|
| TiSpell (semi-mask) | M5 Pro, MPS | 36 s | GPU time |
| ByT5 v12b | M5 Pro, MPS | 67 min | GPU time |
| ByT5 v12b | RTX PRO 6000 (rented) | 6 min | about ₹9 at $1/hr |
| Gemini 3.1-flash-lite | API | 17 min* | ₹48 |
| Gemini 3.6-flash | API | 1 h 4 min* | ₹324 |
*Gemini wall times are from a sequential runner with a 2-second sleep between calls, not API throughput.
The two laptop numbers show the architectural difference: TiSpell does one forward pass per sequence, ByT5 generates byte by byte, roughly 100x apart on the same machine. Both are unoptimised PyTorch. On a rented RTX PRO 6000 the same ByT5 pass drops to six minutes, and batched serving with something like vLLM would cut it further.
The API costs come from the billing console. Gemini 3.6-flash cost seven times Flash-Lite for identical work. Thinking tokens are most of that gap: 3.6-flash rejected thinking_budget=0, and every hidden reasoning token is billed at the output rate.
Side by side, the same 3,839 sentences cost ₹9 on a rented GPU, ₹48 on Flash-Lite, ₹324 on 3.6-flash. Both options scale with how much text you push through them, but they scale differently. GPU time responds to optimisation: compile the model, batch the requests, and the same corpus costs less. Per-token pricing has no such lever, and you are paying someone else’s margin on every sentence. Local also runs offline, has no rate limit, and does not depend on anyone else’s uptime.
What We Learned
More data alone did not move it. Several rounds of extra synthetic pairs, and a Tibetan-pretrained init, all landed near 0.62.
What moved it was matching the corruption mix to the real OCR error distribution. v12b changed both at once, more source text and the realistic mix, so we cannot split the credit cleanly. But volume on its own had already been tried, and had not worked.
Credits
Developed by Dharmaduta based on specifications from the Buddhist Digital Resource Center for the project The BDRC Etext Corpus, funded by the Khyentse Foundation.