Tibetan spelling errors come in two sizes. A wrong component inside a syllable is local: you fix it by looking at that syllable. A whole syllable thatβs been deleted, transposed, or merged is not, because thereβs nothing at that position to correct and you need the surrounding sentence to notice itβs gone.
Standard spell checkers only solve the first kind. As part of the BDRC Etext Corpus project we asked a narrow question: which approach should we build on, what data does it need, and what will it cost at our scale? We compared six approaches, ran the leading candidate end to end, and priced it against the LLM alternative.
The Problem
Our input is already text. OCR happens upstream, and PDFs and EPUBs were never OCRβd at all, so this is text-to-text correction, not a vision task.
| Source | Error character | Signals available |
|---|---|---|
| OCR output | Visually driven. Characters misread from the page. | Text, plus per-token confidence if the engine emits it |
| PDFs / EPUBs | Human typing slips: transpositions, omissions, misspellings | Text only |
Any approach depending on OCR confidence scores can therefore serve at most half our input.
Tibetan syllables are stacks (root letter plus optional prefix, superscript, subscript, vowel, and suffix) separated by the tsheg (ΰΌ). Errors occur both inside a syllable and at the syllable level, where a whole unit goes missing or moves.
The Candidates
Six approaches across three categories. Tibetan-specific: TiSpell (semi-masked dual-head), Cleansing Jewel (Transformer with OCR confidence embeddings), rule-based syllable checkers. Adapted from other languages: DPCSpell (Bangla, detector/purificator/corrector) and ByT5 (byte-level seq2seq). General-purpose: fine-tuned LLMs via API or LoRA.
| Approach | Tibetan-specific | Code available | Syllable-level errors | Cost model | Verdict |
|---|---|---|---|---|---|
| TiSpell | Yes | Yes (MIT) | Yes, dedicated head | Fixed (self-hosted) | Recommended |
| Cleansing Jewel | Yes | No | Partial | Fixed | Blocked: needs confidence scores |
| Rule-based checkers | Yes | Various | No | Negligible | Pre-filter only |
| DPCSpell | No (Bangla) | Yes, with weights | Word-level, not syllable | Fixed | Wrong correction unit |
| ByT5 | No | Yes | Implicit | Fixed | Fallback if tokenization is the problem |
| LLM (API) | No | N/A | Yes, with full context | Per-character | Too expensive as default |
Key Findings
Conventional spell checkers canβt do syllable-level errors at all. In TiSpellβs benchmark, SymSpell, HunSpell and JamSpell all show βββ in the syllable-corruption column. Not a low score, but no capability: they have no mechanism for a deleted syllable, since thereβs no position to correct. That rules out the entire category before any performance comparison.
The gap between the top models is smaller than it looks. TiSpell reports 92.34 F1 on its real-world test set, ahead of RoBERTa+Bi-LSTM (91.61) and Soft-Masked RoBERTa (91.27). But the paperβs own F1-vs-corruption curves show all three within about 0.01 of each other, lines crossing repeatedly, dropping from roughly 0.996 on clean text to 0.90 at 30% corruption. The corpus we train on matters more than which of these we pick.
TiSpell ships no pretrained weights. Training from scratch is mandatory, at roughly 78 hours on an RTX 4090. Neither fact appears in the paper.
The repository needs work before it runs. Getting the pipeline working meant fixing an API key committed to the repo, a README pointing to a nonexistent file, incompatible version pins in requirements.txt, a config option the training script reads but never defines, an empty backbone directory, unreadable dataset archives, and an inference script with hardcoded paths plus an import for a missing module. Also worth noting: the paper reports w_C = 2.0 as optimal while the code defaults to 1.0, so training with defaults means not using the published best configuration. After the fixes, data loading, all nine corruption types, training, checkpointing, and inference all work.
Cost is decided by scale, not model quality. Self-hosting costs a fixed amount; APIs charge per character. Across roughly 30 billion characters, thatβs three orders of magnitude.
The 512-token window is a real constraint. TiSpellβs backbone caps at 512 usable tokens and the repository configures only 96. Syllable-level correction needs full-sentence context, so an error near a chunk boundary can go undetected entirely. Overlapping windows and chunking on the shad (ΰΌ) mitigate this, but never fully match a model that sees the whole document, and Gemini offers 1 million tokens on every tier.
TiSpellβs accuracy was measured on the wrong register. It was trained on Tibet University news articles and tested on web-collected modern text. Our documents are prayers and manuscripts: classical register, religious vocabulary, centuries older. We should not expect 92 F1 on our data.
Data
Training data teaches the model the correction task, and TiSpell can generate it by corrupting clean Tibetan text in nine ways, requiring no annotation. Benchmark data tells you whether the model works, and canβt be generated, because the point is that the errors are real. A model can score well on undoing our own synthetic damage and still fail on what OCR produces.
The annotation workflow gives us real parallel data
The OCR annotation pipeline has annotators producing transcriptions and reviewers correcting them: the annotator output is the erroneous version, the reviewer output the corrected one, and the pair is a real, human-verified example.
Almost every paper in this area, TiSpell included, notes that labelled parallel data is the bottleneck and works around it with synthetic corruption. Synthetic is a workaround for not having real pairs, not a preference. If annotation is already producing pairs as a side effect, we have something most projects donβt.
Three uses: measuring the real error distribution, which drives how we configure the corruption generator; the benchmark set, held out and never trained on; and direct training data, if volume allows.
One caveat: reviewer output isnβt automatically ground truth. Reviewers disagree, miss things, and sometimes edit for preference rather than correctness, so the benchmark set needs spot-checking.
How much data, and which source
This comes down to how many usable real pairs we have. With thousands or more, we train primarily on real pairs and use synthetic only to top up underrepresented error types, since real errors are the target distribution and synthetic only approximates it. With a few hundred, real pairs become benchmark-only and synthetic carries the training, arithmetic, not preference, since a small set canβt both train a model and serve as a held-out test. In between, mix and weight real pairs more heavily.
For reference, TiSpell used 50,000 sentences. Our benchmark set should be at minimum 1,000 sentences of real errors, ideally spanning both OCR-sourced and PDF/EPUB-sourced text.
Splits
| Split | Share | Source | Purpose |
|---|---|---|---|
| Train | 80% | Real pairs and/or synthetic, per the above | Fitting the model |
| Validation | 10% | Same pool as train | Deciding when to stop training |
| Test | 10% | Real annotator/reviewer pairs, held out | Measuring whether it actually works |
TiSpellβs code defaults to a split ratio of 0.999 (99.9% train, 0.1% validation), which is worth changing.
More importantly, the test split shouldnβt come from the same pool as train and validation. Validation tells you when the model stops improving at undoing your own corruptions; only held-out real errors tell you whether it works. We should also evaluate OCR-sourced and PDF/EPUB-sourced text separately rather than reporting one blended number.
Cost
Training is cheap either way. A Vast.ai RTX 4090 runs $0.13β0.40/hr, so a 78-hour run costs $25β30, and three to five runs while tuning comes to maybe $150.
Inference is where the paths diverge. BDRC has digitised over 8,000 volumes and archived more than 15 million pages, roughly 30 billion characters at about 2,000 characters a page.
| Approach | Full corpus, one pass |
|---|---|
| TiSpell, self-hosted | under $100 (GPU rental only) |
| Gemini Flash-Lite | $10,000β56,000 |
| Gemini Flash | ~$179,000 |
And thatβs per pass. Every model improvement repeats the API bill, while the self-hosted bill is another few hours of GPU. TiSpellβs training cost doesnβt appear here and the API needs no training at all β the APIβs real advantage, buried by scale.
Two caveats. Token counts depend heavily on the tokenizer, and general-purpose LLM tokenizers inflate Tibetan badly; one published example had a seven-word phrase expanding to 28 tokens under LLaMA2 and 43 under LLaMA3. We also havenβt benchmarked TiSpellβs throughput, so the under-$100 figure is conservative rather than measured.
Why Not an LLM
Cost and context length are covered above. The argument that weighs more for our content is generation versus editing.
An LLM writes corrected text token by token, so it can rewrite passages, substitute plausible alternatives, and hallucinate. TiSpell structurally cannot, because it edits positions in the input. On prayers and canonical manuscripts, a model that confidently rewrites is more dangerous than one that leaves errors in: an uncorrected error is visible and fixable, a silent βcorrectionβ of a canonical term into a modern equivalent is neither.
Our own layout detection benchmark reached the same conclusion on a different task. Gemini 2.5 Flash scored 28.3% mAP and Gemini 3.1 Pro 26.9%, both near the bottom of eleven models, beaten by small fine-tuned YOLO variants.
One argument here has weakened. We ruled out LLM fine-tuning because it needs a large labelled dataset we donβt have, but if annotation yields pairs at volume that objection loses force, LMSpell found LLMs beat encoder models at spelling correction once the fine-tuning dataset is large. Cost and hallucination still stand; βwe lack the dataβ may not.
Where an LLM would win is long-context correction on a small volume of high-value text. Worth keeping as a targeted tool rather than the default pipeline.
Why Not the Others
Cleansing Jewel is Tibetan-specific and purpose-built for OCR, cutting Googleβs Tibetan OCR error rate from 25% to 12.26% CER via a per-syllable confidence-score embedding. Two blockers: that signal doesnβt exist for PDFs or EPUBs, and thereβs no public code or data.
DPCSpell adds a purificator between detector and corrector, on the argument that correction quality depends on getting the masks right; their ablation shows it lifting mask accuracy to 96.86% exact match and corrector performance by 10.55 points. But it corrects at the word level, and Tibetanβs unit is the syllable.
Rule-based syllable checkers catch structurally illegal syllables with near-perfect precision, need no training, and run instantly. They canβt detect a syllable thatβs legal but wrong in context, which is most of what OCR produces. Good first pass, not a solution.
ByT5 reads raw bytes, so Tibetan tokenization canβt fragment it. No Tibetan pretraining and no syllable modelling, but a useful fallback if tokenization turns out to be the problem.
On copying Chinese or Mongolian. We already are. TiSpell is built on Soft-Masked BERT, a Chinese spell-checker, with the one thing Chinese doesnβt need added: recovering a deleted syllable. Mongolianβs PR2 uses RoBERTa post-processing the same way and reaches 91.2% character recognition, but itβs coupled to the OCR pipeline. Architectures transfer and trained models donβt, so we train from scratch either way and the only question is which blueprint.
The Domain Concern
Beyond the expected accuracy drop on classical text, thereβs a specific failure mode worth naming. A model that doesnβt know classical vocabulary wonβt fail quietly. The encoder flags an error when a token is unlikely given context, and it has no dictionary, so a valid canonical term it has never seen looks identical to an error. It canβt distinguish βunfamiliarβ from βwrong,β and may confidently replace a canonical term with a modern equivalent. That argues for the rule-based pre-filter, and for human review before anything is applied at scale.
The Pipeline
clean canonical Tibetan annotator/reviewer pairs
(prayers, manuscripts) (real wrongβcorrect)
β β
β ββββΊ error distribution
β β (tunes the generator)
β β
β β β β β β β β β β β β β
βΌ β
ββββββββββββββββββββββββββ β
β Corruption generator ββββββββββββββββββββββββββββββββββββββββββ
β ββ OCR-profile errors β β
β ββ typo-profile errorsβ (if volume β
βββββββββββββ¬βββββββββββββ is sufficient)β
βΌ β
training pool ββ β β β β β β β β β β β β β β β β β β β β β β β ββ
(80% train / 10% val)
β
βΌ
ββββββββββββββββββββββββββ
β Train TiSpell (GPU) β
βββββββββββββ¬βββββββββββββ
βΌ
trained model
β
βΌ
ββββββββββββββββββββββββββββββββββββββββ
β 1. Rule-based syllable check β
β 2. TiSpell correction β
β 3. [later] purification stage β
ββββββββββββββββββββ¬ββββββββββββββββββββ
βΌ
corrected Tibetan text
β
βΌ
ββββββββββββββββββββββββββββββββββββββββ
β Evaluate on held-out real pairs β
β OCR-sourced and PDF/EPUB separately β
ββββββββββββββββββββββββββββββββββββββββ
Solid arrows are the default path; the dashed line is conditional.
The annotation output feeds the generator, not just the benchmark. Real pairs tell us which corruptions to generate, and that loop matters more than any single hyperparameter. At high volume they can also join the training pool directly, via the dashed path.
The corruption generator is split by error profile. OCR errors and typing errors have different distributions, and thereβs evidence blending them naively hurts: a 2024 study on training LMs to correct OCR errors found blended models underperformed the baseline, suggesting specialised sub-models instead. We plan to train one model but evaluate the sources separately, and split only if a gap appears.
The rule-based filter runs before the model, not instead of it, removing mechanically impossible syllables cheaply and leaving the modelβs capacity for harder contextual cases.
Corruption should be modelled on observed confusions, not uniform random noise, and set slightly below the real error rate. Over-corrupted training teaches the model that heavy rewriting is normal, which is the failure mode we canβt afford.
If It Underperforms
In rough order of cost: raise context and add overlapping windows if failures cluster at chunk boundaries; add DPCSpellβs purification stage, but only if failures are specifically misplaced or missing masks; continue pretraining the encoder on our canonical corpus, which needs only clean text and the same masked-language-modelling objective; and fall back to ByT5 if tokenization is the problem.
Separately, an LLM stays available as a scoped fallback rather than a replacement. If failures concentrate where full-document context is decisive, routing a small high-value subset through an API is reasonable, the cost argument that rules it out across 30 billion characters doesnβt apply to a few hundred pages. The hallucination risk still does, so anything routed that way needs review.
One caveat on the comparison as a whole. The TiSpell authors state in their limitations that they couldnβt reproduce DPCSpell or LLM-based approaches due to time and GPU constraints, so that comparison has never been run, by them or by us. TiSpell is the best documented fit for Tibetan syllable structure, but βbest availableβ is more accurate than βbest.β
Sources
Tibetan:
- TiSpell, arXiv:2505.08037 and code
- Cleansing Jewel, arXiv:2304.03427, ACM TALLIP 23(5) Art. 73
- Backbone, openpecha/tibetan_RoBERTa_S_e3
- Tibetan NLP survey, arXiv:2510.19144
Other languages and methods:
- DPCSpell, arXiv:2211.03730, Computer Speech & Language 2024
- Soft-Masked BERT, ACL 2020
- PR2 (Mongolian), Alexandria Engineering Journal 2025
- ByT5, arXiv:2105.13626
- LMSpell, arXiv:2512.05414
- Denoising Transformer, arXiv:2105.05977
- Scrambled text (blended corruption levels), arXiv:2409.19735
Credits
Developed by Dharmaduta based on specifications from the Buddhist Digital Resource Center for the project The BDRC Etext Corpus, funded by the Khyentse Foundation.