Preserving manuscript heritage across scripts
How Grantha bridges orthographic barriers, rescues uncatalogued folios, and accelerates archivist cataloguing across 13,358 historic volumes from the Digital Library of India.
The script barrier: Uniting divided traditions
The cultural dilemma: For centuries, Indian literature existed in polyglot coexistence: a Sanskrit philosophical treatise might have commentaries written in Bengali, Tamil, Telugu, or Kannada scripts. Modern multilingual AI embeddings inadvertently create orthographic apartheid, clustering by writing system first. A Sanskrit commentary and its Tamil counterpart sit in isolated clusters, with a meager 2.7% of raw nearest neighbours crossing script lines.
The Grantha solution: By computing and subtracting each language’s empirical centroid vector in the embedding space, Grantha dissolves script-specific clustering while safeguarding genuine semantic and thematic relationships. Script mutual information plummets from 0.642 down to 0.065, boosting cross-script scholarly discovery from 2.7% to 37.2%.
- Raw script NMI (orthographic bias)
- 0.642
- Centred script NMI (after centring)
- 0.065
- Test T2 subject discovery gain
- +14.5 pp (p = 0.0005)
Rescuing unindexed folios and damaged leaves
The cultural dilemma: Physical archives face catastrophic fragmentation. Fragile palm-leaf folios and paper leaves frequently detach from their bindings. Without cover sheets, accession numbers, or legible titles, damaged or skewed scans are routinely misfiled or lost forever in digital repositories.
The Grantha solution: An automated ingest pipeline tests four image orientations, performs optical character recognition via Tesseract across seven historical writing systems, and applies a strict 20-token safety threshold. Degraded scans abstain rather than mislead (2.6% on held-out test), while salvageable single folios are matched to their source book’s topic group 75.2% of the time, compared to 35.9% random chance.
- Single-page to book agreement
- 75.2% (95% CI 74.1% to 76.3%)
- Weak-OCR safety abstention rate
- 2.6% on held-out test
- Digitised manuscript pages analysed
- 77,634
Accelerating archival cataloguing and duplicate detection
The cultural dilemma: Millions of digitised works remain trapped in massive backlog queues because manual cataloguing requires expert palaeographers for each distinct language. Concurrently, identical titles appear multiple times with varying transliterations and degraded metadata, consuming scarce preservation budgets.
The Grantha solution: A distance-weighted voting classifier over 25 nearest catalogue neighbours suggests the correct subject shelf code in its Top-3 recommendations for 91.6% of 1,168 test volumes. Furthermore, dense geometric embedding analysis detects unlinked duplicate editions with 100.0% precision (36.4% recall), providing a turnkey archivist workbench.
- Top-3 catalogue class recommendation
- 91.6% (1,168 test books)
- Duplicate edition detection precision
- 100.0% (36.4% recall)
- Technical documentation
- Download PDF guide
Interactive corpus explorer
Select a language to inspect its usable sample, OCR confidence, and topic-group distribution from the Digital Library of India research corpus.
- Status
- Loading corpus statistics…