Hindi
Hindi pages, signs and notebooks turned into text
Devanagari hangs its letters from a head line that runs unbroken across a whole word, so the shapes a recogniser has to separate sit below a bar rather than beside a space. Add conjunct clusters that fuse two or three consonants into a single new shape and vowel signs that attach above, below, before and after the letter they belong to, and ordinary OCR starts inventing words. Evidano transcribes Hindi as it stands on the page and shows you the image beside the text so every reading can be checked.
Sample pages
Real pages from public collections, shown beside their transcriptions. Pages whose transcription is still being checked are marked.
Digital Library of India, via Wikimedia Commons, Novel first published 1919; this printing Allahabad, Hans Prakashan · Source · Public domain
Transcription
The whole text block, from the end of chapter 3 to the opening of chapter 4, as proofread on Hindi Wikisource against this scan. Paragraphs are given one to a line; the running head (१६ · सेवासदन) is omitted and the centred chapter number is kept on its own line. Spelling, spacing and daṇḍas follow the Wikisource text.
कृष्ण--सुनो, यह रोने धोने का समय नही है। मै कानून के पन्जे में फँसा हूँ और किसी तरह नही बच सकता। धैर्य से काम लो, परमात्मा की इच्छा होगी तो फिर भेट होगी। यह कहकर वह बाहर की ओर चले कि दोनो लड़कियाँ आकर उनके पैरो से चिमट गई। गंगाजली ने दोनो हाथो से उनकी कमर पकड़ ली और तीनो चिल्लाकर रोने लगी। कृष्णचन्द्र भी कातर हो गए। उन्होने सोचा, इन अबलाओ की क्या गति होगी? परमात्मन्! तुम दीनो के रक्षक हो, इनकी भी रक्षा करना। एक क्षण में वह अपने को छुडा़कर बाहर चले गये। गंगाजली ने उन्हें पकड़ने को हाथ फैलाये, पर उसके दोनो हाथ फैले ही रह गये, जैसे गोली खाकर गिरनेवाली किसी चिङिया के दोनो पंख खुले रह जाते है। ४ कृष्णचन्द्र अपने कस्बे में सर्वप्रिय थे। यह खबर फैलते ही सारी बस्ती में हलचल मच गई। कई भले आदमी उनकी जमानत करने आये लेकिन साहब ने जमानत न ली। इसके एक सप्ताह बाद कृष्णचन्द्र पर रिश्वत लेने का अभियोग चलाया गया। महन्त रामदास भी गिरफ्तार हुए। दोनों मुकदमे महीने भरतक चलते रहे। हाकिम ने उन्हें दीरे सुपुर्द कर दिया। वहाँ भी एक महीना लगा। अन्त मे कृष्णचन्द्र को पाँच वर्ष की कैद हुई। महन्तजी सात वर्ष के लिए गये और दोनों चेलो को कालेपानी का दण्ड मिला। गंगाजली के एक सगे भाई पण्डित उमानाथ थे। कृष्णचन्द्र की उनसे जरा भी न बनती थी। वह उन्हें धूर्त और पाखंडी कहा करते, उनके
Transcription from पृष्ठ:सेवासदन.djvu/१७, proofread text on Hindi Wikisource (CC BY-SA 4.0).
National Museum, New Delhi · Source · CC BY-SA 4.0 · Photograph by SpeakingArch, via Wikimedia Commons
Transcription being verified
The lines are caupāīs from the Bālkāṇḍ of the Rāmcaritmānas, printed in every edition of the poem; the manuscript’s own spellings and word joins are being checked against the folio before the text is published here.
What the model is told to watch for: Produce a diplomatic transcription in Hindi.
Source · CC BY-SA 4.0 · Photograph by Psubhashish, via Wikimedia Commons
Transcription being verified
The Commons description records the red line as नशा मुक्ति केंद्र; the surrounding painted text and the number are being read off the full-resolution photograph.
What the model is told to watch for: Produce a diplomatic transcription in Hindi.
Source · CC BY-SA 4.0 · Uploaded to Wikimedia Commons by Mukhtaranis
Transcription being verified
No published transcription of this issue exists; the masthead, standfirst and lead headline are being transcribed and checked against the printed page.
What the model is told to watch for: Produce a diplomatic transcription in Hindi.
What Hindi on the page looks like
Hindi is written in Devanagari, an abugida in which each consonant carries an inherent a that is cancelled or replaced by a vowel sign. The sign for i is written to the left of the consonant it follows in speech, the sign for ā and ī to the right, u and ū underneath, and e, ai, o and au above; ṛ hangs below as a small hook. A dot on the head line, the anusvāra, nasalises the syllable, and candrabindu adds a crescent under it. Above all this runs the shirorekha, the horizontal line that joins the letters of a word into one continuous band and is the first thing a reader sees and the last thing a scribe draws.
Consonant clusters are written as conjuncts. Most are formed by clipping the first letter of its vertical stem and joining it to the next, but a few of the commonest — kṣa, tra, jña, and the stacked forms of ṭṭa, dda and hna — have shapes you cannot derive from their parts. Type founders varied: nineteenth- and early twentieth-century Hindi founts stack conjuncts vertically where modern founts write them side by side, and older books also glue postpositions to the noun, so a line prints kānūnke pañjemẽ where a modern paper prints three separate words. The dot under q, ḵẖ, ġ, z, f, ṛ and ṛh — the nuqtā that marks Perso-Arabic sounds — is set inconsistently and is often left out altogether.
Numerals are the other moving part. Older printing and much handwriting use the Devanagari digits ०१२३४५६७८९, while modern newspapers, price lines and telephone numbers use Latin digits, and one page can carry both, as a masthead dated in Latin above a price in Devanagari shows. Sentences close with a daṇḍa (।) rather than a full stop, verse with a double daṇḍa (॥), and modern prose freely mixes the two with commas, question marks and quotation marks. Devanagari is also the script of Marathi, Nepali, Sanskrit, Bhojpuri and Konkani, so a page in the script is not automatically a page in Hindi.
Why ordinary OCR struggles here
The head line binds a word into one shape
Latin-trained OCR looks for white space between characters. In Devanagari the whitespace is above the letters, not between them, and the shirorekha is a single unbroken stroke over several akṣaras. Segmentation has to cut below the bar at the right places; cut too early and a matra is orphaned, cut too late and two letters are merged into a conjunct that does not exist.
Conjuncts multiply the character set
A recogniser that knows the thirty-odd basic consonants still has hundreds of clusters to learn, many of them rare. When it meets one it does not know, it substitutes the nearest familiar shape, so prakāśa becomes prakasa and vidyā becomes vidhya. Half-forms lose their vertical stem, which is exactly the feature most models use to tell one letter from another.
Vowel signs sit on all four sides
The i-matra is written before the consonant it follows, so the visual order and the reading order differ by one character; the u and ū signs hang beneath the stem and are lost when the scan is contrasty; the anusvāra dot rides the head line and is easily read as a speck of dirt. Losing a single mark turns hai into ha, or a plural into a singular.
Everyday Hindi is rarely clean print
The material people actually upload is a hand-painted wall, a weekly paper printed on absorbent stock, a form filled in by hand, or a photocopy of a 1950s reprint whose type is worn. Hand-painted lettering thickens the head line into a slab, and handwriting often runs the head line on after the letters, or leaves it out, so a word arrives as a row of unattached hooks.
Who works with this material
Newspaper and periodical archives
Hindi dailies and weeklies from the 1920s onwards survive as bound volumes and microfilm. Indexing them needs a transcription that keeps headlines, bylines, datelines and column order apart, and that does not quietly modernise the spelling of the period.
Literary and book-history projects
Editors working on Premchand, the Sarasvatī generation and early Hindi prose want the text as printed, joined postpositions and all, so that variants between an early edition and a later reprint can be compared rather than smoothed away.
Records offices and NGOs
Registers, applications and field forms filled in by hand in Hindi are the daily material of land offices, panchayat records and survey work. What matters there is that names, dates and numbers come out exactly, with anything doubtful flagged instead of guessed.
Language technology and lexicography
Corpus builders need consistent treatment of the nuqtā, of anusvāra against a written half-nasal, and of Devanagari against Latin digits. Fixing those rules once in the prompt keeps a corpus internally consistent across thousands of pages.
Getting the best transcription
Fix the spelling conventions before you start
Say whether the nuqtā should be kept where the page prints it, whether ं should be preserved as written rather than expanded to a half nasal, and whether Devanagari digits should stay as they are or be converted. These are the choices that make a batch consistent.
Ask for the page as laid out
Request a line-for-line or paragraph-for-paragraph transcription with the running head, page number and column breaks marked separately. On a newspaper page, ask for each column to be finished before the next begins, or the story will interleave with its neighbour.
Do not let it modernise old print
Tell the model to keep joined postpositions, stacked conjuncts and old spellings exactly as set, and to add nothing that is not on the page. If you want a modernised reading text as well, ask for it as a second pass so the diplomatic version stays intact.
Test on the worst page you have
Start with the painted wall, the faded reprint or the handwritten form rather than the clean book page. If matras and the anusvāra survive there, the rest of the batch will need very little review.
Frequently asked questions
- Will Devanagari numerals be kept, or turned into 1, 2, 3?
- Either, as you choose. The default is to reproduce what the page prints, so ४.०० stays ४.०० and a Latin date stays Latin. If you need everything normalised for a database, ask for conversion in the prompt and it is applied consistently, including in mixed lines.
- Can it read older Hindi printing where postpositions are joined to the noun?
- Yes. Forms such as kānūnke or apneko are transcribed as printed rather than silently split, which matters when you are collating editions. If you would rather have modern word division, say so and the split is applied as a separate, reversible step.
- How are English words and Latin script inside a Hindi page handled?
- They are transcribed in their own script and kept in place. Mastheads, registration numbers, web addresses and product names constantly appear in Latin inside Devanagari text, and the transcription preserves the switch instead of transliterating one into the other.
- Can it tell Hindi apart from Marathi, Nepali or Sanskrit in the same script?
- It reads what is written, so the script is transcribed either way, and the language shows in the vocabulary and endings rather than the letterforms. If a page mixes a Sanskrit verse with Hindi commentary, ask for the two to be labelled and they are kept apart.
- What about handwriting where the head line is missing?
- Fast Hindi handwriting often draws the shirorekha last, breaks it between letters, or leaves it off entirely, and joins letters that print keeps separate. The model reads the letter bodies and their matras rather than relying on the bar, and marks anything it cannot resolve instead of guessing a plausible word.
- Are conjunct clusters expanded or kept as they are written?
- They are transcribed as the Unicode sequence the cluster represents, so kṣa, tra and jña come out as the letters they are made of and can be searched. Where a stacked conjunct in old type is ambiguous, the reading is flagged for review rather than resolved silently.
Related scripts and pages
Transcribe your Hindi pages
Upload a book page, a newspaper, a form or a photograph of a painted sign and check the Devanagari line by line against the image.