Introduction The RIMES-database (Reconnaissance et Indexation de données Manuscrites et de fac similÉS / Recognition and Indexing of handwritten documents and faxes) comprises handwritten correspondence letters, in French, “sent” by individuals to companies or administrations; all correspondence is fictitious and there is no PII in the records. The database was collected by asking volunteers to write handwritten letters in exchange for gift vouchers. Volunteers were given a fictitious identity (same sex as the real one) and up to 5 scenarios. Each scenario was chosen from among 9 realistic topics: change of personal data (address, bank account), request for information, opening and closing (customer account), change of contract or order, complaint (e.g. poor quality of service), payment difficulties (request for delay, tax exemption, etc.), reminder, claim with other circumstances and a target (administrations or service providers such as telephone, electricity, bank, insurance companies). The volunteers wrote a letter with this information in their own words. The layout was free and the only request was to use white paper and to write legibly in black ink. Contents There are several files in this record; they correpond to the whole dataset and also some subsets that were created during the project. The Communications part is where the full document images and their annotations are stored. We give some detail of what the annotations comprise and briefly describe the other subsets; NB there was a subset comprising images from the cropped logos, which are not distributed here, due to some issues with the annotations. Communications -- Images_Courriers.zip There are in total 5605 communications, each containing from 2 to 3 pages: One correspondence (mandatory) One questionnaire (mandatory) One fax (optional) Filenames are constituted as follows: Communication number [1, 5605] Underscore “_” One letter [F, L, Q] L for correspondence/Letter (Lettre, in French) Questionnaire Fax The images and the corresponding annotations are split into 3 folders: DVD1: images from 1 to 1799 DVD2: images from 1800 to 3699 DVD3: images from 3700 to 5605 There are in total 12610 images. The annotation files are in xml format and support different tasks: Document Structure Identification Handwritten text recognition Writer recognition Information Extraction Document Structure Identification For the document structure, there are 8 "types" to be identified in the Letters for the different blocks of text in the image: Sender address Recipient address Date/Place (each is annotated with it's own tag) Subject Introduction (Ouverture in French) Text body Signature PS / annex Types 1 and 2 have further details, if it is a person or a corporate entity and address can also contain telephone/fax number. For example: Coordonnées Expéditeur (Sender Address) Maxime Granier\n13 Grand rue\n57370 Dames et Quatre Vents In the case of Faxes, the types can be further complemented by the type of text: Dactylographié, which stands for Printed; Manuscrit, for Handwritten text. For instance: Expéditeur_autre / Dactylographié De : Expéditeur_personne / Manuscrit Lucie FOURES Questionnaires have several other types, but there is no remanescent documentation about them. Handwritten text recognition There are annotations for the paragraphs; line breaks are indicated with "\n". The transcriptions are verbatim and contain the same spelling and grammar errors that could be seen in the pages. When there could be more than one possible spelling (j’essaie/j’essaye, événement/évènement, ultrason/ultra son, and writing errors), the options are in the ground truth following a special construction: ¤{alternative_1/alternative_2}¤ Writer recognition There is an identity for each writer in the database, so the usual tasks of identification and verification can be realized. The writer is identified in the usual French form, with family name, in all caps, first followed by the given name, for instance: GRANIER Maxime Information Extraction Nine scenarios are annotated for the different types of communication. We provide some free translations into English. Scenario Free translation Changement de données personnelles Change of personal data Demande d'information Request for information Difficulté de paiement Payment Difficulties Fermeture de compte Account Closure Gestion de sinistre Claims Handling Modification de contrat / Commande Contract Changes / Order Ouverture de compte Account Opening Réclamation Complaint Relance de courrier sans réponse Correspondance Reminder Paragraphs -- images_blocs_de_texte.zip The main body of the letters was cropped from the full page images and stored as grayscale JPEG images and split into 3 folders: DVD1: images from 1 to 1799 (1796 images) DVD2: images from 1800 to 3699 (1899 images) DVD3: images from 3700 to 5605 (1905 images) The transcriptions can be obtained from the corresponding Communications transcriptions; the numbers in the filenames correspond to the communications. Cursive words -- imagettes_mots_cursif.zip The paragraphs were split into lines and each line was further split into words. The images were split into 57 blocks (lot in French) organized in folders named: lot_N_rimes_version_definitive, where N is the block number [1, 57] Each folder has data from 100 letters, further organized into sub-folders following the convention: _L Then each sub-folder has one image per word, with naming: _L__.tiff Where Line Number and Word position start from 0. The transcription should be inferred from the corresponding Communications transcriptions. Character snippets -- imagettes_caracteres.zip Words were split to characters (A to Z) and digits (0 to 9), totalling 95269 images. They are distributed in 3 folders: characters_rimes_DVD1 characters_rimes_DVD2 characters_rimes_DVD3 Each is further divided into 4 blocks (lot in French): lot_1 lot_2 lot_3 lot_4 The naming of the image files follows the following: __.png Where Class is in [A-Z0-9]. Acknowledgments This dataset was originally collected and prepared in 2007 by the following partners: DGA/CTA/DT/GIP - CEP Arcueil; TSP – ARTEMIS Télécom SudParis; and A2iA SA, as part of the Techno-Vision project. This project was funded by the French ministries for Research and Defense (Ministère de la Recherche and Ministère de la Défense). After the acquisition of A2iA SA in September 2018, Mitek Systems, Inc. became a legal owner of the dataset, and decided to release it publicly – which was one of the objectives of the project after its conclusion – under a permissive license in 2024, to encourage open science.