This software is developed for the tokenisation of Picard texts, e.g. splitting sentences into words and ponctuation signs. The tokeniser handles ambiguous separators such as dash, apostrophe, dot. The software is developed in Perl 5.22.1. The installation and the running issues are explained in the script file.

{"references": ["Delphine Bernhard, Amalia Todirascu, Fanny Martin, Pascale Erhart, Lucie Steibl\u00e9, Dominique Huck, Christophe Rey (2017). Probl\u00e8mes de tok\u00e9nisation pour deux langues r\u00e9gionales de France, l'alsacien et le picard"]}