HeLI off-the-shelf language identifier with language models for 200 languages. Usage: java -jar HeLI.jar -r <infile> -w <outfile> The program will read the <infile> and classify the language of each line as one of the 200 languages it knows and writes the results, one ISO 639-3 code per line, into file <outfile>. You can use the -c option to make the program print a confidence score for the identification after each language code. Usage: java -jar HeLI.jar -c -r <infile> -w <outfile> You can give the list of comma-separated ISO 639-3 identifiers for relevant languages after -l option. Usage: java -jar HeLI.jar -r <infile> -w <outfile> -l fin,swe,eng You can give the number of top-scored languages to print after the -t option. (overrides confidence) Usage: java -jar HeLI.jar -r <infile> -w <outfile> -l fin,swe,eng -t 2 If you omit both of the filenames, the program will read the standard input one line at a time and write the result to standard output. It can identify c. 3000 sentences per second using one core on a 2021 laptop and around 3 gigabytes of memory. If you use this program in producing scientific publications, please refer to: @inproceedings{jauhiainen-etal-2017-evaluation, title = "Evaluation of language identification methods using 285 languages", author = "Jauhiainen, Tommi and Lind{\'e}n, Krister and Jauhiainen, Heidi", booktitle = "Proceedings of the 21st Nordic Conference on Computational Linguistics", month = may, year = "2017", address = "Gothenburg, Sweden", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/W17-0221", pages = "183--191", } Producing and publishing this software has been partly supported by The Finnish Research Impact Foundation Tandem Industry Academia -funding in cooperation with Lingsoft.

{"references": ["Jauhiainen, Tommi et al. (2017). Evaluation of language identification methods using 285 languages. https://www.aclweb.org/anthology/W17-0221"]}