• Publicationsclose
• Research dataclose
• Research softwareclose
• Article close
• Slovenščina 2.0: Empirične, aplikativne in interdisciplinarne raziskave close

The current special issue of the journal Slovenščina 2.0 focuses on the newly explored combination of crowdsourcing, language learning and linguistic resource creation. It contains five articles and one project report, providing insightful discussions on several aspects of this combination, as well as results which help us understand its versatile potential and the challenges to address in order to better exploit it.

One way to stimulate the use of corpora in language education is by making pedagogically appropriate corpora, labeled with different types of problems (sensitive content, offensive language, structural problems). However, manually labeling corpora is extremely time-consuming and a better approach should be found. We thus propose a combination of two approaches to the creation of problem-labeled pedagogical corpora of Dutch, Estonian, Slovene and Brazilian Portuguese: the use of games with a purpose and of crowdsourcing for the task. We conducted initial experiments to establish the suitability of the crowdsourcing task, and used the lessons learned to design the Crowdsourcing for Language Learning (CrowLL) game in which players identify problematic sentences, classify them, and indicate problematic excerpts. The focus of this paper is on data preparation, given the crucial role that such a stage plays in any crowdsourcing project dealing with the creation of language learning resources. We present the methodology for data preparation, offering a detailed presentation of source corpora selection, pedagogically oriented GDEX configurations, and the creation of lemma lists, with a special focus on common and language-dependent decisions. Finally, we offer a discussion of the challenges that emerged and the solutions that have been implemented so far.

The popularity of online crowdsourcing platforms was slowly increasing among language learners before the pandemic, but COVID-19 changed the educational systems worldwide. This study aims to uncover whether or not, and if ‘YES’, how the attitudes and habits of language learners concerning the use of crowdsourcing materials in Turkey, Bosnia and Herzegovina, the Republic of North Macedonia and Poland changed during the pandemic. To compare the pre-and during the covid crowdsourcing tool usage, the cross-culturally appropriate questionnaire utilised in the pre-COVID-19 period was used again. The collected data were analysed qualitatively and quantitatively to identify the differences between the periods. The study’s findings showed that the shift from face-to-face to online learning significantly affected the development of crowdsourcing platforms worldwide and their employment in the studied countries. The results also demonstrated that a combination of factors, such as reduced interactions with teachers and peers, an increase in workload, and a lack of support on the part of institutions, led to students taking responsibility for their learning. The number and characteristics of the popular platforms changed from country to country since expectations from students varied.

This work describes a blueprint for an application that generates language learning exercises from parallel corpora. Word alignment and parallel structures allow for the automatic assessment of sentence pairs in the source and target languages, while users of the application continuously improve the quality of the data with their interactions, thus crowdsourcing parallel language learning material. Through triangulation, their assessment can be transferred to language pairs other than the original ones if multiparallel corpora are used as a source. Several challenges need to be addressed for such an application to work, and we will discuss three of them here. First, the question of how adequate learning material can be identified in corpora has received some attention in the last decade, and we will detail what the structure of parallel corpora implies for that selection. Secondly, we will consider which type of exercises can be generated automatically from parallel corpora such that they foster learning and keep learners motivated. And thirdly, we will highlight the potential of employing users, that is both teachers and learners, as crowdsourcers to help improve the material.

While eTwinning is focused on facilitating collaboration among schools in Europe and beyond, the extensive participation of over one million teachers from 44 countries makes the program an extensive educational crowdsourcing activity. In this paper the program which structures the related pedagogical approaches and practices will be analyzed and discussed in light of the crowdsourcing principles. Teachers and students participate in the program voluntarily. All collaborative activities, material production and publication of results which take place online and emphasize language learning fulfil the characteristics of the effective use of crowdsourcing in education. Two kinds of analyses are undertaken, a global analysis of the program features and local analysis of the selected projects. The global analysis relates the crowdsourcing practices to the eTwinning activities. The local analysis is based on the outstanding projects submitted for evaluation for national awards in Poland, further exemplified by activities and reference to the public sites of the projects. The aim of the text is to show that teachers may effectively use crowdsourcing in educational practice even when not primarily focused on its application.

In this study, we investigate theoretical and practical issues connected to differentiating between core and peripheral vocabulary at different levels of linguistic proficiency using statistical approaches combined with crowdsourcing. We also investigate whether crowdsourcing second language learners’ rankings can be used for assigning levels to unseen vocabulary. The study is performed on Swedish single-word items. The four hypotheses we examine are: (1) there is core vocabulary for each proficiency level, but this is only true until CEFR level B2 (upper-intermediate); (2) core vocabulary shows more systematicity in its behavior and usage, whereas peripheral items have more idiosyncratic behavior; (3) given that we have truly core items (aka anchor items) for each level, we can place any new unseen item in relation to the identified core items by using a series of comparative judgment tasks, this way assigning a “target” level for a previously unseen item; and (4) non-experts will perform on par with experts in a comparative judgment setting. The hypotheses have been largely confirmed: In relation to (1) and (2), our results show that there seems to be some systematicity in core vocabulary for early to mid-levels (A1-B1) while we find less systematicity for higher levels (B2-C1). In relation to (3), we suggest crowdsourcing word rankings using comparative judgment with known anchor words as a method to assign a “target” level to unseen words. With regard to (4), we confirm the previous findings that non-experts, in our case language learners, can be effectively used for the linguistic annotation tasks in a comparative judgment setting.

This article reviews the European Network for Combining Language Learning with Crowdsourcing Techniques (enetCollect), an extensive network project created to foster research and innovation (R&I) on the combination of crowdsourcing and language learning. Accordingly, we explain how it began, introduce its overall logic and organization, and discuss its achievements in terms of both (1) creating a new R&I community through a concluded large network project, and (2) fostering R&I on a high-potential and mostly unexplored subject. We also discuss the challenges involved and lessons learned, whether in orchestrating and leading a new R&I community or the challenges we faced and generally observed in the efforts of enetCollect members, as they explored the many facets of such a versatile enterprise.

Texts often express the writer’s emotional state, and it was shown that emotion information has potential for hate speech detection and analysis. In this work, we present a methodology for quantitative analysis of emotion in text. We define a simple, yet effective metric for an overall emotional charge of text based on the NRC Emotion Lexicon and Plutchik’s eight basic emotions. Using this methodology, we investigate the emotional charge of content with socially unacceptable discourse (SUD), as a distinct and potentially harmful type of text which is spreading on social media. We experiment with the proposed method on a corpus of Facebook comments, resulting in four datasets in two languages, namely English and Slovene, and two discussion topics, LGBT+ rights, and the European Migrants crisis. We reveal that SUD content is significantly more emotional than non-SUD comments. Moreover, we show differences in the expression of emotions depending on the language, topic, and target of the comments. Finally, to underpin the findings of the quantitative investigation of emotions, we perform a qualitative analysis of the corpus, exploring in more detail the most frequent emotional words of each emotion, for all four datasets. The qualitative analysis shows that the source of emotions in SUD texts heavily depends on the topic of discussion, with substantial overlaps between languages.

This study addresses categorization issues related to adjective candidates in Estonian, focusing on the category of participles. The aim of the analysis was to assess the ranges of the prototypical adjective and to determine its degree of deviation on the prototypicality scale. The investigation was based on a group of validated adjectives – selected adjectives included in the Basic Estonian Dictionary – and two control groups of more and less lexicalized participles. We tested seven morphosyntactic corpus patterns characteristic of adjectives. The test patterns were based on the prototypical features of the adjective, as well as on observations made in the actual lexicographic analysis. To assess the sample words and determine the significance of the test patterns from the point of view of defining adjectivity, we used deviation analysis. The results of this study can be applied to establish a measure of adjectivity for lexicographic judgments when distinguishing, for instance, lexicalized participles from regular ones. Članek obravnava vprašanja kategorizacije, povezana s pridevniškimi kandidati v estonščini, s poudarkom na deležnikih. Cilj analize je bil oceniti razpon prototipskega pridevnika in določiti stopnjo njegovega odstopanja na lestvici prototipskosti. Raziskava je temeljila na skupini potrjenih pridevnikov – izbranih pridevnikov, vključenih v Osnovni slovar estonščine– in dveh kontrolnih skupinah bolj in manj leksikaliziranih deležnikov. Preverili smo sedem morfosintaktičnih korpusnih vzorcev, značilnih za pridevnike. Testni vzorci so temeljili na prototipskih značilnostih pridevnika in na opažanjih pri konkretni leksikografski analizi. Za oceno vzorčnih besed in določitev pomena testnih vzorcev z vidika opredelitve pridevnika smo uporabili analizo odklona. Rezultate študije je mogoče uporabiti za določitev merila pridevniške ustreznosti pri leksikografskih presojah, na primer za določanje leksikaliziranih deležnikov. 

Poročilo s konference Jezikovne tehnologije in digitalna humanistika (JTDH) 2022, ki je potekala septembra 2022 v prostorih Fakultete za družbene vede Univerze v Ljubljani. Konferenco je priredilo Slovensko društvo za jezikovne tehnologije (SDJT), v soorganizaciji s Centrom za jezikovne vire in tehnologije Univerze v Ljubljani (CJVT), Inštitutom za novejšo zgodovino (INZ) ter raziskovalnima infrastrukturama CLARIN.SI in DARIAH-SI.