• 2015-2024close
• Publications close
• Publikationer från KTH close
• Digitala Vetenskapliga Arkivet - Ac... close
• Digital Humanities and Cultural Her... close
• UArctic close

A neural language model trained on a text corpus can be used to induce distributed representations of words, such that similar words end up with similar representations. If the corpus is multilingual, the same model can be used to learn distributed representations of languages, such that similar languages end up with similar representations. We show that this holds even when the multilingual corpus has been translated into English, by picking up the faint signal left by the source languages. However, just as it is a thorny problem to separate semantic from syntactic similarity in word representations, it is not obvious what type of similarity is captured by language representations. We investigate correlations and causal relationships between language representations learned from translations on one hand, and genetic, geographical, and several levels of structural similarity between languages on the other. Of these, structural similarity is found to correlate most strongly with language representation similarity, whereas genetic relationships—a convenient benchmark used for evaluation in previous work—appears to be a confounding factor. Apart from implications about translation effects, we see this more generally as a case where NLP and linguistic typology can interact and benefit one another.

Uppgifter för behandling av naturliga språk (NLP) har under de senaste åren visat sig vara särskilt effektiva när man använder förtränade språkmodeller som BERT. Det enorma kravet på datorresurser som krävs för att träna sådana modeller gör det dock svårt att använda dem i verkligheten. För att lösa detta problem har komprimeringsmetoder utvecklats. I det här projektet studeras, genomförs och testas några av dessa metoder för komprimering av neurala nätverk för textbearbetning. I vårt fall var den mest effektiva metoden Knowledge Distillation, som består i att överföra kunskap från ett stort neuralt nätverk, som kallas läraren, till ett litet neuralt nätverk, som kallas eleven. Det finns flera varianter av detta tillvägagångssätt, som skiljer sig åt i komplexitet. Vi kommer att titta på två av dem i det här projektet. Den första gör det möjligt att överföra kunskap mellan ett neuralt nätverk och en mindre dubbelriktad LSTM, genom att endast använda resultatet från den större modellen. Och en andra, mer komplex metod som uppmuntrar elevmodellen att också lära sig av lärarmodellens mellanliggande lager för att utvinna kunskap. Det slutliga målet med detta projekt är att ge företagets datavetare färdiga komprimeringsmetoder för framtida projekt som kräver användning av djupa neurala nätverk för NLP. Natural language processing (NLP) tasks have proven to be particularly effective when using pre-trained language models such as BERT. However, the enormous demand on computational resources required to train such models makes their use in the real world difficult. To overcome this problem, compression methods have emerged in recent years. In this project, some of these neural network compression approaches for text processing are studied, implemented and tested. In our case, the most efficient method was Knowledge Distillation, which consists in transmitting knowledge from a large neural network, called teacher, to a small neural network, called student. There are several variants of this approach, which differ in their complexity. We will see two of them in this project, the first one which allows a knowledge transfer between any neural network and another smaller bidirectional LSTM, using only the output of the larger model. And a second, more complex approach that encourages the student model to also learn from the intermediate layers of the teacher model for incremental knowledge extraction. The ultimate goal of this project is to provide the company’s data scientists with ready-to-use compression methods for their future projects requiring the use of deep neural networks for NLP. 

Iron ore mining in the Norrbotten region of Sweden began in the early years of the twentieth century as a commercially uncertain and even dangerous proposition. But even before it began to generate profits, public debate began over the appropriate role of the state and of private capital (including foreign investors). This included whether iron ore should be exported for profit or retained for future processing in Sweden—even though the technology and infrastructure for such domestic industry did not exist. Tracing the evolution of this debate in the Swedish news media through to the First World War, this paper argues that the revenue generated by exports became more attractive than the promise of future domestic industry because that revenue could underwrite pressing political objectives. Although domestic iron ore processing remained linked to visions of future industrial prosperity, uncertain visions of future prosperity lost appeal as the capacity for export revenues to generate prosperity in the present became more potent.

School in the shadow: Private education in Stockholm 1735. During the early eighteenth century, private education was a more significant sector of the educational market than was public education, regarding the number of students and teachers, the presence of female students and teachers, the social background of the students, and the introduction of a more diverse and modern curriculum. Hitherto, little has been known of the actual scope or general conditions of private education, which has been over-shadowed by studies of public education. The article maps private education through the Stockholm Church Consistory’s (Stockholms stads konsistorium) thorough inventory of private teachers in the capital of Sweden during 1734–36, providing information of both suppliers and consumers within the private sector of the educational market, as well as of the practice and functions of private education in early modern time.

Denna uppsats är en studie av Blåsmarks bönhusförenings nedläggning och överlåtelsen avbönhuset till Blåsmarks DUF år 1947. Studien är begränsad till tidsperioden 1945–1948 föratt undersöka föreningarna innan och efter den händelse som är i fokus i uppsatsen och ärbaserad på de mötesprotokoll bägge föreningar förde under tidsperioden 1945-1948. Iuppsatsen undersöks föreningarnas arbete innan bönhusföreningens nedläggning 1947 samtorsakerna bakom dess nedläggning och vilka följder det fick för Blåsmarks DUF. Skeendetundersöks ur ett processperspektiv och följer den analytiskt-rationella modellen. Studiensresultat visade att bönhusföreningens nedläggning och överlåtelse av bönhus till BlåsmarksDUF var ett beslut taget främst på grund av att båda föreningarna bestod av samma medlemmar och arbetet med bönhuset redan till stor del utfördes av Blåsmarks DUF. Vidarevisade även studien att arbetet för Blåsmarks DUF föreföll sig detsamma efter övertagandetav bönhuset men att det utökades till att omfatta frågor gällande skötseln av bönhuset. Detframkom även i studien hur Blåsmarks DUF glädjes av att få ett eget bönhus och att beslutetfrån båda föreningar varit enhälligt och utan opposition.  This thesis offers a study of the closure of the Blåsmarks bönhusförenings and the transfer ofthe prayer house to Blåsmarks DUF in 1947. The study is limited to the period 1945-1948 toexamine the associations before and after the events in question and is based on the protocolswritten during both associations’ meetings between 1945-1948. The essay examines theassociations’ work before Blåsmarks bönhusförenings closure in 1947, the reasons behind itsclosure and what consequences it had for Blåsmarks DUF. The event is examined from a process perspective and follows the analytical-rational model. The results of the study showedthat Blåsmarks bönhusförenings closure and the transfer of the prayer house was a decisionmade mainly because both associations consisted of the same members and the work with theprayer house was already largely carried out by Blåsmarks DUF. Furthermore, the study alsoshowed that the work for Blåsmarks DUF is pleased to have its own prayer house and that thedecision to transfer it to them was unanimous and without opposition. 

The use of Deep Learning methods for Document Understanding has been embraced by the research community in recent years. A requirement for Deep Learning methods and especially Transformer Networks, is access to large datasets. The objective of this thesis was to evaluate a state-of-the-art model for Document Layout Analysis on a public and custom dataset. Additionally, the objective was to build a pipeline for building a dataset specifically for Visually Rich Documents. The research methodology consisted of a literature study to find the state-of-the-art model for Document Layout Analysis and a relevant dataset used to evaluate the chosen model. The literature study also included research on how existing datasets in the domain were collected and processed. Finally, an evaluation framework was created. The evaluation showed that the chosen multi-modal transformer network, LayoutLMv2, performed well on the Docbank dataset. The custom build dataset was limited by class imbalance, although good performance for the larger classes. The annotator tool and its auto-tagging feature performed well and the proposed pipelined showed great promise for creating datasets with Visually Rich Documents. In conclusion, this thesis project answers the research questions and suggests two main opportunities. The first is to encourage others to build datasets with Visually Rich Documents using a similar pipeline to the one presented in this paper. The second is to evaluate the possibility of creating the visual token information for LayoutLMv2 as part of the transformer network rather than using a separate CNN. Användningen av Deep Learning-metoder för dokumentförståelse har anammats av forskarvärlden de senaste åren. Ett krav för Deep Learning-metoder och speciellt Transformer Networks är tillgång till stora datamängder. Syftet med denna avhandling var att utvärdera en state-of-the-art modell för analys av dokumentlayout på en offentligt tillgängligt dataset. Dessutom var målet att bygga en pipeline för att bygga en dataset specifikt för Visuallt Rika Dokument. Forskningsmetodiken bestod av en litteraturstudie för att hitta modellen för Document Layout Analys och ett relevant dataset som användes för att utvärdera den valda modellen. Litteraturstudien omfattade också forskning om hur befintliga dataset i domänen samlades in och bearbetades. Slutligen skapades en utvärderingsram. Utvärderingen visade att det valda multimodala transformatornätverket, LayoutLMv2, fungerade bra på Docbank-datasetet. Den skapade datasetet begränsades av klassobalans även om bra prestanda för de större klasserna erhölls. Annotatorverktyget och dess autotaggningsfunktion fungerade bra och den föreslagna pipelinen visade sig vara mycket lovande för att skapa dataset med VVisuallt Rika Dokument.svis besvarar detta examensarbete forskningsfrågorna och föreslår två huvudsakliga möjligheter. Den första är att uppmuntra andra att bygga datauppsättningar med Visuallt Rika Dokument med en liknande pipeline som den som presenteras i denna uppsats. Det andra är att utvärdera möjligheten att skapa den visuella tokeninformationen för LayoutLMv2 som en del av transformatornätverket snarare än att använda en separat CNN.

This dissertation investigates the role played by Dutch experts in various enterprises and organisations managed or administered by the early modern Swedish state. The work demonstrates how and why Dutch experts were introduced to Sweden, in what manner they were employed by the Swedish state and how their knowledge was utilised in state-controlled organisations. The overall results challenge the impression of an uncomplicated introduction, steered by the state. Instead, the research suggests that the development was multifaceted, where the role and function of state structures, international politics, networks, as well as concepts, including opportunism and pragmatism all played a part in the process which led to the employment of Dutch experts and use of Dutch knowledge. Underlying analytical tools used to demonstrate this include concepts such as collective knowledge, institutional memory, change agents, as well as strategies to minimise uncertainties and knowledge asymmetries in the recruitment process. The investigation consists of four empirical chapters. The first chapter focuses on some of the key Dutch individuals who settled in Sweden during the second half of the sixteenth century. It shows why a Dutch settlement took place, but also how a specific interest in Dutch trade expertise was developed by the Swedish political leadership. The chapter concludes with an analysis of how Dutch expertise was utilised to develop advanced long-distance trade. The second chapter follows the use of Dutch experts in the naval shipbuilding organisation in the first half of the seventeenth century. It shows how an almost complete reliance on Dutch experts and technical knowledge was established during this period. The chapter details how the Dutch introduction and utilisation evolved, when the naval organisation changed from being single-handedly steered by the king, to a situation where it was administered by an independent Admiralty Board. The third chapter investigates the use of Dutch experts in the foreign administration as well as in the administration and management of trade and commercial undertakings. It details how agents and diplomatic networks facilitated exchange and brought knowledge and experts to Sweden. The chapter also challenges an established notion that many of the trade enterprises that were established in Sweden up until the mid-1630s were failures. The results instead demonstrate how successive proposals to develop commercial ventures by Dutch experts created a necessary knowledge base in Swedish society, which in the long run led to the successful formation of advanced commercial enterprises. The final chapter focuses on the establishment of commercial fishing operations. It follows the introduction and growth of a Dutch model for how to develop advanced fishing, and explains why the model only led to a functioning enterprise towards the very end of the period of investigation.