• 2020-2024close
• SE close
• Digitala Vetenskapliga Arkivet - Ac... close
• Publikationer från KTH close
• Digital Humanities and Cultural Her... close
• UArctic close

Uppgifter för behandling av naturliga språk (NLP) har under de senaste åren visat sig vara särskilt effektiva när man använder förtränade språkmodeller som BERT. Det enorma kravet på datorresurser som krävs för att träna sådana modeller gör det dock svårt att använda dem i verkligheten. För att lösa detta problem har komprimeringsmetoder utvecklats. I det här projektet studeras, genomförs och testas några av dessa metoder för komprimering av neurala nätverk för textbearbetning. I vårt fall var den mest effektiva metoden Knowledge Distillation, som består i att överföra kunskap från ett stort neuralt nätverk, som kallas läraren, till ett litet neuralt nätverk, som kallas eleven. Det finns flera varianter av detta tillvägagångssätt, som skiljer sig åt i komplexitet. Vi kommer att titta på två av dem i det här projektet. Den första gör det möjligt att överföra kunskap mellan ett neuralt nätverk och en mindre dubbelriktad LSTM, genom att endast använda resultatet från den större modellen. Och en andra, mer komplex metod som uppmuntrar elevmodellen att också lära sig av lärarmodellens mellanliggande lager för att utvinna kunskap. Det slutliga målet med detta projekt är att ge företagets datavetare färdiga komprimeringsmetoder för framtida projekt som kräver användning av djupa neurala nätverk för NLP. Natural language processing (NLP) tasks have proven to be particularly effective when using pre-trained language models such as BERT. However, the enormous demand on computational resources required to train such models makes their use in the real world difficult. To overcome this problem, compression methods have emerged in recent years. In this project, some of these neural network compression approaches for text processing are studied, implemented and tested. In our case, the most efficient method was Knowledge Distillation, which consists in transmitting knowledge from a large neural network, called teacher, to a small neural network, called student. There are several variants of this approach, which differ in their complexity. We will see two of them in this project, the first one which allows a knowledge transfer between any neural network and another smaller bidirectional LSTM, using only the output of the larger model. And a second, more complex approach that encourages the student model to also learn from the intermediate layers of the teacher model for incremental knowledge extraction. The ultimate goal of this project is to provide the company’s data scientists with ready-to-use compression methods for their future projects requiring the use of deep neural networks for NLP. 

Iron ore mining in the Norrbotten region of Sweden began in the early years of the twentieth century as a commercially uncertain and even dangerous proposition. But even before it began to generate profits, public debate began over the appropriate role of the state and of private capital (including foreign investors). This included whether iron ore should be exported for profit or retained for future processing in Sweden—even though the technology and infrastructure for such domestic industry did not exist. Tracing the evolution of this debate in the Swedish news media through to the First World War, this paper argues that the revenue generated by exports became more attractive than the promise of future domestic industry because that revenue could underwrite pressing political objectives. Although domestic iron ore processing remained linked to visions of future industrial prosperity, uncertain visions of future prosperity lost appeal as the capacity for export revenues to generate prosperity in the present became more potent.

School in the shadow: Private education in Stockholm 1735. During the early eighteenth century, private education was a more significant sector of the educational market than was public education, regarding the number of students and teachers, the presence of female students and teachers, the social background of the students, and the introduction of a more diverse and modern curriculum. Hitherto, little has been known of the actual scope or general conditions of private education, which has been over-shadowed by studies of public education. The article maps private education through the Stockholm Church Consistory’s (Stockholms stads konsistorium) thorough inventory of private teachers in the capital of Sweden during 1734–36, providing information of both suppliers and consumers within the private sector of the educational market, as well as of the practice and functions of private education in early modern time.

Denna uppsats är en studie av Blåsmarks bönhusförenings nedläggning och överlåtelsen avbönhuset till Blåsmarks DUF år 1947. Studien är begränsad till tidsperioden 1945–1948 föratt undersöka föreningarna innan och efter den händelse som är i fokus i uppsatsen och ärbaserad på de mötesprotokoll bägge föreningar förde under tidsperioden 1945-1948. Iuppsatsen undersöks föreningarnas arbete innan bönhusföreningens nedläggning 1947 samtorsakerna bakom dess nedläggning och vilka följder det fick för Blåsmarks DUF. Skeendetundersöks ur ett processperspektiv och följer den analytiskt-rationella modellen. Studiensresultat visade att bönhusföreningens nedläggning och överlåtelse av bönhus till BlåsmarksDUF var ett beslut taget främst på grund av att båda föreningarna bestod av samma medlemmar och arbetet med bönhuset redan till stor del utfördes av Blåsmarks DUF. Vidarevisade även studien att arbetet för Blåsmarks DUF föreföll sig detsamma efter övertagandetav bönhuset men att det utökades till att omfatta frågor gällande skötseln av bönhuset. Detframkom även i studien hur Blåsmarks DUF glädjes av att få ett eget bönhus och att beslutetfrån båda föreningar varit enhälligt och utan opposition.  This thesis offers a study of the closure of the Blåsmarks bönhusförenings and the transfer ofthe prayer house to Blåsmarks DUF in 1947. The study is limited to the period 1945-1948 toexamine the associations before and after the events in question and is based on the protocolswritten during both associations’ meetings between 1945-1948. The essay examines theassociations’ work before Blåsmarks bönhusförenings closure in 1947, the reasons behind itsclosure and what consequences it had for Blåsmarks DUF. The event is examined from a process perspective and follows the analytical-rational model. The results of the study showedthat Blåsmarks bönhusförenings closure and the transfer of the prayer house was a decisionmade mainly because both associations consisted of the same members and the work with theprayer house was already largely carried out by Blåsmarks DUF. Furthermore, the study alsoshowed that the work for Blåsmarks DUF is pleased to have its own prayer house and that thedecision to transfer it to them was unanimous and without opposition. 

The use of Deep Learning methods for Document Understanding has been embraced by the research community in recent years. A requirement for Deep Learning methods and especially Transformer Networks, is access to large datasets. The objective of this thesis was to evaluate a state-of-the-art model for Document Layout Analysis on a public and custom dataset. Additionally, the objective was to build a pipeline for building a dataset specifically for Visually Rich Documents. The research methodology consisted of a literature study to find the state-of-the-art model for Document Layout Analysis and a relevant dataset used to evaluate the chosen model. The literature study also included research on how existing datasets in the domain were collected and processed. Finally, an evaluation framework was created. The evaluation showed that the chosen multi-modal transformer network, LayoutLMv2, performed well on the Docbank dataset. The custom build dataset was limited by class imbalance, although good performance for the larger classes. The annotator tool and its auto-tagging feature performed well and the proposed pipelined showed great promise for creating datasets with Visually Rich Documents. In conclusion, this thesis project answers the research questions and suggests two main opportunities. The first is to encourage others to build datasets with Visually Rich Documents using a similar pipeline to the one presented in this paper. The second is to evaluate the possibility of creating the visual token information for LayoutLMv2 as part of the transformer network rather than using a separate CNN. Användningen av Deep Learning-metoder för dokumentförståelse har anammats av forskarvärlden de senaste åren. Ett krav för Deep Learning-metoder och speciellt Transformer Networks är tillgång till stora datamängder. Syftet med denna avhandling var att utvärdera en state-of-the-art modell för analys av dokumentlayout på en offentligt tillgängligt dataset. Dessutom var målet att bygga en pipeline för att bygga en dataset specifikt för Visuallt Rika Dokument. Forskningsmetodiken bestod av en litteraturstudie för att hitta modellen för Document Layout Analys och ett relevant dataset som användes för att utvärdera den valda modellen. Litteraturstudien omfattade också forskning om hur befintliga dataset i domänen samlades in och bearbetades. Slutligen skapades en utvärderingsram. Utvärderingen visade att det valda multimodala transformatornätverket, LayoutLMv2, fungerade bra på Docbank-datasetet. Den skapade datasetet begränsades av klassobalans även om bra prestanda för de större klasserna erhölls. Annotatorverktyget och dess autotaggningsfunktion fungerade bra och den föreslagna pipelinen visade sig vara mycket lovande för att skapa dataset med VVisuallt Rika Dokument.svis besvarar detta examensarbete forskningsfrågorna och föreslår två huvudsakliga möjligheter. Den första är att uppmuntra andra att bygga datauppsättningar med Visuallt Rika Dokument med en liknande pipeline som den som presenteras i denna uppsats. Det andra är att utvärdera möjligheten att skapa den visuella tokeninformationen för LayoutLMv2 som en del av transformatornätverket snarare än att använda en separat CNN.

This dissertation investigates the role played by Dutch experts in various enterprises and organisations managed or administered by the early modern Swedish state. The work demonstrates how and why Dutch experts were introduced to Sweden, in what manner they were employed by the Swedish state and how their knowledge was utilised in state-controlled organisations. The overall results challenge the impression of an uncomplicated introduction, steered by the state. Instead, the research suggests that the development was multifaceted, where the role and function of state structures, international politics, networks, as well as concepts, including opportunism and pragmatism all played a part in the process which led to the employment of Dutch experts and use of Dutch knowledge. Underlying analytical tools used to demonstrate this include concepts such as collective knowledge, institutional memory, change agents, as well as strategies to minimise uncertainties and knowledge asymmetries in the recruitment process. The investigation consists of four empirical chapters. The first chapter focuses on some of the key Dutch individuals who settled in Sweden during the second half of the sixteenth century. It shows why a Dutch settlement took place, but also how a specific interest in Dutch trade expertise was developed by the Swedish political leadership. The chapter concludes with an analysis of how Dutch expertise was utilised to develop advanced long-distance trade. The second chapter follows the use of Dutch experts in the naval shipbuilding organisation in the first half of the seventeenth century. It shows how an almost complete reliance on Dutch experts and technical knowledge was established during this period. The chapter details how the Dutch introduction and utilisation evolved, when the naval organisation changed from being single-handedly steered by the king, to a situation where it was administered by an independent Admiralty Board. The third chapter investigates the use of Dutch experts in the foreign administration as well as in the administration and management of trade and commercial undertakings. It details how agents and diplomatic networks facilitated exchange and brought knowledge and experts to Sweden. The chapter also challenges an established notion that many of the trade enterprises that were established in Sweden up until the mid-1630s were failures. The results instead demonstrate how successive proposals to develop commercial ventures by Dutch experts created a necessary knowledge base in Swedish society, which in the long run led to the successful formation of advanced commercial enterprises. The final chapter focuses on the establishment of commercial fishing operations. It follows the introduction and growth of a Dutch model for how to develop advanced fishing, and explains why the model only led to a functioning enterprise towards the very end of the period of investigation. 

Educational content recommendation is a cornerstone of AI-enhanced learning. In particular, to facilitate navigating the diverse learning resources available on learning platforms, methods are needed for automatically linking learning materials, e.g., in order to recommend textbook content based on exercises. Such methods are typically based on semantic textual similarity (STS) and the use of embeddings for text representation. However, it remains unclear what types of embeddings should be used for this task. In this study, we carry out an extensive empirical evaluation of embeddings derived from three different types of models: (i) static embeddings trained using a concept-based knowledge graph, (ii) contextual embeddings from a pre-trained language model, and (iii) contextual embeddings from a large language model (LLM). In addition to evaluating the models individually, various ensembles are explored based on different strategies for combining two models in an early vs. late fusion fashion. The evaluation is carried out using digital textbooks in Swedish for three different subjects and two types of exercises. The results show that using contextual embeddings from an LLM leads to superior performance compared to the other models, and that there is no significant improvement when combining these with static embeddings trained using a knowledge graph. When using embeddings derived from a smaller language model, however, it helps to combine them with knowledge graph embeddings. The performance of the best-performing model is high for both types of exercises, resulting in a mean Recall@3 of 0.96 and 0.95 and a mean MRR of 0.87 and 0.86 for quizzes and study questions, respectively, demonstrating the feasibility of using STS based on text embeddings for educational content recommendation. The ability to link digital learning materials in an unsupervised manner—relying only on readily available pre-trained models—facilitates the development of AI-enhanced learning.

The focus of doctoral research is of great significance for the development of an academic discipline and the potential for innovation. A significant number of PhD graduates however expect to be employed as university teachers, and the correlation between competence needs as suggested by the undergraduate curriculum and PhD research appears to be weak. Based on library catalogue data and digital archives, this study investigates interrelations between the initial research orientation of individual scholars and the development of English as a university subject in Sweden. Dissertation topics 1950-1999 indicate a gradual shift from a dominance of linguistics in the earlier decades to a dominance of literary studies towards the end of the period. Dissertations in the field of English-language literature between 2000 and 2019 demonstrate a growing interest in literatures outside England and the United States, a predominance of studies of prose and a move towards contextual modes of criticism centred on social or political theories. Studies of modern or contemporary literature dominate greatly whereas there are few dissertations on older literature. Undergraduate course plans and literature lists for 2020 from the major research universities show a strong connection between first-term literature courses and current research, as indicated by the topics of PhD dissertations from the first two decades of the twenty-first century. The teaching of older literature is not supported by new research to the same extent, however, which means that it may become increasingly difficult to ensure the close links between research and study programmes stipulated in the Swedish Higher Education Act (1992:1434).