
This dataset accompanies the study Beyond Manual Media Coding: Evaluating Large Language Models and Agents for News Content Analysis. It provides a reproducible benchmark for evaluating automated content analysis methods against human-annotated ground truth. The dataset includes: articles.csvContains the 200 news articles collected for this study, each with: id: unique identifier url: source URL of the original article content: full text of the news article codebook.jsonA structured JSON file defining the 26-question analysis codebook used for annotation.Each question entry specifies: questionId: question ID (e.g., Q1) prompt: annotation question text questionAnswerType: type (SINGLE_CHOICE or MULTI_CHOICE) eligibleQuestionAnswers: list of possible tags / codes annotations.jsonContains the complete human annotation data.For each article id, it provides the list of responses to all 26 codebook questions as determined by an expert annotator, establishing the ground truth labels. Intended use Designed for research popuses including natural language understanding, content classification, and LLM evaluation. Please request access with your academic email.
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
