<script type="text/javascript">
<!--
document.write('<div id="oa_widget"></div>');
document.write('<script type="text/javascript" src="https://www.openaire.eu/index.php?option=com_openaire&view=widget&format=raw&projectId=undefined&type=result"></script>');
-->
</script>
Evaluating Language Model Agency through Negotiations This repository contains the raw negotiation transcripts from the paper "Evaluating Language Model Agency through Negotiations" [1]. The data includes both transcripts from self-play (a model plays against itself; corresponding to Section 4.1 of paper) and cross-play (a model plays against another model; Section 4.2). In total, this encompasses 2926 transcripts (942 self-play, 1984 cross-play). The transcripts are structured in the following way: transcripts/ ├── self_play/ │ └── {model_name}/ │ ├── processed_negotiation.csv │ ├── negotiations.csv │ ├── interrogations.csv │ └── .hydra/ │ ├── config.yaml │ ├── hydra.yaml │ └── overrides.yaml └── cross_play/ └── {model_1_name}_{model_2_name}/ ├── processed_negotiation.csv ├── negotiations.csv └── .hydra/ ├── config.yaml ├── hydra.yaml └── overrides.yaml We also include metadata with rules and game setup in the metadata folder. metadata/ ├── agents/ │ └── anon.yaml └── game/ ├── generic-rental-agreement.yaml ├── generic_game_rules.yaml └── issues/ ├── {issue_i} ├── ... Citation Please cite the paper when using data. [1] T.R. Davidson, V. Veselovsky, M. Josifoski, M. Peyrard, A. Bosselut, M. Kosinski, R. West (2023). Evaluating Language Model Agency through Negotiations.
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |