
arXiv: 2207.04958
The opacity of machine learning data is a significant threat to ethical data work and intelligible systems. Previous research has addressed this issue by proposing standardized checklists to document datasets. This paper expands that field of inquiry by proposing a shift of perspective: from documenting datasets towards documenting data production. We draw on participatory design and collaborate with data workers at two companies located in Bulgaria and Argentina, where the collection and annotation of data for machine learning are outsourced. Our investigation comprises 2.5 years of research, including 33 semi-structured interviews, five co-design workshops, the development of prototypes, and several feedback instances with participants. We identify key challenges and requirements related to the integration of documentation practices in real-world data production scenarios. Our findings comprise important design considerations and highlight the value of designing data documentation based on the needs of data workers. We argue that a view of documentation as a boundary object, i.e., an object that can be used differently across organizations and teams but holds enough immutable content to maintain integrity, can be useful when designing documentation to retrieve heterogeneous, often distributed, contexts of data production.
transparency, FOS: Computer and information sciences, Computer Science - Human-Computer Interaction, 300 Sozialwissenschaften::300 Sozialwissenschaften, Soziologie::301 Soziologie, Anthropologie, data work, data labeling, 600 Technik, Medizin, angewandte Wissenschaften::600 Technik::600 Technik, Technologie, Human-Computer Interaction (cs.HC), Computer Science - Computers and Society, machine learning, data production, Computers and Society (cs.CY), dataset documentation, 000 Informatik, Informationswissenschaft, allgemeine Werke::000 Informatik, Wissen, Systeme::004 Datenverarbeitung; Informatik, data annotation
transparency, FOS: Computer and information sciences, Computer Science - Human-Computer Interaction, 300 Sozialwissenschaften::300 Sozialwissenschaften, Soziologie::301 Soziologie, Anthropologie, data work, data labeling, 600 Technik, Medizin, angewandte Wissenschaften::600 Technik::600 Technik, Technologie, Human-Computer Interaction (cs.HC), Computer Science - Computers and Society, machine learning, data production, Computers and Society (cs.CY), dataset documentation, 000 Informatik, Informationswissenschaft, allgemeine Werke::000 Informatik, Wissen, Systeme::004 Datenverarbeitung; Informatik, data annotation
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 24 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Top 10% | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Top 10% | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Top 10% |
