Privacy by design suggests seven principles to embed privacy into systems, but artificial intelligence (AI) and data practice shows a ‘privacy paradox’ in the behaviour of users. The ever-evolving AI capabilities outpace privacy policies and people require more insight into the possible use of their data to make informed consent decisions. Keeping the human in the loop for reconsenting brings huge efforts and is disproportional to agile development. Decentral data management approaches promise efficient ways to handle crowd ownership, rights and their governance involvement, supporting privacy-by-design. This paper illustrates how a DataUnion approach for data-centric AI can efficiently combine decentralised, privacy-preserving features such as data non-fungible tokens (NFTs), federated learning, marketplaces, provenance or value sharing. The discussion concludes on data provenance being a key lever leading to the need for a blockchain protocol that makes privacy contributions an incentivised asset along the data value chain of enriching, verifying and governing data for AI. The quest for explainability in a model-centric approach is the quest for provenance in a data-centric approach.