Downloads provided by UsageCounts
The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent compute centres. Such a distributed data archive requires overarching data management orchestration. Such orchestration comprises of tools which handle data storage and cataloguing, and steering transfers integrating different storage systems and protocols, while being aware of data policies and locality. In addition, it needs a common Authorisation and Authentication Infrastructure (AAI) layer which is perceived as a single entity by end users and provides transparent data access. The scientific domain of particle physics also uses complex and distributed data management systems. The experiments at the Large Hadron Collider\,(LHC) accelerator at CERN generate several hundred petabytes of data per year. This data is globally distributed to partner sites and users using national compute facilities. Several innovative tools were developed to successfully address the distributed computing challenges in the context of the Worldwide LHC Computing Grid (WLCG). The work being carried out in the ESCAPE project and in the Data Infrastructure for Open Science (DIOS) work package is to prototype a Scientific Data Lake using the tools developed in the context of the WLCG, harnessing different physics scientific disciplines addressing FAIR standards and Open Data. We present how the Scientific Data Lake prototype is applied to address astronomical data use cases. We introduce the software stack and also discuss some of the differences between the domains.
4 pages, 1 figure, to appear in the proceedings of Astronomical Data Analysis Software and Systems XXXI published by ASP
astronomy, FOS: Computer and information sciences, Computer Science - Distributed, Parallel, and Cluster Computing, Rucio, FOS: Physical sciences, data management, Distributed, Parallel, and Cluster Computing (cs.DC), Astrophysics - Instrumentation and Methods for Astrophysics, Instrumentation and Methods for Astrophysics (astro-ph.IM)
astronomy, FOS: Computer and information sciences, Computer Science - Distributed, Parallel, and Cluster Computing, Rucio, FOS: Physical sciences, data management, Distributed, Parallel, and Cluster Computing (cs.DC), Astrophysics - Instrumentation and Methods for Astrophysics, Instrumentation and Methods for Astrophysics (astro-ph.IM)
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
| views | 2 | |
| downloads | 11 |

Views provided by UsageCounts
Downloads provided by UsageCounts