Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Presentation . 2019
License: CC BY
Data sources: Datacite
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Other literature type . 2019
License: CC BY
Data sources: ZENODO
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Presentation . 2019
License: CC BY
Data sources: Datacite
versions View all 2 versions
addClaim

Evolution of the S3 service at CERN as a storage backend for infrastructure services and software repositories

Authors: Leon, Jose Castro; Collet, Julien; Mouratidis, Theofilos; Herve Rousseau; Cameselle, Roberto Valverde; Ster, Dan Van Der; Bocchi, Enrico;

Evolution of the S3 service at CERN as a storage backend for infrastructure services and software repositories

Abstract

The S3 service at CERN (S3.CERN.CH) is a horizontally scalable object storage system built with a flexible number of virtual RADOS Gateways on top of a conventional Ceph cluster. A Traefik load balancing frontend (operated via Nomad and Consul) redirects HTTP traffic to the RGW backends, and LogStash publishes to ElasticSearch for monitoring the user traffic. User and quota management is delegated to OpenStack; a novel synchronization mechanism was developed to maintain consistency between Keystone and Ceph. The RADOS backend went through some notable operational exercises, including massive data rebalancing, a FileStore to BlueStore migration, and the addition of SSDs for bucket indexes; these operations gave us increased capacity and up to 50x speedup for some performance metrics. The usage of S3 at CERN has evolved in the recent years. The earliest adopters of S3 were the ATLAS Event Service and CMS BOINC use-cases; the latter takes advantage of S3 pre-signed URLs to offer a secure storage to untrusted remote user PCs. More recently our IT applications demanded S3 object storage: Gitlab and Nexus artifacts, ElasticSearch backups, Kubernetes and others. And our WLCG-specific software is also increasingly taking advantage of the service: CVMFS stratum zero data is a natural fit, and S3 combined with Restic offers a compelling $HOME backup service. Lastly, with the decommissioning of the CERN data centre in Hungary, disaster recovery requirements have motivated a second S3 region on the CERN Prevessin site, with async RGW replication for important buckets.

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
    OpenAIRE UsageCounts
    Usage byUsageCounts
    visibility views 17
    download downloads 16
  • 17
    views
    16
    downloads
    Powered byOpenAIRE UsageCounts
Powered by OpenAIRE graph
Found an issue? Give us feedback
visibility
download
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
views
OpenAIRE UsageCountsViews provided by UsageCounts
downloads
OpenAIRE UsageCountsDownloads provided by UsageCounts
0
Average
Average
Average
17
16
Green