Store and Share data

Intro

SD4H has a large Object Store. Objects are accessible through a web API via the Rados Gateway (radosgw) service. Both the S3 and swift API standards are supported by the radosgw.

In an Object Store, an object is the equivalent of a file on a posix file system. The object store gives users a lot of flexibility, but the steps to do simple tasks like sharing and transferring data involve somewhat of a learning cure. We propose a procedure here so this curve is as gentle as possible. Once done, the procedure will be both more secure and more flexible than sharing data on a share HPC platform or on some VM owned by your group.

Configuring S3 access

You will need to install and configure the openstack client on your machine.

Fedora/RedHat LinuxpipOSX

You can install the client form you package manager

sudo dnf install python-openstackclient-doc.noarch
# test the installation
openstack --version

This will get you the latest version of the client

pip install python-openstackclient
# test the installation
openstack --version

You need to have homebew installed, then the client can be install using pipx

brew install pipx
pipx install python-openstackclient
# test the installation
openstack --version