Store and Share data: Difference between revisions

← Older edit

Store and Share data (view source)

Revision as of 17:38, 20 May 2025

1,188 bytes added , 20 May

m

Fonts.

Dbrownlee

28

edits

@@ Line 6: / Line 6: @@
 = Configuring S3 access =
-==Install the openstack client==
+# You first need to have your [[OpenStack Client]] installed and configured.
-You will need to install and configure the [https://pypi.org/project/python-openstackclient/ openstack client] on your machine.
+# Then, with the client you generate an e2c/S3 id and secret
-<tabs>
+With the client is installed and the RC files downloaded in step 1 you can create the S3 ID and secret.
-<tab name="Fedora/RedHat Linux">
-You can install the client form you package manager
-{{Code
-  |lang="bash"
-  |contents=
-sudo dnf install python-openstackclient-doc.noarch
-# test the installation
-openstack --version
-}}
-</tab>
-<tab name="Debian/Ubuntu Linux">
-You can install the client form you package manager
-{{Code
-  |lang="bash"
-  |contents=
-sudo apt-get install python python-dev python-pip
-sudo pip install python-openstackclient
-# test the installation
-openstack --version
-}}
-</tab>
-<tab name="pip">
+<syntaxhighlight lang="bash" line>
-This will get you the latest version of the client
+$ source  $HOME/id/myproject-openrc.sh
-{{Code
+Please enter your OpenStack Password for project po-test as user poq:
-   |lang="bash"
+# Use the same password that you used to connect to the [https://juno.calculquebec.ca/ Juno web page].
-  |contents=
+# you can now create the credentials
-pip install python-openstackclient
+$ openstack ec2 credentials create
-# test the installation
++------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
-openstack --version
+| Field      | Value                                                                                                                                                                       |
++------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+| access     | XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX                                                                                                                                            |
+| links      | {'self': 'https://juno.calculquebec.ca:5000/v3/users/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/credentials/OS-EC2/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'} |
+| project_id | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx                                                                                                                                            |
+| secret     | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx                                                                                                                                            |
+| trust_id   | None                                                                                                                                                                        |
+| user_id    | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx                                                                                                            |
++------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
+</syntaxhighlight>
-}}
+The important values here are <code>access</code> and <code><code>secret</code>which are the S3 [aws_]access_key_id and [aws_]secret_access_key respectively. AWS stands for Amazon Web Services, they are the creator of the S3 API.
-</tab>
-<tab name="OSX">
-You need to have [https://brew.sh/ homebew] installed, then the client can be install using pipx
+= Manage you S3 buckets with Globus =
-{{Code
-  |lang="bash"
-  |contents=
-brew install pipx
-pipx install python-openstackclient
-# test the installation
-openstack --version
-}}
-</tab>
-<tab name="windows">
+See the [[Globus]] documentation
-Install your favorite linux distribution
-with [https://learn.microsoft.com/en-us/windows/wsl/install-manual Windows Subsystem Linux]
-and follow the precedure from another tab :)
-</tab>
-</tabs>
+= Use a S3 client to manage your bucket =
+There are a few clients that can be used to access Ceph S3 api. We recomend [[rclone]], it is fast and and flexible.
-==Get the RC file from Juno==
-Once the client is install you need to set credentials to login. You can get these credentials by connecting to the [https://juno.calculquebec.ca/ juno web page]. Select the right project to store your data and download the RC file:
-[[File:Openstack rc.gif|thumb|Download|left| the RC file from the Juno website]]
-==Create the S3 (ec2) Credentials==
-==Manage you S3 buckets with Globus==