Poq: Created page with "We recommend using Globus to share data Store in our Object Store. But you can also use bucket policies to share data with groups that are also tenants on our platform. = Share data with Bucket Policies = == Create the right policy == Note that we are only documenting the use of the S3 API, but note that setting will also be [https://docs.ceph.com/en/latest/radosgw/bucketpolicy/#swift transfer to the swift API]. We will show you you can create a [https://docs.c..."

2025-10-07T17:25:52Z

Created page with "We recommend using Globus to share data Store in our Object Store. But you can also use bucket policies to share data with groups that are also tenants on our platform. = Share data with Bucket Policies = == Create the right policy == Note that we are only documenting the use of the S3 API, but note that setting will also be [https://docs.ceph.com/en/latest/radosgw/bucketpolicy/#swift transfer to the swift API]. We will show you you can create a [https://docs.c..."

New page

We recommend using [[Globus]] to share data Store in our Object Store. But you can also use bucket policies to share data with groups that are also tenants on our platform.

= Share data with Bucket Policies =

== Create the right policy ==

Note that we are only documenting the use of the S3 API, but note that setting will also be [https://docs.ceph.com/en/latest/radosgw/bucketpolicy/#swift transfer to the swift API].

We will show you you can create a [https://docs.ceph.com/en/latest/radosgw/bucketpolicy/ bucket policy] to share selected objects from that bucket to other tenants of the platform.

First ask the other group what their project number on the Juno is. It is a 33 digit hexadecimal number [https://juno.calculquebec.ca/identity located here]. We will denote that number as <code><Remote-Project></code>. Here is a <code>policy.jon</code> example file that can share the content of <code>bucket-to-share</code> with all members of <code><Remote-Project></code>. Note that you cannot share the bucket with a specific member of that project; you share it with the project as a whole.

<source lang="json">{
"Version": "2012-10-17",
"Id": "S3PolicyId1",
"Statement": [
{
"Effect": "Allow",
"Principal": {"AWS": ["arn:aws:iam::<Remote-Project>:user/<Remote-Project>"]},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucket-to-share",
"arn:aws:s3:::bucket-to-share/*"
]
}
]
}</source>
Note that you need to give access to <code>bucket-to-share</code> itself, so you can read information about it. You also need to give access to the objects with the glob's notation <code>*</code>. It also means that you can give access to a specific list of objects like this:

<source lang="json">{
"Version": "2012-10-17",
"Id": "S3PolicyId1",
"Statement": [
{
"Effect": "Allow",
"Principal": {"AWS": ["arn:aws:iam::<Remote-Project>:user/<Remote-Project>"]},
"Action": [
"s3:ListBucket",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucket-to-share",
"arn:aws:s3:::bucket-to-share/prefix-*",
"arn:aws:s3:::bucket-to-share/specific-file.txt"
]
}
]
}</source>
== Apply the Policy to the Bucket ==

You first need to [[create an s3 key and secret pair]]. Then use the client of you choosing to interact with the S3 API. We like the official <code>aws</code> client, since it is flexible and well documented.

<source lang="bash">aws s3api put-bucket-policy --bucket bucket-to-share --policy file://policy.json</source>

== How can the other group can read the data now? ==

That part is a bit more complicated... Every tenant in our Ceph cluster lives in its own namespace, which means that [https://stackoverflow.com/questions/24112647/why-are-s3-and-google-storage-bucket-names-a-global-namespace unlike the AWS] object store, out [https://docs.ceph.com/en/latest/radosgw/multitenancy/#accessing-buckets-with-explicit-tenants namespace configuration is not global]. It uses multi-tenancy... to an extent. When you are accessing the object store of your own tenant, it does not make a difference if the namespace is global or not. When accessing the data from another tenant, your tenant’s id is prepended to the bucket name like this : <code><Remote-Project>:<bucket-name></code>. This means that you need to share your bucket <code>project id</code> with the group you want to share the data with. Lest say that your <code>project id</code> is <code><my-project></code>, your collaborator will be able to use the <code>rclone</code> client out of the box like this:

<source lang="bash">rclone ls rclone-config-name:<my-project>:mybu/ </source>

However some clients will not accept the <code>:</code> in the bucket name. If you are using <code>boto3</code>, the official <code>aws</code> python <code>s3</code> client, you will need to add the <code>:</code> character to the VALID_BUCKET list of accepted `character before using it:

<source lang="python">import boto3
from botocore.config import Config
import botocore.handlers
import re
botocore.handlers.VALID_BUCKET = re.compile(r'^[:a-zA-Z0-9.\-_]{1,255}$')

[...]
# Specify the bucket name including the tenant
bucket_name = "tenant:bucketname"

# Example: List objects in the bucket
response = s3_client.list_objects_v2(Bucket=bucket_name)
[...]</source>

Finally, the <code>aws</code> client is also built on top of the <code>boto3</code> library, and changing the VALID_BUCKET value in the <code>awscli/botocore/handlers.py</code> installed with you <code>aws</code> client would also let you use that client. While it is a working hack, we would not recommend that last solution as a sustainable way of using that tool.

Share Object Store Data - Revision history