Backing up Object Store

Object Store data, while stored redundantly via Ceph, is not backed up. Object Store buckets are backed up to the TSM tape system upon request only by following the procedures listed on this page.

The following contents and policies apply to backups by default:

What is in the backup?

* Only the bucket data is backed up.  We are not currently backing up the IAM policies of the buckets or objects.
* Only the current version of the data is seen by the backup system.  Object chunks or versioned objects are not seen by the backups system.

What is the backup policy?

* Backups are run on a daily basis.
* The current object and one modified version of object are kept (this is different than full bucket versioning).    
* The modified version is kept for 6 month - after that period only the current object is kept. 
* Deleted objects are kept for 6 months.

Backup Procedure

Please follow this procedure to request backups of your buckets.

List bucket

Send a list of buckets to back up to sd4h support with the name and ID of the project where the buckets live.

Give us permission

You need to configure the iam policy statement of all the buckets you want to back up so your TSM robot user in charge of the backup can access them. Here is the policy that needs to be added.

For example, using the aws cli, apply the policy on my-bucket using the my-profile identity.

First, we make sure that my-bucket currently has no policy.

$aws s3api --profile my-project  get-bucket-policy --bucket  my-bucket

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

If that command returns something, you need to add the new statement the existing policy. But we are not covering that here.

Adding policy.json to my-bucket

File : policy.json

{
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/tsm"]},
    "Action": [
      "s3:ListBucket",
      "s3:GetObject"
    ],
    "Resource": [
      "arn:aws:s3:::my-bucket/*",
      "arn:aws:s3:::my-bycket"
    ]
  }
]
}

Then loading the policy to the bucket:

$aws s3api --profile my-profile  put-bucket-policy --policy file://my-policy.json --bucket my-bucket

Restore Procedure

List bucket

Send us list of buckets or object to restore to sd4h support.

Give us permission

You will be asked to create a bucket for each bucket you want to restore to restore with the -restore suffix.

File : policy.json

{
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/tsm"]},
    "Action": [
      "s3:ListBucket",
      "s3:GetObject",
      "s3:PutObject",
      "s3:PutObjectAcl",
      "s3:AbortMultipartUpload"
    ],
    "Resource": [
      "arn:aws:s3:::my-bucket-restore/*",
      "arn:aws:s3:::my-bycket-restore"
    ]
  }
]
}

Once it is done we will restore you data to that folder.