Backing up Object Store

It is possible to have our Object Store data backed up to our TSM tape system. Here are the default policies and the content of our backups.

What is in the backup?

* Only bucket data is backed up, we are currently not backing up the buckets or object IAM policies.
* Only current version of the data is seen by the backed up system, object chunk or object versiond, are not seen by the backups system.

What is the backup policy?

* We run backup on a daly basis.
* Current object and one modified version of object are kept (this is different than bucket versioning).    
* The modified version is keept for 6 month, after that period only the current object is kept. 
* Deleted objects are keept for 6 months.

Backup Procedure

If you want to have backups of your bukets, here is the formal procedure to do so.

List bucket

Send a list of buckets to back up to sd4h support with the name and ID of the project where the buckets live.

Give us permission

You need to configure the iam policy statement of all the buckets you want to back up so your TSM robot user in charge of the backup can access them. Here is the policy that needs to be added.

For example, using the aws cli and apply the policy on my-bucket using the my-profile identity.

First, we make sure that my-bucket has currently no policy.

$aws s3api --profile my-project  get-bucket-policy --bucket  my-bucket

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

If that command returns something, you need to add the new statement the existing policy. But we are not covering that here.

Adding policy.json to my-bucket

File : policy.json

{
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/tsm"]},
    "Action": [
      "s3:ListBucket",
      "s3:GetObject"
    ],
    "Resource": [
      "arn:aws:s3:::my-bucket/*",
      "arn:aws:s3:::my-bycket"
    ]
  }
]
}

Then loading the policy to the bucket:

$aws s3api --profile my-profile  put-bucket-policy --policy file://my-policy.json --bucket my-bucket

Restore Procedure

List bucket

Send us list of buckets or object to restore to sd4h support.

Give us permission

You will be asked to create a bucket for each bucket you want to restore to retore with the -restore suffix.

File : policy.json

{
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/tsm"]},
    "Action": [
      "s3:ListBucket",
      "s3:GetObject",
      "s3:PutObject",
      "s3:PutObjectAcl",
      "s3:AbortMultipartUpload"
    ],
    "Resource": [
      "arn:aws:s3:::my-bucket-restore/*",
      "arn:aws:s3:::my-bycket-restore"
    ]
  }
]
}

Once it is done we will restore you data to that folder.