Backing up Object Store

Backup

It is possible to have backed up of object store data to our TSM tape system. Here is the formal procedure to do so.

List bucket

Send a list of buckets to back up to sd4h support with the name and ID of the project where the buckets live.

Give us permission

You need to configure the iam policy statement of all the buckets you want to back up so your TSM robot user in charge of the backup can access them. Here is the policy that needs to be added.

For example, using the aws cli and apply the policy on my-bucket using the my-profile identity.

First, we make sure that my-bucket has currently no policy.

$aws s3api --profile my-project  get-bucket-policy --bucket  my-bucket

An error occurred (NoSuchBucketPolicy) when calling the GetBucketPolicy operation: The bucket policy does not exist

If that command returns something, you need to add the new statement the existing policy. But we are not covering that here.

Adding policy.json to my-bucket

File : policy.json

{
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/tsm"]},
    "Action": [
      "s3:ListBucket",
      "s3:GetObject"
    ],
    "Resource": [
      "arn:aws:s3:::my-bucket/*",
      "arn:aws:s3:::my-bycket"
    ]
  }
]
}

Then loading the policy to the bucket:

$aws s3api --profile my-profile  put-bucket-policy --policy file://my-policy.json --bucket my-bucket

Restoring Data

Send us list of buckets or object to restore to sd4h support.

Give us permission

You will be asked to create a bucket for each bucket you want to restore to retore with the -restore suffix.

File : policy.json

{
"Statement": [
  {
    "Effect": "Allow",
    "Principal": {"AWS": ["arn:aws:iam:::user/tsm"]},
    "Action": [
      "s3:ListBucket",
      "s3:GetObject",
      "s3:PutObject",
      "s3:PutObjectAcl",
      "s3:AbortMultipartUpload"
    ],
    "Resource": [
      "arn:aws:s3:::my-bucket-restore/*",
      "arn:aws:s3:::my-bycket-restore"
    ]
  }
]
}

Once it is done we will restore you data to that folder.