Bucket Object Versioning

Object Store data is not versioned by default. Bucket Object Versioning can be useful for protecting against accidental deletions or overwrites, for tracking changes over time and for allowing data recovery to previous states. It's not an alternative to backups, but rather a complement.

Procedure using S3 API

Our Object Store provides the S3 API and Bucket Object Versioning can be configured like it would be done in AWS.

When enabled on a bucket, deleting an object doesn’t actually remove its data immediately. Instead, S3 adds a delete marker that will now represents the latest version of the object. This object will then be hidden from a normal object listing. In the case of overwriting an object, it creates a new version for it that will now represents the latest version of the object.

Enabling for a bucket:

aws --endpoint-url https://objets.juno.calculquebec.ca s3api put-bucket-versioning --bucket <BUCKET> --versioning-configuration Status=Enabled

Verifying if it's enabled for a bucket:

aws --endpoint-url https://objets.juno.calculquebec.ca s3api get-bucket-versioning --bucket <BUCKET>

Note: the MFADelete attribute should be ignored as it's not compatible with our Object Store.

Listing object versions:

aws --endpoint-url https://objets.juno.calculquebec.ca s3api list-object-versions --bucket <BUCKET>

Lifecycle

Lifecycle processing rules can be configured to automatically clean up old object versions. Without such rules, every version of every object remains stored indefinitely which could lead to a huge storage usage over time.

Setting a lifecycle configuration on a bucket:

aws --endpoint-url https://objets.juno.calculquebec.ca s3api put-bucket-lifecycle-configuration --bucket <BUCKET> --lifecycle-configuration file://lifecycle.json

lifecycle.json example:

{
  "Rules": [
    {
      "ID": "ExpireOldVersions",
      "Status": "Enabled",
      "Filter": {
        "Prefix": ""
      },
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 1,
        "NewerNoncurrentVersions": 3
      }
    }
  ]
}

In this example, it means that we want to automatically permanently delete any noncurrent version that is older than 1 day and not among the 3 most newest versions.

Verifying the lifecycle configuration of a bucket:

aws --endpoint-url https://objets.juno.calculquebec.ca s3api get-bucket-lifecycle-configuration --bucket <BUCKET>

Note: lifecycle processing occurs once per day at midnight.

Procedure using Swift API

Some methods for Bucket Object Versioning are technically possible with the Swift API, but not supported on our Object Store.