Create Write-Once-Read-Many Archive Storage with Amazon Glacier

Many AWS customers use Amazon Glacier for long-term storage of their mission-critical data. They benefit from Glacier’s durability and low cost, along with the ease with which they can integrate it in to their existing backup and archiving regimen. To use Glacier, you create vaults and populate them with archives (either directly or by using S3 lifecycle rules).

In certain industries, long-term records retention is mandated by regulations or compliance rules, sometimes for periods of up to seven years. For example, in the financial services industry (including large banks, broker-dealers, and securities clearinghouses), SEC Rule 17a-4(f) specifies that “electronic records must be preserved exclusively in a non-rewriteable and non-erasable format.” Other industries are tasked with meeting similar requirements when they store mission-critical information.

Lock Your Vaults
Today we are introducing a new Glacier feature that allows you to lock your vault with a variety of compliance controls that are designed to support this important records retention use case. You can now create a Vault Lock policy on a vault and lock it down. Once locked, the policy cannot be overwritten or deleted. Glacier will enforce the policy and will protect your records according to the controls (including a predefined retention period) specified therein.

You cannot change the Vault Lock policy after you lock it. However, you can still alter and configure the access controls that are not related to compliance by using a separate vault access policy. For example, you can grant read access to business partners or designated third parties (as sometimes required by regulation).

The Locking Process
Because the locking policy cannot be changed or removed after it is locked down (in order to assure compliance), we have implemented a two-step locking process in order to give you an opportunity to test it before locking the vault down for good. Here’s what you need to do:

1. Call InitiateVaultLock and pass along your Vault Lock retention policy. Glacier will install the policy and set the vault’s lock state to InProgress, and then return a unique LockID (keep this value around for 24 hours, after which it will expire).
2. Test your retention policy thoroughly. The vault will behave as though subject to the policy during the testing period. During this period you should thoroughly test the operations that are prohibited by the policy and ensure that they fail as expected. For example, you will probably want to test DeleteArchive and DeleteVault attempts from the root account, all IAM users, and from any users with cross-account access. You should also let the 24 hour testing period go by and then test that users with proper permissions can delete the archives.
3. If you are satisfied that the policy works as expected, call CompleteVaultLock with the LockID that you dutifully saved in step 1, dust off your hands and stroll off into the sunset. The vault’s state will be set to Locked, and the policy will remain in effect until the heat death of the universe.
4. If the policy does not work as expected, call AbortVaultLock during the 24 hour window (or wait until it passes) in order to remove the in-progress policy. Refine the policy and start over at step 1.

In most cases, you should plan to create a vault, apply your locking policy, and then upload archives to the vault where they will be governed by the policy. This is because Vault Lock does not backdate existing archives or other activities that were performed before the vault was locked. As such, there is no way to ensure that they are in compliance with the policy.

Here’s a sample policy that implements a 365 day retention rule:

As you can see this policy denies everyone (AWS:*) the ability to use Glacier’s DeleteArchive function on the resource named vaults/BusinessCritical for a period of 365 days.

More Control
After you apply and finalize your locking policy, you can continue to use the existing vault access policy as usual. However, the vault access policy cannot reduce the effect of the retention controls in the locking policy.

In certain situations you may be faced with the need to place a legal hold on your compliance archives for an indefinite period of time, generally until an investigation of some sort is concluded. You can do this by creating a vault access policy that denies the use of Glacier’s Delete functions if the vault is tagged in a particular way. For example, the following policy will deny deletes if the LegalHold tag is present and set to true:

Once the investigation is over, you can remove the hold by changing the LegalHold tag to false.

From Our Customers
Records retention in the Financial Services industry is governed by strict regulatory requirements. In advance of today’s launch, several AWS Financial Services customers were given a sneak peek at Vault Lock to review the new features. Here’s what they had to say:

Matt Claus (Vice President and CTO of eSpeed at Nasdaq):

“Records retention in the financial services industry is subject to specific regulatory requirements that govern how and for what period of time certain data is stored. From the details we’ve reviewed with the AWS team, Amazon Glacier Vault Lock will provide a storage solution that can help regulated entities within the Nasdaq group, like eSpeed, meet data storage and retention compliance obligations in a cost-effective manner. We look forward to using this new offering. With the launch of Glacier Vault Lock, AWS continues to add solutions that provide highly available and scalable infrastructure to financial services firms.”

Rob Krugman (Head of Digital Strategy, Broadridge Financial Solutions):

“Amazon is moving into the regulatory archival space in a big way with WORM storage capability now available on Amazon Glacier, which gives us more options in product development as we design and build solutions to support the future of financial services.”

Lock up Today
This new feature is available now and you can start using it today. To learn more about this feature, read about Getting Started Amazon Glacier Vault Lock.

— Jeff;