HomeCrunch

Setup Recycle Bin

Learn how to recover accidentally deleted objects.

The Granica Crunch Recycle Bin persists data even after it has been deleted so that you can quickly and easily recover accidentally deleted objects. The Recycle Bin is enabled via Crunch policies.

Recycle Bin basics

  • Object Deletion: When you delete an object using the S3 rm command Crunch automatically "moves" the deleted object to the Recycle Bin. The Recycle Bin is a virtual bucket called 'n-virtual-recyclebin'. There is only one Recycle Bin bucket per Crunch deployment. Objects placed by Crunch in the Recycle Bin are no longer visible in the crunched/source data bucket.

  • Object Location: The Recycle Bin preserves the full object path and prepends it with the crunched/source data bucket name. Here are some illustrative object and bucket paths before and after an object is deleted:

    • Before Delete: BUCKET=my_bucket OBJECT=pictures/tahoe.jpg
    • After Delete: BUCKET=n-virtual-recyclebin OBJECT=my_bucket/pictures/tahoe.jpg

  • Object Recovery: Via the Granica S3 endpoint you can perform LIST, HEAD and GET operations on the Recycle Bin and its objects just as you would for regular objects and buckets. This allows you to recover objects as needed.

    note

    You can not PUT objects into the Recycle Bin.

  • Emptying the Recycle Bin: Crunch automatically removes the Recycle Bin objects after a configurable time period set in Crunch policy. You can also manually empty objects from the Recycle Bin using a DELETE request.

  • Reported Object Size: When Crunch moves an object from its original bucket to the Recycle Bin the size of the original bucket is not updated. When the object is ultimately emptied from the Recycle Bin the size of the original bucket is decremented. This helps you to more accurately track the storage usage and cost of your objects.

Setup Recycle Bin

The Recycle Bin is a Crunch policy and is disabled by default. You can get, set and delete the Recycle Bin policy using the granica policy edit command.

The recycle-bin field requires a sub-field delete-after set to a positive non-zero value which is the expiry period for objects in the Recycle Bin. Objects will be automatically and permanently deleted from the Recycle Bin after this period has expired. The following example shows the Recycle Bin enabled with an expiry period of 30 days.

standard:
recycle-bin:
delete-after: 30d

Restore objects

At present, we support the following procedures to restore an object from the Recycle Bin:

GET-OBJECT S3 API

Download the object to be restored from the Recycle Bin. This object will be downloaded and saved locally under the name “tahoe.jpg” in the example below.

aws --endpoint-url https://<BOLT_URL> s3api get-object --bucket n-virtual-recyclebin \
--key my_bucket/pictures/tahoe.jpg ./tahoe.jpg

PUT-OBJECT S3 API

Upload the object (i.e. tahoe.jpg) to any crunched bucket and any name of choice

aws --endpoint-url https://<BOLT_URL> s3api put-object --bucket new_bucket \
--key pictures/tahoe.jpg --body tahoe.jpg
note

When using GET-OBJECT and PUT-OBJECT operations to restore objects, after restoration succeeds the objects in the Recycle Bin are permanently deleted.

RESTORE-OBJECT S3 API

When Crunch receives a RESTORE-OBJECT s3 client request where the bucket is the Recycle Bin (n-virtual-recyclebin) the original crunched bucket name is extracted from the object name. Note that the original crunched bucket name is the prefix of the objectname. The object is logically “moved” from the Recycle Bin back to the crunched bucket. Here the object “pictures/tahoe.jpg” is restored to the crunched bucket called "my_bucket":

aws --endpoint-url https://<BOLT_URL> s3api restore-object --bucket n-virtual-recyclebin \
--key my_bucket/pictures/tahoe.jpg

COPY-OBJECT S3 API

When Crunch receives a COPY-OBJECT S3 client request where the source bucket is the Recycle Bin (n-virtual-recyclebin) and the destination bucket is the same as the original crunched bucket, then the object is logically "moved" from the Recycle Bin to the original crunched bucket. The COPY-OBJECT operation is different from RESTORE-OBJECT because it allows the restored object to have a different name.

aws --endpoint-url https://<BOLT_URL> s3api copy-object \
--copy-source n-virtual-recyclebin/my_bucket/pictures/tahoe.jpg \
--key lake_tahoe.jpg --bucket my_bucket

Permanently delete Recycle Bin objects

Objects in the Recycle Bin can be manually deleted before their automatic removal as defined by Crunch policy.

DELETE-OBJECT S3 API

aws --endpoint-url https://<BOLT_URL> s3api delete-object --bucket n-virtual-recyclebin \
--key my_bucket/pictures/tahoe.jpg

FAQ

What happens when an object is deleted that is already present in the Recycle Bin?

If an object in the Recycle Bin is deleted, it is permanently deleted. If the corresponding source object is deleted, no action is taken as from all usage perspectives, the object does not exist in Crunch. The Recycle Bin object will be deleted based on the policy set.

How is Recycle Bin data partitioning handled?

Recycle Bin object metadata and data is partitioned the same way it was before the object was deleted and moved to the Recycle Bin. The data continues to be in the same partition-specific target bucket as before.

Can objects be restored to any location?

If an object is restored using the manual GET-OBJECT/PUT-OBJECT procedure, then it can be restored to any bucket. If an object is restored using the RESTORE-OBJECT S3 API it will be restored to the same bucket and object name as before. If an object is restored using the COPY-OBJECT S3 API it will be restored to the same bucket (and will cause an error if a different bucket is chosen) but the object name can be different.

See also

Was this page helpful?

Questions? Contact us
© 2024 Granica. All rights reserved.
Read our privacy policy and cookie policy.