Phased Rollout Checklist

Use this checklist to help ensure a smooth deployment of Granica Crunch.

When you’re done developing your Granica Crunch integration and you’re ready to go live, refer to this checklist to make sure you cover all the critical steps. Some checklist items are only applicable for specific integrations.

1. QA functional testing

For the QA testing, we’ll start by crunching one test bucket.

  1. For the test bucket, add the s3:Get*, s3:List*, s3:DeleteObject* and s3:PutObject* permissions to the Crunch data-cruncher role
  2. Update the Crunch policy for the bucket to enable cleaner
  3. Update the app to send reads and deletes through Granica
  4. Crunch the test bucket
  5. Ensure the objects are cleaned in the test bucket in S3
  6. Ensure the app is able to read the objects via the Granica endpoint

2. QA performance testing

  1. Crunch a test bucket with a few GBs of data
  2. Measure read throughput and latency - the app should run with high concurrency and long enough to get accurate measurements. We suggest 100 workers running for 30 mins

3. Production testing

We will first test on a production bucket without cleaner so that there are no changes to the objects in the source bucket

  1. For one production bucket, add the s3:Get* and s3:List* permissions to the Crunch data-cruncher role
  2. Set cleaner to disable in Crunch policy
  3. Set crunch_delay as needed in the Crunch policy
  4. Update the bucket inclusion and exclusion filters and enable auto-discover in Crunch policy. This starts crunching all buckets that match the inclusion criteria.
  5. Update the production app to read objects through Granica
  6. Ensure the app is able to read the objects through the Granica endpoint
  7. Update the bucket policy to include the s3:DeleteObject permission to the Crunch data-cruncher role
  8. Update the production app to send deletes to the Granica endpoint
  9. Ensure the deletes objects are also deleted in S3
  10. App performance - the production app should be successful when it's run with typical parameters (number of workers, runtime)

4. Go-live!

  1. Run 'granica update' to pick up the software that enables cleaning source objects
  2. Enable cleaner for the bucket that was crunched
  3. Enjoy the savings!
  4. Update the inclusion and exclusion filters in policy.yaml, to include additional buckets to crunch

See also