Phased Rollout Checklist
Use this checklist to help ensure a smooth deployment of Granica Crunch.
When you’re done developing your Granica Crunch integration and you’re ready to go live, refer to this checklist to make sure you cover all the critical steps. Some checklist items are only applicable for specific integrations.
1. QA functional testing
For the QA testing, we’ll start by crunching one test bucket.
- For the test bucket, add the
s3:Get*
,s3:List*
,s3:DeleteObject*
ands3:PutObject*
permissions to the Crunch data-cruncher role - Update the Crunch policy for the bucket to enable
cleaner
- Update the app to send reads and deletes through Granica
- Crunch the test bucket
- Ensure the objects are cleaned in the test bucket in S3
- Ensure the app is able to read the objects via the Granica endpoint
2. QA performance testing
- Crunch a test bucket with a few GBs of data
- Measure read throughput and latency - the app should run with high concurrency and long enough to get accurate measurements. We suggest 100 workers running for 30 mins
3. Production testing
We will first test on a production bucket without cleaner so that there are no changes to the objects in the source bucket
- For one production bucket, add the
s3:Get*
ands3:List*
permissions to the Crunch data-cruncher role - Set
cleaner
to disable in Crunch policy - Set
crunch_delay
as needed in the Crunch policy - Update the bucket inclusion and exclusion filters and enable
auto-discover
in Crunch policy. This starts crunching all buckets that match the inclusion criteria. - Update the production app to read objects through Granica
- Ensure the app is able to read the objects through the Granica endpoint
- Update the bucket policy to include the
s3:DeleteObject
permission to the Crunch data-cruncher role - Update the production app to send deletes to the Granica endpoint
- Ensure the deletes objects are also deleted in S3
- App performance - the production app should be successful when it's run with typical parameters (number of workers, runtime)
4. Go-live!
- Run 'granica update' to pick up the software that enables cleaning source objects
- Enable
cleaner
for the bucket that was crunched - Enjoy the savings!
- Update the inclusion and exclusion filters in policy.yaml, to include additional buckets to crunch