You’re 20…30…80 minutes into a production outage, and it’s Christmas Day. This year, your number came up when the powers that be spun the roulette wheel of on-call duty and instead of enjoying mulled wine with your family by the fire, you’re sweating with dry-mouth behind your laptop. Many scenarios have raced through your mind, such as, “can I run away?”, “will my second on-call pick-up?”, “should we try a patch?” (it looks so straightforward!). In the meantime, you’ve watched time plunge forward as ever-more users are being affected by, say, an automated failure on your check out flow. Now, after the ill-fated patch has done more harm than good, and you and your second on-call are faced with a dreaded manual rollback scenario, you might be asking the spirit of Christmas “why me?”. In this Christmas Blog Post Special, we invite you on a Christmas journey to answer a more constructive question….
Did it have to be this way?
The short answer to that question is NO! But for the long answer, let’s rewind the scene and see how it would play out if you had an intelligent release orchestration solution supporting your application at the time of the failure.
You are chilling with your family
In this scenario, you are enjoying mulled wine with your family by the fire. Meanwhile, under the hood of your application, the latest version of your checkout flow is being continuously validated by your release orchestration solution.
You receive an alert that there has been a failure. You login to your laptop and see that the failure has been corrected by an automated rollback. Under the hood, your release orchestration solution had detected an anomaly in the checkout flow: a health condition in the release policy for that service was no longer being met. That automatically triggered an on-failure procedure in the same policy you and your team had set weeks ago.
But what about downtime?
On your laptop, you can see how much downtime there was, and what proportion of users were affected. This was minimal. And behind the scenes? Your friendly release orchestrator had prompted your CD pipeline to deploy the previous version of the checkout flow and its dependencies. It verified the health of the previous checkout flow version and then routed users safely to it, and away from the failing version.
But what about incident response?
You see on your laptop that the release orchestrator has already identified and secured all the data you and your team need to start fixing the issue on the next normal working day. You calmly close your laptop and decide to have a bit more wine and get on with a cozy Christmas. And your second on-call? Probably sleeping off a good dinner…
...And you are 0 minutes into a production outage.
But wait! We haven’t finished telling our Christmas tale yet!
Releasing a new feature to increase global conversion rates…even on Christmas!
Now let’s rewind the scene again and see how it would play out if the business had decided to roll out a new feature to increase conversion rates going out to all your global customers at the start of their Christmas day – from Australia, to APAC, EMEA, and North America.
You are chilling with your family
Again, you are enjoying mulled wine with your family by the fire. Meanwhile, under the hood of your application, the latest version of your new feature is being rolled out by your release orchestration solution (spoiler alert: we recommend Vamp!).
Then quietly a release is set in motion
While you put your feet up, a release plan that you set in advance triggers a canary release to production. The release plan controls when and in what order you’ll release the feature to which environment, starting with Australia, then rolling out to APAC, EMEA, and finally, North America.
And intelligent automation completes it
At this point, an automated canary-release policy takes over for each service. It first tests technical health on a small sub-segment of users and then adds conditions like the number of baskets converted into sales by, for instance, logged in iPhone users in Brisbane. If the conditions of the release are met for Brisbane, Vamp intelligently scales up user traffic to release to all users in Australia and continues to scale intelligently and automatically across your global environments.
… And you are still 0 minutes into a production outage. With no pager alerts in sight.
Merry Christmas from Vamp!
At Vamp, we know what it’s like to work on slow, risky releases. We’ve been there ourselves! That’s why we built Vamp as a release orchestration solution that will take over release decisions for you. If you would like to experience how Vamp Cloud-Native Release Orchestration works, view our demo video, download our whitepaper on how Vamp release policies work, or book a guided tour of the product with one of our engineers.