It's Friday at 4.15pm and the weekend is just around the corner... are you ready to push the release button and go home? While there are many potentially nerve-wracking moments in any software delivery process, none can fill a team with white-hot fear quite like the last step: releasing to production. In the previous part of this series, we explained the differences between Continuous Delivery and Progressive Delivery. Now it's time to explore different strategies and ways to release software safely in production. Let's dive in!
There are different types of release strategies, but it all depends on your use case. For example, when you want to roll out your application’s new version for testing only, or any specific features for specific groups of users/customers just like beta versions. In this post, we'll discuss the following release strategies:
- Rolling update
- Blue-Green release
- Canary release
- Policy-based release
Rolling updates are the default deployment strategy for Kubernetes deployments. However, uncontrolled rolling updates might not be something you desire.
In this strategy, the Deployment selects a Pod with the old programming, deactivates it, and creates an updated Pod to replace it. The Deployment repeats this process until no outdated Pods remain. The advantage of the rolling update strategy is that the update is applied Pod-by-Pod so the greater system can remain active.
Blue-green releases uses a second, identical, but inactive production environment. This is where the upgrade is done, while the active production environment continues to serve all traffic. When the upgrade is tested and deemed fit for production, all traffic is switched over. At any time, only one of the environments is live, with the active environment serving all production traffic. In this scenario, cut-over time is minimized because most work is done on the inactive environment, where it doesn’t matter how long the upgrade takes.
- It's a strategy ;)
- Less risk relative to big bang strategy
- Rapid rollback (flip traffic back)
- Replicating production environments is complex and expensive
- Lose user transactions when flip is made
- All users are exposed to issues (when they occur)
Canary releasing is a methodology that reduces the impact of a failed release. It limits the number of users exposed to a bad release (known as limiting the blast radius) by gradually rolling out a new version of a service to a small, representative subgroup of real user traffic before rolling it out to the entire user base. The benefits of this approach are that you can carry out a gradual release, observe its behavior and then scale up or roll back.
Having said that, the downside is that performing a “classic” canary release is a manual and subjective process, with huge variation across release operators and teams. Bespoke decision-making and subjectivity are a major reason teams are stuck at re-inventing the release management wheel in isolation. They have little other choice than to create their own release automation code, which unfortunately means toil and sub-optimal code quality. This highly customized approach reduces re-usability of release automation code across teams.
- Release in small phases (2%, 10%, 25% etc.)
- Cheap and fast rollback
- The ability to test in production with real users
- Scripting can be complex and it doesn't scale
- Toil: manual configuration and monitoring
- Release decisions are subjective - when is a release (un)successful?
- Variation: every engineer or operator does it differently
So how do we go beyond having to custom-code everything, guesstimating what works and hand-holding our software releases?
The answer is to move towards a policy-based release methodology. Policy-based release management separates setting conditions – both technical and business – from enforcing the conditions. The key is to determine how you want your services to perform up-front, with input from all relevant stakeholders. You translate that input into release objectives and metrics (Service Level Objectives and Service Level Indicators), and implement these as code in a release policy. As a codified set of rules that manages the release, the policy lays the foundation for repeatability, observability and scale.
- Decoupling decision-making from the releasing process by settings objectives (SLOs) in advance helps to remove manual monitoring of each release, resulting in releases without human errors.
- Codifying the conditions around which to set the release test makes the release objective; add intelligent automation and the release process autonomously decides what to do next based on real-time metrics and pre-set conditions.
- No more untraceable decisions; release policies give you a data trail for incident response, post-mortems or compliance audits.
- Less toil and rework. Policies are reusable across teams, removing duplicate work so teams can spend more time on creating new code and features.
- Manually configure dynamic traffic shaping (or use Vamp and turn it into an automated process in seconds)
- You're introducing a new process
Release with confidence - even on a Friday afternoon (with a little help from Vamp)
Vamp will protect you and your users from the outages that come from replacing “good code” with “worse code”. With Vamp, each release is controlled by a release policy which contains precise criteria for determining whether the new version is “good enough” to replace the current version.
As a user, you set which release policy will be used for different types of release, for example emergency fixes, planned improvements or new features. You can select from a library of predefined policies. You can use these policies out-of-the-box or you can customise them to suit your application.
Let's be honest - it’s already hard enough to automate a rollback or a canary release for one service. But without a structured and standardized set of conditions on which to base your release test, multiple teams testing multiple services across dependent services will run into huge issues and downtime.
That’s where release policies make the difference!
They lay the foundation for a robust automated release management process that scales – allowing you and your release team to push the release button – even on a Friday afternoon. Try Vamp for free right here or request a demo.