Dev vs Ops, product vs IT, senior management vs product management – there’s nothing like the fear of releasing new software into production for exposing the interdepartmental tensions bubbling under the surface of any organization. Even in organizations already using containerized apps and, in theory, working in agile principles, sprint cycles drag, releases get crammed with features, risk increases, and when decision-time comes, it can look like a game of hot potato. In our last blog post, we talked about how, if done in a managed way, testing in production isn’t scary but is a safe way of releasing faster without breaking things. But software releasing isn’t just about software, it’s also about people. The release process touches multiple roles in any organization. In this post, we’ll look at how safely managing testing in production can relieve the fear of “big bang” releases and make the jobs of all those involved in the release process, from product owner to support roles, easier.
How testing in production soothes silos, turf wars, and empire building
When the “release moment” is the first time you expose a big batch of changes to actual real users in production, it’s understandably nerve-wracking. With no way of testing a portion of live traffic against a new version, development cycles drag, and releases get bloated. Developers and product owners want to move faster, operations want to maintain the status quo, and other stakeholders like marketing or senior management add more changes to the list in order to push their changes through. How does testing in production break silos and turf wars?
- Testing in production means exposing a new feature or update to live production traffic, ideally to a small sub-set of users, which supports a more iterative, agile way of working
- Testing in production through iterative releasing allows you to control the impact of a release and scale up traffic if its successful, or rollback if it fails
- Smaller releases also mean a smaller number of changes in each release, so, if something goes wrong, it’s easier to pinpoint it and fix it
Now everyone involved has more confidence in releasing! How to restructure your release process to get there:
- Have a method in place to release incrementally, to a small sub-set of users (via a canary release)
- Specify the conditions under which you will segment users to create the sub-set (for instance, location, device, cookie, IP address, basket value). Use your business KPIs
- Specify the steps to increase user traffic if the release is successful (at Vamp, we codify this and the above step in flexible release policies)
- Specify the condition under which to rollback if the release fails. Automate rollback
- Release in a series of automated steps
- Make sure to monitor the release and observe. Specifying the release in advance creates health checks for both the tech and business impact of a release
- Share knowledge across teams so everyone knows what good looks like in production
Release orchestration touches multiple roles in your organization: everyone benefits!
Benefits for the application owner:
- Safer, reliable releases mean product owners can safeguard the customer experience for their service and the brand
- Tying business KPIs to releasing means POs can relate technical metrics to business metrics/outcomes
- This allows for safe experimentation like A/B testing to test and improve the business value of releases
Benefits for IT architects and IT/delivery managers:
- Structure, codification and feedback enable easier collaboration between IT and business
- Reduces the cost and length of testing and acceptance environments: better time-to-market and better-time-to value!
- Lays the foundation for a standardized and scalable way of releasing across teams
- Makes metrics actionable across the release process
Benefits for development teams:
- Developers can quickly validate their code
- This creates a positive feedback loop with less rework
- Teams can focus more on building new functionality with more time for innovation
Benefits for DevOps/Ops/SRE teams:
- Less time spent on firefighting, while more efficient at trouble shooting
- Less time spent on rework
- More time to remove technical debt created by patches made under pressure
- More time to contribute to valuable work
Add intelligent automation and you have stress-free, self-driving releases, even on a Friday
Having controlled, incremental releases tested in production on live user traffic lays the foundation for more confidence in the release process. When everyone in the process can trust they are delivering better quality software and rely on a rollback procedure if something goes wrong in production, people relax. Self-defensive silos break down and teams collaborate better across IT and business to drive the product forward. Vamp was born from our personal experiences of working on “scary” big bang releases and the need to safely test in production and smartly automate the process. Our platform is designed to relieve the stress of releasing software for people and teams, and the risk for business.
If you are interested in seeing how Vamp Cloud-Native Release Orchestration can help you safely test in production for stress-free Friday releases, book a guided tour, with or without your own containers!