Rollback Retrospective: Questions You Should Ask Before a Release

Software Releases Gone Wrong

Stories of software releases gone wrong riddle the Internet.

A critical security vulnerability in MacOS High Sierra uncovered in late 2017 allowed anyone to gain system administrator privileges without a password.

In May 2017, the WannaCry ransomware attack targeted vulnerabilities in Microsoft operating systems, crippling organizations around the world.

A major software bug released by Cloudflare in February 2017 triggered data leaks across thousands of websites.

Some companies never recover from these public failures. Others are able to rebound by rolling back and fixing their software bugs. Yet, they often still pay a significant price for these market flops.

As IDC’s Application Development Software Program Director Al Hilwa notes in a guest post on Developers Alliance, “Project failure is expensive. Sunk costs in a failed project add to the enormous missed opportunity cost of a successful project. Furthermore, these costs escalate towards the end of the project because failure is invariably preceded by failed attempts to avert it. These attempts come at late points in the project where fixes are more complex, time-consuming and expensive than adjustments made at an earlier stage.”

Doomed from the Start

Hilwa cites a report that explores the changing nature of the software development industry. Among the top reasons for software project failures: “poor team or organizational management” and “insufficient time allocated to testing.” Another key reason was “time constraints and premature software release.”

According to Project Smart, “When the pressure to deliver is on, it is often testing that suffers. All the testing gets left until the end of the development cycle and only lip service paid to it. Often, the result is a product filled with bugs and an unhappy customer.”

Organizations can rewrite this story. Think of a software release failure as misalignment between expectations and final delivery. That misalignment is often due to a disconnect between the quality assurance (QA) team and development. By taking steps to close the gap, organizations can ensure software is only released when it’s ready and safe for consumption.

10 Steps to a Quality Release

First and foremost, organizations should make security part of testing as well as development. Beyond that, they can refer to the following checklist to avoid QA and testing oversights.

1.What is our release strategy? The most common strategies include:

  • Big Bang – roll out the feature to all users
  • Incremental – roll out the feature to a subset of users and then to more users over time
  • Parallel – release the feature in both the legacy and new code

2. How are we improving existing code? Besides adding new features and enhancements, how are we strengthening our existing code? How do we determine the risk posed by new bugs in old code, and what is our policy for handling these bugs?

3. Do our processes support our release calendar? Since the goal is to release software with full confidence, it’s best not to set release dates in stone. If there is no leeway in the schedule, it’s essential that all processes are in place to enable the team to launch on time.

4. Have we written the necessary unit tests? QA should support every new and upgraded feature with unit tests whenever possible.

5. Has DevOps taken steps to support the release? Make sure development and operations personnel have addressed all needed infrastructure and dependencies.

6. How do we ultimately determine when it’s okay to release software? Some organizations embrace an 80/20 testing approach: if 80 percent of tests pass, the organization feels its safe to launch. Yet when developers apply a code fix to a found defect, it’s possible to introduce a new issue. As a result, it’s a must to conduct thorough testing.

7. Does the software satisfy all requirements? Make sure all key stakeholders (the product owner, engineers, customer) review and agree to the planned release.

8. Has QA approved the release? Quality Assurance has the ultimate say over whether software is ready for release. Using a project-tracking tool like Jira in conjunction with an integrated test case management tool can make it easy to determine the answer to this question.

9. How will we communicate the release to customers? Determine who manages release notes, and confirm these notes are ready at the time of release.

10. What is our rollback plan? How will we know if a feature becomes problematic, and how will pull it back and respond? Can we anticipate the downtime if we rollback?

Finally, QA leaders should ensure their teams have the appropriate tools to execute against these other 10 questions. Tricentis’s software testing platform, Tricentis qTest, embeds testing within the software development lifecycle to help your teams release higher quality software and prevent the need for a rollback. With Tricentis qTest, your team is enabled to detect and resolve defects faster, test and track continuously, from requirement design to production release and increase code coverage for more confident releases. Learn more.

Continue Reading

Please register for access