Pull up the last three articles that you read about software development and testing and do a quick CTRL+F search for DevOps on each of them.
Chances are, it showed up at least once in every article.
It seems DevOps is an inescapable topic in the world of software development, and everyone is talking about moving toward a DevOps approach.
Tricentis is just as guilty of this DevOps obsession as everyone else. But there’s a good reason for it when you consider how DevOps helps teams release higher quality software at a faster pace. But amid all the excitement, a lot of what’s put out about DevOps boils down to jargon that can be difficult to understand, especially for those of us just hopping on the DevOps bandwagon.
When it comes to really learning about DevOps, there are a lot testers need to know, but one of the most important questions that arises time and again is: How can testers make sure they’re moving as fast as they need to in a DevOps environment without sacrificing quality or traceability?
Having seen countless Tricentis customers go through this transition, we’ve witnessed many successes and just about as many stumbles. And from those experiences, we can confidently deduce that the secret sauce for testing teams to successfully embrace DevOps lies in behavior-driven development (BDD).
BDD best practices – Why BDD is the secret sauce to maintaining traceability and deploying quickly
DevOps is all about collaboration and speed, but in a world that’s moving so quickly, how can testers maintain imperatives like traceability? It isn’t easy when taking a more traditional or even an agile approach, let alone a DevOps approach. And in the fast-paced DevOps world, the answer lies in Behavior-Driven Development (BDD). A complement of test-driven development (TDD), BDD is a test-first methodology that promotes collaboration between “the three amigos” (testers, developers and product owners) in order to better understand business and user needs and to allow for more efficient and effective user experience testing.
So why exactly is BDD the secret sauce for testing in DevOps? It’s all about the fluidity that BDD offers. Specifically, there are three reasons why BDD is such a perfect match for testing in DevOps:
1. It promotes collaboration
Traditionally, the software development process looks something like this: The stakeholder tells the product owner what the business needs. The product owner then writes requirements. The developer translates those requirements into code and the tester translates those requirements into test cases.
There are a lot of silos within this process and a lot of room for error and miscommunication, what with all the telling and translating. And in a DevOps world where everything is moving quickly and multiple things are happening at once, this lack of collaboration just doesn’t cut it.
BDD, on the other hand, provides a much higher level of collaboration. When following BDD, the software development process looks something like this: The stakeholder and product owner talk about business needs. The product owner, developer and tester (aka the three amigos) then collaborate around requirements, which they define as scenarios. The developer then uses those scenarios for automated tests, the tester uses those same scenarios as the basis for their tests and those tests report back against features and the scenarios.
Unlike the traditional approach, BDD emphasizes collaboration. In doing so, it eliminates miscommunications since everyone is in the same room talking together rather than translating requirements that get passed down from one person to the next. In turn, this shared understanding helps increase efficiency and effectiveness, ultimately allowing the team to bring a better product to life in less time.
2. It gets straight to the point
Unlike a traditional approach where needs and requirements get passed down, often at a high level, BDD gets straight to the heart of what’s really necessary through collaborative discussions about specific examples, or scenarios. These discussions and the use of specific scenarios offer several benefits for testing in DevOps.
Scenarios make it easier for testers to understand the product and write test cases that properly support the requirements and roll up to business needs.
Because the test cases are tied directly to scenarios, and because those scenarios are aligned to business needs, BDD makes it easier for all stakeholders, including those from the business, to provide feedback in a timely manner.
And all of that collaboration keeps testing extremely focused. In BDD, testers know exactly what the end result should be, and that means they can develop tests that focus on proving whether or not the expected behavior happens. This setup helps testers run only the necessary tests, and more quickly get to the point– whether or not the software performs as it should.
This style helps teams avoid taking on unnecessary tasks that don’t contribute to understanding and perfecting software performance.
3. It’s easy to understand, follow, and replicate
BDD calls for the use of the Gherkin syntax when writing scenarios. The Gherkin syntax is plain English that follows a Given-When-Then format. Its purpose is to define the context (Given), the action (When) and the outcome (Then) of different scenarios.
The facts that every BDD scenario should be written following the Gherkin syntax and that the Gherkin syntax is plain English are extremely valuable. The standard use of the Gherkin syntax is so valuable because it keeps everyone speaking the same language (which goes back to BDD’s goals of boosting collaboration and eliminating ambiguity). This standardization keeps all stakeholders on the same page and helps avoid any duplication of efforts throughout the team. It also allows for easy replication of scenarios and test cases built off those scenarios across releases, which allows the team to move faster down the line.
Meanwhile, the use of plain English makes it easy for everyone, including business users, to understand each scenario. Once again, this shared understanding goes back to BDD’s goals of boosting collaboration and eliminating ambiguity. In fact, not only does this use of a standard syntax that’s written in plain English help keep all stakeholders, including business users, on the same page but it also makes it easier for technical programs to translate those scenarios into test cases. An added bonus: Even computers speak plain English!
At the end of the day, BDD’s high level of collaboration turns out to be a perfect match for DevOps, which itself calls for a high level of collaboration in order to churn out new releases quickly and on a regular basis. Specifically, BDD’s emphasis on collaboration eliminates ambiguity, gets straight to the heart of the product’s performance and keeps everyone, even business users and computers, on the same page, making the software testing process far more efficient and effective.
Along the way, this approach makes it much easier for testers to maintain a high level of traceability without losing speed. That’s because all user requirements get clearly defined upfront through the creation of scenarios (and, best of all, testers are part of this discussion), and testers then develop their test cases based on those scenarios. This setup ensures that testers have documentation on requirements, stay true to those requirements and aren’t left guessing what type of test cases they need to write in order to ensure proper performance. This last point means testers can work faster when developing test cases, and that they have the information they need to get those test cases right the first time around, both of which help save time to keep the process moving quickly.
The history of BDD
Now that we’ve covered why BDD is the secret sauce to testing in DevOps, let’s take a step back to understand the bigger picture of how we got here. First up: The history of BDD.
BDD was actually born out of TDD. In TDD, developers write unit tests to fail before code gets written. In BDD, the same principles apply, but everything happens at the business level rather than the technical level. Whereas TDD focuses on technical tests, the requirements for BDD are written based on human behavior. As a result, BDD goes straight to the functionality of the software and focuses on its ability to fulfill user and business needs.
Also unlike TDD, BDD follows the standard, plain English Gherkin syntax for writing scenarios. These scenarios then become the basis for the automated tests and test cases. In BDD, code then gets refined until it passes those test cases that were written based on the scenarios. This method once again comes from BDD’s TDD roots.
Dan North is credited as the founder of BDD. He famously summed up its purpose in a single tweet:
BDD vs TDD: How BDD helps overcome challenges with TDD
If BDD is a subset of TDD, why is BDD the secret sauce to testing in DevOps as opposed to TDD?
1. BDD focuses on behaviors, not technicalities
BDD emphasizes behaviors and business needs whereas TDD focuses on the more technical aspects of the product. For example, when writing user stories in a TDD setup, the user persona and the feature take priority, with the benefit of that feature to the user persona getting tacked on at the end. BDD flips this model, starting with the benefit for a user persona. The feature that delivers that benefit that becomes the last piece of the puzzle. This flip ensures that the benefit remains in focus at all times, so it doesn’t get lost in the shuffle, and so that features without any benefit don’t get developed.
2. BDD emphasizes collaboration more than TDD
The collaborative nature of BDD aligns well with the shift left in testing that a DevOps environment necessitates. The three amigos model of BDD helps ensure that testers are brought in earlier, which kicks off testing earlier in the development lifecycle. Furthermore, because this collaboration helps eliminate ambiguity and leaves testers with a clear path to follow when developing test cases, it ultimately helps the team deliver better quality software within that shorter timeframe.
3. BDD supports a more rapid development cycle than TDD
BDD also eliminates unnecessary steps and is designed for reusability, which supports a more rapid development cycle, even compared to TDD. TDD’s focus on unit tests makes it more procedure-driven and still allows room for ambiguity that can lead testers to focus on too wide a scope and do unnecessary work. In contrast, the three amigos collaboration in the BDD model leaves testers with very clear behavior-driven scenarios, which allows testers to write more meaningful tests that get right to the point. It also means that testers don’t waste time on tests that don’t confirm product performance (or don’t review performance for scenarios in which the product won’t actually be used).
This specificity, along with the standard use of the Gherkin syntax across all scenarios against which test cases are created, also allows for reusability of test cases. This reusability enables more rapid development, which can help testers keep pace with the need for speed in DevOps during future deployments.
4. BDD improves traceability over TDD
Finally, BDD improves traceability over TDD because of its structured nature and allowance for self-documentation. In terms of structure, the standard Gherkin syntax that all BDD scenarios follow becomes important once again. The standard use of this syntax along with the fact that it’s written in plain English provides structure to develop more standard test cases as well as more standard test case steps, which can then be reused across different test cases. Furthermore, this structure helps create simpler and more reliable test case steps, which not only help with reusability but also help eliminate duplication, cutting down on wasted time and effort.
Looking at self-documentation, plain English of the Gherkin syntax makes it easier to write, review, manage and share tests. All of these points help testers work faster without sacrificing traceability, even compared to TDD. Additionally, the ease of sharing tests becomes important to collaboration, since tests are now based on a language that even business users can understand, which removes many of the technical obstacles that come with collaborating in a TDD environment.
Basics of BDD: Putting the secret sauce to work
So how can you put BDD to work? Let’s dive into the basics of kick-starting BDD.
The collaboration model
As noted previously, BDD emphasizes collaboration through what’s known as the three amigos, and this collaboration changes how requirements get set and tests get written.
In a non-BDD environment, the stakeholder tells the product owner the business needs, and the product owner uses that information to write the requirements. Then the developer and the tester translate those requirements into code and test cases.
In a BDD environment, a lot changes. Instead of the stakeholder telling the product owner the business needs, the two meet and have a discussion about those needs. The product owner then takes that discussion to the tester and the developer (these are the three amigos), and they all collaborate on what the requirements should be based on those business needs. The three amigos also write scenarios in the plain English Gherkin syntax to support those requirements. The developer and tester then use those scenarios to write automated tests and test cases. The idea is that the scenarios are set and used as an airtight guide for writing tests. And those tests stay static the whole time, as code gets written and rewritten until it passes those tests.
This collaboration removes the silos that typically exist and eliminates the ambiguity that comes with those silos and often leads to a “broken telephone” situation where information gets warped as it gets passed down from person to person.
At the end of the day, this collaboration should also help the teamwork together to create a stronger product that better fits the needs of users. Notably, it should allow the team to do so more efficiently and effectively due to the high level of collaboration and lack of ambiguity that often slows down the process when taking other approaches to development and testing. Additionally, it should allow teams to work quickly without sacrificing traceability due to the use of the scenarios developed by the three amigos.
Writing test cases
At this point, we’ve thrown around the term Gherkin syntax a lot. But what exactly does it mean to write test cases following the Gherkin syntax?
To recap, the Gherkin syntax is a plain English style of writing scenarios that follows the format of Given (this is the context), When (this is the action), Then (this is the outcome).
In practice, it looks something like this for the scenario of shopping online:
- Given I am in my shopping cart
- When I place my order
- Then I should see an order confirmation
Another common example is that of a coffee vending machine:
- Given I put money into the coffee machine
- When I press the brew button
- Then I should be served coffee
As these examples illustrate, the Gherkin syntax is very easy to understand, even for business users with no technical background. This simplicity makes it easy to explain these test cases to everyone and to avoid duplication. It also makes it easy to use these scenarios to write clear test cases that are designed to validate whether or not the product behaves as it should.
As you might have deduced already, BDD uses a lot of terms with which many teams that are new to the practice might not be familiar. Key terms to know include:
BizDevOps: Brings business stakeholders into DevOps in order to confidently validate that business processes work end-to-end across single or multiple applications.
Continuous integration and delivery: The idea that software can be released into production at any time and that there is a continuous feedback loop ensuring its quality is always ready for production.
Continuous testing: The process of ensuring that software is always ready for production as part of a continuous integration and delivery model.
Cucumber: A framework for running tests that supports scenario creation for Gherkin formatting to streamline automated test case creation. Cucumber is considered “the grandfather” of BDD. It was created first in Ruby, but now offers a version for JAVA as well.
easyb: A framework for running tests that supports scenario creation for Gherkin formatting to streamline automated test case creation. Uses Groovy.
jBehave: A framework for running tests that supports scenario creation for Gherkin formatting to streamline automated test case creation. Primarily JAVA based, but also uses a jRuby format.
Gherkin syntax: The plain English format for writing scenarios in a BDD environment. Follows the Given-When-Then format that indicates the context (Given), the action (When) and the outcome (Then).
Scenarios: Very clearly indicate how software should behave in order to eliminate any ambiguity among testers and developers. These scenarios should be written by the three amigos, should follow the Gherkin syntax and should be used to create automated test cases.
The three amigos: The product owner, developer and tester. The three amigos should collaborate to develop scenarios.
BDD best practices
When adopting BDD, keep in mind the following best practices:
Write declarative, not imperative, scenarios: Declarative scenarios are simpler, easier to follow and better represent the user perspective, all of which makes them easier to maintain. A declarative scenario would be:
- Given I am in my shopping cart
- When I place my order
- Then I should see an order confirmation
An imperative scenario would be:
- Given I am in my shopping cart
- When I hit the “place my order” button
- And I fill in my billing information
- And I fill in my shipping information
- And I hit the “submit order button”
- Then I should see an order confirmation
Keep step definitions simple: Testers should write step definitions to translate the Gherkin scenario into actions the system should take during testing. It’s important to keep these step definitions simple by limiting them to one action per step and by creating unique step definitions. This simplicity allows for better reusability and avoids confusion within the system about when to use which step definition.
Recycle steps whenever possible: Recycling step definitions helps increase efficiency by making it so testers don’t have to write new steps every time. This recycling simplifies maintenance and makes it faster to implement changes since testers can change a step definition once and have that change apply across all scenarios in which that step is used.
Give your DevOps efforts a leg up with BDD
As DevOps continues to take the software development world by storm, testers are trying to figure out where they fit in. For many testers, keeping up with the speed of DevOps without sacrificing traceability proves particularly challenging.
BDD is important for testing in DevOps because it adds a layer of traceability on top of what everyone really wants these days, which is to operate quickly. BDD does this by promoting collaboration among product owners, developers and testers (aka the three amigos), eliminating ambiguity (and therefore time spent focusing on unnecessary work) from the development and testing process, creating documentation for testers and developers to follow without spending a significant time on planning and allowing for reusability and easy maintenance of tests.
Through it all, BDD allows for the shift left in testing and close collaboration that DevOps requires, improves efficiency and effectiveness to help testers keep up with the need for speed and helps deliver better software by focusing on specific user needs.
qTest Scenario was designed specifically for BDD practitioners in enterprise DevOps environments. qTest Scenario enables enterprise testing organizations to embrace BDD, with a test-first approach that facilitates test scenario creation and ensures feature traceability.