Given Something needs testing
When Someone suggests using BDD/Cucumber/SpecFlow
Then Some other people get really vocal – why?
The test community seems to have a love hate relationship with BDD, but what actually is it, and how can it be used effectively?
In software engineering, behaviour-driven development (BDD) is an Agile software development process that encourages collaboration among developers, QA and non-technical or business participants in a software project. It encourages teams to use conversation and concrete examples to formalize a shared understanding of how the application should behave. It emerged from test-driven development (TDD)
https://en.wikipedia.org/wiki/Behavior-driven_development
Whereas TDD would apply at the unit testing level, requiring an overhead in upfront work to build failing tests before development can commence, BDD captures the conversations about requirements for a user story into ‘given when then’ syntax, and those sentences can be translated into tests by writing the code for step definitions that implement those behaviours.
Tools like Cucumber, SpecFlow, etc. provide utilities to interpret given/when/then statements, execute the necessary steps and perform assertions, and build a report on test execution.
User stories
Here is a simple example to demonstrate a test scenario and the corresponding code implementation derived from a user story. I’m using simple maths operations as an example, but in the real world this would almost certainly be a more complex user facing GUI application. Firstly, here is the test case written in plain language:
Scenario: Multiplication
Given I have 2 and 6
When I multiply the values
Then The result should be 12Here is the (partial) Java implementation of the step definitions:
@Given("I have {int} and {int}")
public void i_have(int first, int second) {
value1 = first;
value2 = second;
}
@When("I multiply the values")
public void multiply() {
actual = value1 * value2;
}
@Then("the result should be {int}")
public void the_result_should_be(int expected) {
Assert.assertEquals(expected, actual, 0);
}There’s some more code required which glues everything together, which I’ll detail in an upcoming post.
To extend the test suite to include a different operation, only the new step definition need be created – the others should be written to be as reusable as possible. In the above example, adding:
@When("I add the values")
public void add() {
actual = value1 + value2;
}Would allow additions to be tested – the Given and Then step definitions are reused by both tests. Subsequent test cases should reuse available steps, and a rich set of step definitions would slowly evolve.
Data tables
Where multiple runs of the same test are required, rather than repeat the example scenario for each set of input and output values, BDD frameworks provide data table tooling to express the tests in a neater form, with the appropriate ‘glue’ to actually run the tests as intended. For example, here is the test case to perform the same multiplication operation on several sets of data, in a very readable format:
Scenario: Multiplication Table
When I multiply a series of values
|first|second|result|
| 1| 1| 1|
| 2| 3| 6|
| 4| 5| 20|
| 6| 7| 42|
| 8| 9| 72|Here is the Java implementation. If you’re unfamiliar with programming, it’s a loop to go through all the rows of values in turn:
@When("I multiply a series of values")
public void i_multiply_a_series_of_values(DataTable dt) {
List<List<String>> list = dt.asLists(String.class);
//i starts from 1 if there is a header
for (int i = 1; i < list.size(); i++) {
value1 = Integer.valueOf(list.get(i).get(0));
value2 = Integer.valueOf(list.get(i).get(1));
multiply();
System.out.println(actual);
the_result_should_be(Integer.valueOf(list.get(i).get(2)));
}
}This allows larger sets of repetitive test to be concisely defined and executed.
API tests
It may be tempting to continue using BDD throughout your tests, but while applying those principles may make sense, this usually results in a lot of unnecessary abstraction and extra code.
Firstly, specifying a simple API test involves a lot of repetition for the assertions:
Scenario: Get First Post
Given I get post "1"
When The request is received
Then The status code is 200
And The content of "title" is "my first blog post"
And The content of "content" is "this is the content"
And The content of "author" is "webmaster@example.com"The partial Java implementation would include:
@Then("The status code is {int}")
public void checkStatus(int status) {
Assert.assertEquals(status, response.statusCode());
}
@And("The content of {string} is {string}")
public void checkContent(String path, String expected) {
Assert.assertEquals(expected, response.getBody().path(path));
}Considering that it’s unlikely (although not impossible) that the product owner will be contributing to these types of test, a more readable way to implement these tests whilst retaining GWT like syntax would be to use RESTAssured’s fluent syntax. For example the entire test could be implemented in little more than:
@Test
public void getFirstPost() {
given().when().get("/posts/1").then()
.body("title", equalTo("my first blog post"))
.body("content", equalTo("this is the content"))
.body("content", equalTo("webmaster@example.com"))
.statusCode(200);
}I’ll be exploring RESTAssured, along with other popular test libaries and tools in future posts.
Defining rules
One of the reasons that the usage of BDD is hotly debated is because when implemented without guidance, test packs built using it can quickly become complicated and the tests devolve into test procedure like documents rather than succinct test cases. Maintenance starts to become a problem as complexity increases, and eventually tests begin to fail and confidence in the test pack reduces. Test cases should be broken down if they have multiple steps and complex preconditions in order to keep the tests as simple as possible.
Bad GWT
This is human readable, but a very vague (and contrived) test case:
Scenario: Image search
Given the user opens a web browser
And the user navigates to a search engine
When the user enters a word into the search bar
Then the results are shown
And the user clicks on the "Images" link at the top of the page
Then images of the search are shownThis is the sort of thing you may have encountered in manual test cases – it’s unclear, and thuis open to interpretation, and the implementation or execution may not actually meet with the desired intent.
Defining rules – keep it clean(er)
Here’s an example of some simple and very strict rules to try and keep things under control:
- 5 lines maximum
- ‘Given’ statement clearly captures preconditions
- ‘When’ statement has no ‘And’ clauses – keep to a single action
- Clear ‘Then’ statement that has a definitive pass or fail outcome
Better GWT
Re-writing the previous example, in an attempt to make it more readable and easier to implement:
Scenario: Basic search
Given The user opens a search engine site in a web browser
When The user searches for “cheese”
Then Search results for “cheese” are shown
Scenario: Image search
Given The user opens a search engine site in a web browser
And The user searches for “cars”
When The user clicks on the "Images" link at the top of the page
Then Images of the search are shownEach test is has clearly defined preconditions and actions, and a distinct outcome. Some elements of the step definitions can be reused. Test failures will be much easier to diagnose where the problem occurred.
Conveying intent
The use of GWT syntax need not imply that it will be turned into test cases using BDD tooling. It is actually very useful only as a communication medium, to elaborate a complex user story with some corresponding ‘given when then’ statements to help describe a complex business process to people who aren’t subject matter experts in the business domain.