Thursday, 22 September 2016

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

This final installment demonstrates how to start implementing some automated acceptance criteria. To follow along, you can clone the github repository for the sample project.

Alternatively, if you want to create your own brand new Thucydides test suite and write your own tests, you can use the Maven archetype. From the command line, run mvn archetype:generate -Dfilter=thucydides-jbehave, as illlustrated below:

mvn archetype:generate -Dfilter=thucydides-jbehave
...
Choose archetype:
1: remote -> net.thucydides:thucydides-jbehave-archetype
     (Thucydides automated acceptance testing project using Selenium 2, JUnit and JBehave)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Choose net.thucydides:thucydides-jbehave-archetype version:
1: 0.9.87
2: 0.9.88
3: 0.9.89
4: 0.9.90
Choose a number: 4:
Define value for property 'groupId': : com.acme.myapp
Define value for property 'artifactId': : mywebtests
Define value for property 'version':  1.0-SNAPSHOT: :
Define value for property 'package':  com.acme.myapp: :
Confirm properties configuration:
groupId: com.acme.myapp
artifactId: mywebtests
version: 1.0-SNAPSHOT
package: com.acme.myapp
 Y: :

In both cases, the project structure should look something like the one in Figure 1. - See more at:

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Figure 1: The sample project directory structure.

This directory structure follows standard Maven conventions. The JBehave stories are stored in the src/test/resources/stories directory, in a simple directory structure that reflects the organization of the requirements into capabilities and features. In the src/test/java directory, you will find the JBehave test harness (the AcceptanceTestSuite class) as well as any JBehave step implementation classes.

In the src/main/java directory, we find the Thucydides step classes and page objects. Placing these classes in the src/main/java is a relatively arbitrary choice, based on the following reasoning:
  • This maven project (or module) is dedicated to acceptance tests - if the acceptance tests are stored in the same project as the application code, this choice obviously makes less sense.
  • Page Objects and Thucydides steps are occasionally reused between related acceptance test projects, and can therefore be viewed as deliverables rather than test code: for example, page objects and steps related to integration with PayPal might be reused for several projects.


Automating Requirements and Acceptance Criteria


While JBehave helps document and automate the acceptance criteria themselves, Thucydides also reports on the features and capabilities being delivered. This helps give better context for the individual acceptance criteria as well as more relevant aggregate reporting. In this project, the capabilities are represented as directories underneath the src/test/resources/stories directory. The features are represented by JBehave stories directly in these directories; for a more complexe application, we might have a sub-directory for each feature, with JBehave stories in these sub-directories.

Inside each capability directory, a text file called narrative.txt describes the feature. The contents of this file are free text, with the first line serving as a title in the reports. An example is shown here:
Learn the meaning of a word
In order to learn the meaning of a word that I don't know
As an online reader
I want to be able to find out the meaning of the word
Acceptance criteria in JBehave take the form of simple text files with the ".story" suffix. Each ".story" file contains the acceptance criteria for one feature or story, along with an optional narrative section that describes the feature or story in question. The examples above are all written in JBehave notation: you will find them in the src/test/resources/stories directory of the sample project. A simple example is shown here:
Narrative:
In order to understand what I am reading about
As a reader
I want to be able to look up definitions of words

Scenario: Looking up the definition of a simple word
Given the user does not know the meaning of the word 'banana'
When the user looks up the definition of the word 'banana'
Then they should obtain a definition containing the words 'An elongated curved fruit'
Once we have a set of ".story" files, we can automate them with JBehave. Thucydides provides a convenient test harness class, called ThucydidesJUnitStories, that you can extend to run all of the JBehave stories in in the src/test/resources/stories directory. Just create a class in your project that extends this class, like the following:

import net.thucydides.jbehave.ThucydidesJUnitStories;

public class AcceptanceTestSuite extends ThucydidesJUnitStories {}

Implementing the Tests


Now that we have some acceptance criteria, we can start to automate them. When work starts on a feature or story, the acceptance criteria will be pending, with no implementation. As our understanding of the problem grows, we will be able to flesh out the implementation. We start off with a high-level sketch of what needs to be done to demonstrate the feature, without going into the details of how this will be done. This is done using a Thucydides Step class to add a layer of abstraction between the "how," described in the JBehave steps, and the "what," detailed inside the Thucydides step methods. An automated acceptance criterion is a guide for developers and living documentation for testers and other team members as much as it is a test, so it is important that at each stage we focus on clearly communicating what the test is trying to do. The following code illustrated one possible implementation of the JBehave story listed above using this approach:

public class DefinitionSteps {

    @Steps
    ReaderSteps reader;

    @Given("the user does not know the meaning of the word '$word'")
    public void givenTheUserDoesNotKnowAWord () {
        reader.consults_the_online_dictionary();
    }

    @When("the user looks up the definition of the word '$word'")
    public void whenTheUserLooksUpTheDefinitionOf(String word) {
        reader.looks_up_the_definition_of(word);
    }

    @Then("they should obtain a definition containing the words '$definition'")
    public void thenTheyShouldSeeADefinitionContainingTheWords(String definition) {
        reader.should_see_a_definition_containing(definition);
    }
}

In the next step, we start to implement the Thucydides step methods, fleshing out the details of "how" this feature should be demonstrated. In the following example, we use Page Objects to isolate and centralize the UI logic from the business logic described in the test steps. Within these step methods, we focus on what each step is doing in logical terms, without committing too much to a particular UI implementation. Indeed, if this test involves the UI, we may well wait until the UI is reasonably stable before implementing these methods in order to avoid too much rework.

public class ReaderSteps extends ScenarioSteps {
   
    DictionaryPage dictionaryPage;

    public ReaderSteps(Pages pages) {
        super(pages);
        dictionaryPage = getPages().get(DictionaryPage.class);
    }

    @Step
    public void consults_the_online_dictionary() {
        dictionaryPage.open();
    }

    @Step
    public void looks_up_the_definition_of(String term) {
        enters(term);
        starts_search();
    }

    @Step
    public void should_see_a_definition_containing(String terms) {
        List displayedDefinitions = (List)dictionaryPage.getDefinitions();
        assertThat(displayedDefinitions, hasItem(containsString(terms)));
    }

    @Step
    public void enters(String keyword) {
        dictionaryPage.enter_keywords(keyword);
    }

    @Step
    public void starts_search() {
        dictionaryPage.lookup_terms();
    }
}

When the UI is stable enough to work with, or even once we have a reasonably stable HTML design, we can implement the Page Object for this web page. Page Objects isolate the HTML implementation details of a web page behind a friendlier and more implementation-neutral API, making the test code more reusable and easier to maintain. Selenium 2/WebDriver provides excellent support for the Page Object pattern out of the box. Here, we extend the Thucydides PageObject class, which provides a number of convenience methods, but the code should look reasonably familiar to any developer having worked with Selenium 2/WebDriver.

@DefaultUrl("http://en.wiktionary.org/wiki/Wiktionary:Main_Page")
public class DictionaryPage extends PageObject {

    @FindBy(name="search")
    private WebElement searchTerms;

    @FindBy(name="go")
    private WebElement lookupButton;

    @FindBy(css="ol li")
    private List<WebElement> definitionList;

    public DictionaryPage(WebDriver driver) {
        super(driver);
    }

    public void enter_keywords(String keyword) {
        $(searchTerms).type(keyword);
    }

    public void lookup_terms() {
        $(lookupButton).click();
    }

    public Iterable<String> getDefinitions() {
        return extract(definitionList, on(WebElement.class).getText());
    }
}

Reporting and Story Telling


Acceptance tests are as much about communication as they are about testing; indeed, testing is almost a secondary aspect of the whole BDD process. The Thucydides reports are naturally designed to record test results (see Figure 2), but they also provide more detailed reporting at other levels. Firstly, Thucydides records and reports on the steps that were executed during each test (see Figures 3 and 4), thus documenting how a particular acceptance criteria was demonstrated. If the test uses the web UI, screenshots are also recorded at each step (see Figure 5).

Secondly, it is important to know what features are to be implemented, and how many of these have currently been completed. Agile projects measure progress in terms of working software, and in a BDD project, working software can be demonstrated by a working acceptance test. Thucydides provides aggregate reporting that can help judge what features and capabilities are potentially ready for delivery (see Figure 6).

You can try this out for yourself by running the full test suite and generating the Thucydides reports in the demo project. To do this, just run the mvn verify in the project root directory. This will produce a set of reports in the target/site/thucydides directory.

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Figure 2: A summary of test results

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Figure 3: A simple test report

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Figure 4: A table-based test report

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Figure 5: Tests involving the UI record screenshots for each step

BDD and Automated Acceptance Testing for Java Web Apps: Creating a Suite

Figure 6: What capabilities and features have been specified and tested?