Agile Traceability - Moving from Stories to Tests in a Way That Keeps Stakeholders Engaged

  

Clarifying the requirements of a User Story is a problem of communication. The Development Team requires unambiguous requirements, defined in a form that translates easily into testable code. So developers would prefer requirements that are defined in a technical language such as UML. However the ultimate source of requirements is the Product Owner, who is rarely qualified to deliver requirements in a technically precise language.

The approach to communication outlined below provides one way to iteratively evolve requirements into technically precise and unambiguous specifications.

The Difference Between "Conditions of Satisfaction" and "Acceptance Criteria"

The Conditions of Satisfation (CoS) define the boundaries of a User Story and are usually written on the back of the story. These conditions are rarely detailed enough to fully define what needs to be coded (nor should they be) but will be used to start the conversation.

The outcome of the conversation should be more detailed Acceptance Criteria. The Acceptance Criteria are used to provide a testable way to confirm that a story is completed and working as intended. The CoS defines what the PO expects, while the Acceptance Criteria define how the CoS will be demonstrated.

• CoS capture the expectations of the PO - and more importantly the intent behind the story. They exist at various levels (Epic, story and frequently even Iteration).
• At the story level, the CoS are unlikely to completely define how the story should be coded. As always in the Agile process, this will be done at the last responsible moment (see "The Right Conversation at The Right Time" below).
• Acceptance Criteria define how the PO will confirm that a story has been implemented in line with expectations. Thus the acceptance criteria are a formal and less ambiguous definition of the CoS.

Incorporating Acceptance Criteria into your user stories and tasks has several benefits:

• they get the team to think through how a feature or piece of functionality will work from the user’s perspective
• they remove ambiguity from requirements
• they form the basis of tests that will confirm that a feature or piece of functionality is working and complete

How To Write Conditions of Satisfaction

A Condition of Satisfaction:

• Helps clarify objectives and intent. 
• Applies to a specific product backlog item
• Defines a condition that must be true for that product backlog item to be considered done.

For example, a user story such as, "As a user, I want to login so that I can use the site functions," might include these conditions of satisfaction:

• User starts logged out
• User is logged in only when proper credentials are provided
• User has access based on user role
• User is locked out after three failed attempts

These conditions are clearly not sufficient to code from, they are intended as a starting point for the conversation. It is rare to find that exhaustive acceptance criteria and tests were defined as part of story creation. The iteration down from Conditions of Satisfaction to Acceptance Criteria and tests usually occurs at or close to the last responsible moment. 

 

Why do we leave this final clarification until so late? Because in Agile we try to avoid wasted effort:

Sometimes we learned things from the last story that will help define Acceptance Criteria for the next story 

• Sometimes the PO decides not to do the story.
• Sometimes the PO decides to change the story
• Sometimes we discover in the course of doing a previous story that not all of the product owner’s conditions of satisfaction can be met, and we will need to negotiate or compromise - thus changing the Acceptance Criteria.

Some of the success of the Agile process comes down to minimising wasted effort by delaying effort until it is definitely needed. This waste minimisation is sometimes termed "Maximising the Work Not Done". 

 

The Right Conversation At The Right Time

Exploration of the product owner’s conditions of satisfaction is a highly iterative process. 

As an example: During Project Initiation (perhaps at a Discovery Workshop), the team and product owner will collaboratively explore the product owner’s Conditions of Satisfaction. These Conditions will go beyond scope to include schedule, budget, and quality. The team and product owner look for ways to meet all of the conditions of satisfaction. If no feasible approach can be identified, then the conditions of satisfaction must be renegotiated. For example, if schedule is the issue then the product owner may prefer a release in one month that includes one set of user stories rather than a release five months later that includes additional user stories.

Subsequent conversations may occur at various points. This iterative process allows you to have the right conversations at the right time. 

The best times for the team to be thinking about the detailed implementation of stories and tasks is just prior to implementing these stories, as this ensures that the understanding is still fresh in mind during coding. Consequently Acceptance Criteria are frequently written in conjunction with the PO during Backlog Refinement. Acceptance Tests may be written at the same time as Acceptance Criteria, or delayed until the moment that the task is actually picked up from the board, at which point the developer seeks to work in conjunction with a tester and PO or BA to define tests based on the Acceptance Criteria.  

How To Write Acceptance Criteria

Acceptance Criteria provide a communication tool allowing the PO, Tester and Coder to agree on an unambiguous definition of a task or story. This is a communication tool for crossing the boundary between technical and non-technical, so Acceptance Criteria must be written in simple language, but should be easily translated into automated tests.

Defining a simple scenario for each task is an easily understood way to achieve this goal, as scenarios are usually specific enough to provide the user action (input) and expected outcome (output) in a testable way while being clear enough for non-technical POs to understand.

The Given-When-Then formula is an approach that works well at both generating scenarios and providing a good basis for formulating the tests.

Using the Given-When-Then Formula to Write Acceptance Criteria

Although primarily used as a tool for helping to define Acceptance Criteria tests, the Given-When-Then formula is a template that can be used at every level from Epic down to task:

• (Given) some context
• (When) some action is carried out
• (Then) a particular set of observable consequences should obtain

 

An example (From Dan North: http://dannorth.net/introducing-bdd/ ):

• Given the account is in credit
• And the card is valid
• And the dispenser contains cash
• When the customer requests cash
• Then ensure the account is debited
• And ensure cash is dispensed
• And ensure the card is returned

Notice the use of “and” to connect multiple givens or multiple outcomes in a natural way.

BDD is intended to guide the writing of acceptance tests for a User Story or task. Tools such as JBehave, RSpec or Cucumber encourage use of this template.

 

If your Given-When-Then scenario is fine-grained enough to allow a definition of input and output, then you are ready to write your test.

How It Works

The system works like this:

1.        The initial Conditions of Satisfaction should have been written when the Story was created. The Conditions of Satisfaction are then refined and will probably be elaborated down to Acceptance Criteria during Backlog Refinement, in order to reduce ambiguity and thus enable the estimation of the story. Tests may be defined at this time.

2.        During the Stand-Up a developer claims one or more tasks (as always). At that point, if acceptance criteria have not been defined down to the test level then the coder agrees to meet with tester and (ideally) the PO or BA right after the Stand-Up to define the acceptance tests. The tester works with the coder and BA or Product Owner to write an appropriate scenario for each task (or more than one scenario, if needed, but keep the number to the minimum possible). This scenario should be simple enough to convert into the Given-When-Then form. When the scenario(s) have been written and converted into given-when-then test(s), the PO should confirm that this fully illustrates the task and this confirmation should be captured. (Note: Automated tests provide protection against unexpected code interactions or behaviors, however tests are also an overhead. As a guideline: At this stage, the coder should only code the tests required to demonstrate the agreed scenarios. This provides enough tests to provide complete requirements traceability. This traceability is important because if the application passes the automated tests, all requirements are demonstrated to be present and functioning as expected - an automated "Traceability Matrix".)

3.        The coder can choose to implement tests first (if the coder is using TDD or BDD) or write the code first and confirm function by implementing the agreed tests. As soon as the coder completes the task and tests the coder calls in the tester and any other interested party, such as Scrum Master and/or PO for a “Desk Check”, in which the code is shown with functionality running and passing the Acceptance Tests. The coder receives immediate feedback concerning whether the code is performing as expected from the tester, and other stakeholders present.

4.        The tester may then run further tests on the code (Edge conditions, Usability, Accessibility, etc) and then “sign off”.

5.        Final PO sign-off occurs, as always, at the end of the sprint after the review or showcase.