Xebia Background Header Wave

This is the first blog in the series after my last week’s introduction to the five quality patterns in Agile development, to deliver the right software with great quality. This blog is about a pattern that I think is absolutely necessary for a team to accelerate, deliver with quality and keep on doing this over time as an application expands. Your acceptance criteria should be your tests. That sounds simple enough, but it’s often not the case. By changing the way you refine stories your acceptance criteria can become your tests as well. Let me guide you through it.

Optimize to become Agile

Agile development means iterating in short cycles. In these short cycles, we need to optimize as much as possible to reap the benefits. I’ve been at companies where we would have four development sprint, only to have two sprint behind those to integrate everything and to perform acceptance tests. This is not efficient. It looked like the teams were able to work in short sprints, but in practice, they could still only deliver software about every quarter.

Fast feedback is important

Accelerating means basically to get in more control. Think about it. We only did the acceptance testing in the last weeks before a release because we, or the business, did not trust that all requirements were met. How could it be that we were not sure? We were not in control. We waited until the end to verify that what we said we would deliver was actually what the business wanted. At the time of the sprint review we often still had discussions about tests, requirements or even the scope. We did not get to a common understanding and therefore no trust from the business. We need to pull this feedback forward, speak the same language and find a way so that they would not need the manual acceptance tests anymore. Fast feedback is key here if we want to treat our acceptance criteria as tests.


Requirements are everything

It all starts with the requirements or acceptance criteria. Requirements tell us what (and what not) to implement. When all requirements are implemented, the story is done. It’s that simple. Unfortunately, we have a few problems here:

  • The whole team and the business is not always involved in creating the requirements.
  • We skip detailing the requirements since refinements take too long.
  • Therefore; our requirements do not tell the full story.

After our refinement, we are still in flux with our requirements. Everyone involved creates their own opinion on these requirements. Developers look at them and interpret the requirements for their needs. At the same time, the tester(s) are looking at them and add details and edge-cases. And after this, the business is taking a look at them during the sprint review and find that things should have been slightly different. While we say that we are Agile, we still created mini handovers.

The requirements are what should bind all people involved. Constantly checking against these requirements gives us the feedback we need to know if we’re still on track. That we are building and delivering what is expected. Not at the end of a sprint or even multiple sprints, but constantly! This does mean one very, very important thing. We need to spend enough effort upfront to make the requirements clear, unambiguous and suitable for development, testing, acceptance and even documentation.

Spend your effort once!

Teams often don’t like refinement. It takes a long time, there is too much discussion and at the end, we miss things anyway. Why bother?! But refinement is for me the most important aspect of agile software development. It’s our contract with each other. For everything we do! We just need to be smart about it. A couple of things we need to achieve:

  1. Make sure that the story is small enough; when stories are too big we cannot refine them quickly. But when is it too big? We need to make that visible.
  2. We need to think in scenarios or rules of the application; when I do this with that, then something should happen.
  3. And we need to add specific examples of these scenarios; when I insert two euro in a vending machine and I buy a Mars of 1.50 then I should be returned 50 cents.

So how do we with minimal effort get to these scenarios?

Refine using Example Mapping

Example Mapping is a very visual way of refining your story. It’s tremendously helpful in a concept call Specification by Example. This concept, originally thought of by Gojko Adzic, focusses on creating clearly defined scenarios with specific examples for your requirements. These examples lead one-on-one to test cases. Because of this, there is no discussion anymore on what a feature should and should not do. Test coverage is also taken care of. If you use this technique and use it right. You should have 100% functional test coverage. This does not mean that you shouldn’t perform any other tests like performance or unit tests, but there should be no discussion anymore on functionality and extra tests.

Getting started

So what does an Example Mapping session actually look like?

  1. Start with the PO presenting the story to the group. What are the ideas behind it? What are the goals?
  2. Write the Story on a yellow post-it Keeping with the earlier used vending machine example; we have a story that says; “As a client, I want to receive change when I put more money in the machine than the product that I bought cost”. Or in short: “Receiving Change”.
  3. Give the rest of the group blue, green and red post-its and a Sharpie each. This is so that everyone can write down his/her ideas immediately. No discussions without having something on the board.
  4. Discuss and define rules. Rules are written down by using the blue post-its. A rule is a clearly defined function of the feature. In this case, we can have the rules: “Change should be returned in the lowest amount coins” and “Change can only be returned in coins, not in notes”.
  5. These rules by themselves can still be a bit unclear. Ask yourself, how you would test these rules? These tests will become examples. Examples are written on the green post-its and put under its related rule. Create as many examples that are necessary to cover the rule.
  6. During the discussion about the rules and examples questions can pop-up. When this question cannot be answered do not spend extra time on it. Write the question on a red (or a pink…) post-it instead. The PO can gather these questions at the end of the session and discuss these with the relevant stakeholders so that they can be discussed during the next session.

Example of Example Mapping


Finding problems

Example Mapping makes problems with user stories very visible.
If you find yourself creating too many rules, you likely have a story that is too big! Split up the story to limit the number of rules.
Too many examples can indicate a modelling problem. Ask yourself if the rule can be split or if there is a way to make this rule simpler. More than 6 examples often indicate a problem.

Doing this right results in a very good overview of what needs to be built. It also helps in reducing the size of the story. This limits confusion limits the scope and helps the team focus on bringing this small amount of functionality live in a shorter timeframe.

How to put this in practice

You don’t need to refine stories with the whole team all the time. Use the concept of the three amigo’s where you have 3-4 people together in small groups working on a story. All the disciplines that are needed (e.g. development, testing, business, PO, …) should be involved and a decision-maker should be present. This will avoid having a large number of open questions in the end or assumptions being made.
These small sessions can be scheduled during the time of your original refinement where you can run multiple sessions in parallel on different or the same stories.
After these small sessions, make sure you bring the whole team and business together before the planning to make sure everybody is aligned on the story. When you discussed the same stories make sure to discuss the differences the teams found and focus on clarifying those. When you’ve looked at different stories, quickly present them to each other to make sure you all have a common understanding. This should not take more than 5-10 minutes.

Next steps

We now have clearly defined and small stories we can start planning and pull the stories into our sprint. If done right, then we now have acceptance criteria that we can use as our tests.

In my next blogs, I will guide you to start using these scenarios and examples as guidelines for development, test automation and to generate your documentation from them. See you in my next blog!

I am a specialist at Qxperts. We empower companies to deliver reliable & high-quality software. Any questions? We are here to help! www.qxperts.io

Erik Zeedijk
As an Agile Test Consultant Erik always has the drive to improve the quality of software being delivered. This includes setting-up test automation and using CICD to get the fastest possible feedback on quality. It also means introducing the mindset change that is needed to think differently about testing and the way we measure quality.

Get in touch with us to learn more about the subject and related solutions

Explore related posts