Note: although this post focusses on mobile app development using Xamarin it also applies to other native mobile apps built in Swift, Java or even web apps. it’s 2017! whatever you are building get started with Test Automation! As a consultant working for Xpirit i get to see a lot of different customers which I help with my expertise in building mobile applications to improve their mobile apps. Something I noticed in the previous year is that continuous delivery is a hot topic and companies and teams focus on deploying apps automatically to their testers through hockeyapp or even to the stores in beta and / or production. In agile scenario’s (and come on who isn’t doing that currently? Every company or project I visit is saying they are agile or doing Scrum although some only do dailies and call that scrum ) In the current world it is really important to be able to release often because you want to be able to adapt to customer needs which are almost always changing and evolving.
Implementing a Shift left Quality ModelTest Automation is a process that does not belong to the developers or testers alone. It’s something that has to be in everyone’s mind from Product Owner to Developer and Tester. Automated tests can help you lower regression test effort but investing in Test Automation can really help you make a shift left focussing on quality earlier in your application development process. Traditional Quality Model In the traditional quality model most attention to quality happens in the phases AFTER development. This is a pattern that originates from the waterfall era and does not work that well with short iterations. When will you test or verify that you’re matching the expectations of your stakeholders? Quality issues will arise in the last few days of the sprint and there will be barely time to fix them in the current sprint. This is what happens at a lot of companies i visit so lets look at what we should do to improve this. Shift left Quality Model So how do we make this shift to the left and get earlier feedback on the software quality that we are building? A common pitfall i see at companies is that they focus on the deployments only and then test manually which often ends up in testing a lot of functionality a sprint after the actual development happend. instead of having an Agile Test Pyramid in place I often come across the Software Testing Cupcake.
Agile test pyramidThe Agile Test Pyramid is a concept created by Mike Cohn. It gives a good description of how your automated test landscape should look like for any type of application. The Software Testing Cupcake anti pattern is often a Agile Test Pyramid turned up side down or just 3 different teams not working together building their own tests. Often this starts with plain old manual testing of the application. after a couple of releases and regression testing taking more and more time teams start to invest in test automation and invest in automating their manual tests by writing automated UI tests. This may work for a while but automated UI tests are really expensive to build, break often and give very little information of what exactly is going wrong. Keeping these tests up to date can be hard whenever the UI is changing often and also take quite some time to execute.
Unit testsThe Agile Test Pyramid focusses on a solid base of Unit tests. Although unit tests do not focus on the end to end scenarios that the manual testers might be executing they do have a lot of advantages over Automated UI tests: They are cheap to build and maintain, give really exact error messages of where some piece of code is not working as expected and most of all they are (or should be) executing fast. Another benefit of unit tests is that they ensure proper isolation of dependencies leading to a better cleaner software architecture. I’ve witnessed to many projects where was asked to help to write unit tests for a certain piece of software which was full of spaghetti code that made it nearly impossible to write tests. Writing unit tests immediately from the start while writing your app code (preferably even in TDD style) will certainly lead to cleaner and better testable code. As an example of fast feedback take a look at the live unit testing feature of Visual Studio 2017 where you immediately see feedback of your unit tests while editing your code. In my role as consultant I’m often ask to come up with some guidelines and rules around unit testing. My approach is that just setting certain rules around unit testing is not going to work it makes developers focus to much on these rules and not on the actual purpose: building better software. An example of this is setting rules on code coverage. A question that is often raised is: “Shall we set a rule for code coverage on 80 / 90 / 100%?” Doing this often leads in developers aiming for this number of code coverage and makes them think they are finished when reaching this “goal”. If you can reach the 80% code coverage with just testing all the simple code and leave 20% of complex code untested you are clearly missing the point of why we are writing unit tests.
“Shall we set a rule for code coverage on 80 / 90 / 100%?” Doing this often leads in developers aiming for this number of code coverage and makes them think they are finished when reaching this “goal”.The only way to make unit testing work for your team is to have the full team believe writing unit tests is good for them as a team and it gives them the ability to make changes easier in existing code saving time and money. If you are building a stable base of the testing pyramid out of unit tests make sure these tests run fast and don’t have any external dependencies. if tests become slow only 1 thing will happen: people will stop running them and all it’s value will be lost. If you set up a good software architecture that focusses on loosely coupling for example using dependency injection your code will be a lot easier to unit test because it’s also easier to create mocks for your dependencies. So what kind of tools do i prefer?