This article by me was originally published on TheServerSide.
There’s a common phrase in testing: if you do something more than once – automate it. Software testing, where we routinely perform similar actions, is a perfect base for automation. In modern software development, with the use of microservices and continuous deployment approach, we want to deliver features fast and often. Therefore, test automation becomes even more important, yet still facing some common problems. Based on my experience, here is my list of top 5 mistakes that teams make in acceptance test automation.
Stability
False Fails? Always-red plans? We all know that. Stability of automated tests is one of the most obvious issues, yet most difficult to obtain. Even big players like LinkedIn or Spotify admit to have struggled with it. While designing your automation, you should put extra attention to stability, as it is the most frequent cause of failure and test inefficiency. Writing tests is just a beginning, thus you should always plan some time in sprint for maintenance and revisions.
UI perspective
Majority of modern applications are web-based, therefore the most preferable way for functional testing is from the UI perspective. Despite their usefulness, browser tests also have some substantial problems like slow execution time or stability randomness. Since we’re in the world of microservices, it’s worth to consider dividing tests into layers – testing your application features directly through webservices integration (or backend in general) and limiting UI tests to a minimal suite of smoke tests would be more efficient.
Overmocking
Because of many dependencies over the systems, mocking services become a popular pattern and are also often forced over test environment limitations. However, you should pay great attention to the volume of your mocked checks – mocks can miss the truth or be outdated, so your development would be held on false assumptions. There’s a saying: “Don’t mock what you don’t own”, which means you can stub only these pieces of architecture that you’re implementing. This is a proper approach when you test integration with the external system, but what if you want to assume stable dependencies and test only your implementation? Then yes, you mock everything except what-you-own. To sum up, the mocking and stubbing strategy can differ depending on test purposes.
Tight coupling with framework
That’s a tricky one. Developers often tend to choose frameworks and tools based on the current trends. The same applies to test frameworks where we have at least a few great frameworks to use just for a test runner, not to mention the REST client, build tool and so on. While choosing a technology stack, we should bear in mind the necessity to stay as much independent from tools as we can – it’s the test scenario that is the most important, not the frameworks.
Keep it simple
Depending on your team structure, acceptance tests are implemented by developers or testers. Usually the developers are better in writing code, while the testers have a more functional approach (it’s not a rule, though). Automated test are not a product itself, but rather a tool, therefore I would put functionality over complexity. Your test codebase is nearly as big as the tested system? Try to categorize tests according to their domain or type. Adding new tests requires a time-consuming code structure analysis? Sometimes a more verbose (but more readable) code is better for your tests than complex structures.
Summary
The worst-case scenario that can happen to your automated tests is abandoning them due to relatively simple, yet common issues. Time saved by automating simple test cases can be used for executing more complex scenarios manually, and that leads to better software quality and higher employee motivation in general. If you have some other interesting experiences with test automation, let me know!
