In the world of development, testing is considered a necessary evil and it is the first thing to be sacrificed when deadlines are knocking at the project manager’s door. However, without proper QA, applications are prone to failure, bad reviews, dissatisfied customers and ultimately bad marketing for the company. What is the middle line in this case?
The answer lies in the difference between two approaches to software development, the classic waterfall and the modern agile. In the first approach, testing was done sequentially, at the end of predetermined stages and was performed manually, which took a considerable amount of time. In the agile methodology, a minimal viable product is designed, tested, released to a small market segment, retested, reworked and so on. Since in the Agile way there is a lot more testing and refining involved, automation emerged as a necessity.
Automatic Software Testing Benefits
When it comes to automation, there is a tradeoff between speed and costs. Since automation requires top programmers, it’s not cheap, and sometimes you are better off performing tests manually. Yet there are some hard to ignore benefits, as an article on test automation by A1QA describes.
Speed & Accuracy
Automatic testing saves between 70-90% of the time necessary for the same tests to be performed by manually. The machine is never tired, doesn’t get bored and can run 24/7 without asking for a raise or time off. This gives reliability, consistency, and accuracy.
Once the test is set-up, human input is no longer necessary, and the results are even more reliable than those produced by QA testers since there is no psychological factor to slow down the process.
Scope & Scale
Automatic testing can also go a lot more in-depth compared to humans. It becomes beneficial in the case of multi-language software and can even go to some limit cases that general manual testing could easily oversee.
Automatic testing is also capable of simulating large volumes of simultaneous users, an aspect that is important to ensure the server and the network are ready to handle this and which is impossible to replicate by human testers.
Reusable code and Time to market
The most important thing is that pieces of code can repeatedly be used through multiple stages of polishing the product. This feature speeds up the process and increases the quality of the product iteratively. The pay once, use forever feature is also tempting especially when there are a lot of test cases to be performed repetitively.
Having code snippets that speed up testing and can even be used by non-technical staff accelerates time to market and can even cut costs in the long run.
What should be automatically tested?
Of course, not all features should be tested automatically, as this could, in fact, prolong the time to delivering the final product and make costs increase exponentially.
The first candidates for automation are the bits that are dull and repetitive. Just imagine testing the scrolling or the buttons on a page. It is easy, but tedious, yet not having this functionality will be regarded as a major flaw by clients.
Regression testing, namely performing the same test over and over again on new versions of the same software are some of the parts that could benefit most from automating the test scripts.
Included in the same category are the parts that are very time-consuming and prone to errors when performed by humans. Just imagine testing a long sequence and the tester making a small mistake just before the end and being forced to start over. Such a process should be high on the list for machine-powered testing.
Another noteworthy category is that of tests that are critical to the program, such as the initial user login or payment gateways. Since most platforms use a third party for payments, one of the most important aspects to be tested is integration.
What is better to test manually?
Regardless of the benefits brought by automation, it should not be applied blindly. A test should never be automated by default before it is performed a few times by a human tester. This brings insight into the major flaws and the best way to treat the problem. After manual testing, the conclusion could be that it needs to be more detailed, or it is not a candidate for automation.
Also never automatize tests that are related to items that change often. It is a waste of time and of good resources to do so. For example, during user experience testing a small number of real tests should be performed with real users that are able to give developers feedback instead of letting a machine choose.
It is also less advisable to automate testing when the problem is very complex, and it can unfold using different scenarios. With AI’s progress and deep learning, this could become a reality, but the market is not ready yet.
Rely on manual testing when you still are in development mode and have too many ideas to stick to and perfect.
Barriers to Automated Tests
There are three main obstacles to applying automated software testing. The first is related to set-up costs which can be intimidating and sometimes do not fit in the initial development budget of the app. The second one, which is the hardest to overcome is corporate culture. Manual QA testers are afraid of losing their jobs and most of the times lack the skills to create automated testing. The last aspect is related to having a viable strategy and a way to keep all the code snippets up to date.
Automation testing is becoming a helpful tool for different test branches including functional testing, integration testing, live-monitoring, and multi-platform testing. However, it should be used with care and only for situations that are already fully understood and documented, not for new ones. Manual QA testers should not worry about their jobs, as machines are not ready to take over more delicate aspects like debugging, user experience and dealing with complex cases.