All of my articles are free to read. If you don’t have a Medium subscription, you can still read by clicking this link.
I started wondering why many test automation scripts continue to fail infrequently and daily? After a deep analysis with more 20000+ tests, I found that it is not largely to do with the tools or scripting techniques and neither with the build. So what causes more failures? The wrong selection of tests that are automated is the biggest pain today.
Most importantly the manual testers, who spend tirelessly testing the product or project with almost daily build today. I personally have experienced my testing team start losing interest in testing creatively over time due to repeated manual (boring) tasks.
With test automation in place, testers could be much prepared for testing the inevitable changes deployed in the new builds. Agree?
Where to Start?
Start automating the tests that are manually tested against each build. Do you mean smoke tests first? Yes, the build verification tests. Let’s see how does it make a difference.
Assume, a test case is about 20 steps and with few verifications on it. Let’s say, the manual testing effort takes 5 minutes to complete for a single iteration and the test automation effort for the same test (using Selenium WebDriver) for the single browser (~chrome) may take about 120 to 180 minutes or more including development, testing, integration, etc.
With that said, unless you run your automated test minimally 24 times to get the returns with the hope that the automated test does not break before you achieve your returns. Okay?
Per industry data, 5% of the overall test contributes towards the smoke and that sounds a minimal effort. Correct, it is the beginning of the automation success.
Can developers use my automated smoke tests?
If the smoke tests are successful and used on daily tests inside test environments and can be as well promoted for use with developers too. Why not? Shift Left.
What next?
Then, move on to sanity tests — the most the business-critical tests that need to be automated to ensure the business cases do not fail in the production environment. It may be a good idea to automate them for multiple test data as well. Per industry data, roughly 15% of the overall test contributes towards the sanity and that sounds like a decent effort. Together with smoke and sanity, you reach the Pareto rule (roughly 80% of the effects come from 20% of the causes). Make sense?
Sanity tests are developed and what then?
You may be tempted to go next. But stay there for a while by executing sanity tests against different browsers, platforms, and different data to confirm the automation stability. Great idea?
If all goes well, then what?
If all goes well, your automation is working great for you now. Time to step up! Go for the minimal regression suite and maybe new functionalities as well, if you are part of in-sprint automation.
In sum, start automating the minimalism — the smoke and sanity and then grow your automation suite consistently. Remember, the tree does not bear fruit overnight and neither success happens over a day. Start small and stay consistent with your test automation.
