In today’s business environment can software development and software testing be isolated activities? Given the need of faster go-to-market and frequent product upgrades, looking at development and testing as isolated activities can often be a recipe for disaster. As the complexities in IT infrastructures increase because of the sheer number of integrated systems, the number of required tests grows too. This can often lead to delayed releases or worse, releases with bugs which the customers discover later. The costs that stem from software quality defects can make a huge dent in an organization's financial as well as social health. Software defects cost over USD$60 billion in the US alone. Research estimates that the cost of correcting software defects and bugs during the coding phase is USD $937 while correcting them post release amount to over USD $14,102!
One of our customers, a leading provider of recreational safety courses, had an excellent development team responsible for all development and testing. However, over time, there was an uptrend in issues reported by customers and also frequent slippages in release dates. That’s when they identified the need for a better testing strategy and a dedicated testing team. This is the story of how we helped them integrate testing within the development cycle and streamlined the testing processes.
One of the biggest challenges for our client was the stringent go-to-market timelines. They were also under the pressure to roll out frequent upgrades of their flagship products. However, in the absence of a robust testing strategy or a dedicated QA team, the releases were getting delayed and it also led to the users reporting bugs and issues! Given the competitive environment that we exist in today where users are spoilt for choice, these issues could easily lead to their customers switching loyalties. This is where we began our journey with our client.
Before we set out to provide a solution, our team thought it was imperative to first understand the workings of our client’s business. In order to give them the right solution, given the specialized nature of their product, we had to first understand what our client does and what they hoped to achieve with their product. Then, we went on to identify the challenges at hand and discovered that along with the absence of a defined testing process, our client also needed a thorough documentation process. We discovered that we were working with legacy systems with large gaps in knowledge of the applications’ behaviour. Our investigations also revealed that all of their applications had to be tested across a wide variety of platforms. The application had over 15+ browser extensions and had to work across different operating systems such as Win XP, Windows 7, Windows 8.1 and MAC 10.7, MAC 10.8, Mac 10.10 and needed to display correctly on different form factors such as iPads and PC’s. Clearly, we had our work cut out for us.
We knew we had to take a very structured approach to solving this problem, so we started by breaking down the goals into three phases that covered release stability, test case coverage, and test automation. We identified the need to integrate testing in every sprint along with the need to introduce documentation of each function as a test case. We also realized that for the successful completion of this project, we needed to take project overview in a timely and structured manner.
A strong foundation is needed to build a strong building - We took the same approach towards this project and started by designing a robust testing strategy that included Compatibility Testing across various OS and Browser combinations.
- We suggested the use of Selenium as a test automation tool as it made an allowance for wide adoption needed for continuous regression testing.
- We also appointed two test automation engineers whose main function was to automate all the critical application which helped in reducing regression time.
- While we went along with the test automation initiatives, we also employed one manual tester whose main scope of work was to document test cases and to provide consistent feedback on the release quality.
- Along with this, in order to ensure the timely execution and completion of the project, we designed a Requirements Traceability Matrix that was updated on a weekly basis.
- We also had monthly governance calls at the leadership level to review the project performance and to ensure that all parameters discussed were being met. We also discussed the areas of improvement during those calls.
Now that the project is complete, we can look back on the result of our hard work with pride.
- We successfully managed to streamline the testing process for all their courses and completed automation for courses across 50 states.
- We managed to increase the test coverage of all applications by 90% and reduced the regression testing time by over 70% through our automation initiatives. The automated regression suite saved 40 man-hours per release cycle.
- We ensured that all issues and enhancements to the product were listed systematically and tracked according to priority.
The icing on the cake was that our testing initiatives helped our client reduce their development time, cost and the associated stress for all stakeholders in development and business operations.
Using a planned and systematic approach, within a period of three months, we were regression testing the release candidates of our client’s flagship products. Our testing strategies were so comprehensive that we were detecting bugs even before the product reached production. We ensured that we had built a robust process for product testing to ensure that the final product would achieve consistent customer satisfaction.
Within a period of twelve months, we had completely streamlined their testing process and today the Clarion testing team runs like a well-oiled machine for our client. Needless to say, our client is more than thrilled with our professionalism and efficiency that helped in increasing the effectiveness of their product.
Client Speak -
"Clarion had the expertise and resources that we sought, and the ability to hit the ground running. They have wasted no time diving into our flagship product. Despite the very specialized nature of our business and products, they were quick to grasp what we do and quickly demonstrated their expertise. We were amazed at the speed of their progress. Clarion has been extremely easy to work with, always welcoming our feedback, and staying true to our process. Needless to say, we’re huge fans of Clarion.”