Software testing: Good Practice vs. Bad Practice

Published 17th December 2008

Defining a testing approach has to account for many factors, a lot of which are not within control. It is therefore extremely important that the testing decided upon should be aligned with the business objectives, and to attain this, adapting to the best practice is the defined solution, which is seldom a reality.

This article offers pragmatic thoughts around adopting good testing practice as opposed to best practice to actually suit the needs of the business. While the importance of ‘Best Practice’ is echoed throughout the IT industry, good practice in certain cases can be a more practical solution.

There are various standards in place which organizations can adhere to in order to deem themselves observers of best practice. For example, the Capability Maturity Model Integration (CMMI) is a process improvement approach that “provides organizations with the essential elements of effective processes.” Attaining the level 5 of certification verifies that the practices of the organization are exemplary.

The Testing Maturity Model (TMM) designed to complement CMMI and is based on best industry practices, and the Test Process Improvement model from Sogeti supports the improvement of test processes by looking at 20 key areas and has different levels therein to enable insight into the state of the key areas.

In order to satisfy the criteria stipulated in the best practice guidelines, organizations must be committed and must invest time and money to implement and adhere to the processes as defined by such guidelines.

The motivations behind testing are usually related to reducing costs that may be incurred post launch, improving quality of applications and reducing time to market. The overall objective might be to keep what are labeled ‘IT’ costs as low as possible. Adopting best practice in such a case is impractical and more expensive. Yet if the goal is to reduce costs in long-term, best practice would perhaps be more suitable.

In this case, a cost analysis of good practice versus best practice would reveal the disparity. At what point, then do the added costs related to undertaking best practice transcend into necessary expenditure? This happens when success of the project or application is of critical importance. A retail website would not necessitate the same level of defect elimination that the launch of a NASA rocket would; one would obviously require more rigorous testing than the other. The level of acceptable risk of the application therefore becomes a crucial determining factor in whether to adopt best or good practice. Where quality is of the utmost importance and cost is secondary, best practice should be chosen.

One of the other drivers behind testing is to reduce time to market. This would be necessary, for example, within the financial services industry where first mover advantage of a new policy system can give huge benefits in terms of reputation and profits. Good practice may ensure a faster testing process due to having fewer criteria to fulfill. There are exemplary practices and guidelines to follow, but again the suitability of them for the project should be considered.

Choosing good practice over best practice can also depend where you are in the development lifecycle. There may be tactical or strategic solutions adopted to best fit the project.

In these challenging times, there is an increased pressure on businesses to get products to market quicker, reduce costs and increase margins. It is imperative that testing helps the business achieve this by being more streamlined and efficient, and must effectively balance risk against benefits. In some cases, organizations are willing to accept a higher level of risk because the benefits of getting their product to market will outweigh the cost of resolving issues in production. The key is to understand those risks and to develop a flexible approach which maintains the agreed quality standards.

Good practice is not about cutting corners, nor should it be considered sub standard. Instead, when defining the testing approach, cost analysis, risk analysis and process definition should all be considered. If properly developed, appropriate testing should be implemented for the organization at all times.

It could be argued that whatever an organization chooses to do that meets their business requirements and achieves a quality deliverable into the organization is ‘best practice’, even if it cannot be found in a book!

To read the complete Industry Comment, visit: http://www.applabs.com/html/SoftwareTestingGoddpracticevsBadPractice_514.html