Automated testing – The bottom line why it’s needed early and often

You need automated testing early and often because it reduces the cost of defects. “Defect Cost Increase (DCI) is one of the few empirical verified truths about software development: the sooner you find a defect, the cheaper it is to fix”(Beck, 2005). In a traditional waterfall project, all testing happens at the end of the project.

The waterfall based late testing approach results in high defect cost increase. As a former SQA analyst, I have experienced the pain that goes along with this process. Late nights working in WAR rooms, constant stress and continual red status reports. It’s the classic project death march experience.

In Agile and Lean software development, testing happens early and often. The team builds working software in short iterations. Testing is part of the software development process. In the below caption, Figure 14 shows the waterfall style of testing, compared to figure 15 which shows a more agile testing process.

FullSizeRender

One of the more popular and effective approaches is test driven development (TDD). Using TDD, the developer will first write an automated test. This test will ensure the basic functionality of the code works. After writing the test, he’ll first run it to see that it fails. Then, he’ll write his code, and run the test again to ensure the test passes.

By using TDD, the developer has already taken steps to ensure quality is built up front. The code then goes to QA, where integration, user experience, or performance testing happens. Just as the developer has created his automated test, so should the QA resource.

A high performing QA team will have an automated regression suite ready to run as soon as new code is ready. It is crucial that core functionality regression tests run first, before new feature testing. The idea is, make sure the new code hasn’t broken anything first, then begin the new feature testing. Most organizations have this backwards, doing all the regression testing at the end.

To put this all together you need the right tools, technology and resources. Whether it be JUnit, QTP, or Selenium, find the automation technology that’s right for your organization. Remember, the goal is to reduce defects and their cost.

Many organizations have avoided implementing test automation due to the expense. Ask any software quality team that uses automation and they’ll tell you the ROI is well worth it. If your organization builds software, you must use test automation. By not using it, you’re operating on an unsustainable model. It’s unsustainable because your regression test suite will continue to grow.  You cannot sustain a growing regression suite without automation.

For more content like this, subscribe to the MacIsaac Consulting Blog.

To contact us about our services, click here.

References: Beck, K (With Gamma, E.). (2005) Extreme Programming Explained, Upper Saddle River, NJ, Pearson Education, Inc

 

 

 

 

 

Previous

Leading cause of failed Agile projects? Company culture not aligned with core Agile values

Next

Work life balance – What does “success” mean to you?

2 Comments

  1. Defect Cost Increase (DCI) is one of the few empirical verified truths about software development: the sooner you find a defect, the cheaper it is to fix”(Beck, 2005).

    This is a myth and has been discussed several times in the past few years. Here are two articles for you to read and update your commonly held misinformation.

    http://blog.paluno.uni-due.de/semat.org/wp-content/uploads/2012/03/a_short_history_of_the_cost_per_defect_metric.doc

    http://www.developsense.com/blog/2009/08/tyranny-of-always/

    • Interesting points. I think the main take away is that it’s better to build quality into the product, opposed to trying to inspect quality in

Leave a Reply

Your email address will not be published. Required fields are marked *

Powered by WordPress & Theme by Anders Norén