Tag: Software Testing

System integration testing, a 6 step survival guide

system integration testing

So you’ve gotten through your planning, design, development and application testing. Everything looks good, the team feels confident, and the project is on track. There’s only one problem. Up until this point you’ve been looking at your code changes with tunnel vision. Application testing is like fixing a bike tire while it’s not connected to the bike. The real testing begins when the wheel gets reconnected to the bike. Once reconnected, then you’ll see the how the fixed tire works with all other parts of the bike.

I’ve been through many large software projects, both as a project manager and as a tester. Often the most difficult part of completing a software project is end to end system integration testing. This is particularly true if you’re not using an Agile model.

System integration testing is where the rubber meets the road. If you work for companies like Facebook, Spotify, or Twitter you are lucky. These companies use modern Agile software delivery methods that make integration testing less painful. Automation, DevOps, and continuous delivery is the way of the future.

Unfortunately most companies still build software the old fashion way. They create one huge pile of software then throw it over the fence to QA nearing project end. This leaves integration testing only a matter of weeks to complete and meet a tight deadline.

If you don’t do a great job planning for integration testing, you’ll be in for real trouble. If you do a great job planning, you’ll still most likely be in for some pain.

Here are 6 steps to help you survive system integration testing so you can be successful.

  1. Plan enough contingency time – This might be the most important point. Give the team enough time to do the test effort. If you know it takes 1-2 weeks just to get your environment and data working right, plan for that time. In system integration testing, many roadblocks occur. Data, configuration and environment issues can  block testing. Plan for the unexpected.
  2. Create test scenarios with the help of system experts – Once you know all the affected systems, find out who the technical experts are for each system. Then schedule white board meetings to draw out end to end flow test scenarios. The system experts will be invaluable helping you to write end to end scenarios.
  3. Get all your data setup in advance – There’s nothing more frustrating than finding out none of your data works on day 1 of testing. Setup your data in advance and check to see its setup right before you do your testing.
  4. Run connectivity tests – For large projects, take at least 1 week to test environment connections before the start of your SIT. Often companies use shared testing environments and configurations are always being changed. Make sure all your configurations and connections are setup. If you don’t do this, you’ll think you’re running into defects only to discover an application is pointing to the wrong environment.
  5. Use a war room – All throughout system integration testing you should have a room where all key resources sit together, all day. The key players usually consist of developers, testers, a PM, and system and environment experts. As soon as a test runs into an issue, a developer should be there to look into it immediately.
  6. Have a test lead manage the effort – Last but not least is having a good test lead. In my opinion, test leads have one of the most difficult jobs when it comes to managing integration testing. The job requires both testing expertise, project management skills, and tremendous grit. Often test leads have a lot of pressure put on them to meet deadlines. A good test lead will rise to the challenge.

For more content like this, subscribe to the MacIsaac Consulting Blog.

To contact us about our services, click here.

Root cause analysis – Why toddlers are better at it than us

root cause analysis

If you’ve ever talked to a toddler, you’ve been hit with an endless amounts of whys…

Dad, why do you go to work? So I can make money. Why do you need to make money? So we can eat. Why do we need to eat? And on and on it goes.

Well, it turns out we need to learn from toddlers. Their relentless, and annoying, asking of why is what we should be doing for root cause analysis.  Working in technology, we deal with problems all the time. When we’re confronted with problems, the majority of us don’t take the time to do a good root cause analysis. This results in problems coming back to haunt us.

For example, say an application stops working. The team begins to triage the issue, and discovers a service stopped running. The service gets restarted, application relaunched and all is well. Or so the team thought. The next day the application stops working again. After inspection, the team finds again the agent needed to be restarted.

The agent stopping was only a symptom, and not the root cause. This is why we need to use “the 5 whys” when confronted with a problem. By asking why 5 times, you’ll dig deep enough to get to the core of the problem.

Repeating why five times is a process that was first used by Taiichi Ohno in the Toyota Production System. It is now a common practice in Lean software development.

In the example I gave above, the 5 why scenario might look like this:

  1. Why did the application stop working? Because the service stopped.
  2. Whey did the service stop? Joe performed an install and forgot to restart all the agents.
  3. Why did Joe forget to restart all the agents? Because there is no documented process about restarting agents.
  4. Why is there no documented process? Nobody took the time to do it.
  5. Why didn’t anyone take the time to do it. Everyone is busy fighting fires, nobody has the time.

In the above scenario, by asking why 5 times you get to the root of the problem.

Creator of XP Kent Beck writes: “After Five Whys, you find the people problem lying at the heart of the defect (and it’s almost always a people problem).

For more content like this, subscribe to the MacIsaac Consulting Blog.

To contact us about our services, click here.

References: Beck, K (With Gamma, E.). (2005) Extreme Programming Explained, Upper Saddle River, NJ, Pearson Education, Inc

 

 

 

 

Automated testing – The bottom line why it’s needed early and often

You need automated testing early and often because it reduces the cost of defects. “Defect Cost Increase (DCI) is one of the few empirical verified truths about software development: the sooner you find a defect, the cheaper it is to fix”(Beck, 2005). In a traditional waterfall project, all testing happens at the end of the project.

The waterfall based late testing approach results in high defect cost increase. As a former SQA analyst, I have experienced the pain that goes along with this process. Late nights working in WAR rooms, constant stress and continual red status reports. It’s the classic project death march experience.

In Agile and Lean software development, testing happens early and often. The team builds working software in short iterations. Testing is part of the software development process. In the below caption, Figure 14 shows the waterfall style of testing, compared to figure 15 which shows a more agile testing process.

FullSizeRender

One of the more popular and effective approaches is test driven development (TDD). Using TDD, the developer will first write an automated test. This test will ensure the basic functionality of the code works. After writing the test, he’ll first run it to see that it fails. Then, he’ll write his code, and run the test again to ensure the test passes.

By using TDD, the developer has already taken steps to ensure quality is built up front. The code then goes to QA, where integration, user experience, or performance testing happens. Just as the developer has created his automated test, so should the QA resource.

A high performing QA team will have an automated regression suite ready to run as soon as new code is ready. It is crucial that core functionality regression tests run first, before new feature testing. The idea is, make sure the new code hasn’t broken anything first, then begin the new feature testing. Most organizations have this backwards, doing all the regression testing at the end.

To put this all together you need the right tools, technology and resources. Whether it be JUnit, QTP, or Selenium, find the automation technology that’s right for your organization. Remember, the goal is to reduce defects and their cost.

Many organizations have avoided implementing test automation due to the expense. Ask any software quality team that uses automation and they’ll tell you the ROI is well worth it. If your organization builds software, you must use test automation. By not using it, you’re operating on an unsustainable model. It’s unsustainable because your regression test suite will continue to grow.  You cannot sustain a growing regression suite without automation.

For more content like this, subscribe to the MacIsaac Consulting Blog.

To contact us about our services, click here.

References: Beck, K (With Gamma, E.). (2005) Extreme Programming Explained, Upper Saddle River, NJ, Pearson Education, Inc

 

 

 

 

 

Powered by WordPress & Theme by Anders Norén