Friday , 22 September 2017
Home / Test Process & Best Practices / How Do You Know If Your Testing Is Effective?
How-Do-You-Know-If-Your-Testing-Is-Effective

How Do You Know If Your Testing Is Effective?

Yesterday I saw a very interesting post in one of the LinkedIn groups. Someone asked the question, “If you’re in an interview and asked ‘how do you know if your testing is effective’, what would your answer be? I think it is a very interesting and at the same time a very tricky question.  Along those same lines, the questions I ask in interviews are, “How do you measure your success in testing?”, and “How do you know you achieved good results?”.  My statistics show that 90% of the candidates struggle to answer these questions correctly. Here are some of my recommendations to answer this questions correctly.

In order to understand if something is effective you need to measure it against a baseline. You need metrics. The effectiveness of the testing can be measured by the following metrics (there more but the ones below are the ones I personally like the most):

  1. Test Effectiveness %– this metrics shows the efficiency of removing defects by internal testing before delivering to customer. Internal testing means (static testing, SAT, SIT, UAT).
  2. % of Defect Leakage– how many defects were missed during System Testing. For this you have to measure UAT defects or combined ST + UAT and measure leakage in production. For more information please refer to this link: http://www.qamentor.com/wisdom-center/sample-deliverables-showcase/defect-leakage-analysis
  3. Defect Removal Efficiency %– comparison of internally reported defects with total defects, which includes those that customer reported from the production Environment.
  4. Test Execution Productivity– productivity of executors, the number of test cases per tester per unit of time. If you are lazy you are not effective. Simple theory.
  5. Defect Rejection Rate– this shows how well the testing team understood requirements and how many rejected defects are there due to misunderstanding of requirements.
  6. Error Discovery Rate– It is defined as ratio of defects per test case. The higher the discovery rate the more structured and more complete your test case coverage is in general. Yes, it’s possible to argue that if the code is perfect then this metric wouldn’t make sense. I would agree, but it is still a better metric than no metric.

But one thing you must remember is that effectiveness needs to be measured and so we should have metrics that will help us to analyze and come up with decisions about our effectiveness.

About the Author

The author, Ruslan Desyatnikov, is the CEO & Founder of QA Mentor. He created QA Mentor to fill the gap he has witnessed in QA service providers during his near 20 years in QA. With Ruslan’s guidance, unique services and methodologies were developed at QA Mentor to aid clients in their QA difficulties while still offering a high ROI. Ruslan offers monthly seminars aimed at imparting his extensive testing knowledge that can be applied to start-ups as well as large companies. To learn more about QA Mentor and testing services please visit www.qamentor.com or contact Ruslan directly by sending email to [email protected]

Check Also

Habbits

Five Common Habits Keeping You from Success

Success means something different to different people. For some, it’s starting and raising a family. …

7 comments

  1. I absolutely love your blog and find the majority of your post’s to be precisely
    what I’m looking for. Does one offer guest writers
    to write content for you personally? I wouldn’t mind producing a post or
    elaborating on many of the subjects you write with regards
    to here. Again, awesome weblog!

  2. Hey there, You have done a fantastic job. I’ll certainly digg it and personally recommend to my friends.

    I’m sure they will be benefited from this web site.

  3. Your style is so unique in comparison to other folks
    I’ve read stuff from. Thank you for postijg whewn you have the opportunity,
    Guess I’ll just book mark this web site.

  4. Thank you for any other informative site. Where else may I get that kind of info written in such
    a perfect method? I’ve a undertaking that I’m simply now operating on,
    and I have been on the glance out for such info.

  5. Nice post. I used to be checking continuously this blog and I am impressed!
    Extremely useful information particularly the last
    section 🙂 I deal with such information a lot. I used to be
    looking for this particular information for a long time.
    Thanks and good luck.

  6. I like looking at and I believe this website got some truly useful
    stuff on it!

  7. Τhis pieсe of writing is genuinely a good one itt helps new internet people, who are wishing for blogցing.

Leave a Reply

Your email address will not be published. Required fields are marked *

*