If there’s anything that the US healthcare website failure taught those of us in the software industry, it’s the value of testing. More specifically, the value of performance and stress testing. The government website’s failure was in large part due to its inability to handle the number of users that were on it simultaneously. This exact scenario is what QA Mentor’s Performance and Capacity Planning Services were designed to prevent.
You want your company to grow and you need more people to visit, share, and use your website, especially in today’s online world. But, you need to be sure that your site can handle the traffic you currently have and are trying to generate. Your website infrastructure and architecture needs to be able to grow with you and handle your new and existing customers without errors or long waiting times.
When you find issues in production, the costs to fix them increase exponentially. Not only do you lose money in the way of future or current business and your online image, but it’s more complicated and time consuming to fix issues in a production environment. It’s far more efficient to find and fix them prior to a live release. No matter what the US government does at this point, no matter how much they improve the health care website, everyone will remember that it failed on release and their faith in the services offered by the site are already diminished. This is a valuable lesson and one that QA Mentor can help you avoid learning on your own.
Performance testing includes multiple, distinct facets to fully exercise the software and hardware and identify any weaknesses, as well as benchmark where and when the issues arise. Every site has its breaking point and QA Mentor’s Performance testing services can help you find yours so you can fix inefficiencies prior to release, add additional hardware, or be prepared for future scalability needs with our Capacity Planning.
So, how do we do it?
We perform several distinct types of tests either simultaneously or consecutively to establish industry defined benchmarks, monitor performance and stress the system to find that magic breaking point.
During this part of the performance testing process, we first determine a normal load expectancy for your site by using usage statistics provided by your or gathered online. With that information, we can find the optimal configuration of concurrent users that your site should handle easily. Next, we determine a heavy load capacity, and then the expected load based on to projected growth. Starting with the lighter load, we steadily increase the concurrent users until the system starts to respond more slowly. We continue the increase until the system stops responding, thereby discovering the failure threshold. This information gives us the optimal load boundaries and the data gathered helps to identify where the bottlenecks are.
Generally, this test will uncover buffer overflow issues, memory leaks, or load balance problems. Performance optimization can then be completed and hardware changes or additions can be made to increase the failure thresholds as needed.
This part of the test focuses on data volume. Similar to the concurrent users’ threshold, we slowly increase the amount of data being processed or transferred by the systems under test. With this test we can determine the amount of data your site can handle before it starts to display errors or stop responding at all.
During stress testing, we find the site’s breaking point. We give the site more users and data than we know it can handle and see what it does in response. This type of stress is considered abnormal for the system, but it’s important to know and understand how your software responds and most importantly, how it recovers from these unusual, but not impossible stressors. Can your system take a very heavy, load that buckles it and then return to normal operating processes on its own? This test will tell you exactly that information so we can use it in our Capacity Planning for you.
This part of the performance testing shows how well your site can receive a normal load and maintain an optimal performance level. Using the usage statistics gathered before the tests, we determine what an ‘everyday’ load is for your site. We increase the concurrent users and data transfer or processing to that point and hold steady for a predetermined period of time, watching for any errors or unusual behavior. This test can help identify memory leaks or other issues that a dramatic load test may miss, and since this is the normal, everyday operating capacity of your site, it’s very important to feel confident it can maintain that consistency without error.
Using leading industry tools such as LoadRunner, LoadStorm, Cloudtest, and Rational Performance Tester, we gather data during all parts of the performance testing. This data is vital to help identify performance bottlenecks, point to infrastructure weaknesses, and help you to make plans for future scalability due to your imminent growth. We will be able to tell you exactly how many users your site can handle before it starts to respond more slowly. We can point you to exactly how many transactions you can process per day, hour, or even per second before the site begins to exhibit stress. We can then work with you to fix the performance issues and make sure your site is ready for production, and ready for the growth of your company.