When to start Performance Testing?

-

While, it is ideal to start performance testing as soon as there is a confidence in stability of the system under test; that shouldn't be an entry criteria always. With that being said, doing performance testing after achieving the desired stability will keep the testing outcome clear and precise.


In case of modern, distributed systems; doing performance testing at independent component level shall be right thing to do. For example: Backend can be tested for its overall response/latency measurement and GUI can be test doubled and verified for its overall responsiveness under various conditions, which you can create in your test environment (Ex. high latency network links, slow responding backend under certain load conditions etc.). This strategy shall give you the performance assessment at individual unit level and its good to know baseline & benchmark of each of the component. So when you run end-2-end performance tests after integration, identifying bottlenecks could be relatively easy.

Now about performance goals; there could be different goals to measure for your application depending on what usecase it caters to. Here are few diverse examples:

  • WebApp, Mobile App: Responsiveness, loading of application, across support/interoperability matrix.
  • Cloud storage: Latency, IOPs, Throughput of file/object transfer upstream and downstream.
  • Virtual machines, VDI: Latency, Responsiveness of the VDI/VM, Individual application loading time on each of the instances etc.
  • Databases: Transactions per second

In summary, having goals pertaining to application's performance SLAs would be an ideal thing.

In my atomic talk  - "Outperforming in the Chaotic world!" in last year' TestFlix, I presented a strategy to combine performance with Chaos engineering to test application's performance in all possible, real-world conditions. Since conditions will not be the same always; it becomes important to benchmark performance of your applications during steady state as well as turbulent situations.

Happy Testing!

Comments

Post a Comment

Popular posts from this blog

Docker: What is it? How can it be used in Testing?

-
Everyone's talking about Docker now a days! There are plenty of use cases it has in various different areas of technology domain. First let's get to know what Docker is! Docker is a container virtualization technology, which does not require any hypervisor to run it. Strange, isn't it? Well, not really… here is how it works! Docker relies on Linux Containers (a.k.a. LXC), which is a pretty old implementation of container technology. LXC uses two of the Linux Kernel capabilities: i) Cgroups ii) Namespaces Cgroups enables the system resource allocation, isolation; such as Memory and CPU etc. while Namespaces let each of the container to run in an isolated space with its process isolation capability, without interfering with other containers on the same system. The result - You can run large number of containers on your system. The only limiting factor will be your host system's resources! Now what makes Container Virtualization differ from Virt
Read more

What DevOps mean to QA/Test?

-
DevOps - as it stands for "Development" + "Operations" is a buzz word these days and everyone is trying to get some insight into it. As exciting it may sound, many in the test department has a common question - "Where do they stand in the whole premise of DevOps?". The question is very valid and it needs to be dissected thoroughly to know what really is there in store for QA engineers. Traditionally QA has been divided into two categories - 1) Manual 2) Automation. While large population of the QA fraternity was involved in manual work initially, many started doing automation in their respective domains. Whether to do manual testing or automation is largely dependent on type of technology being used and other factors such as available skills within the test organisation, budget and most importantly organisational culture; in this case, how well the innovation is promoted. Now going back to our original question; Test Automation certainly has a very
Read more

Smart Testing – In the age of Machine Learning!

-
Image
                 Artificial Intelligence and Machine Learning are becoming wide spread across various industries. It plays a vital role in developing sci-fi sounding stuff such as Self driving cars and more practical things that touch human lives almost every day, like Voice assistants, Chatbots, Smart power grids etc.   In Software testing, transition from traditional approach to CI/CD driven DevTestOps model is essential to cope up with increased release cadence. Test automation has helped keeping up with the required delivery speed, however adapting AI/ML capabilities in test automation will be a go-forward approach.   Evolution of Software testing To begin with, let’s review how Software testing has evolved over past few years.   As testing evolved, so does the tools & techniques. However to keep up with the pace of change in the technology, testing strategies must reinvent and bring in required change.    Current challenges in Software Testing From days of slow paced deliverie
Read more