Automated performance testing with Pensum
Michael de Graaff
Arjan van Hugten
Michael de Graaff & Arjan van Hugten
 4 Minutes
 28 October 2020

Automated performance testing with Pensum

How do you ensure the availability, continuity and speed of your applications? This question is becoming increasingly important when when developing websites, online stores, intranets, client portals, or custom software. It is why we have already described which five technical tests are part of our development process. Although each one offers valuable insights, there is a drawback: designing, executing, analyzing, and maintaining tests is time consuming. We came up with a solution and are excited to share it with you!

How convenient would it be to test performance fully automatically and, as a result, save a lot of time? Our ideas for a solution grew from that thought. The result: our self-developed tool Pensum, which makes it possible to automatically test the performance of your applications. It is both fast and reliable. We explain how it works in three steps.

Step 1: scenarios

The first step of a performance test is usually laying out the possible scenarios. You ask questions such as: Which actions do users perform in the application? What are the most traveled paths? How many users do I expect? And what should my application be able to handle? The answers often emerge from data analyses conducted with tools for application performance management. Based on the results, a scenario may be tested which is as similar as possible to what will actually happen on application servers.  

Pensum takes almost the entire process off your hands and saves a good deal of time. The tool generates load test scenarios based on data from Google Analytics. As a result, the most current scenarios are always available and you are free to focus on implementation. 

Step 2: Testing

Once you generated your scenarios, you can start testing. This sounds simple, but in practice it means that you have to set up your entire infrastructure as it will be in production. You require the right tooling to store the results, appropriate insights into the performance of application servers, and a visual representation of the correct performance indicators.

In Pensum, we integrated three powerful tools for this: k6/LoadImpact for generating the load on application servers, InfluxDB for saving the results and Grafana for visual representation. 

Step 3: reporting and analysis

Now that the test results have been collected, you want to know whether the performance of the application has improved, weakened, or stayed the same. In this phase, you determine what good performance indicators are and, if results show weakening, where the delays occur.

Pensum also makes this final step much simpler because you are able to make comparisons between test runs easily. If you conduct a test with every release, it becomes even more convenient as the results can be placed side-by-side for clear, one-to-one comparisons. Moreover, Pensum generates the test results in a report automatically as a pdf.

Fast and flawless

You now know how Pensum makes it possible to automate every aspect of a technical quality test – design, testing, and analysis – which not only provides swift insights into results and red flags but also saves time while minimizing errors: the test scenarios are always based on the actual behavior of the application.

And because Pensum reveals potential weaknesses in performance immediately rather than at the end of the development process, you are able to respond instantly. By making Pensum a part of quality control for new releases, you save a lot of time with minimum effort. 

Curious about the technical background? Want to try out Pensum for yourself? Check out our opensource repositories on GitHub or the Pensum extractortool which distils load test scenarios from Google Analytics data. 

  • performance
  • testing
Michael de Graaff
Michael de Graaff

Software testing and test management that is Michael's area of ​​expertise. He is happy to share his lessons learned and how he comes up with solutions for automated testing, performance tuning and quality gatekeeping. Helping organizations set up and / or improve their testing process, makes him very passionate about his work.

Arjan van Hugten
Arjan van Hugten

After his research into generating and performing load tests based on historical data, Arjan was part of the setting up of Pensum. A way to generate automatic load tests. He also works as a back-end developer with certification for Kentico, among others.