We recently published a post on our blog which walked through how to use ChatGPT to analyze your JMeter test results. You can find the link in the description below for that article.
Today, we are going to show you in real-time some of these methods interacting with this popular AI assistant service. As we discussed in our blog post, getting the most out of ChatGPT comes down to being both creative and specific with your prompt, which is the technical term for each line of user input.
ChatGPT has powerful contextual capabilities, but these must be controlled with specific actions if we want the most useful output from the model.
Before we delve into asking ChatGPT to analyze our JMeter test results, let’s gather some sample data that we want for analysis. If you are using a cloud-based load test provider like Redline 13, this is an easy task. We’ll navigate to our test results, find our output files, and then download them. We’ll specifically need the JTL file as this contains all the sampled results from our JMeter test.
If you have run your JMeter tests locally or using a different cloud testing provider, you should be able to access the JTL file in a similar way.
Let’s get started with some basic prompts. We can demonstrate the ability for ChatGPT to interpret JMeter test results at a high level by using the basic prompt: ‘Can you analyze this JTL file?’ and pasting the content of the JTL file directly into the chat interface.
One limitation of the current iteration of ChatGPT is the inability to attach files, which is why we must paste the file contents as part of our prompt. A future release may allow us to attach files directly to be considered as part of the context.
After a few seconds of think time, ChatGPT responds with basic information about the structure of our JTL data and some basic conclusions about that data. All in all, this is not very useful for a results analysis perspective. However, it does highlight the importance of asking for specific information.
Another approach that yields better results is asking ChatGPT to create scripts for us that make our analysis easier. This has two benefits. First, it helps accomplish our immediate goal of analyzing this test. Secondarily, we could save the script for future tests where we don’t need to interact with ChatGPT at all to reuse the scripted as written for us.
Let’s follow with another prompt, taking advantage of the fact that our JTL file is in context with the current session. We can ask, ‘Can you write a Python script that graphs this data over time?’ Note the specificity in our request.
ChatGPT enthusiastically creates a Python script which is up to the task. It even gives us a commentary about the code and what lines we might need to change in order to use it.
Now, let’s take this sample code and run it in our favorite Python code editor. As noted, we may need to install publicly available dependencies using pip or an equivalent installer.
Here we have modified our script slightly to conform to our data in our use case. And here are the results of running our Python script.
This particular example uses Matplotlib as a dependency for graphing. The resultant graphs generated from the script can be exported, and the script, of course, can be further modified to suit more particular use cases.
As mentioned earlier, we can save this script for reuse. Using this method, we can ask ChatGPT to generate a variety of scripts which can be programmatically joined to run powerful custom reports on your JMeter tests.
All of this, of course, could be attained without the use of ChatGPT. The primary advantage is we have leveraged the AI system to perform the hard work of coding the scripts which generate these reports.
If you want to learn more about using ChatGPT to augment your JMeter load testing, please be sure to check out the links in our blog in the description below. Our blog also contains many other resources for JMeter load testing in the cloud.
