Performance testing is crucial field of modern software development. Taking into account that in today’s world majority of app’s communication is web-based, it turns out to be even more important. However, it still enjoys less interest than automated functional testing, and publications on load testing subject usually focus on mature and complex tools like JMeter or Gatling. In this post I’d like to introduce command line usage of a super simple and lightweight tool for performance testing of HTTP services, which is Vegeta.
Who is Vegeta
Besides Dragon Ball’s character, Vegeta is a simple load testing tool written in GO, that can be used both from command line and as an external library in your project. In order to use Vegeta, download executables from here. It is recommended to set environment variable for ease of use.
For Mac users, you can download vegeta directly from Homebrew repositories:
$ brew update && brew install vegeta
Vegeta’s Arsenal
Usage is pretty straight foreward, you just need to keep with scheme:
vegeta [global flags]
Global flags let’s you customize number of CPUs to use, or profile cpu and heap settings. You can also check version of your vegeta executable (-version).
Next you choose a command, which is one of three: attack, report and dump, and then set command flags, which are options for each command. Attack command is to deal with your attack target. You can set request body here, attach string with server certificate, set headers, define req/s rate and much more. Report command let’s you create report of your tests, which can be in simple stdout string or for example some neat html plots. Dump is a very handy command, where you can log information of all your request to stdout or save it to file. You can view full option list with:
$ vegeta
(yes, that simple!)
Prepare to Battle
Enough theory, let’s make some kamehame-ha! We will perform some load testing on simple GET/POST service on our local machine (and yes, load testing your localhost isn’t the brightest idea, but it’s for educational purpose only). We will use standalone wiremock stubs from one of my previous posts, that creates REST application mock. Basically, we want to test two methods:
GET /user/1
POST /addNewUser
{
“login” : “lukaszroslonek”,
“www” : “www.google.com”
}
First Clash
Let’s say we want to perform 3 seconds attack with default req/s number, using 2 CPUs. We also want to analyse test results in form of simple text output in console. In order to do so, we’ll execute following command in console:
$ echo “GET http://localhost:8080/user/1” | vegeta -cpus 2 attack -duration=3s | vegeta report
In the first part of above command, we define HTTP method and endpoint under test. In second part, we set global flag of CPUs number and declare the attack to last 3 seconds. Last part is where we tell Vegeta to prepare test report, which is printed to stdout by default. If everything went successful, we should see output:
$ echo “GET http://localhost:8080/user/1” | vegeta -cpus 2 attack -duration=3s | vegeta report
Requests [total, rate] 150, 50.34
Duration [total, attack, wait] 2.981607925s, 2.979999934s, 1.607991ms
Latencies [mean, 50, 95, 99, max] 1.291686ms, 1.4065ms, 1.842859ms, 1.994408ms, 2.132038ms
Bytes In [total, mean] 10350, 69.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:150
Error Set:
We can familiarize ourselfs about some important metrics from report output like success ratio, latency or request rate.
Second Clash
For the second method, we want to perform 1 second attack with 100 req/s and default number of CPUs. HTTP method and enpoint should be read from text file, and we want to produce nice, html file with plot report. To do so, first of all we need to prepare two files.
target.txt:
POST http://localhost:8080/addNewUser
body.json:
{
“login” : “lukaszroslonek”,
“www” : “www.google.com”
}
Now we can execute following command:
$ vegeta attack -targets=target.txt -body=body.json -rate=100 -duration=1s | vegeta report -reporter=plot -output=report.html
In this test, we provide targets list in target.txt file, and request body in body.json. We also set report type to plot and redirect output to report.html file, which creates for us nice looking, interactive plot:
Final Flash
Results from single test are stored in results.bin file, which is created after attack execution. We can use it for further analysis of test output. Let’s say we want to create a histogram with requests duration times. To achive that, we need to redirect output of cat results.bin to vegeta report:
$ cat results.bin | vegeta report -reporter=”hist[0,100ms,200ms,300ms]” -output=hist.txt
…and we’ve just created text file hist.txt with following content:
$ cat hist.txt
Bucket # % Histogram
[0, 100ms] 4 100.00% ############################################################
[100ms, 200ms] 0 0.00%
[200ms, 300ms] 0 0.00%
[300ms, +Inf] 0 0.00%
Continue reading
If you want to continue reading and expand your knowledge in area of REST and microservices, I recommend you these books:
- Building Microservices – one of the most important books for me, everything you want to know about microservices is here
- Java For Testers: Learn Java fundamentals fast – test automation does not require complex programming knowledge. Learn fundamentals of Java for test automation. From tester to testers!
- RESTful Web APIs – another great book about REST architecture. Lots of practical knowledge about designing and consuming RESTful APIs
Summary
Vegeta is a very simple tool that let’s you perform simple load tests of your HTTP service. Although it does not offer as many possibilities like JMeter, LoadUI or Gatling, it’s