Why even Spray-can is Way Too Slow (for my purposes)

In a previous blog I discussed the speed of the Spray-can web-server and mentioned some measurements I did. My co-worker Age Mooij, committer on the Spray project, pointed me at ‘weighttp’ (see weighttp at github) a tool for benchmarking web servers. Cool! Of course I now had to do more experiments and so I did. I found out Spray-can is way too slow for my purposes and here’s why.

Recall I am after handling peak-loads on the net. Part of the solution (no details here, sorry, maybe later) I have in mind is handing out an integer number. Therefore the Spray-can server just issues a number at this time. Age suggested this weighttp command:

weighttp -n 100000 -c 100 -t 4 -k 'https://xebia.com/blog:8080/dispatcher'

And so I did. I got:

finished in 2 sec, 302 millisec and 196 microsec, 43436 req/s, 5594 kbyte/s

There is no typo in this, I got a throughput of 43k requests per second. Now that is impressive. When doing ten times more:

weighttp -n 1000000 -c 100 -t 4 -k 'https://xebia.com/blog:8080/dispatcher'

I even get:

finished in 13 sec, 797 millisec and 56 microsec, 72479 req/s, 9420 kbyte/s<span style="background-color: #ffffff; color: #333333; font-family: Georgia, 'Times New Roman', 'Bitstream Charter', Times, serif; font-size: 16px;">That's almost a throughput of 73k requests/sec!!! On a laptop! And I'm not even using the latest Spray-can version which is supposedly even faster.</span>

BUT
Note the usage of the ‘-k’ switch. It is the "keep alive" switch. Here’s what happens when you do not use it:

weighttp -n 100000 -c 100 -t 4 'https://xebia.com/blog:8080/dispatcher'

I get:

finished in 217 sec, 972 millisec and 939 microsec, 458 req/s, 68 kbyte/s

Que? A throughput of only about 450 requests / second? What the F? So I dug into this "keep alive" some more. My good friend Google came up with persistent http connections and I read some more here: KeepAlive Nonsense
Where does this leave me?

Well, I learned Spray-can has beyond excellent performance when keeping the connection open. For my purpose, peak load handling, this is irrelevant, as I want to handle about 200k requests within 10 seconds but these requests will come in from different clients. Therefore the experiment without the ‘-k’ switch is closer to reality. Ergo even Spray-can is way too slow for my purposes. Back to the drawing table. I learned a lot today.