Best of Xebia tech list August 2007: Magic number 28232

15 Aug, 2007
Xebia Background Header Wave

We have had a number of very interesting discussions on our techlist in the last week or so, so I thought it best to get started as soon as possible. The reason I choose this particular thread is that it is a very technical thread, but it’s very hard to find any information about it online. Oh, and it’s not about Java or Agile for a change!
This thread quickly rose to about 20 mails, so I have made a selection to keep it readable.

FromWilco Koorn
SubjectMagic number 28232??
DateAugust 13, 2007 18:48:40

I'm stress testing a system using socket communication.
I have a client that uses this unix command:
for i in `seq 1 100000`; do nc -v 8910 & echo $i; done
Or in other words, I'm trying to make 100.000 connections to something listening on box, port 8910
I have a way (through a web interface) at the box to count connections. All is well until I hit the number 28232. If I fire this command from another box, I see more than 28.232 connections
I want to know what this number is. Must be some limit in the "client" machine (its not the max number of open files, nor the max number of processes). Perhaps in the network?

FromErik Rozendaal
SubjectMagic number 28232??
DateAugust 13, 2007 21:50:40

What's the output at the client? One thing that could be done is the maximum command line length... limited to ~64k on some unixen. So it could be the `seq 1 100000` is only expanded up to 28.232 times before hitting that limit.

FromErik Jan de Wit
SubjectMagic number 28232??
DateAugust 14, 2007 7:49:42
On unix socket connections are limited by the number of files there can by simultaneous be open (e.g. number of open files) There are a couple of places where you can find this limit:
1. In /etc/security/limits.conf
* soft nofile 1024
* hard nofile 65535
2. Increase ulimit by "ulimit -n 65535"
3. echo 65535 > /proc/sys/fs/file-max
4. In /etc/sysctl.conf
But why do you want 100.000 connections to be open at the same time usually connections are pooled?

FromWilco Koorn
SubjectMagic number 28232??
DateAugust 14, 2007 9:52:42
It is consistent. It is also NOT a file handling thingie (I had that before at 1024 and we put it to the max).
I've seen this limit with two different test methods. Even appears when we have two boxes running the clients where those two are on the same subnet, which might indicate network/router/firewall problems.
I'm particularly interested in 28232 as number. It is not 32k - 4k. This might give me a clue what's happening.
Thanks anyway, Wilco.

FromSilvester van der Bijl
SubjectMagic number 28232??
DateAugust 14, 2007 10:10:09

It's the port range available to the OS. See
Short version:
/proc/sys/net/ipv4/ip_local_port_range is "32768 61000"
61000 - 32768 = 28232 maximum ports available for outgoing connections.
You can increase the port range to allow for more simultaneous outgoing connections. If I remember correctly from your presentation it wouldn't be a problem for the production setup (client only opens 1 connection to a single server port).

FromWilco Koorn
SubjectMagic number 28232?? Solved!!
DateAugust 14, 2007 10:17:14

The magic number 28232 where I bump into is indeed caused by the range of ports available to netcat.
sysctl -a | grep port
net.ipv4.ip_local_port_range = 32768 61000
And, 61000 - 32768 = 28232!!!!
Thanks everybody for the help, regards, Wilco


Get in touch with us to learn more about the subject and related solutions

Explore related posts