Blog

How to create the smallest possible docker container of any image

30 Jun, 2015
Xebia Background Header Wave

Once you start to do some serious work with Docker, you soon find that downloading images from the registry is a real bottleneck in starting applications. In this blog post we show you how you can reduce the size of any docker image to just a few percent of the original. So is your image too fat, try stripping your Docker image! The strip-docker-image utility demonstrated in this blog makes your containers faster and safer at the same time!

We are working quite intensively on our High Available Docker Container Platform  using CoreOS and Consul which consists of a number of containers (NGiNX, HAProxy, the Registrator and Consul). These containers run on each of the nodes in our CoreOS cluster and when the cluster boots, more than 600Mb is downloaded by the 3 nodes in the cluster. This is quite time consuming.
[bash]
cargonauts/consul-http-router latest 7b9a6e858751 7 days ago 153 MB
cargonauts/progrium-consul latest 32253bc8752d 7 weeks ago 60.75 MB
progrium/registrator latest 6084f839101b 4 months ago 13.75 MB
[/bash]
The size of the images is not only detrimental to the boot time of our platform, it also increases the attack surface of the container.  With 153Mb of utilities in the  NGiNX based consul-http-router, there is a lot of stuff in the container that you can use once you get inside. As we were thinking of running this router in a DMZ, we wanted to minimise the amount of tools lying around for a potential hacker.
From our colleague Adriaan de Jonge we already learned how to create the smallest possible Docker container  for a Go program. Could we repeat this by just extracting the NGiNX executable from the official distribution and copying it onto a scratch image?  And it turns out we can!

finding the necessary files

Using the utility dpkg we can list all the files that are installed by NGiNX
[bash]
docker run nginx dpkg -L nginx

/.
/usr
/usr/sbin
/usr/sbin/nginx
/usr/share
/usr/share/doc
/usr/share/doc/nginx

/etc/init.d/nginx
[/bash]

locating dependent shared libraries

So we have the list of files in the package, but we do not have the shared libraries that are referenced by the executable. Fortunately, these can be retrieved using the ldd utility.
[bash]
docker run nginx ldd /usr/sbin/nginx

linux-vdso.so.1 (0x00007fff561d6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd8f17cf000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fd8f1598000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fd8f1329000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fd8f10c9000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fd8f0cce000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd8f0ab2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd8f0709000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd8f19f0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd8f0505000)
[/bash]

Following and including symbolic links

Now we have the executable and the referenced shared libraries, it turns out that ldd normally names the symbolic link and not the actual file name of the shared library.
[bash]
docker run nginx ls -l /lib/x86_64-linux-gnu/libcrypt.so.1

lrwxrwxrwx 1 root root 16 Apr 15 00:01 /lib/x86_64-linux-gnu/libcrypt.so.1 -> libcrypt-2.19.so
[/bash]
By resolving the symbolic links and including both the link and the file, we are ready to export the bare essentials from the container!

getpwnam does not work

But after copying all essentials files to a scratch image, NGiNX did not start.  It appeared that NGiNX tries to resolve the user id ‘nginx’ and fails to do so.
[bash]
docker run -P –entrypoint /usr/sbin/nginx stripped-nginx -g "daemon off;"

2015/06/29 21:29:08 [emerg] 1#1: getpwnam("nginx") failed (2: No such file or directory) in /etc/nginx/nginx.conf:2
nginx: [emerg] getpwnam("nginx") failed (2: No such file or directory) in /etc/nginx/nginx.conf:2
[/bash]
It turned out that the shared libraries for the name switch service reading /etc/passwd and /etc/group are loaded at runtime and not referenced in the shared libraries. By adding these shared libraries ( (/lib//libnss) to the container, NGiNX worked!

strip-docker-image example

So now, the strip-docker-image utility is here for you to use!
[bash]
strip-docker-image -i image-name
-t target-image-name
[-p package]
[-f file]
[-x expose-port]
[-v]
[/bash]
The options are explained below:
[bash]
-i image-name to strip
-t target-image-name the image name of the stripped image
-p package package to include from image, multiple -p allowed.
-f file file to include from image, multiple -f allowed.
-x port to expose.
-v verbose.
[/bash]
The following example creates a new nginx image, named stripped-nginx based on the official Docker image:
[bash]
strip-docker-image -i nginx -t stripped-nginx \
-x 80 \
-p nginx \
-f /etc/passwd \
-f /etc/group \
-f ‘/lib//libnss‘ \
-f /bin/ls \
-f /bin/cat \
-f /bin/sh \
-f /bin/mkdir \
-f /bin/ps \
-f /var/run \
-f /var/log/nginx \
-f /var/cache/nginx
[/bash]
Aside from the nginx package, we add the files /etc/passwd, /etc/group and /lib//libnss shared libraries. The directories /var/run, /var/log/nginx and /var/cache/nginx are required for NGiNX to operate. In addition, we added /bin/sh and a few handy utilities, just to be able to snoop around a little bit.
The stripped image has now shrunk to an incredible 5.4% of the original 132.8 Mb to just 7.3Mb and is still fully operational!
[bash]
docker images | grep nginx

stripped-nginx latest d61912afaf16 21 seconds ago 7.297 MB
nginx 1.9.2 319d2015d149 12 days ago 132.8 MB
[/bash]
And it works!
[bash]
ID=$(docker run -P -d –entrypoint /usr/sbin/nginx stripped-nginx -g "daemon off;")
docker run –link $ID:stripped cargonauts/toolbox-networking curl -s -D – https://stripped

HTTP/1.1 200 OK
[/bash]
For HAProxy, checkout the examples directory.

Conclusion

It is possible to use the official images that are maintained and distributed by Docker and strip them down to their bare essentials, ready for use! It speeds up load times and reduces the attack surface of that specific container.
Checkout the github repository for the script and the manual page.
Please send me your examples of incredibly shrunk Docker images!

Mark van Holsteijn
Mark van Holsteijn is a senior software systems architect at Xebia Cloud-native solutions. He is passionate about removing waste in the software delivery process and keeping things clear and simple.
Questions?

Get in touch with us to learn more about the subject and related solutions

Explore related posts