Blog

How to create the smallest possible docker container of any image

30 Jun, 2015

Once you start to do some serious work with Docker, you soon find that downloading images from the registry is a real bottleneck in starting applications. In this blog post we show you how you can reduce the size of any docker image to just a few percent of the original. So is your image too fat, try stripping your Docker image! The strip-docker-image utility demonstrated in this blog makes your containers faster and safer at the same time!

We are working quite intensively on our High Available Docker Container Platform  using CoreOS and Consul which consists of a number of containers (NGiNX, HAProxy, the Registrator and Consul). These containers run on each of the nodes in our CoreOS cluster and when the cluster boots, more than 600Mb is downloaded by the 3 nodes in the cluster. This is quite time consuming.
[bash]
cargonauts/consul-http-router latest 7b9a6e858751 7 days ago 153 MB
cargonauts/progrium-consul latest 32253bc8752d 7 weeks ago 60.75 MB
progrium/registrator latest 6084f839101b 4 months ago 13.75 MB
[/bash]
The size of the images is not only detrimental to the boot time of our platform, it also increases the attack surface of the container.  With 153Mb of utilities in the  NGiNX based consul-http-router, there is a lot of stuff in the container that you can use once you get inside. As we were thinking of running this router in a DMZ, we wanted to minimise the amount of tools lying around for a potential hacker.
From our colleague Adriaan de Jonge we already learned how to create the smallest possible Docker container  for a Go program. Could we repeat this by just extracting the NGiNX executable from the official distribution and copying it onto a scratch image?  And it turns out we can!

finding the necessary files

Using the utility dpkg we can list all the files that are installed by NGiNX
[bash]
docker run nginx dpkg -L nginx

/.
/usr
/usr/sbin
/usr/sbin/nginx
/usr/share
/usr/share/doc
/usr/share/doc/nginx

/etc/init.d/nginx
[/bash]

locating dependent shared libraries

So we have the list of files in the package, but we do not have the shared libraries that are referenced by the executable. Fortunately, these can be retrieved using the ldd utility.
[bash]
docker run nginx ldd /usr/sbin/nginx

linux-vdso.so.1 (0x00007fff561d6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fd8f17cf000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fd8f1598000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fd8f1329000)
libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fd8f10c9000)
libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fd8f0cce000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fd8f0ab2000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd8f0709000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd8f19f0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd8f0505000)
[/bash]

Following and including symbolic links

Now we have the executable and the referenced shared libraries, it turns out that ldd normally names the symbolic link and not the actual file name of the shared library.
[bash]
docker run nginx ls -l /lib/x86_64-linux-gnu/libcrypt.so.1

lrwxrwxrwx 1 root root 16 Apr 15 00:01 /lib/x86_64-linux-gnu/libcrypt.so.1 -> libcrypt-2.19.so
[/bash]
By resolving the symbolic links and including both the link and the file, we are ready to export the bare essentials from the container!

getpwnam does not work

But after copying all essentials files to a scratch image, NGiNX did not start.  It appeared that NGiNX tries to resolve the user id ‘nginx’ and fails to do so.
[bash]
docker run -P –entrypoint /usr/sbin/nginx stripped-nginx -g "daemon off;"

2015/06/29 21:29:08 [emerg] 1#1: getpwnam("nginx") failed (2: No such file or directory) in /etc/nginx/nginx.conf:2
nginx: [emerg] getpwnam("nginx") failed (2: No such file or directory) in /etc/nginx/nginx.conf:2
[/bash]
It turned out that the shared libraries for the name switch service reading /etc/passwd and /etc/group are loaded at runtime and not referenced in the shared libraries. By adding these shared libraries ( (/lib/*/libnss*) to the container, NGiNX worked!

strip-docker-image example

So now, the strip-docker-image utility is here for you to use!
[bash]
strip-docker-image -i image-name
-t target-image-name
[-p package]
[-f file]
[-x expose-port]
[-v]
[/bash]
The options are explained below:
[bash]
-i image-name to strip
-t target-image-name the image name of the stripped image
-p package package to include from image, multiple -p allowed.
-f file file to include from image, multiple -f allowed.
-x port to expose.
-v verbose.
[/bash]
The following example creates a new nginx image, named stripped-nginx based on the official Docker image:
[bash]
strip-docker-image -i nginx -t stripped-nginx \
-x 80 \
-p nginx \
-f /etc/passwd \
-f /etc/group \
-f ‘/lib/*/libnss*’ \
-f /bin/ls \
-f /bin/cat \
-f /bin/sh \
-f /bin/mkdir \
-f /bin/ps \
-f /var/run \
-f /var/log/nginx \
-f /var/cache/nginx
[/bash]
Aside from the nginx package, we add the files /etc/passwd, /etc/group and /lib/*/libnss* shared libraries. The directories /var/run, /var/log/nginx and /var/cache/nginx are required for NGiNX to operate. In addition, we added /bin/sh and a few handy utilities, just to be able to snoop around a little bit.
The stripped image has now shrunk to an incredible 5.4% of the original 132.8 Mb to just 7.3Mb and is still fully operational!
[bash]
docker images | grep nginx

stripped-nginx latest d61912afaf16 21 seconds ago 7.297 MB
nginx 1.9.2 319d2015d149 12 days ago 132.8 MB
[/bash]
And it works!
[bash]
ID=$(docker run -P -d –entrypoint /usr/sbin/nginx stripped-nginx -g "daemon off;")
docker run –link $ID:stripped cargonauts/toolbox-networking curl -s -D – http://stripped

HTTP/1.1 200 OK
[/bash]
For HAProxy, checkout the examples directory.

Conclusion

It is possible to use the official images that are maintained and distributed by Docker and strip them down to their bare essentials, ready for use! It speeds up load times and reduces the attack surface of that specific container.
Checkout the github repository for the script and the manual page.
Please send me your examples of incredibly shrunk Docker images!

guest
14 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Michiel Sens
Michiel Sens
6 years ago

Very nice and very useful!! Tx!! 🙂

Brian
Brian
6 years ago

Great utility, thanks! I’ve been using Alpine Linux to try and achieve the same results but it seem your utility does the trick.

Vlad
Vlad
6 years ago

Well this is impressive! I’ll have to give it a try soon. Thank you for sharing.

Manuel Weiss
Manuel Weiss
6 years ago

Great article Mark! Thanks for the write-up. As you asked for other examples:
One of our engeineers, Nick Gauthier, explains how we shrink our Docker images in this article. In this case he created a 10MB Docker image for a Go Web Server:
http://blog.codeship.com/building-minimal-docker-containers-for-go-applications/
You might enjoy the read 🙂

Per Wiklander
Per Wiklander
6 years ago

It would be awesome to get a minimal JVM container. Sure, it wouldn’t be very minimal given the normal size of Java stuff, but it should at least be possible to get rid of the cruft from the base images. I don’t really know where to start digging though.

Per Wiklander
Per Wiklander
6 years ago

And a comment with the correct email address, so that I get updates…

Olivier Fayau
Olivier Fayau
6 years ago
Reply to  Per Wiklander

I’ve made very small JRE images (no JDK), based on Busybox and available on Docker hub.
It goes from 21MB to 115MB : only 8MB were added from JVM package.
Most use embedded JRE and/or Java 8 compact profiles, so it would not fit any usage (so some package may be truncated).
But 2 last are full Java SE 8 compliant : OpenJDK (115MB) and Oracle SE Embedded (88MB).
See list and size here : https://registry.hub.docker.com/u/ofayau/busybox-jvm/

Michael Mercier
Michael Mercier
6 years ago

There a similar tool that use the AUFS diff capabilities to automatically grab the necessary libraries from the last AUFS layer executables:
https://github.com/djosephsen/skinnywhale
Quite experimental but definitly a good idea too!

sss
sss
6 years ago

Good tool, but how to use it ?
I downloaded /bin/strip-docker-image from your github, when I ran it , it shows:
[root@vmcentos ~]# ./strip-docker-image
./strip-docker-image: line 5: syntax error near unexpected token `newline’
./strip-docker-image: line 5: `’
I dont know why ? Could u tell me please?

Sss
Sss
6 years ago
Reply to  sss

Oz. I solved it.
That’s a good tool. Thanks.

Kyle Quest
Kyle Quest
6 years ago

DockerSlim [1] is conceptually similar where it tries to discover what your application really needs. It has other capabilities too… It also has a set of sample apps (python, node, ruby, Java) showing how much you can shrink your images.
[1] http://dockersl.im

Alex
Alex
1 year ago

You didn’t include the libraries from the ldd output in your strip-docker-image command line. How did the libraries end up in the stripped-down nginx image nonetheless?

Explore related posts