Building an OS container

consistent with your VM builds

In a previous blog entry I described some of the controls that are needed if you want to use a container as a VM. Essentially, if you want to use it as a VM then you must treat it as a VM.

This means that all your containers should have the same baseline as your VM OS, the same configuration, the same security policies.

Fortunately we can take a VM and convert it into a container. Mostly.

In my lab setup (“lab” be a grandious word; it’s a single machine running too many KVM virtual machines :-)) I’ve created a process for building a new VM via kickstart. I can run virt_build test2 and it will create the LVM volumes, start an install of CentOS 7, patch it, apply my local configs (create my account, install SSH keys, etc etc). A few minutes later I can then ssh test2 and, just like that, I have a new VM.

Many enterprises will have a similar build process; whether it uses PXE to do the initial boot and [Cobbler]( to dynamically build the kickstart file, or does it the simplistic way I do doesn’t really matter. The result is an automated consistent build.

Side note: Some artifacts of the automated build process might need to be split into a “build” and “first boot” section; for example a server may register itself to Active Directory for authentication, or to Tivoli for monitoring… these are now “first boot” type activities and can’t be done at build time. But let’s ignore this complexity :-)

The important parts of my kickstart file are:

network --onboot yes --device eth0 --bootproto dhcp --ipv6 auto

rootpw  --iscrypted yeahyeahyeah
authconfig --enableshadow --passalgo=sha512
firewall --disabled
selinux --disabled

clearpart --all --initlabel
part /boot --fstype=ext4 --asprimary --size=500
part swap --asprimary --size=512
part / --fstype=ext4 --asprimary --grow --size=1

There’s a %packages and %post section (which creates my account) and so on. This should be perfectly familiar to anyone used to kickstart.

Now the result of my build process is an LVM disk image consisting of three partitions

  1. /boot
  2. swap
  3. /

This, in my case, is /dev/Raid10/vm.test2.

So now we need to take this image and convert it to a docker container.

So we need to make the image accessible to the host OS. Then we can mount it. Then send it to the docker server, where it can be converted to an image.

$ sudo kpartx -av /dev/Raid10/vm.test2
add map Raid10-vm.test2p1 (253:21): 0 1024000 linear /dev/Raid10/vm.test2 2048
add map Raid10-vm.test2p2 (253:22): 0 1048576 linear /dev/Raid10/vm.test2 1026048
add map Raid10-vm.test2p3 (253:23): 0 6313984 linear /dev/Raid10/vm.test2 2074624
$ sudo mkdir /tmpmount
$ sudo mount -r /dev/mapper/Raid10-vm.test2p3 /tmpmount
$ cd /tmpmount
$ sudo tar cf - . | ssh dockerserver docker import - c7test2

$ cd
$ sudo umount /tmpmount/
$ sudo kpartx -d /dev/Raid10/vm.test2

Now this may through up a couple of errors about sockets (in my case it complained about postfix sockets), but we can ignore them.

tar: ./var/lib/gssproxy/default.sock: socket ignored
tar: ./var/spool/postfix/private/virtual: socket ignored
tar: ./var/spool/postfix/private/bounce: socket ignored
tar: ./tmp/ssh-IB3AMGEQUc/agent.1873: socket ignored

If we now look on the docker server:

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
c7test2             latest              374dc7777cfc        3 minutes ago       1.185 GB

Notice how large this is, compared to the official docker centos image    latest              970633036444        3 weeks ago         196.7 MB

But we now have an OS container that we can run!

$ docker run --name container1 c7test2 /sbin/init &

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
0e640e27a91b        c7test2             "/sbin/init"        26 seconds ago      Up 22 seconds                           container1

And we can see the container is running a full OS tree

$ docker top container1 | head -4
UID                 PID                 PPID                C                   STIME               TTY                 TIME                CMD
root                30732               9504                0                   19:49               ?                   00:00:00            /sbin/init
root                30755               30732               0                   19:49               ?                   00:00:00            /usr/lib/systemd/systemd-journald
root                30793               30732               0                   19:49               ?                   00:00:00            /usr/sbin/rsyslogd -n

I haven’t done anything clever with the networking here, but

$ docker inspect container1 | grep IPAddress

tells me this container is

So we can now ssh into it!

$ ssh                            
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is 32:31:44:a0:9b:61:1b:ff:2f:db:76:ae:a9:a5:36:2b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.


My OS image is now a (mostly) working container with a funky hostname :-)

What doesn’t work? Well, I don’t seem to have a getty process running on the console. And commands like dmesg don’t work. And I haven’t joined it to my main network (using the default docker0 network). But for all intents and purposes, this is a working OS container.

(For clean-ness we should create a derived container based on this image that automtically runs init and lets the docker command return, but that’s a refinement).


Now I’ve walked through how I could convert my build process to creating a docker OS container. Naturally you’ll need to modify your own processes to work in your environment, and the more complex the environment the more tweaks necessary. But this is the concept you can follow; take a known good build from your standard build process and convert it to a container.

I’m not really a fan of the “container as a VM” model. I can understand that some people might want to do it and, as previously written, I think that the lines between a container and a VM will blur (at which time the build process will also become easier).

But if you’re going to do it, building a process that maintains consistency between your VM and container builds is a massive step forwards towards controlling the estate and, with automation, can simplify your compliance stance; you only have the one build process to worry about :-)