Setup Docker on Proxmox VE Using ZFS Storage

4
Proxmox Docker Portainer Title
Proxmox Docker Portainer Title

At STH, we have been using Proxmox VE as an open source hyper-converged solution for years. Underlying the project is a standard Debian installation with KVM virtualization and LXC containers. About a year ago we provided a guide to creating the Ultimate Virtualization and Container Setup (KVM, LXC, Docker) with Management GUIs. This setup allowed one to use bare metal Docker containers on Proxmox VE for maximum performance. It also allows one to utilize Portainer.io as a graphical management interface. Due to some software ecosystem changes, there is a new Docker and ZFS interaction that can cause the setup to fail upon reboot if the root pool is set as a ZFS pool. Today we have a method to fix this and get everything working as normal.

Setup Docker on Proxmox VE Using ZFS Storage

If you install Docker CE as normal, and Proxmox VE was setup with the installation on a ZFS rpool, then the system will fail to boot if you make a container using default settings. After installing Docker, we will see that it automatically selects the ZFS storage driver:

# docker info | grep Storage
Storage Driver: zfs

The issue is that this will interfere with the Debian Linux ZFS boot. Another concern is that the if you are installing on a boot device such as a SATA DOM you may want to change the Docker container storage directory to a zfs pool using the same procedure.

What we need to do now is stop Docker and change the configuration to mount to a different location. We are using “rpool/docker” in this example, however, you will likely want to move this to a different ZFS pool. If you have Proxmox VE installed on a single ZFS pool in the system, then you can use our location as a template.

# service docker stop
# rm -rf /var/lib/docker
# zfs create -o mountpoint=/var/lib/docker rpool/docker
# mkdir /etc/systemd/system/docker.service.d

The next step is that we want to create a storage-driver.conf file:

# nano /etc/systemd/system/docker.service.d/storage-driver.conf

We then want to define storage driver usage in the service:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --storage-driver=zfs -H fd://

Once this is complete, we can restart daemons and Docker:

# systemctl daemon-reload
# service docker start

There may be more elegant ways, and there are certainly recommended changes such as moving the Docker storage directory to another ZFS pool, but this configuration will let you at least reboot the system.

The Setup Docker on Proxmox VE Using ZFS Storage Video

If you want to see the above in action, we have a video you can find here:

That should show you a bit more of the process, but the basics are all in the text here.

Final Words

Although we do not recommend this in production, if you just want a simple development system, this will work. We recommend that if you follow our original guide and use a ZFS root, that you apply this fix. Even if you are not using a ZFS rpool, we strongly suggest using this guide and changing the storage location to something other than the rpool for your Docker container storage.

SHARE
Previous articleNetSpectre Slow and Remote Vulnerability
Next articleSamsung V-NAND 5th Generation In Mass Production
Patrick has been running STH since 2009 and covers a wide variety of SME, SMB, and SOHO IT topics. Patrick is a consultant in the technology industry and has worked with numerous large hardware and storage vendors in the Silicon Valley. The goal of STH is simply to help users find some information about server, storage and networking, building blocks. If you have any helpful information please feel free to post on the forums.

4 COMMENTS

  1. The proxmox guys are too stubborn. They should also support the docker and all this mess would be over. It’s unfortunate that to setup a few docker containers on proxmox I have to crate a kvm machine and then setup the docker inside that if I don’t want a complete mess.

  2. Nice withthe sharing of the knowledge. Would it not be easier to have a LXC dedicated to containing docker apps? Would this offer pretty much the same performance?

    But still, shows yet again the power of using a solution that is, at the end of the day, just a Debian platform.

  3. I had to remove -H fd:// to make it work for me. It might have something to do with the latest version of docker.

LEAVE A REPLY

Please enter your comment!
Please enter your name here