I have always been a fan and user of open-source software and enjoy running Linux on my laptop, so I was pretty stoked to learn about and be able to attend All Systems Go!, a conference focused on foundational user-space Linux technologies, this past weekend. It was unlike a lot of conferences that I have attended in the past and a lot of the topics were over my head, but I met a lot of nice people and learned about a lot of things that I could dive deeper into afterwards and I really enjoyed the small community atmosphere. The themes of the talks seemed to be containers, systemd (a system and service manager for Linux), and BPF (which is supposed to replace iptables). I also sensed a lot of excitement over Flatpak (which provides a sandbox environment in which users can run applications in isolation from the rest of the system).

I will highlight one of the talks that I found interesting:

Dan Walsh from RedHat gave a presentation about replacing Docker with Podman. He is on the team that develops container engines and worked on taking apart what Docker did into a series of sub-components. Why do this? Docker has popularized the concept of containers to the point where people often use the terms interchangeably (like Kleenex!), but containers are actually processes created from combining a bunch of Linux kernel features together and Docker is just one way of managing containers and he believes that its popularity can also be a hindrance to innovation.

Dan ran through six items that are necessary in order to experiment with container engines:

1. A standard definition of what makes up a container image 
(the [Open Container Initiative](https://www.opencontainers.org/) has standardized what it means to be a container image)
2. A mechanism to pull images from a container registry to the host
3. The ability to explode images onto [CopyOnWrite](https://drawings.jvns.ca/copyonwrite/) file systems on disk
4. A standard mechanism for running a container
5. A standard way to setup networking for containers (Container Networking Interface)
6. A tool to monitor containers

Podman is a new tool that works with containers differently than Docker. It can run and manage pods (a process that has one or more containers on them) as well as regular containers and does it all without a daemon. Podman is written in Go and basically uses the same command line interface as Docker. You can just set an alias ( alias docker = ‘podman’ ). It also supports the Dockerfile format and OverlayFS, a union-mount file system which facilitates the layered building blocks important to Docker images. It is part of the libpod effort, a library that allows other tools to manage pods/containers, and is its default CLI tool.

The Docker daemon runs as root by default because that is necessary in order to create new cgroups and many commands within the container expect to be able to alter the filesystem (i.e. when using apt, yum, etc. to install packages). Normally, only the root user can create another root user process and not run into problems with file permissions and ownership. Setting up networking also requires root access if you don’t want to use tricks and workarounds that can have performance impacts.

Podman does not run as root by default and allows one to use the Docker CLI without such privileges. It does this by taking advantage of the user namespace support in the Linux kernel. This is a feature that enables privilege isolation and user identification across different processes. If you give a container a completely new user namespace, it can have its own internal/virtual root user which is separate from the host system’s root user. Root processes could then be run in a container without having to have root on the system.

According to Dan, Podman is different because it does not run a “big fat daemon”. Running a daemon in Linux is not generally wrong, but may be viewed as overkill for when you just want to containerize some processes. The Podman model uses OCI runtimes to launch the container like Docker, but the processes are direct children of the Podman process (i.e. fork-exec model), with cgroups still in control of the container processes. There can also be proper integration with systemd by putting Podman into a unit file.

Podman is still in development and has a lot of items on its roadmap. For now, it does not support healthchecks, Docker volumes, docker-compose, among other things. Overall, Podman seems like a cool idea and it is good that people are thinking up new ways to innovate with containers with the mindset to start with high security first. It will be interesting to follow along with its progress.