Bootstrapping rootless containers with Ansible
Some deployments can be easily automated with SSH and shell scripts, but once a deployment reaches a certain (not-so-big) size, one may start searching for easier to manage alternatives. In this post, I will summarize what I ended up using so far and describe an automated way to bring up a machine with rootless Podman containers using the official nginx image as an example.
Podman
First, let’s cover the container configuration, and then move to the configuration of the managed node. For simplicity, this part assumes that you run a Linux system and have Podman installed.
Why Podman?
I prefer Podman for a couple of reasons:
- Daemonless and rootless approach that reduces the attack surface.
- Systemd integration that simplifies the whole setup. You can basically run your containers as (almost) regular systemd services.
I can also recommend the book Podman in Action that covers Podman in more detail.
Quadlets
Nowadays systemd supports quadlets that make running containers somewhat easier compared to the more manual podman systemd generate approach. The idea is that you can describe Podman flags in the [Container] section of the quadlet, and systemd will generate a service unit file for it. You can also enable automatic updates using AutoUpdate=registry and systemd will be updating the container image for you. Nice!
We can create ~/.config/containers/systemd/nginx.container with the following contents:
[Container]
Image=docker.io/library/nginx:latest
AutoUpdate=registry
PublishPort=8080:80
Volume=%h/nginx/content:/var/www/html:Z,ro
Volume=%h/nginx/nginx.conf:/etc/nginx/nginx.conf:Z,ro
[Service]
Restart=always
[Install]
WantedBy=default.target
This container needs a config. Let’s create the simplest ~/nginx/nginx.conf:
worker_processes auto;
events {
worker_connections 64;
}
http {
server {
server_name localhost;
listen 80;
listen [::]:80;
location / {
root /var/www/html;
index index.html;
}
}
}
We also need some content to serve, so let’s put the following index.html in ~/nginx/content/:
<h1>Hello from Podman!</h1>
Finally, before our user can run the container, we need to allocate user and group id ranges. You can read more in the rootless Podman tutorial, but to keep this post short you can just run this command to allocate the 100000-165535 id ranges for your user in /etc/subuid and /etc/subgid:
usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $USER
…and that’s almost it! To test the container locally, run these commands:
systemctl --user daemon-reload
systemctl --user start nginx
The first command generates a service file based on the quadlet, and the second one starts the new service. If the service for some reason fails to start, you can use journalctl -xe to see the most recent failures in the journal.
You should now see the index page if you open http://localhost:8080.
Ansible
Now that we have a working local setup, we can replicate it on a managed node. This section assumes that the host runs Debian 13, but except for the apt module the configuration should be mostly the same for any other distribution.
Why Ansible?
Ansible is by no means a new tool, but it’s still maintained and I find it intuitive to use for simple use cases like this. My first attempt at automating setting up a new VM involved a few shell scripts, and when I replaced all of them with a single Ansible playbook, the setup has become much simpler.
Another seemingly good option is Butane. I like the idea of running a minimal Fedora CoreOS, but unlike Debian not every VPS provider has a default image for it, and the instructions to install it manually look a bit tedious. Maybe I’ll try it some other time and write a post about how to achieve the same with CoreOS and Butane :)
Playbook
Ansible allows you to describe the configuration as a YAML playbook. Playbooks use modules to describe the end state, e.g. a copied file or installed package.
Playbooks can be run using ansible-playbook <playbook.yaml> -u <user> -i <inventory>, where user is the user to use and inventory can either be an inventory file or a comma-separated list of hosts, e.g. 1.2.3.4, (note a comma in the end!).
Here is the complete example. The rest of the post explains it section by section. I’m not going into too much detail though, refer to the official documentation to clarify specific module options.
Let’s start with defining a playbook that will install the required Debian packages:
- name: Install nginx
hosts: all
become: true
vars:
nginx_user: nginx
nginx_home: /home/nginx
tasks:
- name: Install packages
ansible.builtin.apt:
pkg:
- podman
- systemd-container
state: present
update_cache: true
cache_valid_time: 3600
tags: packages
Now, let’s create a new user. Here we also allocate the id ranges and enable lingering to make sure the user services start after a reboot:
- name: Create user
ansible.builtin.user:
name: "{{ nginx_user }}"
state: present
create_home: true
home: "{{ nginx_home }}"
shell: /sbin/nologin
system: true
tags: user
- name: Allocate user ids
ansible.builtin.lineinfile:
path: /etc/subuid
line: "{{ nginx_user }}:100000:65536"
create: true
tags: user
- name: Allocate group ids
ansible.builtin.lineinfile:
path: /etc/subgid
line: "{{ nginx_user }}:100000:65536"
create: true
tags: user
- name: Enable lingering
ansible.builtin.command:
cmd: "loginctl enable-linger {{ nginx_user }}"
changed_when: false
tags: user
Once we have the user configured, we can copy all the configuration files from the control node to the remote user’s home directory:
- name: Create directories
ansible.builtin.file:
path: "{{ item }}"
state: directory
owner: "{{ nginx_user }}"
group: "{{ nginx_user }}"
mode: '0755'
loop:
- "{{ nginx_home }}/.config/containers/systemd"
- "{{ nginx_home }}/nginx/content"
tags: config
- name: Copy files
ansible.builtin.copy:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: "{{ nginx_user }}"
group: "{{ nginx_user }}"
mode: '0644'
loop:
- src: ~/nginx/nginx.conf
dest: "{{ nginx_home }}/nginx/nginx.conf"
- src: ~/nginx/content/index.html
dest: "{{ nginx_home }}/nginx/content/index.html"
- src: ~/.config/containers/systemd/nginx.container
dest: "{{ nginx_home }}/.config/containers/systemd/nginx.container"
tags: config
Finally, we can reload daemons and start the service:
- name: Reload daemons
ansible.builtin.command:
cmd: "systemctl --machine={{ nginx_user }}@ --user daemon-reload"
changed_when: false
tags: service
- name: Start service
ansible.builtin.command:
cmd: "systemctl --machine={{ nginx_user }}@ --user start nginx"
changed_when: false
tags: service
Note that Ansible has a systemd module. However, I found it very cumbersome to use for the user scope. It requires setting the XDG_RUNTIME_DIR environment variable that contains a user id. It’s possible to query it (e.g. using ansible.builtin.getent), but I think it’s way easier just to call systemctl --machine=nginx@.
If your firewall does not block port 8080, you should be able to see the page served by your server.
Summary
That’s finally it! You should now have a working rootless setup that supports automatic updates and can be easily re-created on a new Debian machine.