Molecule-Kubevirt Driver

Running Ansible unit tests in Kubernetes is painful. In this note, I detail existing solutions, expose my alternative proposal and explain how to use it.

Ansible is a well-known provisioning tool

When you want to test your ansible code, you need to ensure automation for creating anything that could be as close as possible to what a server is.

This means some systems where you can ssh, use sudo commands, have access to systemd daemons, …

Molecule is Ansible test framework

Many so-called drivers do exist for molecule, including the creation of real Virtual Machines on cloud-providers or on virtualization tools.

Molecule and Kubernetes

You can use Docker or Podman drivers:

  • They will start some fake servers running inside containers but : you’ll have to run them with high privileged and specific SYS_CAP_ADMIN capabilities and other horrible things for security.
  • Also running in containers means you won’t have access to user privileges escalations : you cannot sudo.

Else you can use non-container solutions like Vagrant or libvirt. This means installing using a tool from outside the cluster. Not good what we’re looking for.

Starting VMs in a Kubernetes cluster

It’s a great tool. I use it daily to test my own Ansible code.

But you’ve just said you can’t test Ansible code in Kubernetes ?!?

But I can’t use it for unit testing roles I code (pieces of the playbooks).

Simply because there is no molecule (unit test tool) driver for Kubevirt (good VMs in Kubernetes).

Please welcome Molecule-Kubevirt

This brand new driver makes use of Kubevirt in order to:

  1. create VM according to unit test platforms definition
  2. ensure molecule gets access to the VM : create a ssh key, inject its public part in cloud-init config and create a Kubernetes Service for ssh
  3. And… well.. that’s all 😊

Project repo is here but if you want to give a try, just read next section.

Showtime

Sample usage : migrating from Docker to Kubevirt

Let’s git clone this repo, install requirements, switch its molecule tests from Docker to new molecule-kubevirt driver and test.

Install Kubevirt

Change few lines of code

Also change converge playbook to execute tests as root user via become directive:

Few notes about those changes :

  • You can see I use a self-built container, for convenience, containing the Virtual Disk of the Virtual Machine. Documentation explains how you can build your own,
  • Switching to Kubevirt implies we now run tests on real Virtual Machines. The nginx role will install some system packages. On real VMs this can only be done by root user (or sudoer),
  • Pro-tip: don’t be confused on the Virtual Disk. Tests will not run inside this container. Kubevirt will start a Pod with Qemu virtualization tool, Qemu will load the Virtual Disk from the container hosting the disk file. But this container hosting the disk file will run no process.

Build an image to run tests inside the cluster

Create a Dockerfile:

Build the image and push it to a container registry (change with your own repository) :

# build
docker build . -t molecule_kubevirt_runner:latest
# push to any registry
docker tag molecule_kubevirt_runner:latest my-registry/molecule_kubevirt_runner:latest

Add authorization to the Pod

Create as molecule-kubevirt-sa.yaml file containing these lines :

Then apply the file:

kubectl apply -f molecule-kubevirt-sa.yaml

Start the test Pod

Start the pod and look at its logs:

kubectl apply -f molecule-pod.yaml
# wait for Pod to start
kubectl logs -l app=molecule -f

We can see the VM being created with a valid ssh access.

And a bit later, molecule runs the tests inside the VM and tests the role, just as usual.

Voilà. We unit tested the role with a pure Kubernetes solution, without any compromise in security.

Pitfall

Not only beautiful to look at, console log can give great help in case of trouble in your CI.

Take a look at my Git Actions workflow for more information.

Is this really working ?

Only CentOS 7 was tested right now but this should work with any well built Kubevirt container image.

Next Steps ?

  • make use of Kubevirt’s Containerized Data Importer for another option than self-building container disk
  • some kind of offload mode (launch molecule in your local env and starts VMs on remote Kubernetes cluster)
  • any Pull Request you might submit :)

End of the story

Please also give claps👏, github stars ⭐ or issues / Pull Requests.

Thanks

Thanks to Blur rock band, especially for their album “Parklife”. Coding with cool songs gives me a sense of enormous well-being.

Great songs for great coding.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store