Installing Piwik with Docker and Ansible

It’s my second article on Ansible. Ansible is perfect to manage Docker containers. I like to learn by applying tools to real usage, so here I will show how I deployed Piwik(open-source equivalent of Google Analytics) with Docker and Ansible.

Ansible installation

I talked about Ansible installation in install vector with Ansible. So here I will just do the short description. You only need to install python package manager pip and install Ansible with it :

sudo aptitude install -y python-pip
sudo pip install --upgrade pip
pip install ansible

Ansible roles

In the previous article, I used a unique playbook file :

- hosts: testlab
  sudo: yes
  tasks:
    - name: Install pcp package
      apt: name=pcp update_cache=yes state=present

    - name: Install pcp-webapi package
      apt: name=pcp-webapi state=present

It’s fine when you have few tasks to perform and when they are related but when you start to have more complex tasks it’s becoming messy quickly.

Ansible has a good documentation on how to organize your files : http://docs.ansible.com/ansible/playbooks_roles.html. It’s a must read.

I try to completely configure my web server with Ansible. My playbook is available on github. It’s a work in progress. All comments are welcome :)

I didn’t use Ansible Galaxy. It’s really great to have so many playbooks and roles available but I want to learn Ansible and I think it’s better for me to do everything from scratch as an exercise.

environments

I have two hosts files. One file named dev for development :

[dev]
vm1

And for production :

[prod]
djouxtech

Ansible use ssh. So you can specify any hostname or use an alias you defined in your ~/.ssh/config. For example, my vm vm1 will use this section :

Host vm1
  HostName 192.168.121.27
  User vagrant
  Port 22
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /home/adejoux/testlab/.vagrant/machines/vm1/libvirt/private_key
  IdentitiesOnly yes
  LogLevel FATAL

I used again Vagrant to provision my test environment. It’s so great :). More details about it in vagrant with libvirt.

Ansible variables

You can specify variables by group of hosts by creating files with the same name in the folder group_vars.

I tried to keep most variables common for both environments. All hosts are included in the all group.

So I put this common variables in group_vars/all:

mysql_data_dir: "/docker/data/piwikdb"
pg_data_dir: "/docker/data/postgres"
piwik_port: "127.0.0.1:50000"
mariadb_image: "mariadb:latest"
piwik_image: "adejoux/piwik:latest"
pg_image: "postgres:latest"
nginx_ssl_dir: "/home/adejoux/ssl"

I assigned my systems to two different groups dev and prod.

It allows me to have dedicated variables for each environment.

I put my variables for dev in group_vars/dev:

piwik_url: "stats.vm1.com"
piwik_cert: "/etc/nginx/ssl/nginx.crt"
piwik_key:  "/etc/nginx/ssl/nginx.key"

And my variables for prod in group_vars/prod:

piwik_url: "stats.krystalia.net"
piwik_cert: "/etc/nginx/ssl/krystalia.cer"
piwik_key:  "/etc/nginx/ssl/krystalia.key"

It’s only the ssl keys I use and the url for piwik statistics.

Installing Docker with Ansible

My target system will be an Ubuntu 14.04. Docker is available in the official repository but it’s a pretty old version :

root@vm1:~# docker version
Client version: 1.0.1
Client API version: 1.12
Go version (client): go1.2.1
Git commit (client): 990021a
Server version: 1.0.1
Server API version: 1.12
Go version (server): go1.2.1
Git commit (server): 990021a

It’s better to install the latest version from Docker official repository :

I put the tasks in the role docker by creating roles/docker/tasks/main.yml :

- name: Add key for Docker repository
  apt_key: keyserver=keyserver.ubuntu.com id=36A1D7869245C8950F966E92D8576A8BA88D21E9 state=present
- name: Add docker repository
  apt_repository: repo='deb http://get.docker.io/ubuntu docker main' state=present
- name: Install Docker
  apt: name=lxc-docker update_cache=yes state=present
- name: Install Docker python library
  pip: name=docker-py

A really strong point of Ansible is clarity. Just by reading the main.yml file I know what it does. It’s self-documentation.

To use this role I create a playbook named setup.yml :

- hosts: all
  sudo: yes
  roles:
    - common
    - docker

This playbook will only use two roles common and docker. The role common only install pip if not installed. I run the playbook on my dev environment :

# ansible-playbook setup.yml -i dev

PLAY [all] ********************************************************************

GATHERING FACTS ***************************************************************
ok: [vm1]

TASK: [common | Install Python Package Manager] *******************************
changed: [vm1]

TASK: [docker | Add key for Docker repository] ********************************
changed: [vm1]

TASK: [docker | Add docker repository] ****************************************
changed: [vm1]

TASK: [docker | Install Docker] ***********************************************
changed: [vm1]

TASK: [docker | Install Docker python library] ********************************
changed: [vm1]

PLAY RECAP ********************************************************************
vm1                        : ok=6    changed=5    unreachable=0    failed=0

For each task you have also his role displayed. Great :)

Ansible Vault

In the next steps, I will use variables which will store passwords. I still want to be able to put all my files in github. So I will use ansible-vault to create a secret file named secret :

$ ansible-vault create secret
Vault password:
Confirm Vault password:

I will store the following fields :

ansible_sudo_pass: foobar
mysql_root_password: foobar2

The file is encrypted. I still require a password for sudo on my production system. I store the password here.

Deploying the Mariadb Container

I want to have a dedicated database for Piwik.

I create the role mariadb and put tasks in roles/mariadb/tasks/main.yml :

- name: create mariadb data directory
  file: path="{{ mysql_data_dir }}" state=directory

- name: Install MariaDB container
  docker:
    name: piwikdb
    docker_api_version: "1.18"
    image: "{{mariadb_image}}"
    state: started
    env:
      MYSQL_ROOT_PASSWORD: "{{ mysql_root_password }}"
    volumes:
      - "{{ mysql_data_dir }}:/var/lib/mysql"

I plan to store my data outside of the container. So the first task is to create the directory on the host.

The second task is more interesting because I am using the docker module.

The parameter name allow me to choose the container’s name.

The parameter image will pull the image I defined in mariadb_image variable.

The parameter env pass environment variables to the docker container. Here it’s the mysql root password. It will only be used to setup MariaDB if no database exist.

The parameter volume map a host directory into the Docker container. It will map the value of mysql_data_dir to /var/lib/mysql inside the container.

If you are using SElinux you need to enable permissions on the directory at the host level else you will have a “permission denied error” :

chcon -Rt svirt_sandbox_file_t /data/piwikdb

I use docker_api_version because at the moment where I am writing the article the docker python library doesn’t support the client library(1.19).

I will not make any network port available at host level for this container. I plan to have only other containers accessing it.

Deploying the Piwik container

I create another role for piwik installation in roles/piwik/tasks/main.yml :

- name: Install Piwik
  docker:
    name: piwik
    docker_api_version: "1.18"
    image: "{{piwik_image}}"
    state: started
    links:
      - "piwikdb:db"
    ports:
      - "{{ piwik_port }}:80"

- name: deploy PIWIK configuration
  template: src=piwik.conf.j2 dest=/etc/nginx/conf.d/piwik.conf owner=root group=root mode=0644
  notify:
    - restart nginx

The first task is using the docker module again.

The parameter links allows network communications between piwikdb and this container. piwikdb container will be know as db by this container. The /etc/hosts file will also be populated accordingly.

The parameterports is used to make available the http port on localhost. Here the piwik_port variable will be 127.0.0.1:50000. I use 127.0.0.1 to have the container listening only on the localhost interface.

The second task use the [template module] to generate the NGINX http configuration file.

The template is in roles/piwik/templates/piwik.conf. It’s a standard NGINX configuration file where you can use Ansible variables. For example, here :

server {
  listen 80;
  server_name {{ piwik_url }};

This task has a handler :

  notify:
    - restart nginx

The handler will perform an action when notified. I defined it in roles/common/handlers/main.yml :

- name: restart nginx
  action: service name=nginx state=restarted

This handler will restart nginx if the template is deployed successfully.

piwik playbook

The playbook is pretty simple :

- hosts: all
  vars_files:
    - secret
  sudo: yes
  roles:
    - mariadb
    - piwik

The new parameter is vars_files. It specifies to use the secret file generated by ansible-vault.

When running ansible-playbook, we need to add the –ask-vault-pass option. It would be possible to use –vault-file instead and it will read the master password from a file. But I prefer to be asked to provide the password. I feel safer ;).

Here the result :

╰─$ ansible-playbook piwik.yml -i dev --ask-vault-pass
Vault password:

PLAY [all] ********************************************************************

GATHERING FACTS ***************************************************************
ok: [vm1]

TASK: [mariadb | create mariadb data directory] *******************************
changed: [vm1]

TASK: [mariadb | Install MariaDB container] ***********************************
changed: [vm1]

TASK: [common | Install Python Package Manager] *******************************
ok: [vm1]

TASK: [piwik | Install Piwik] *************************************************
changed: [vm1]

TASK: [piwik | deploy PIWIK configuration] ************************************
changed: [vm1]

PLAY RECAP ********************************************************************
vm1                        : ok=2    changed=4    unreachable=0    failed=0

NGINX setup

I am only adding a configuration file for piwik here. So here all requests will be proxied and redirected to my piwik container. An improvement would be to put nginx itself in a container. I am thinking about it currently. I want to finish my container logging and monitoring implementation first.

If you want to see how to setup a basic NGINX with Ansible, you can look at my Ansible repository.

It’s not the one I am using on my web server. It’s a work in progress allowing me to setup nginx quickly in dev :) For information, I didn’t include my ssl keys in the repo. I have a folder in my home directory ssl with these files :

# ls /home/adejoux/ssl
dhparam.pem  nginx.crt  nginx.key

This files are copied by Ansible.

NGINX configuration

The full configuration file is visible here.

In this configuration, I wanted to have all http connection redirected to https :

server {
  listen 80;
  server_name {{ piwik_url }};
  server_tokens off;

  location / {
    return         301 https://$server_name$request_uri;
  }
}

In the https section, I will set an auth_basic authentication to protect my piwik installation :

  location / {
    proxy_pass         http://127.0.0.1:50000/;
    auth_basic "Restricted";
    auth_basic_user_file htpasswd;
  }

But we need to allow request to piwik.php and piwik.js. They need to be accessible for everyone if we want Piwik to work. So I will disable authentication for them :

  location ~ piwik\.(php|js) {
    proxy_pass http://127.0.0.1:50000;
    auth_basic "off";
  }

PS: you need to generate a htpasswd file in /etc/nginx if you keep the authentication part in my template.

piwik

If the playbook finish properly, you will have this screen when connecting on your url (here https://stats.vm1.com) :

When prompted to provide database hostname use db :

If you use my docker image for piwik, you will have Geo location modules pre installed :

the end

Using Docker with Ansible is pretty great. I can manage my containers in my workflow naturally. And it’s very fun to do :)