Using Librarian-Ansible to install Ansible roles from Gitlab

ansible_logo_black_square

I have some Ansible roles which I try to keep up2date and these are on Github and on my personal Gitlab instance. Sometimes this takes a little bit longer that I want to, but other projects needs some attention to.

For my own personal environment, I use Ansible too and this is in an seperate git repository of my Gitlab server (Repository: environment/ansible.git). There is one thing that buggers me: My Ansible roles differs from the one used in my personal Ansible setup. At moment of writing, the ‘dj-wasabi.zabbix-agent’ role is at tag 0.2.1, but I use ‘0.0.2’ in my own Ansible setup (Oh, really that old?? 🙂 ).

There should be an solution for this. But before we continue, the solution should met my goals:

  • All Ansible roles should have their own git repository in Gitlab,
  • All Ansible roles have their own Jenkins job, documentation and test cases,
  • I want to make use of tags or versions.

With this I can create specific tags/version of the role and we can run some tests via Jenkins like ‘testkitchen’. With ‘testkitchen’ we run the role on an vagrant/docker and see if everything runs fine. But for know, ‘testkitchen’ is out of scope for this.

I first looked at ‘ansible-galaxy’. It has the possibility for using an ‘requirements.yml’ file which holds all information. Like location and even an version, so we can specify the correct role. After some testing it only work when you have the repository at Github.com.
Also the repository should exists on the Galaxy itself. So for the Zabbix roles this could work, but I also have some roles created just for my own environment. These are specific and there is no need to upload them to the Galaxy or github, so the ‘ansible-galaxy’ will not work for me.

I found “librarian-ansible’. This could be something which might work for me, but didn’t found information on the web. Yes, I did found something that the Ansible creator Michael DeHaan isn’t an very big fan of this (https://groups.google.com/forum/#!msg/ansible-project/TawjChwaV08/3p6Zv24rMWgJ). So lets try it anyways, maybe it creates a fan out of me ;-).

Installation is very simple, we have to install 1 gem:

wdijkerman@curiosity [ ~/git/environment/ansible ] (14:03:56 - Sat Aug 15)
 (master) > sudo gem install librarian-ansible
Successfully installed librarian-ansible-1.0.6
1 gem installed

Now we have to create the Ansiblefile. The Ansiblefile is used for declaring the roles and where these roles can be found. We can do this with the following command:

wdijkerman@curiosity [ ~/git/environment/ansible ] (14:24:19 - Sat Aug 15)
 (master) > librarian-ansible init
      create  Ansiblefile

It creates the “Ansiblefile” in the current directory. It already has some basic roles specified, but I don’t use them.

#!/usr/bin/env ruby
#^syntax detection

site "https://galaxy.ansible.com/api/v1";

role "kunik.deploy-upstart-scripts";

role "pgolm.ansible-playbook-monit",
  github: "pgolm/ansible-playbook-monit";

With the default Ansiblefile it shows you, that you also can make use of the Ansible Galaxy. The site is configured to use Ansible Galaxy API and when you only have specified the “role” (In this case kunik.deploy-upstart-scripts”, it will be downloaded from the Galaxy.

2nd example is downloding the git repository “pgolm/ansible-playbook-monit”, which will be installed with the role name “pgolm.ansible-playbook-monit”. Nice, but I only need to make use of the “git” option. I start with the following:

#!/usr/bin/env ruby
#^syntax detection

role "zabbix-javagateway",
    git: "git@gitlab.dj-wasabi.local:ansible/zabbix-javagateway.git",
    ref: "0.1.0"

When I run “librarian-ansible install’ it will clone the git repository and checkouts the tag “0.1.0”. The role is now installed in the ‘ librarian_roles/’ directory with the name “zabbix-javagateway”. But I want it in my roles directory, so I have to run the ‘librarian-ansible’ command again, but with the config option:

librarian-ansible config path roles --global

This sets the path to my “roles” directory. This is specified in my ansible.cfg and I want them in this directory. So running again the ‘librarian-ansible install’ command and the role is installed again. But, what I didn’t know (or didn’t read in the very few sites that exists about librarian-ansible) is it will delete the content of the directory. So all my roles which were in the ‘roles/’ directory are deleted. So, ‘git checkout roles/’ and moving all roles to their own git repository and start again! 🙂

Maybe add the “roles” directory in my .Gitignore file. We don’t want to store all the roles in this repository too.

I think I’m going to be an fan for librarian-ansible. 🙂

 

wdijkerman-zabbix version 1.5.0 puppet module released; not puppet 4 compliant yet, but it can automatically install Zabbix Templates!

Puppet LogoZabbix Logo

I just released an new version of the wdijkerman-zabbix puppet module: 1.5.0.

I had hoped to make this release puppet 4 compliant, but when I was investigating this I found that most of the modules that the wdijkerman-zabbix module is depending on isn’t puppet 4 compliant yet. With puppet 4, the only issue I encountered so far is related to the puppetlabs-apache module (It restarts the apache service before all files are present in the conf.d directory).

With this release you can automatically install Zabbix Templates! You need to use the define ‘zabbix::template’ and specify where the XML can be found and the template will be installed.

The changelog for this release:

  • Fix for: Inherting params #93
  • Fix for: new postgresql instance #91; Also update metadata for postgresl module version
  • Fix for: Need to overide php_values #89
  • 2nd fix for: Zabbix-proxy install database population #62. Also for postgresql now.
  • Added support to Amazon Linux with epel 6. #96 (By pull request: Wprosdocimo (Thanks!))
  • import templates and create hostgroup if missing #95 (By pull request: 1n (Thanks!))
  • Added Support For Zapache monitoring script #94 (By pull request: elricsfate (Thanks!))
  • merge of hiera hashes from entire hierarchy #98 (By pull request: szemlyanoy (Thanks!))
  • Added property script_ext for: File extensions of Userparameters scripts #97
  • Updated documentation in README.md

The “#<NUM>” is a reference to an Github issue. The release can now also be found on and downloaded from the forge.

If you find an bug or have an improvement for this Zabbix puppet module, please create pull requests!

wdijkerman-zabbix version 1.4.0 puppet module released

Puppet LogoZabbix Logo

I just released an new version of the wdijkerman-zabbix puppet module: 1.4.0.

The original idea was using this version to make the puppet module puppet 4 compliant, but I didn’t had any time for this. So there were some issues which needed to be fixed, so I created this release.

The changelog:

  • Adding “apt” as dependency.
  • Adding ‘script_dir’ parameter for userparameters define.
  • Fix documentation: iptables is set to false (not true).
  • Fix illegal comma separated argument list #81 (By pull request: IceBear2k (Thanks!))
  • Fixes #80 setting Hostname and HostnameItem causes a warning on agentd s… #82 (By pull request: f0 (Thanks!))
  • Allow to not purge include dir. #79 (By pull request: altvnk (Thanks!))
  • Correct typo in ‘manage_resources’ documentation. #77 (By pull request: rnelson0 (Thanks!))
  • Added zabbix_hostgroup #87 (By pull request: hkumarmk (Thanks!))

The “#<NUM>” is a reference to an Github issue. The release can now also be found on and downloaded from the forge.

If you find an bug or have an improvement for this Zabbix puppet module, please create pull requests!

wdijkerman-zabbix version 1.3.0 puppet module released, support for Apache 2.4 and more!

Puppet LogoZabbix Logo

I just released an new version of the wdijkerman-zabbix puppet module: 1.3.0.

This release contains mostly pull requests from other users, so many thanks for this!

The changelog:

  • bugfix for vhosts in apache 2.4 #67 (By pull request: ju5t (Thanks!))
  • Update apt key to full 40characters #66 (By pull request: exptom (Thanks!))
  • rename ListenIp => ListenIP (By pull request: sbaryakov (Thanks!))
  • Fix manage_repo parameter on the zabbix class (By pull request: roidelapluie (Thanks!))
  • minor typo (By pull request: andresvia (Thanks!))
  • better default parameter for userparameter (By pull request: sbaryakov (Thanks!))
  • Fix for: Multi-node Setup: Web class does not properly configure database port #69
  • Fix for: Zabbix-proxy install database population #62

The “#<NUM>” is a reference to an Github issue. The release can now also be found on and downloaded from the forge.

If you find an bug or have an improvement for this Zabbix puppet module, please create pull requests!

The next release will probably be used for puppet 4 or maybe Zabbix 3.0, depends on when they are released. 🙂

Zabbix nodata trigger, really a lifesaver

Zabbix has a lot of triggers you can use for your environment. But when it comes to the most important checks, I’ll update the current trigger and create an “or” statement with the “nodata” trigger.

It happened for me a few times when Zabbix didn’t notify me about some checks that went into PROBLEM while the service/program wasn’t responding. In all of these cases, I had it configures with the “last” trigger. All of them said it was ok, but the latest update was at least 30 minutes ago (Some even longer!). I slept well though, but my start of the day wasn’t really good. 🙂

I did find that most of these kind of checks were items that used scripts which were created by myself or by my colleagues. These were simple scripts in bash or python, which had no proper way of exiting due to an timeout.

So the first and most easy way is to update the trigger with the “nodata” configuration. I use an “default” of 5 minutes, if I don’t have any data the trigger can be fired. An good example is the trigger for the zabbix-agent:

{Template App Zabbix Agent:agent.ping.nodata(5m)}=1

When there is no data retrieved in the last 5 minutes, it will be fired. So I’ve updated the trigger for my Apache template like this:

{Template Apache:apache.run.status[{HOST.NAME},'localhost'].last()}=0 or {Template Apache:apache.run.status[{HOST.NAME},'localhost'].nodata(5m)}=1

If the last value of the item apache.run.status is 0 or there is no data retrieved in the last 5 minutes, then we can assume Apache is down.

But these scripts needs to be able to exit within the TimeOut parameter in the configuration file. So each script should have an timeout and when this timeout occurs, it should print some information which Zabbix can handle. (Not an python stacktrace for example. 🙂 )

I know the “nodata” trigger has an few drawbacks. It do add some extra load on the server for these checks. I believe the “nodata” triggers are checked every 30 seconds, but I try to use the nodata only for those critical (Or Disaster in Zabbix terms 🙂 ) triggers.

Another one, but happens (hopefully) not that much, when the Zabbix-server is in some kind of maintenance or had some problems (Or when you have an Zabbix proxy which doesn’t send the data to the server) a lot of triggers are fired when the times has past. So you just updated the Zabbix-server with the latest (linux/kernel) patches and after say 10 minutes the Zabbix-server is up and running again, all of those nodata triggers are fired. 🙂

But anyways, it is an lifesaver! 🙂

Installing zabbix-agent with Ansible

ansible_logo_black_squarezabbix_logo

Not only I have an puppet module for installing Zabbix, I also have some Ansible roles for this. At the moment there are 4 roles:

In this blog item, we talk about the “zabbix-agent” role. The latest version is 0.2.0.

Defaults

Installing this role is very easy:


ansible-galaxy install dj-wasabi.zabbix-agent

It will be installed in your roles directory. Default is “/etc/ansible/roles” or whatever you have configured in the ansible.cfg file. After installation there is only 1 (or 2 when you make use of active items) parameters needed for making this role work:

agent_server: <IP_-_FQDN_OF_ZABBIX_SERVER>
agent_serveractive: <IP_-_FQDN_OF_ZABBIX_SERVER>

This will need the ip address or the FQDN of the “zabbix-server”.

OS?

This role works on several operating systems/families:

  • RedHat
  • Debian
  • Ubuntu
  • OpenSuse

If you have an operating system/family which isn’t in the list above, you can create an issue at the Github page and please fill in the request. I can’t make any guarantee that it will come, but I can try it. Or if you do have some Ansible skills, please create an Pull request and I would be happy to accept it. 🙂

Playbook

So, how does the playbook looks like? Like this:

- hosts: all
  sudo: yes
  roles:
   - role: dj-wasabi.zabbix-agent
     agent_server: <IP_-_FQDN_OF_ZABBIX_SERVER>
     agent_serveractive: <IP_-_FQDN_OF_ZABBIX_SERVER>

As you see it is very basic and does the job very good. This only installs the agent on the specific server and configures the configuration file. But we really want to automate everything right?

Cove

Few weeks ago I found this pull requests for the “ansible-modules-extra” repository. This pull requests had an few ansible modules which made sure that you can use the Zabbix API to create or update hosts configuration. In the pull requests there were something like 5 modules, but this Ansible role only use 3 of them. With this role, you can create the following:

  • host groups
  • Host itself.
  • Macros for the host

For now, when the host is created, it will only create the “zabbix interface”. Maybe with the next release I’ll make sure you can also create SNMP, JMX and IPMI interfaces.

How do we have to configure it? Something like this. You will have to change it to your environment.

- hosts: wdserver00
  roles:
     - role: zabbix-agent
       agent_server: 192.168.1.1
       agent_serveractive: 192.168.1.1
       zabbix_url: http://zabbix.example.com
       zabbix_api_use: true
       zabbix_api_user: Admin
       zabbix_api_pass: Zabbix
       zabbix_create_host: present
       zabbix_host_groups:
         - Linux servers
       zabbix_link_templates:
         - Template OS Linux
       zabbix_macros:
         - macro_key: apache_type
           macro_value: reverse_proxy

I’ll skip the first 2 parameters, as these are described earlier on this page.

zabbix_url: The url on which the Zabbix web interface is available.
zabbix_api_user: The username which will connect to the API.
zabbix_api_pass: The password for the “zabbix_api_user” user.
zabbix_create_host: present if we want to create the host, absent if we want to delete it.
zabbix_host_groups: List of hostgroup where this host belongs to.
zabbix_link_templates: List of templates which will be linked to the host.
zabbix_macros: key, value pair of macros that will be used by the host. 

When we run Ansible, we will see at the end of the run:

.. <skip> ..
TASK: [zabbix-agent | Create hostgroups] **************************************
ok: [wdserver00 -> 127.0.0.1]

TASK: [zabbix-agent | Create a new host or update an existing host's info] ****
changed: [wdserver00 -> 127.0.0.1]

TASK: [zabbix-agent | Updating host configuration with macros] ****************
changed: [wdserver00 -> 127.0.0.1] => (item={'macro_key': 'apache_type', 'macro_value': 'reverse_proxy'})

Nice! If you check the Web interface, you’ll see that the host is created with the correct host groups and templates. If not, you’ll see some error messages in the Ansible output which will say what went wrong.

This role isn’t perfect, so if you encounter an bug or found/have and enhancement, please create an Pull request at Github and I’ll accept it. We can all make this role beter. 🙂

Side note:

There are more parameters which can be overridden, please check the “defaults/main.yml” file or the README.

Zabbix agent installation and configuration with puppet exported resources

puppet zabbix_logo

Who doesn’t like to automate tasks when this is easy and can save a lot of time/troubles. When you are just starting with Zabbix, it seems that the only thing you can automate is installing the components with rpm/deb packages on the servers/clients. After installation of the packages, you’ll still need to create the host in the web interface which takes some time (And boring to if you have a lot of hosts to configure..). There is an possibility to use (auto) discovery rules, but this requires some configuration in the web interface and this only works once when the host isn’t configured in the web interface (Initial setup only).

Luckily, the Zabbix has an API which can be used for some tasks that can be automated, like creating hosts! The wdijkerman-zabbix module can use this API for the following tasks:

  • Create host
  • Update host with Templates

There is an requirement when you want to do this: Puppet Master needs the PuppetDB configured. If you want to configure an PuppetDB, please use Google for this as I won’t discuss this here.

Idea

The idea is that when an installation is done via the “zabbix::agent” class on the agent, it send some information via puppet to the PuppetDB. This is some basic information like ipaddress, name of the host and the hostgroups for this host. When the puppet agent on the “zabbix-server” runs, it will retrieve the information from the PuppetDB and will create via the Zabbix API the host(s). With this setup, everything is automated and you don’t have to worry for forgetting something.
So, how do you have to configure the zabbix module?
class { 'zabbix':
  zabbix_url       => 'zabbix.example.com',
  manage_resources => true,
}

This is an very basic configuration method mentioned above which will automatically configure the hosts in the web interface when a new “zabbix-agent” is installed. You don’t have to do anything else (yes ok, you’ll have to run some puppet runs on several machines. :-).)

This assumes that the password for the “Admin” user is still “zabbix”. You can override this of course, just like the database credentials which are default set to “zabbix-server”. (Don’t forget to change it to something non default 😉 )

Userparameters

There is an other nice feature in the “wdijkerman-zabbix” module which uses the Zabbix API. The “zabbix::userparameters” define can make use of the API. The goal for this define is installing an UserParameters file in the zabbix-agent includedir. This file consists of 1 or more UserParameter entries which explains what commands needs to be executed for an key.

An example:

zabbix::userparameters { 'Exim':
 content => 'UserParameter=exim.mailq,sudo /usr/sbin/exim -bpc',
}

There is an item in a template which executes the “sudo /usr/sbin/exim -pbc” command with the “exim.mailq” key. When the above 3 lines are in the exim puppet module and this module installs the UserParameter file, you still need to update the host in the web interface to make sure the “Template Exim” is assigned.

But stop, we can use it like the following now:

zabbix::userparameters { 'Exim':
 content  => 'UserParameter=exim.mailq,sudo /usr/sbin/exim -bpc',
 template => 'Template Exim',
}

We also supply the name of the Zabbix Template in this “user::parameters” define. When configured we first have to execute an puppet run on the agent, so it sends some data to the PuppetDB.

When the Puppet Agent runs on the “zabbix-server”, it first checks if the hosts exits and then it will assign the template “Template Exim” to the host automatically. Off course when the template does not exists, the Puppet run will fail.

Woot! 🙂