Tag Archives: gmetad

SELinux && Ganglia / Multicast && Apache && RRDs

SELinuxWhile setting up test server with Ganglia triad (gmetad, gmond, gweb over Apache) I had only 2 SELinux alerts to solve. First one regarding to denial of /var/lib/ganglia/rrd access by httpd process (while accessing ganglia-web interface):

In order to resolve that I just added SELinux contexts to rrdtool binary and /var/lib/ganglia/rrds dir:

The last SELinux alert was regarding to Apache trying to access port 8652 by socket connection (on ganglia-web I saw error like “fsockopen 8652 access denied”):

So I had to create local policy for this particular occasion as there is no ganglia modules for SELinux (maybe I should create one…?). Firstly let’s check that there’s no even one module:

So let’s create this policy:

After this everything should work like a charm. To confirm that We have this new policy working:

Ganglia, multicast && KVM on CentOS

http://ganglia.sourceforge.net

This is just a short note – I have to post it as this problem was really annoying and I couldn’t find any solutions in Google, so had to resolve it by myself.

Don’t know what Ganglia is? Check here: http://ganglia.sourceforge.net/ – it just kicks ass :)

Problem? I installed gmond on all our hosts / guests (CentOS 5/6, KVM virt, latest Ganglia daemons), also configured properly gmetad daemons and started this whole stuff using multicast. And it was working – for a while. After about 10-20 minutes it just stopped working on KVM guests. I saw no charts for those machines – but gmonds (even in debug mode) didn’t reveal any problems. And KVM hosts’ charts were fine (mostly..).

One more thing – in KVM guests I always set “deaf = yes” (just don’t want to have too much multicast traffic – i set it to “no” only on some bare hosts).

Ok so the problem.. I hung for some time on tcpdump / strace and came to the root of this problem – somehow there was no multicast traffic on KVM guests (I turned off iptables on KVM guests for the time of this whole issue – resolving). After some time I found 2 possible root causes:

  1. On KVM hosts by default there is multicast filter set on: no-ip-multicast (You can check if You have it turned on with following command: 

    If it’s turned on – You can turn it off with:

    And that should do this part of the trick
  2. And also – on CentOS KVM guests we have to turn off rp_filter in /etc/sysctl.conf:

    and:

    (You can try setting it to “loose mode” (so value: 2) instead of 0 – it can work for You and it’s always safer

Thats all for now. My sources for this one?