Bandwidth monitoring with Neutron and Ceilometer

OpenStack Telemetry (aka Ceilometer) gives you access to a wealth of information – and a particularly interesting stat I wanted access to was outgoing bandwidth of my Neutron networks. Out of the box, Ceilometer gives you a rolled up cumulative stat for this with the following:

ceilometer sample-list --meter network.outgoing.bytes

This produces output like the following

ceilometer-sample-list

This is fine, but when you’re trying to break down the network stats for something useful like billing since beginning of month, this makes it tricky – even though I’m sure I shouldn’t have to do this! I followed this excellent post http://cjchand.wordpress.com/2014/01/16/transforming-cumulative-ceilometer-stats-to-gauges/ which describes this use-case brilliantly (and goes further with Logstash and Elastic Search) – where you can use Ceilometer’s Pipeline to transform data into a format that suits your use case. The key pieces of information from this article was as follows:

1. Edit the /etc/ceilometer/pipeline.yaml of your Controller and Compute hosts and add in the following lines

-
 name: network_stats
   interval: 10
   meters:
     - "network.incoming.bytes"
     - "network.outgoing.bytes"
   transformers:
     - name: "rate_of_change"
     parameters:
       target:
         type: "gauge"
         scale: "1"
   publishers:
     - rpc://

2. Restart the ceilometer-agent-compute on each host

restart ceilometer-agent-compute

That’s it. We now have a “gauge” for our incoming and outgoing neutron – which means they’re pieces of fixed data applicable for the time range associated with it (e.g. number of bytes in the last 10 second sample set).

ceilometer-sample-list-gauge

 

I wanted to see if I could answer the question with this data: how much outgoing bandwidth was used for any given network over a period of time (i.e since the start of the month). It could be I misunderstand the cumulative stats (which tease me), but this was a useful exercise nonetheless! Now here is my ask of you – my bash fu below can be improved somewhat and would love to see it! My limited knowledge of Ceilometer, plus the lure of Bash, awk and sed, provided me with a script that outputs the following:

neutron-bandwidth-output

The (OMFG WTF has Kev wrote now) script, link below, loops through each of your Neutron networks, and outputs the number of outgoing bytes per Neutron network. Just make sure you’ve got ‘bc’ installed to calculate the floating numbers. If I’ve used and abused Ceilometer and awk too much, let me know!

https://raw.githubusercontent.com/OpenStackCookbook/OpenStackCookbook/icehouse/bandwidth_report.sh

Advertisements

Tagged: , , , , ,

2 thoughts on “Bandwidth monitoring with Neutron and Ceilometer

  1. Sam April 29, 2015 at 8:16 pm Reply

    Hi, Thanks for the informations !

    it might be the wrong place, but did you notice that your ressource ID are formated like the following :

    –tap

    giving result like that :

    instance-00001535-6b1bd401-4bb8-404b-9bae-0a3da79c068b-tap844d2678-83

    where :
    instance-00001535 is instance_name
    6b1bd401-4bb8-404b-9bae-0a3da79c068b is instance_id
    844d2678-83 is 11_first_char_of_interface_id

    Do you know what to change to have an resource_id that is only the interface_id and not this crappy string chain please ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: