Category Archives: Scripting

Vagrant OpenStack Plugin 101: vagrant up –provider=openstack

Now that we have a multi-node OpenStack environment spun up very easily using Vagrant, we can now take this further by using Vagrant to spin up OpenStack instances too using the Vagrant OpenStack Plugin. To see how easily this is, follow the instructions below:

git clone https://github.com/mat128/vagrant-openstack.git
cd vagrant-openstack/
gem build vagrant-openstack.gemspec
vagrant plugin install vagrant-openstack-*.gem
vagrant box add dummy https://github.com/mat128/vagrant-openstack/raw/master/dummy.box
sudo apt-get install python-novaclient

With the plug-in installed for use with Vagrant we can now configure the Vagrantfile. Remember the Vagrantfile is just a configuration file that lives in the directory of the working environment where any artifacts related to the virtual environment is kept. The following Vagrantfile contents can be used against the OpenStack Cloud Computing Cookbook Juno Demo environment:

require 'vagrant-openstack'
Vagrant.configure("2") do |config|
  config.vm.box = "dummy"
  config.vm.provider :openstack do |os|    # e.g.
    os.username = "admin"          # "#{ENV['OS_USERNAME']}"
    os.api_key  = "openstack"      # "#{ENV['OS_PASSWORD']}"
    os.flavor   = /m1.tiny/
    os.image    = /trusty/
    os.endpoint = "http://172.16.0.200:5000/v2.0/tokens" # "#{ENV['OS_AUTH_URL']}/tokens"
    os.keypair_name = "demokey"
    os.ssh_username = "ubuntu"
    os.public_network_name = "ext_net"
    os.networks = %w(ext_net)
    os.tenant = "cookbook"
    os.region = "regionOne"
  end
end

Once created, we can simply bring up this instance with the following command:

vagrant up --provider=openstack

Note: The network chosen is a routable “public” network, which is accessible from the Vagrant client and is a limitation at this time for creating these instances. Also note that vagrant openstack seems to get stuck at “Waiting for SSH to become available”. Ctrl + C at this point will drop you back to the shell.

Bandwidth monitoring with Neutron and Ceilometer

OpenStack Telemetry (aka Ceilometer) gives you access to a wealth of information – and a particularly interesting stat I wanted access to was outgoing bandwidth of my Neutron networks. Out of the box, Ceilometer gives you a rolled up cumulative stat for this with the following:

ceilometer sample-list --meter network.outgoing.bytes

This produces output like the following

ceilometer-sample-list

This is fine, but when you’re trying to break down the network stats for something useful like billing since beginning of month, this makes it tricky – even though I’m sure I shouldn’t have to do this! I followed this excellent post http://cjchand.wordpress.com/2014/01/16/transforming-cumulative-ceilometer-stats-to-gauges/ which describes this use-case brilliantly (and goes further with Logstash and Elastic Search) – where you can use Ceilometer’s Pipeline to transform data into a format that suits your use case. The key pieces of information from this article was as follows:

1. Edit the /etc/ceilometer/pipeline.yaml of your Controller and Compute hosts and add in the following lines

-
 name: network_stats
   interval: 10
   meters:
     - "network.incoming.bytes"
     - "network.outgoing.bytes"
   transformers:
     - name: "rate_of_change"
     parameters:
       target:
         type: "gauge"
         scale: "1"
   publishers:
     - rpc://

2. Restart the ceilometer-agent-compute on each host

restart ceilometer-agent-compute

That’s it. We now have a “gauge” for our incoming and outgoing neutron – which means they’re pieces of fixed data applicable for the time range associated with it (e.g. number of bytes in the last 10 second sample set).

ceilometer-sample-list-gauge

 

I wanted to see if I could answer the question with this data: how much outgoing bandwidth was used for any given network over a period of time (i.e since the start of the month). It could be I misunderstand the cumulative stats (which tease me), but this was a useful exercise nonetheless! Now here is my ask of you – my bash fu below can be improved somewhat and would love to see it! My limited knowledge of Ceilometer, plus the lure of Bash, awk and sed, provided me with a script that outputs the following:

neutron-bandwidth-output

The (OMFG WTF has Kev wrote now) script, link below, loops through each of your Neutron networks, and outputs the number of outgoing bytes per Neutron network. Just make sure you’ve got ‘bc’ installed to calculate the floating numbers. If I’ve used and abused Ceilometer and awk too much, let me know!

https://raw.githubusercontent.com/OpenStackCookbook/OpenStackCookbook/icehouse/bandwidth_report.sh

Easy OpenStack Folsom with VirtualBox and Vagrant

Testing OpenStack is now as easy thanks to VirtualBox and Vagrant. To run a mini test environment with Compute, Cinder, Keystone and Horizon you just need the following tools:

  • VirtualBox
  • Vagrant
  • Git client

Getting Ready

To set up a sandbox environment within VirtualBox to run OpenStack Folsom you will need to download:

Installation of these tools are simple – follow the on-screen prompts.

When ready we need to configure the VirtualBox “Host-Only” Networking. This networking mode allows us to communicate with our VirtualBox guest and our underlying host.
We will set up the following:

  • Host-Only Network: IP 172.16.0.254; Network 172.16.0.0/255.255.0.0; Disable DHCP
  • Host-Only Network #2: IP 10.0.0.254; Network 10.0.0.0/255.0.0.0; Disable DHCP

(Hint: there is a bash script @ https://raw.github.com/uksysadmin/OpenStackInstaller/folsom/virtualbox/vbox-create-networks.sh to create these for you).

How To Do It

To create a VirtualBox VM, running Ubuntu 12.04 with OpenStack Folsom from Ubuntu’s Cloud Archive, carry out the following

1. Clone the GitHub OpenStackInstaller scripts

git clone https://github.com/uksysadmin/OpenStackInstaller.git

2. Make the scripts the ‘folsom’ branch

cd OpenStackInstaller
git checkout folsom

3. Run ‘vagrant’ to launch your OpenStack instance which will come up with IP 172.16.0.201

cd virtualbox
vagrant up

4. After a short while your instance will be ready. Note that on the first run, Vagrant will download a 384Mb Precise64 “box”. Subsequent launches will not require this step.

Launch a web browser at http://172.16.0.201/horizon and log in with:

Username: admin
Password: openstack

(Note, to edit the IP it is assigned, modify virtualbox/vagrant-openstack-bootstrap.sh (Warning its a bit of a sed hack!).