Monthly Archives: April 2014

Home Rackspace Private Cloud / OpenStack Lab: Part 6 – Upgrading

Minor Release Upgrade of Rackspace Private Cloud

By now you must know that OpenStack gets released every 6 months. We are due the 9th release of OpenStack, Icehouse, this week.

In the meantime, at Rackspace we continue to push out updates and fixes to the Havana release. In Rackspace Private Cloud, this is our 4.2 release of our product.  Currently, my lab is running v4.2.1 and the latest is v4.2.2 which was released on March 31st 2014. This blog post outlines the steps to go from v4.2.1 to v4.2.2 using the Rackspace Private Cloud Chef Recipes.

The general process is as follows and is extremely simple to do:

  1. Edit Chef environment to tell it to upgrade packages
  2. Download v4.2.2 recipes and upload to our Chef Server
  3. Apply Cookbook recipes to our environment

Upgrade Steps

1. Edit Chef environment on openstack1 (

vi /opt/base.env.json

then edit the following override_attributes:

"override_attributes": {
 "osops": {
 "do_package_upgrades": true
 "glance": {
 "image_upload": false

2. Load the new environment changes into Chef

knife environment from file /opt/base.env.json

3. Grab the updated Cookbooks from Github

git clone /opt/chef-cookbooks
pushd /opt/chef-cookbooks
git checkout ${COOKBOOK_VERSION}
git submodule init
git submodule sync
git submodule update

4. Upload the new cookbooks to Chef

knife cookbook upload -o /opt/chef-cookbooks/cookbooks -a
knife role from file /opt/chef-cookbooks/roles/*.rb

5. As I run HA Controllers, the updates are done on each Controller in turn (openstack1, then openstack2), ensuring no loss of service. To do this we ensure services have failed over to our second controller, and then updates are run as follows:

service keepalived stop
service keepalived start

6. We repeat the process on the second controller

ssh openstack2              # Second controller HA Pair
service keepalived stop
service keepalived start
exit                        # Back to the first controller

7. We then chef-client all the compute nodes to update them. Note that our instances don’t stop or have interrupted service throughout this in my lab (although it is recommended to reboot the instances when convenient to pick up any performance improvements).

knife ssh "role:single-compute" chef-client

And that’s it, we’re all done! I ran Nagios, as instances, throughout my upgrade monitoring each instance, my router and the services and didn’t get any interrupted service during this v4.2.1 to v4.2.2 upgrade in my lab, as you’d expect.