Running OpenStack under VirtualBox – A Complete Guide (Part 2)

In the previous post, we very simply got OpenStack running under VirtualBox. This next part takes this further by getting multiple compute nodes installed to spread the load of your instances. It also paves the way for Part 3 where we will then start to look at Swift, OpenStack’s storage implementation of S3.

Part 2 – OpenStack on a multiple VirtualBox VMs with OpenStack instances accessible from host

We will be using the same set up as for Part 1.

The proposed environment

  • OpenStack “Public” Network:
  • OpenStack “Private” Network:
  • Host has access to its own LAN, separate to this on and not used for this guide
  • One VirtualBox VM running the software needed for the controller
  • One VirtualBox VM running the software needed for a compute node

This guide will assume you have followed Part 1. We are simply adding in a compute node now, so that will make the virtual machine you created in Part 1 will become the Cloud Controller (CC). Also, OpenStack has been designed so that any part of the environment can run on a separate server. For this guide we will have the following

  • Cloud Controller running MySQL, RabbitMQ, nova-network, nova-scheduler, nova-objectstore, nova-api, nova-compute
  • Compute node running: nova-compute

The Guide

  • Add a new VirtualBox Guest
    • Name: cloud2
      • OS Type: Linux
      • Version: Ubuntu (64-Bit)
    • 2048Mb Ram
    • Boot Hard Disk
      • Dynamically Expanding Storage
      • 8.0Gb
    • After this initial set up, continue to configure the guest
      • Storage:
        • Edit the CD-ROM so that it boots Ubuntu 10.10 Live or Server ISO
        • Ensure that the SATA controller has Host I/O Cache Enabled (recommended by VirtualBox for EXT4 filesystems)
      • Network:
        • Adapter 1
          • Host-only Adapter
          • Name: vboxnet0
        • Adapter 2
          • NAT
          • This will provide the default route to allow the VM to access the internet to get the updates, OpenStack scripts and software
      • Audio:
        • Disable (just not required)
    • Boot the guest and install Ubuntu as per normal
  • Assign static IPs to the cloud controller
    • Ensure that the Cloud Controller you created in Part 1 has static addresses for eth0 and eth1.
      • For the sake of this guide, I’m assuming you have assigned the following
        • eth0:  This address is your Cloud Controller address (CC_ADDR) and will be the API interface address you will be communicating on from your host.
        • eth1: stays as dhcp as it is only used for NAT’d access to the real world
    • Your compute nodes don’t need to be set statically, but for the rest of this guide it is assumed the addresses are as follows
      • Cloud2
        • eth0:
        • eth1: stays as dhcp as it is only used for NAT’d access to the real world
  • Grab this script to install OpenStack
    • This will set up a repository (ppa:nova/trunk) and install MySQL server where the information regarding your cloud will be stored
    • The options specified on the command line match the environment described above
    • wget --no-check-certificate
  • Run the script (as root/through sudo) specifying that you want a compute node and that you’ve specified the IP address of the Cloud Controller
    • sudo bash ./ -A $(whoami) -T compute -C
  • No further configuration is required.
  • Once completed, ensure that the Cloud Controller knows about this new compute node
    • On the Cloud Controller run the following
      • mysql -uroot -pnova nova -e 'select * from services;'
      • sudo nova-manage service list
      • You should see your new compute node listed under hosts
      • If you don’t have a DNS server running that resolves these hosts add your new compute node to /etc/hosts
        • cloud2
  • On the new compute node run the following
    • sudo nova-manage db sync
  • As you copied your credentials from the cloud controller created in Part 1, you should just be able to continue to use this from your host – but this time you can spin up more guests.
    • If you changed the eth0 address of your cloud controller, ensure your cloud/creds/novarc environment file has the correct IP.
  • Repeat the steps above to create further compute nodes in your environment, scaling seamlessly

Tagged: , , , ,

14 thoughts on “Running OpenStack under VirtualBox – A Complete Guide (Part 2)

  1. […] Next: Part 2 – OpenStack on a multiple VirtualBox VMs with OpenStack instances accessible from hos… […]

  2. […] Installazione di OpenStack Nova (2) Articoli Blog Add comments mar 222011 Hello there! If you are new here, you might want to subscribe to the RSS feed for updates on this topic.Powered by WP Greet Box WordPress Plugin In questo secondo HowTo eseguiremo una installazione di un nodo computazionale di OpenStack Nova utilizzando Ubuntu Server 10.10 nella versione a 64 bit e un comodo script di autoinstallzione. Questo HowTo è liberamente tratto da questo articolo. […]

  3. raaverook October 11, 2011 at 6:38 pm Reply

    Nice tutorial. Is there a part 3 coming ?

    • Kevin Jackson October 11, 2011 at 7:28 pm Reply

      Yes, I want to do one on including Glance and Dashboard, which really gives a good feature set to running it under VirtualBox. And maybe something along the lines of Crowbar or Orchestra for deployment – but is overkill for vbox. Vagrant is more suited here but in the real world you’d want to use an enterprise deployment mechanism, but either way I’ll put some new guides up soon.

      • Haneef Mubarak January 8, 2012 at 9:06 am

        So are you actually going to write the tutorial? And if you do, could you also specify your host system’s specs in more detail (GHz, etc.)?

        Haneef Mubarak

      • Kevin Jackson January 8, 2012 at 11:53 am

        I’ve other conflicting priorities at the moment to continue this tutorial but I’ll get around to updating the methods in-line with fixes seen in Diablo and the forthcoming Essex release.


  4. fabio October 20, 2011 at 11:37 am Reply

    hi there i m not able to fully run the image on the node2. i can see in the console:
    Sending discover…
    wget: can’t connect to remote host ( Network is unreachable

    how can i fix it?

  5. dara October 24, 2011 at 5:46 pm Reply

    @fabio – I had the wget: can’t connect to remote host ( too. The guest on the compute node was not getting the dhcp offer back from the network node. I changed the VirtualBox nics for the eth0s to allow promiscuous mode and then it got the IP.

    Before that I had to change from using
    to just

  6. Fabio October 25, 2011 at 2:44 pm Reply

    @dara thank you vary much for your reply.
    i cannot still get it work. could you pls clarify a couple of more doubts i have?
    1) on the compute-node did you assign an ip to the bridge? some manual says that i also have to bind br100 with the eth card ( under Configuring Multiple Compute Nodes). did you follow also all those steps or not?
    2) when you say to change eth0 nic to allow promiscous mode (ifconfig eth0 promisc): do you mean on the compute-node, on the controller or on my machine maybe? (i actually turned all the interfaces on the node to promisc but no success)
    3) somewhere else people suggests to load an iptables rule (iptables -t nat -A PREROUTING -d -p tcp -m tcp –dport 80 -j DNAT –to-destination $nova_api:8774). did you issue also that command on the node, or turning eth0 to promisc was enough?
    again thanks, i ‘m not far from having a cloud fully working, but now is already one week struggling with this issue.

    • dara October 25, 2011 at 4:44 pm Reply

      1. I did not assign an IP to the bridge on the compute node.

      2. No it is a setting on the VirtualBox NIC “hardware”. Shutdown the VirtualBox VMs. Select VM, goto settings, network, select the adapter that corresponds to guest eth0, in advanced, you should see a setting called “promiscuous mode” that defaults to deny – change this to “allow all”.

      I did this on both controller and compute VMs. I didn’t need to do ‘ifconfig eth0 promisc’

      I used tcpdump on the various interfaces and found the DHCP offers were not getting back up into the compute node – for no good reason. This could be something very specify to virtual NICs and NIC drivers. So VirtualBox version and Linux guest version. What kind of VBox NICs are you using? I’m using virtio-net

      3. I didn’t have to do that. If the guest on the compute node gets an IP on, it will get a default gateway which is the IP of the bridge on the crontroller/network node, which should forward it on. Can your network-node get to this address?

  7. Fabio October 25, 2011 at 5:25 pm Reply

    you are my angel. if anytime you come to Amsterdam, i’ll offer you all the beers you can drink.

  8. John Sanabria January 2, 2012 at 6:05 pm Reply

    I can run a new instance in the compute node (cloud2) however I can not get access to that instance.

    clouduser@cloud1:~$ euca-describe-instances
    RESERVATION r-j0r8006n myproject default
    INSTANCE i-00000007 ami-00000003 running openstack (myproject, cloud2) 0 m1.tiny 2012-01-02T17:00:48Znova aki-00000001 ari-00000002

    Here is my try,

    clouduser@cloud1:~$ ssh -i cloud/creds/openstack.pem root@
    ssh: connect to host port 22: No route to host

    Thanks for your help.

  9. April 24, 2012 at 3:43 am Reply

    Hi, I’m also having issue with the DHCP offers not making it back to the guest VM. The offers are seen on eth0 (using tcpdump) but don’t seem to be forwarded to the bridge so that the guest VM could see them (i.e. not seen on vlan100, vnet0 and other bridge interfaces). The promiscuous setting change on eth0s didn’t help (using virtio-net for eth0). Any tips would be great, thanks!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: