Crowbar & OpenStack Install Part 1

A few weeks ago I decided to investigate OpenStack.  The core components are a collaboration and resulting love child of NASA and RackSpace and is an Open Source solution to build a blaah blaah cloud.  It seems to be gaining good traction and is often mentioned in the same breath as VMware.  In fact it appears everyone not in VMwares back pocket is jumping on the bandwagon and so seemed worthy of a look.

This is still a moving target for me, its not 'up' and has certainly not been painless and smooth, but I'm writing up Pt.1 as much for me as for the blogosphere.

OpenStack is, well, just that, a stack of different products which together produce an end solution.  In my eyes and how I'd describe it to 'management' is its comparable to LAMP.  Linux, Apache, MySQL and PHP.

I won't go into the details you can read yourself - http://en.wikipedia.org/wiki/OpenStack

I started by reading the install docs and attempting to install each piece individually and then I guess hook them all together.

This didn't go well and a twitter reply from Matthew McClean suggested I take look at Dells Crowbar.

The rest of this post will be the process, troubleshooting and useful links I've come across as I attempt to install Crowbar on my journey to OpenStack nirvana ..

Crowbar background

https://github.com/dellcloudedge/crowbar/wiki
http://robhirschfeld.com/2011/03/14/how-openstack-installer-works/

Resources

4 VMware guests - Each has two NICS each on a vlan, eth0 vlan 66 and eth1 99.  eth0 will be re-configured from the default so its available on my corp LAN.

• Admin Node - 2GB RAM, 20GB disk
• Node 1 - 1GB RAM, 15GB  disk
• Node 2 - 1GB RAM, 8GB disk
• Node 3 - 1GB RAM, 8GB disk

Installation

The documented install procedure - https://github.com/dellcloudedge/crowbar/wiki/Install-crowbar

If only it were that simple.

• Build or grab the correct ISO - in this case v1.1 http://crowbar.zehicle.com/openstack111111.iso

NB//Crowbar.iso is a base install, it has none of the OpenStack components. 

• Boot your first server on the ISO
• If the default networking works for you then great. Otherwise edit network BEFORE installing.  In my case I changed the Admin network settings.
• Run install - https://github.com/dellcloudedge/crowbar/wiki/Install-crowbar

This will build, pretty seamlessly, your 'admin' node which is then used to build the other components.

Here things went a little squiffy.

Theres a  nice YouTube video of building the environment which I started to follow :

But with little success.  The first barclamp (name assigned to the Crowbar packages, which actually in turn directly represent Chef cookbooks) which the video has you assign is Glance, which for me failed.  Attempting to resolve this I joined the Crowbar lists and co-coincidently someone else asked about the order to install applications and had another link :

http://lists.us.dell.com/pipermail/crowbar/2011-November/000308.html

MySQL
Keystone
Nova Dashboard
Glance
Nova

Very different to the order on the video ! - But its still worthwhile watching.

Another little find went into some further details - http://kb.dtosolutions.com/wiki/Deploying_a_small_OpenStack_test_environment_using_Crowbar

From these I eventually cooked up the following install method which seems to be more successful than anything else so far.

Due to a bug in the dashboard when removing or adding nodes I found you had to create all your nodes upfront before you start to apply barclamps.  Create 3 nodes, power them all on so they run the discovery image and become available in the Crowbar console (unassigned, yellow).

Click bulk edit and assign all three.  They will install the OS, reboot a few times and then go 'assigned' in the console with a green icon.

Assuming you have the admin node built and three green nodes.

• Build MySQL proposal and assign server to node 1 and client to node 2 and 3
• Build Keystone and assign to node 1
• Nova Dashboard to node 1
• Glance to node 1
• Nova.  

Nova requires a 'controller' and then compute nodes.  Trying to do all three failed for me so I did them individually.  In my case I'm using VMware for this POC so the nova proposal needs to be changed.  Edit the line :

"libvirt_type": "kvm", -> "libvirt_type": "qemu",

And assign Controller to node 1 and then compute to 2 and 3.

Throughout all of this I had a tail -f running to the admin log :

/opt/dell/crowbar_framework/log/production.log

By all accounts I should now have a working environment. I don't, everything appears to be installed but the learning curve continues.

Nova dashboard is not working which I've traced to a routing issue on node 2. It has the admin interface up but the default gateway is using one of the networks I didn't change at the install time.

Nagios is not working. Watching the production.log during the barclamp installs showed Nagios errors but I'm ignoring them/that for now.

Troubleshooting / Hints 'n' tips : 

Barclamps sometimes report failure to deploy but repeating the same task will be successful.

If your running your POC on hypervisors or something else that has the technology make good use of snapshots.