TREllisNet logo

Random musings on life & free software

Road Warrior 101

I’ve been very fortunate enough to have travelled a lot in the past 6 years in my roles as a Consultant at both Red Hat, Canonical and now Eucalyptus, but also for holidays. Those countries have included: Philippines, India, Australia, USA (San Francisco, Santa Barbara, Boston, New York, Washington DC, Florida), Canada (Montreal), South Africa, Finland, Sweden, Netherlands, Spain, Germany, Italy, Belgium, Ireland, Poland… to name a few!

Suffice to say, when you spend a bit of time “on the road”, you learn a few tricks to help make life a bit easier. This blog post is a break from my technical posts to impart a few of those tricks on you, the public.

do you have any tips? I’d love to hear them, stick them in the comment box below!

Adventures in NoSQL, Part 2

In Part 1 of this blog series, “Adventures in NoSQL”, I deployed a single instance of MongoDB and used Python’s tweetstream module to fill a collection with a data feed from Twitter.

In the real world you wouldn’t ever use a single instance of MongoDB (or twitter data :–) ) as there is no redundancy if an instance fails, all your data is gone or you need to take some time to restore it from a backup.

However, we can harness the power of a private Eucalyptus IaaS Cloud to use as our infrastructure, this means we can quickly scale out resources using direct EC2 API calls, the euca2ools command line utilities or the Eucalyptus Web interface.

In this post, I’ll explore using Replication to spread your data across multiple MongoDB servers for redundancy.

Backing Up a Eucalyptus Cloud Controller

Introduction

When you first start using Eucalyptus, you are concerned with the intricacies of building images, configuring virtual instances and using all the variety of API features and tools that Eucalyptus offers.

However, when Eucalyptus becomes a tool your business relies on you need to consider making sure you can recover the system from a catastrophic failure. In the enterprise this usually means making adequate backups and being able to restore the whole system from those backups using an automated procedure perhaps with tools such as Ansible, Puppet and Chef or via a documented manual restore procedure.

Of course, like most things – it’s never as easy as it seems it should be! :–)

This article will cover backing up and manually restoring the Eucalyptus Cloud Controller (CLC) to a known state.

FOSDEM 2013

This last weekend I was over at FOSDEM 2013, where 5,000 geeks descended onto the ULB university in Brussels, Belgium for a huge free and open source software conference.

It was my first time at FOSDEM after many years of nearly attending and the first thing that struck me was the size. 5,000 people in attendance over the two days, 390hrs of talks and every talk was absolutely packed. It reminded me of the Ubuntu Developer Summit (UDS), but on steroids.

Adventures in NoSQL, Part 1

You’ve deployed and setup a private Cloud platform but now what? You need an application!

I’ve been experimenting with a number of technologies to generate workloads and give some demos to prospective Eucalyptus customers. A NoSQL database seems like a great use-case to demo as the technology benefits from being designed for scale-out workloads and this happens to be exactly what an IaaS Cloud does best.

There are an abundance of NoSQL implementations (Cassandra, MongoDB, Couchbase, Neo4j…), written in different programming languages and with slightly different takes on which two parts of the CAP theorem they choose to implement and which method they will use to store and display data.

For this post I’m going to be using MongoDB, which is in the “CP” camp, it handles Consistency and Partition Tolerance whilst forgoing Availability (Every request may not see a response), although MongoDB still provides some great availability options.

A Keepalived VIP for Eucalyptus Node Controllers

Background

In a Eucalyptus HA configuration there are two Cluster Controllers’s (CC) which are in an active-passive state. One is in “ENABLED” mode and one is “DISABLED” mode. If a failure occurs, the active CC services moves to the secondary CC system.

If you combine this with Eucalyptus MANAGED or MANAGED-NOVLAN networking configuration with a private back-end network your Node Controllers (NC) will require a default gateway for access to external networks and to the Walrus service to download Eucalyptus Machine Images.

Eucalyptus Sosreport Plugin

After some discussion on the Eucalyptus Community Mailing list, I’m happy to annouce I’ve had a crack at a python based plugin for sosreport.

Sosreport is a cool support data collection utility written in python and availble for RHEL/CentOS, Fedora, Ubuntu & Debian. It can collect lots of useful system files and stats that you might use to debug a server or send to a support organisation to help you.

I’ve packaged up my plugin for review by the community before I submit upstream to the sosreport project.

Monitoring Eucalyptus IaaS Availability With Nagios

This article describes one way to monitor your Eucalyptus cloud with Eutester, a framework for testing clouds and Nagios, a popular open source infrastructure monitoring tool. I’ve also posted this over on the Eucalyptus wiki for future reference.

We use a simple instance testcase from Eutester to spin up an instance in Eucalyptus with a custom ssh key and security group, ping the instance, ssh into it and then terminate the instance. We then report to Nagios on the overall success of the test.

It’s Alive!

I’ve decided to bring back my blog from near death.

I’ve had a blog for the past 10 years with very little content, so time to publish some things I’ve been working on.

I have a ton of small articles written as notes, I’ll try to write some of these up as I go along. For some reason I’ve never managed to get around to posting them online. I blame actually working with customers to solve problems.

This blog is powered by Octopress and hosted on Amazon S3 storage as I like cloudy services. I’ve even moved my DNS over to Route 53.

There are plently of docs out there if you are interested…

Lucid Cluster Testing

In the past I’ve worked quite a lot with Red Hat Cluster Suite and other Linux based active/passive failover clustering technologies. As technology has moved on, so have the available options for cluster software. Pacemaker + Corosync are two of these fantastic technologies.

Ante Karamatić recently sent a mail to the Ubuntu server list, to ask for help testing the latest packages… If you are interested in cluster technologies in Ubuntu then this is a great opportunity to get involved, all instructions are provided on the wiki page so get testing!

See the UDS Lucid blueprint for the cluster stack for more info on the progress: https://blueprints.edge.launchpad.net/ubuntu/+spec/server-lucid-cluster-stack