Friday, October 29, 2010

Ubuntu Cloud Community needs You

"I'm interested in Ubuntu and the cloud, how do I get involved" is a question I got a few times already. I thought it would be a good idea to answer this as a blog post. I believe one of the very first things you'd want to do, is to make sure you're on the main communication channels, talking to the community, asking questions, seeing other questions being answered, trying to answer some yourself, sharing opinions and generally "connecting" with the rest of the community. That is a great first step. So I'll highlight the main communication venues for the Ubuntu cloud community, as well as way to get kick-started.

Places to be
  • Ubuntu Cloud Forums, while pretty young, there has been some pretty good stir in the forums. While IRC and mailing lists may be more focused on "asking questions", the Forums are a great way to get in touch with other community members. To share your experience building your private clouds, the hardware used, software configuration, tuning and optimization, challenges faced ...etc. Come join in, if you would like to ask questions, or if you would like to share opinions, tips or tricks, get on the forums and make some splash :)
  • The Ubuntu-Cloud mailing list is a great technical resource where most of the experts and developers are subscribed. For very technical discussions, questions, feature suggestions, RFEs, development discussions the mailing list is a great resource.
  • The EC2Ubuntu mailing list is a great resource that focuses on running Ubuntu in the Amazon EC2 public cloud. This list is active with a wealth of info on the topic
  • IRC chat has long always been a primary real-time communication tool used by free software enthusiasts. The Ubuntu cloud IRC room is (surprise, surprise) #ubuntu-cloud on Freenode. Jump in, and engage
Once connected, things you can do include playing with the latest technology such as creating yourself a private UEC cloud, verifying latest features work as advertised, report and fix bugs, suggest features, design and implement new projects to advance the state of Ubuntu on the cloud. While the community is very welcoming, I definitely understand we need to create better new-comer friendly engagement paths, more hand-holding if you will. A better mentoring program from senior members as well as low hanging fruit are things the Ubuntu cloud and server communities need to identify and improve to make it easier to attract and engage fresh talent

Wednesday, October 27, 2010

PointnClick guide to running Ubuntu in the cloud

My previous post about running uec on ec2, drew some comments as being a fairly complex process. That may be true because essentially you're attempting to hack together a configuration which is not entirely supported. Anyway, that led me to want to demo a "point-n-clink" guide to running your first Ubuntu server in the EC2 cloud. i.e. no command line allowed, just point and click :) It doesn't get any easier than this, so let's hit it

Assuming you have setup an account with Amazon and could login to the Amazon console, you should see
pointnclick-guide-ubuntinthecloud-1

Before we begin, let's set ourselves a "key-pair". Which is basically a ssh public/private key-pair that enables you to login to your instance once launched. Click on "Key Pairs" and click "Create key pair", I'm gonna name my key "ubuntu" and click create

pointnclick-guide-ubuntinthecloud-2

The key is promptly created and pushed for download through your browser. Let's proceed with another "preparatory" setup. By default, EC2's firewall denies all inbound traffic to the instance, which means you cannot even ssh into your instance. Let's open ports 22 for ssh, and 80 just in case we wanna test by running an apache2 server. So, add your from-to ports and source IPs as per the next screenshot and "Save" it

pointnclick-guide-ubuntinthecloud-3

Great, now we're ready to actually launch an instance. Click "EC2 Dashboard" link, click the button "Launch Instance", a wizard starts asking for which AMI we would like to use. An AMI is a template that will be copied to your instance and used to start it. Click "Community AMIs", it may take a moment for the AMIs to load. Now here's a trick, to quickly "zoom-in" on the official ubuntu AMIs, use the following search string "ubuntu-images/" in order to locate EBS based images, and "ubuntu-images-us/" in order to locate instance store based images. This is in no way a "supported" feature of neither Amazon nor Ubuntu, it's just a convenient hack that works today and may not work tomorrow. We choose an EBS based AMI because we plan on launching a micro instance and those require EBS AMIs

pointnclick-guide-ubuntinthecloud-4

click the "Select" button beside it. Click continue and choose a t1.micro instance and click continue a couple of times

pointnclick-guide-ubuntinthecloud-5

pointnclick-guide-ubuntinthecloud-6

pointnclick-guide-ubuntinthecloud-7

You're now on the key-pair page, we simply choose our previously generated "Ubuntu" key pair and click continue

pointnclick-guide-ubuntinthecloud-8

On the firewall configuration page, we choose the "default" security group, since this is what we had configured to open ports 22 and 80 previously

pointnclick-guide-ubuntinthecloud-9

Voila, we're ready. Review and confirm the settings on the page and click "Launch"

pointnclick-guide-ubuntinthecloud-10

The cloud starts deploying your virtual server and you get the following message

pointnclick-guide-ubuntinthecloud-11

In a minute your instance should be up and running. Let's locate it and login to it. Clicking the "Instances" link, then clicking our only instance so far and locating its "Public DNS" entry setting allows us to ssh to it

pointnclick-guide-ubuntinthecloud-12

Now we're all set, let's jump to our terminal to ssh into the instance. It seems the "ubuntu.pem" key-pair the browser downloaded gets by default permissions that are too open for ssh's taste. Thus we need to "chmod" it to 700, this is the part where I lied about not using any CLI, but hey chmod doesn't really count ;) let's then ssh straight into our instance using the ubuntu.pem key-pair

pointnclick-guide-ubuntinthecloud-13

Awesome! We're in! let's do what the greeting message tells us to "sudo tasksel --section server" and choose to install a LAMP Server. We get asked for a MySQL password twice, the system installs and configures a full LAMP stack for us. Let's visit the apache web server

pointnclick-guide-ubuntinthecloud-14

That concludes this graphical guide. As you can see launching your first Ubuntu server instance in the cloud can't really get much easier than this :) Before you go for a cup of coffee, do not forget to "Terminate" your instance. If you don't, you keep on getting billed by the hour for it.

pointnclick-guide-ubuntinthecloud-15

Let me know in the comments if a step was unclear. Also, let me know if there are other topics you'd want me to demo. If you're interested in running Ubuntu in a cloud context and have any doubts, drop by on IRC channel #ubuntu-cloud on freenode, and grab me "kim0"

Friday, October 22, 2010

Cloud on Cloud, UEC on EC2

So you wanted to play with Ubuntu Enterprise Cloud (UEC), but didn't have a couple of machines to play with ? Want to start a UEC instance right now, no problem. You can use an Amazon EC2 server instance as your base server to install and run UEC on! Of course the EC2 instance is itself a virtual machine, thus running a VM inside that would require nested virtualization which AFAIK wouldn't work over EC2. The trick here is that we switch UEC's hypervisor temporarily to be qemu. Of course this won't win any performance competitions, in fact it'd be quite slow in production, but for playing, it fits the bill just fine.

If you're thinking doing all that is gonna be complex, you have a point, however it won't! In fact it'll be very easy due to efforts of Ubuntu's always awesome Scott Moser. Scott has written a script that automates all needed steps. But wait, it gets better, we're not even going to run this script ourselves, we're passing it as a parameter to the EC2 instance invocation and due to Ubuntu's cloud-init technology, that script is going to be run upon instance boot-up, doing its work automagically. Now let's get started

On your local machine, let's install bzr and get the needed script

cd /tmp
sudo apt-get install bzr -y
bzr branch lp:~smoser/+junk/uec-on-ec2
cd uec-on-ec2/

The file "maverick-commands.txt" contains the script needed to turn the generic Ubuntu image on ec2 into a fully operational single-node UEC install. If you don't have "ssh keys" (seriously?) generate some

ssh-keygen -t rsa

Now let's do a neat trick to import the keys into EC2, marking them with the name "default"

for r in us-east-1 us-west-1 ap-southeast-1 eu-west-1; do ec2-import-keypair --region $r default --public-key-file ~/.ssh/id_rsa.pub ; done

Let's open a few needed ports in EC2's default security group

for port in 22 80 8443 8773; do ec2-authorize default -p $port ; done

Very well .. We now launch our EC2 instance, passing in the "maverick commands" file. What happens is, the server instance is created, booted, Ubuntu's cloud-init reads up the maverick commands script we passed to it, and executes it, it starts downloading, installing and configuring UEC in the background while you ssh into your new EC2 instance

ec2-run-instances ami-688c7801 --instance-type m1.large -k default --user-data-file=maverick-commands.txt

Give it a minute to boot, then try to ssh in
ec2-describe-instances
ssh ubuntu@ec2-a-b-c-d.compute-1.amazonaws.com

Replace the DNS name for the EC2 instance, with the proper one you get from ec2-describe-instances. Once logged into the EC2 instance, I would start byobu and tail the log file to monitor progress. The whole thing takes less than 5 minutes

byobu
tailf uec-setup.log

The script once finished configuring UEC, actually downloads a tiny linux distro and registers its image in UEC, so that you can start your own instances! You know the script has finished when you see a line that looks like
emi="emi-FDC21818"; eri="eri-53721963"; eki="eki-740D19EC";

UEC is now up and running, let's check the web interface! You login with the default credentials admin/admin

uec-on-ec2-1

Let's navigate to the Images tab, to get the EMI ID (Equivalent of an AMI ID)

uec-on-ec2-2

Is that cool or what, Hell Yes! Now let's start our own VM inside UEC that's inside EC2. Remeber to replace emi-FE03181A with the EMI ID you got from the web interface

euca-run-instances --key mykey --addressing private emi-FE03181A

You can use "euca-describe-instances" to get the new instance internal IP address and ssh to that

ubuntu@domU-12-31-38-01-85-91:~$ ssh -i euca/mykey.pem ubuntu@172.19.1.2
The authenticity of host '172.19.1.2 (172.19.1.2)' can't be established.
RSA key fingerprint is db:9b:47:a4:06:81:26:d7:cf:38:a4:0e:6c:05:54:0d.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.19.1.2' (RSA) to the list of known hosts.

Chop wood, carry water.

$ uname -r
2.6.35-16-virtual
$ df -h                                                                                                                                                                                                                                     
Filesystem                Size      Used Available Use% Mounted on
/dev/sda1                23.2M     14.1M      7.9M  64% /
tmpfs                    24.0K         0     24.0K   0% /dev/shm

et voila, you're ssh'ed into a ttylinux instance running inside qemu managed by UEC running over EC2 :) If you do find that cool, what about contributing back. How about you start hacking on that script to make it even more awesome, such as maybe installing over multiple nodes or whatever crazy idea you can think of! If you're interested to start hacking, drop me a hi in #ubuntu-cloud at Freenode

Free Ubuntu Server for a year at Amazon

Yes, you can get your very own Free Ubuntu server in the clouds for one full year! The folks at Amazon have just announced

"Beginning November 1, new AWS customers will be able to run a free Amazon EC2 Micro Instance for a year, while also leveraging a new free usage tier for Amazon S3, Amazon Elastic Block Store, Amazon Elastic Load Balancing, and AWS data transfer. AWS’s free usage tier can be used for anything you want to run in the cloud: launch new applications, test existing applications in the cloud, or simply gain hands-on experience with AWS"

You can find the details of the offer at: http://aws.amazon.com/free/

Ubuntu being arguably the most popular Amazon guest image is also available to you for free today! Get the list of Official Ubuntu images created by the Ubuntu's very own server team at

Maverick http://uec-images.ubuntu.com/server/maverick/current/
Lucid http://uec-images.ubuntu.com/server/lucid/current/

For the free offer, you will want to launch a t1.micro instance, and it seems you will want to wait till Nov 1st for registering your account (credit card needed). This is great news! If you ever wanted to play with Ubuntu server on the cloud, now is the best time to get started!

Update: The terms page mentions "Only accounts created after October 20, 2010 are eligible for the Offer"

Update2: Please read Scott's comment below, on why you will be charged 0.5$/mo if you run the standard Ubuntu image (or Amazon's own AMIs)

Hello Planet-Ubuntu

Hello World! I just became an Ubuntu member yesterday, which means I will no longer be on mute ;) I'm working as the Ubuntu Cloud community liaison, which means I finally bring some cloud and server content to this space instead of all being around fluffy desktops :) If you're interested in using or contributing to Ubuntu in a "cloud" context, feel free to grab me (kim0 on Freenode #ubuntu-cloud). Feel free to drop in, say hi, ask questions or whatever you need. Besides this blog, you may also be interested in my tweets. I'm very proud being part of this great community. Rock on!

Friday, October 15, 2010

OpenWeek Introduction to Cloud IRC session logs

If you couldn't attend the Ubuntu Open Week session live on IRC, don't despair! You can read the logs at
http://irclogs.ubuntu.com/2010/10/13/%23ubuntu-classroom.html#t17:01