Showing posts with label PROGRAMMING. Show all posts
Showing posts with label PROGRAMMING. Show all posts

Thursday, April 16, 2009

Programmatically monitoring your EC2 instances

The EC2 C# library provides several classes that allow you to monitor and pass commands to your running instances and volumes. Each request class is coupled with a response class that is returned as an instance when you make a functions call using an instance of the EC2Service class.

The EC2Service class is the controller trough which requests and responses are passed. Classes used to monitor running instances use the Describe prefix, such as the DescribeVolumesRequest or DescribeInstancesRequest classes. Note that both the DescribeInstancesRequest and the DescribeVolumesRequest class expect lists of IDs. This means that you can request status reports from multiple as well as single instances and volumes.

I have created a wrapper class that can be used for the following:

1. Initialize an EC2 instances using an existing AMI.
2. Terminate an EC2 instance using an existing AMI.
3. Attach an EBS volume.
4. Detach an EBS volume.
5. Monitor the EBS volume attachment process.
6. Monitor the EBS volume detachment process.
7. Monitor the EC2 instance startup process.
8. Monitor the EC2 instance closing process.

EC2 wrapper:



using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Amazon.EC2;
using Amazon.EC2.Model;

namespace consoleEBS_And_EC2
{
class EC2Wrapper
{

private AmazonEC2 service;
private string _instance;
public string instance { get { return _instance; } set { _instance = value; } }
private bool _isRunning = false;
public bool isRunning { get { return _isRunning; } set { _isRunning = value; } }
private bool _volAttached;
public bool volAttached { get { return _volAttached; } set { _volAttached = value;} }
private string _volID;
public string volID { get { return _volID ;} set { _volID = value;} }
private string _runReservationId;
public string runReservationId { get { return this._runReservationId; } set { this._runReservationId = value;} }
private bool _detaching = false;
public bool detaching { get { return _detaching;} set {_detaching = value;} }
private bool _attaching = false;
public bool attaching { get { return _attaching; } set { _attaching = value ;} }





public EC2Wrapper(string accessKeyId, string secretAccessKey,string amiId)
{
try
{
service = new AmazonEC2Client(accessKeyId, secretAccessKey);


runYourInstances(amiId);



}
catch (Exception ex)
{
//Call logger here, then throw user friendly message.
//throw new Exception("Your instance did not start. Check your eventlog for details.");
throw ex;
}
}

private void runYourInstances(string amiId)
{
RunInstancesRequest request = new RunInstancesRequest();
////Set allowed number of instances:
request.MinCount = 1;
request.MaxCount = 1;
request.InstanceType = "m1.small";
//request.
Placement p = new Placement();
p.AvailabilityZone = "us-east-1a";
request.Placement = p;
request.ImageId = amiId;
RunInstancesResponse risp = this.service.RunInstances(request);
this.runReservationId = risp.RunInstancesResult.Reservation.ReservationId;
this.instance = (from r in risp.RunInstancesResult.Reservation.RunningInstance where risp.RunInstancesResult.Reservation.ReservationId == this.runReservationId select r.InstanceId).FirstOrDefault();

}
/// <summary>
/// Monitors running request until it either the timeout has elapsed (12 minutes) or the status is running.
/// </summary>
/// <param name="id"></param>
/// <returns></returns>
public bool monitorStartingProcess()
{

int timeOutIterations = 0;
try
{

while (true)
{
RunningInstance ri = getRunningInstance();
if (ri != null)
{
this.isRunning = true;
break;
}
System.Threading.Thread.Sleep(1000);
timeOutIterations += 1;
if (timeOutIterations == 720)
break;

}
}
catch (Exception ex)
{
throw ex;
}
return this.isRunning;
}

private RunningInstance getRunningInstance()
{
DescribeInstancesRequest rs = new DescribeInstancesRequest();
rs.InstanceId = new List<string>() { this.instance };
DescribeInstancesResponse rsp = service.DescribeInstances(rs);
Reservation res = (from r in rsp.DescribeInstancesResult.Reservation where r.ReservationId == this.runReservationId select r).FirstOrDefault();
RunningInstance ri = (from r in res.RunningInstance where r.InstanceId == this.instance && r.InstanceState.Name == "running" select r).FirstOrDefault();

return ri;
}
/// <summary>
/// Terminates running instance.
/// </summary>
public void terminateRunning()
{
try
{
TerminateInstancesRequest rq = new TerminateInstancesRequest();
rq.InstanceId.Add(this.instance);
TerminateInstancesResponse rsp = this.service.TerminateInstances(rq);
this.isRunning = false;
}
catch (Exception ex)
{
throw ex;
}
}

public bool Attach_EBS_Volume(string id)
{
bool result = false;
AttachVolumeRequest vrq = new AttachVolumeRequest();
this.volID = id;
vrq.VolumeId = this.volID;
vrq.InstanceId = this.instance;
vrq.Device = "/dev/sda";
AttachVolumeResponse vrp = this.service.AttachVolume(vrq);
if (vrp.AttachVolumeResult.Attachment.Status == "attaching")
{
result = true;
this.attaching = true;
}
else
result = false;
return result;
}



public bool monitorVolumeAttachment()
{

int timeOutIterations = 0;
try
{

while (true)
{
DescribeVolumesRequest rs = new DescribeVolumesRequest();
rs.VolumeId = new List<string>() { this.volID };
DescribeVolumesResponse rsp = service.DescribeVolumes(rs);
Volume vol = (from v in rsp.DescribeVolumesResult.Volume where v.VolumeId == this.volID && v.Status == "in-use" select v).FirstOrDefault();
if (vol != null)
{
this.volAttached = true;
this.attaching = false;
break;
}
System.Threading.Thread.Sleep(1000);
timeOutIterations += 1;
if (timeOutIterations == 720)
break;

}
}
catch (Exception ex)
{
throw ex;
}
return this.isRunning;
}

public bool Detach_EBS_Volume()
{
bool result = false;
DetachVolumeRequest dvr = new DetachVolumeRequest();
dvr.InstanceId = this.instance;
dvr.Device = "/dev/sda";
dvr.Force = false;
dvr.VolumeId = this.volID;
DetachVolumeResponse dvs = this.service.DetachVolume(dvr);
if (dvs.DetachVolumeResult.Attachment.Status == "detaching")
{
result = true;
this.detaching = true;
}
else
result = false;
return result;
}


public bool monitorDetachVolumeRequest(string id)
{
int timeOutIterations = 0;
try
{

while (true)
{
DescribeVolumesRequest rs = new DescribeVolumesRequest();
rs.VolumeId = new List<string>() { id };
DescribeVolumesResponse rsp = service.DescribeVolumes(rs);
Volume vol = (from v in rsp.DescribeVolumesResult.Volume where v.Status == "available" && v.VolumeId == id select v).FirstOrDefault();
if (vol != null)
{
this.volAttached = false;
this.detaching = false;
break;
}
System.Threading.Thread.Sleep(1000);
timeOutIterations += 1;
if (timeOutIterations == 720)
break;

}
}
catch (Exception ex)
{
throw ex;
}
return this.volAttached;
}

public static bool isInstanceRunning(string id,string accesskey,string secretkey)
{
bool result = false;
AmazonEC2 ae = new AmazonEC2Client(accesskey,secretkey);
DescribeInstancesRequest rs = new DescribeInstancesRequest();
rs.InstanceId = new List<string>() { id };

DescribeInstancesResponse rsp = ae.DescribeInstances(rs);


var res = (from r in rsp.DescribeInstancesResult.Reservation select new { state =
(from ri in r.RunningInstance where ri.InstanceId == id && ri.InstanceState.Name == "running" select ri.InstanceState.Name.ToString()).FirstOrDefault()});
foreach (var inst in res)
if ((string) inst.state == "running")
result = true;
return result;

}
}
}


Invocation of wrapper:



using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace consoleEBS_And_EC2
{
class Program
{

static void Main(string[] args)
{

//Ubuntu 8.10 image, can use the small instance type which will save us money.
String ami = "ami-7dfd1a14";
string EBS_id = null;
try
{

Console.WriteLine("Type accessKeyId: ");
string accessKeyId = Console.ReadLine();
Console.WriteLine("Type secretAccessKey: ");
string secretAccessKey = Console.ReadLine();
EC2Wrapper wrp = new EC2Wrapper(accessKeyId, secretAccessKey, ami);
Console.WriteLine("Instance start request sent");
loopDescribeInstance(wrp, ami);
if (wrp.isRunning)
{
Console.WriteLine("Type the Id of the EBS volume you want to attach: ");
EBS_id = Console.ReadLine();
if (wrp.Attach_EBS_Volume(EBS_id))
{
Console.WriteLine("Volume attach request sent");
if (wrp.monitorVolumeAttachment())
{
Console.WriteLine("Volume attached successfully");
System.Threading.Thread.Sleep(20000);
if (wrp.Detach_EBS_Volume())
{
Console.WriteLine("Volume detached request sent");
if (!wrp.monitorDetachVolumeRequest(EBS_id))
{
Console.WriteLine("Volume detached successfully");
}
else
{
Console.WriteLine("Detach operation failed");
}
}
else
{
Console.WriteLine("Detach request failed");
}
}
else
{
Console.WriteLine("Attach operation timed out");
}
}
else
{
Console.WriteLine("Volume attach request failed");
}
}

Stop(wrp);
Console.ReadLine();
}
catch (Amazon.EC2.AmazonEC2Exception aex)
{
Console.WriteLine(aex.ToString());
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}

}

static void loopDescribeInstance(EC2Wrapper w, string id)
{

try
{

if (w.monitorStartingProcess())
{
Console.WriteLine("Instance started");

}
else
{
Console.WriteLine("Instance not started");

}

}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}

}

static void Stop(EC2Wrapper srvc)
{
srvc.terminateRunning();
Console.WriteLine("Running instance stopped");
}

}
}


Note that the EC2Wrapper class only allows us to start one instance, and that it has two properties called attaching or detaching which are used to indicate whether an instance is in the process of attaching or detaching an EBS volume. Once the volume is attached or detached, we leave the transitional state and the attaching and detaching properties are set to 'false'.

A drawback to the EC2 C# library is that it only contains one exception class, Amazon.EC2.AmazonEC2Exception, but the error codes of this class are fairly informative so it would be totally feasible to trap AmazonEC2Exceptions, read the error code and then throw our own custom exceptions based on the error code.

Wednesday, March 18, 2009

Initializing and controlling your own instance on EC2

When it comes to cloud computing, there are two paths you can go by as Robert Plant sang in 1971: One is SaaS, and the other is virtualization. Amazon's EC2 offering uses the latter. A virtual machine is a software implementation of a computer that executes programs like a real machine. Amazon and it's two competitors GoGrid and Flexiscale all use Xen,an open source virtual machine monitor. This reduces the risk for vendor lock-in and a customer can also download his image and run it in his own Xen environment. This posting focuses on how to get started with Amazon EC2.

First of all you need to open an account, if you have ever bought something from Amazon you are already halfway there since Amazon already have the billing info, otherwise you need to sign up for AWS (Amazon Web Services) which includes EC2 and a bunch of other stuff that I will write about in another posting. Secondly, you need to sign up for the EC2 service, and thirdly you need to obtain a private and a public key or a X.509 certificate which both are provided by Amazon. In the case of the latter you can also bring your own certificate.

Once you have finished registration you are ready to launch and control your own instance using one or several of the tools provided by Amazon:

For those who want to avoid installing non-Microsoft products, the best alternative is of course the EC2 console but I prefer using Elasticfox which is a plugin for Firefox although the difference in functionality between the two is rather insignificant. For those of you who are into self-flagellation, download the command line tool. Here's how you launch an instance with Elasticfox:



Amazon also provides libraries for several programming languages. In this post I will show how to launch and terminate images using the C# Library for EC2. The library is using the .NET 2.0 framework and wraps the Amazon APIs. The C# library includes mock-up functionality which simulates responses from the Amazon EC2 service. The mock-up is helpful if you want to make sure your code does not contain bugs before you go live.

To initialize two instances of the same image, in Visual Studio 2005 or 2008, set a reference to the C# Amazon.EC2 library and add the following using statements:
......
using Amazon.EC2;
using Amazon.EC2.Model;
......
Initialize an instance of the AmazonEC2 class passing the access key and the secret access key as string variables:

AmazonEC2 service = new AmazonEC2Client(accessKeyId, secretAccessKey);

Initialize an instance of the RunInstanceRequest class:
......
RunInstancesRequest request = new RunInstancesRequest();
// set request parameters here
request.ImageId = "your image id goes here";
//Set allowed number of instances:
request.MinCount = 1;
request.MaxCount = 2;
//If we are using SQL Server and Windows 2003 we need the
//m1.large instance type:
request.InstanceType = "m1.large";
RunInstancesResponse response = service.RunInstances(request);
....

In order to stop an instance you simply create an instance of the TerminateInstancesRequest class:
TerminateInstancesRequest request = new TerminateInstancesRequest();
//Set the id to the instance you want to terminate, InstanceID is a collection
//so you can pass more than one instance id:
request.InstanceId.Add("your instance id goes here");
TerminateInstancesResponse response = service.TerminateInstances(request);

The code examples above just show a fraction of all the things you can do with the EC2 API. Remember that bugs can be costly when working with EC2, you don't want to be charged for starting more instances than you actually planned. Sample code can be found here.