Let's say you have managed to get an instance running SQL Server 2005 on a Windows 2003 operating system up and running. You are running concurrent CRUD operations against a database, but you decide you need to install a service package and restart the instance. That should be no problem right?
Wrong!
When you terminate a running instance, any data changes made during the session are lost unless they were saved on Amazon Elastic Block Store (henceforth referred to as EBS). The only way to save any data on your instance that is not stored on EBS is by bundling the image. Bundling an image means saving the instance and it's current configuration to Amazons's Simple Storage Service (henceforth referred to as S3). Typically what you would do is to create your image, test it and save it again as part of an iterative process moving towards perfection. Once all the necessary applications and configuration settings have been made and you have made sure everything works, you should only need to save production data, and that should be stored on an EBS volume. One of the biggest advantages with EBS volumes is that if your image becomes corrupt, you can simply attach the EBS volume to a new image and retrieve the data. You can also create snapshots of EBS volumes.
The easiest way to create and save an image is to launch one of the images in Amazon's public image directory, install any additional software and service packages, then bundle it using the management console or Elasticfox, unless you are John Wayne or somebody who insists on doing it the hard way by using Amazon's command line tools. The image you bundle can be registered as either private or public. A public image can be accessed and launched by anyone with an AWS account. You can also create images on your own machine and upload to Amazon. In this posting however, we will focus on creating new images from running instances initalized using existing images already registered with AWS.
To create an image from a running instance using Elasticfox:
1. Select the 'Instances' tab
2. Right click the instance and select the 'Bundle into an AMI' option.
3. Type a bucket name and an image name for your S3 folder (a new bucket will be created if a bucket with the specified name does not exist, then click ok.
4.Click the 'Bundle Task' tab and you will see the progress of the bundle task you just created. It can take a while so be patient. Bundling actually consists of two steps, bundling or packaging the image, then saving it to S3. The progress will be displayed in the Bundle State column. Once the process is finished the Bundle State column will display as 'complete'.
5. Once the 'complete' status is reached, you need to right click the image file in the Bundle Tasks view and select 'Register AMI' to register it before you can use it. Adding a descriptive tag is optional but useful if you need to keep track of a lot of images.
6. The image you created is now registered and will appear among your images with the Visibility value set to 'Private'.
7. To launch the image you created, right click and select the 'Launch instance(s) of this AMI' option.
As mentioned above you need to have an EBS volume to persist data between sessions. So how do you create an EBS volume?
To create an EBS volume using Elasticfox:
1. Click on the Volumes and Snapshots tab, then click the add(+) button to create your EBS volume.
2.Type desired size (anything between 1 GB and 1 TB), add a tag, click create, and Bob's yer uncle.
Attaching the EBS volume is even simpler:
1.In Elasticfox, right click your running instance and select 'Attach an EBS volume'.
2.In the popup, select the id of the volume you want to attach and click 'Attach'.
Assuming that you attached the EBS volume to a Windows instance, all you need to do now is to log on to your running image, then go to Administrative Tools, select Computer Management and format the volume. Note that your instance and your image must be running in the same availability zone, otherwise you will get an error.
The C# library also offers functions for handling EBS volumes. The following code example shows how to create an EBS volume and attach it to a running instance (for an example on how to launch an instance, see my previous post Initializing and Controlling your instance on EC2):
First of all, make sure you are running your instance in the same availability zone
as the EBS volume. If the volume is located in the us-east-1a zone, your instance must be launched there as well by setting the Placement property for an instance of the RunInstancesRequest class:
Placement p = new Placement();
p.AvailabilityZone = "us-east-1a";
request.Placement = p;
Assuming that we have launched the instance successfully, we can now attach our EBS volume (for initialization of the service class see Initializing and Controlling your instance on EC2):
AttachVolumeRequest request = new AttachVolumeRequest();
request.VolumeId = "vol-e38d6c8a";
request.InstanceId = "i-a05e3bc9";
request.Device = "xvdg";
AttachVolumeResponse response = service.AttachVolume(request);
If the request was successful, the response.Attachment.Status property will be set to 'attaching'. You can also check the result by opening Elasticfox and clicking the Volumes and Snaphots tab and checking the Attachment Status field of the EBS volume specified in the function call. To monitor your instances and volumes use the DescribeInstancesRequest and DescribeVolumesRequest classes, but more about that some other time. Sampai nanti as they say in Indonesian.
Monday, March 23, 2009
Wednesday, March 18, 2009
Initializing and controlling your own instance on EC2
When it comes to cloud computing, there are two paths you can go by as Robert Plant sang in 1971: One is SaaS, and the other is virtualization. Amazon's EC2 offering uses the latter. A virtual machine is a software implementation of a computer that executes programs like a real machine. Amazon and it's two competitors GoGrid and Flexiscale all use Xen,an open source virtual machine monitor. This reduces the risk for vendor lock-in and a customer can also download his image and run it in his own Xen environment. This posting focuses on how to get started with Amazon EC2.
First of all you need to open an account, if you have ever bought something from Amazon you are already halfway there since Amazon already have the billing info, otherwise you need to sign up for AWS (Amazon Web Services) which includes EC2 and a bunch of other stuff that I will write about in another posting. Secondly, you need to sign up for the EC2 service, and thirdly you need to obtain a private and a public key or a X.509 certificate which both are provided by Amazon. In the case of the latter you can also bring your own certificate.
Once you have finished registration you are ready to launch and control your own instance using one or several of the tools provided by Amazon:
For those who want to avoid installing non-Microsoft products, the best alternative is of course the EC2 console but I prefer using Elasticfox which is a plugin for Firefox although the difference in functionality between the two is rather insignificant. For those of you who are into self-flagellation, download the command line tool. Here's how you launch an instance with Elasticfox:
Amazon also provides libraries for several programming languages. In this post I will show how to launch and terminate images using the C# Library for EC2. The library is using the .NET 2.0 framework and wraps the Amazon APIs. The C# library includes mock-up functionality which simulates responses from the Amazon EC2 service. The mock-up is helpful if you want to make sure your code does not contain bugs before you go live.
To initialize two instances of the same image, in Visual Studio 2005 or 2008, set a reference to the C# Amazon.EC2 library and add the following using statements:
......
using Amazon.EC2;
using Amazon.EC2.Model;
......
Initialize an instance of the AmazonEC2 class passing the access key and the secret access key as string variables:
AmazonEC2 service = new AmazonEC2Client(accessKeyId, secretAccessKey);
Initialize an instance of the RunInstanceRequest class:
......
RunInstancesRequest request = new RunInstancesRequest();
// set request parameters here
request.ImageId = "your image id goes here";
//Set allowed number of instances:
request.MinCount = 1;
request.MaxCount = 2;
//If we are using SQL Server and Windows 2003 we need the
//m1.large instance type:
request.InstanceType = "m1.large";
RunInstancesResponse response = service.RunInstances(request);
....
In order to stop an instance you simply create an instance of the TerminateInstancesRequest class:
TerminateInstancesRequest request = new TerminateInstancesRequest();
//Set the id to the instance you want to terminate, InstanceID is a collection
//so you can pass more than one instance id:
request.InstanceId.Add("your instance id goes here");
TerminateInstancesResponse response = service.TerminateInstances(request);
The code examples above just show a fraction of all the things you can do with the EC2 API. Remember that bugs can be costly when working with EC2, you don't want to be charged for starting more instances than you actually planned. Sample code can be found here.
First of all you need to open an account, if you have ever bought something from Amazon you are already halfway there since Amazon already have the billing info, otherwise you need to sign up for AWS (Amazon Web Services) which includes EC2 and a bunch of other stuff that I will write about in another posting. Secondly, you need to sign up for the EC2 service, and thirdly you need to obtain a private and a public key or a X.509 certificate which both are provided by Amazon. In the case of the latter you can also bring your own certificate.
Once you have finished registration you are ready to launch and control your own instance using one or several of the tools provided by Amazon:
For those who want to avoid installing non-Microsoft products, the best alternative is of course the EC2 console but I prefer using Elasticfox which is a plugin for Firefox although the difference in functionality between the two is rather insignificant. For those of you who are into self-flagellation, download the command line tool. Here's how you launch an instance with Elasticfox:
Amazon also provides libraries for several programming languages. In this post I will show how to launch and terminate images using the C# Library for EC2. The library is using the .NET 2.0 framework and wraps the Amazon APIs. The C# library includes mock-up functionality which simulates responses from the Amazon EC2 service. The mock-up is helpful if you want to make sure your code does not contain bugs before you go live.
To initialize two instances of the same image, in Visual Studio 2005 or 2008, set a reference to the C# Amazon.EC2 library and add the following using statements:
......
using Amazon.EC2;
using Amazon.EC2.Model;
......
Initialize an instance of the AmazonEC2 class passing the access key and the secret access key as string variables:
AmazonEC2 service = new AmazonEC2Client(accessKeyId, secretAccessKey);
Initialize an instance of the RunInstanceRequest class:
......
RunInstancesRequest request = new RunInstancesRequest();
// set request parameters here
request.ImageId = "your image id goes here";
//Set allowed number of instances:
request.MinCount = 1;
request.MaxCount = 2;
//If we are using SQL Server and Windows 2003 we need the
//m1.large instance type:
request.InstanceType = "m1.large";
RunInstancesResponse response = service.RunInstances(request);
....
In order to stop an instance you simply create an instance of the TerminateInstancesRequest class:
TerminateInstancesRequest request = new TerminateInstancesRequest();
//Set the id to the instance you want to terminate, InstanceID is a collection
//so you can pass more than one instance id:
request.InstanceId.Add("your instance id goes here");
TerminateInstancesResponse response = service.TerminateInstances(request);
The code examples above just show a fraction of all the things you can do with the EC2 API. Remember that bugs can be costly when working with EC2, you don't want to be charged for starting more instances than you actually planned. Sample code can be found here.
Tuesday, March 10, 2009
The creative destruction unleashed by cloud computing
What can we learn from history? A lot actually. At the outset of WWI the opposing armies were all led by men who favoured using the manouevrability of cavalry to achieve decisive victories instead of fighting costly wars of attrition. Technological innovations forced them to rethink their strategy completely as the armies became bogged down in trench warfare.
When WWII broke out, the French and the British thought they would be fighting WWI all over again. Trying to avoid the heavy casualties of the fighting in the trenches the French built a line of defensive positions with fixed artillery and concrete bunkers along the German border called the Maginot Line. But the strategic thinking of German general Heinz Guderian which emphasized making the most of modern telecommunications and the mobility and agility of Germany's Panzer forces changed things completely. All the costly investments the French put into the Maginot Line proved futile as the German armored divisions simply circumvented it by making a detour trough Belgium.
Cloud computing is similar to the German armor divisions that conquered France, because just like armor, cloud computing is a resource that you can concentrate where and when it is needed and thus fits hand in glove with the lean enterprise philosophy. Traditional computing is more like the Maginot Line, massive resources tied up in fixed positions waiting to handle a crisis that might or might not occur. Businesses who remain in their bunkers will allow their competitors to reap the benefits.
So how do we define cloud computing? The narrow definition is the serving up of data and applications in the same way we plug a electric cord into the wall. At the end of each month or quarter there will be a bill charging us for the usage. The application and the data can be served by either a software as a service solution or a virtual server.
Nicholas Carr compares the evolution of Cloud Computing with the emergence of large-scale electric utilities during the first half of the 20th century in his book The Big Switch and how this affected manufacturing businesses. The early 20th century manufacturers adopted the new electric grid technology out of economic necessity as the utilities achieved economies of scale that individual manufacturers producing their own electricity could not match. Carr argues that the economic forces that drove businesses to swap their own electrical power sources for the new electric grid, are the same forces that will drive the businesses of today to swap their own servers for cloud computing.
So what are the economic benefits of cloud computing from a business perspective? Among the more important ones are cost reduction and time to market,the benefits of a utility payment model, and resilience.
Economies of scale allow cloud computing utility companies to lower costs.
Time to market (TTM) is shortened because companies will spend less time on procurement of hardware. You just need to sign up with a vendor and you have all the computing power you need at your disposal.
The utility payment model used by companies such as Amazon means that you will only pay for the services you use, not for excess capacity.
Resilience, cloud computing allows you to scale up your application platform in a way that is not possible using a traditional hardware setup or a hosting solution. An example: Using Amazon's Elastic Compute Cloud (EC2) Animoto, were able to scale up from 50 EC2 servers to 3500 in a week. This means that it is easy for companies to handle fast growth or peaks in demand but, as Amazon's founder Jeff Bezos put it: "Don't try this at home".
What are the drawbacks from a business perspective? The ones that immediately spring to mind are lack of maturity, lack of applications, and lack of subscriber competence.
Lack of maturity: Cloud computing is still in its infancy. Even though many companies profess an interest in the benefits, few have any experience.
Lack of applications: Most existing software applications have been designed with traditional desktop or server-client environments in mind. Even if traditional applications can be run on virtual servers in the cloud, a paradigm shift is required to fully harness the power of cloud computing.
Lack of subscriber competence: Even companies that would reap benefits trough the implementation of cloud computing are vacillating due to a perceived sense of loss of control and a perceived security threat.
So what does the future hold?
The current economic downturn is affecting the entire IT industry and although I think it is reasonable to expect fewer IT-related bankruptcies than during the dot bomb crisis, I expect to see cost cutting, lower hourly rates for consultants and staff reductions. But this gloom aside, I hope that the the crisis will drive technological change, particularly when it comes to the adoption of cloud computing.
The businesses most likely to adopt revolutionary cloud computing strategies are start ups and although starting an IT-business has never been cheaper, it is quite possible that the lack of venture capital will be a millstone around the neck of many innovators. Even so I still believe that the technology is here to stay and we better get used to it.
Cloud computing is a formidable force of creative destruction that in the next five years will to do traditional client-server computing what the effective use of armor did to the Maginot line. This means that players like Microsoft, Oracle, and SAP will have to catch up with the likes of Google, Zoho, GoGrid, Salesforce and Amazon. The question is, will they succeed?
When WWII broke out, the French and the British thought they would be fighting WWI all over again. Trying to avoid the heavy casualties of the fighting in the trenches the French built a line of defensive positions with fixed artillery and concrete bunkers along the German border called the Maginot Line. But the strategic thinking of German general Heinz Guderian which emphasized making the most of modern telecommunications and the mobility and agility of Germany's Panzer forces changed things completely. All the costly investments the French put into the Maginot Line proved futile as the German armored divisions simply circumvented it by making a detour trough Belgium.
Cloud computing is similar to the German armor divisions that conquered France, because just like armor, cloud computing is a resource that you can concentrate where and when it is needed and thus fits hand in glove with the lean enterprise philosophy. Traditional computing is more like the Maginot Line, massive resources tied up in fixed positions waiting to handle a crisis that might or might not occur. Businesses who remain in their bunkers will allow their competitors to reap the benefits.
So how do we define cloud computing? The narrow definition is the serving up of data and applications in the same way we plug a electric cord into the wall. At the end of each month or quarter there will be a bill charging us for the usage. The application and the data can be served by either a software as a service solution or a virtual server.
Nicholas Carr compares the evolution of Cloud Computing with the emergence of large-scale electric utilities during the first half of the 20th century in his book The Big Switch and how this affected manufacturing businesses. The early 20th century manufacturers adopted the new electric grid technology out of economic necessity as the utilities achieved economies of scale that individual manufacturers producing their own electricity could not match. Carr argues that the economic forces that drove businesses to swap their own electrical power sources for the new electric grid, are the same forces that will drive the businesses of today to swap their own servers for cloud computing.
So what are the economic benefits of cloud computing from a business perspective? Among the more important ones are cost reduction and time to market,the benefits of a utility payment model, and resilience.
Economies of scale allow cloud computing utility companies to lower costs.
Time to market (TTM) is shortened because companies will spend less time on procurement of hardware. You just need to sign up with a vendor and you have all the computing power you need at your disposal.
The utility payment model used by companies such as Amazon means that you will only pay for the services you use, not for excess capacity.
Resilience, cloud computing allows you to scale up your application platform in a way that is not possible using a traditional hardware setup or a hosting solution. An example: Using Amazon's Elastic Compute Cloud (EC2) Animoto, were able to scale up from 50 EC2 servers to 3500 in a week. This means that it is easy for companies to handle fast growth or peaks in demand but, as Amazon's founder Jeff Bezos put it: "Don't try this at home".
What are the drawbacks from a business perspective? The ones that immediately spring to mind are lack of maturity, lack of applications, and lack of subscriber competence.
Lack of maturity: Cloud computing is still in its infancy. Even though many companies profess an interest in the benefits, few have any experience.
Lack of applications: Most existing software applications have been designed with traditional desktop or server-client environments in mind. Even if traditional applications can be run on virtual servers in the cloud, a paradigm shift is required to fully harness the power of cloud computing.
Lack of subscriber competence: Even companies that would reap benefits trough the implementation of cloud computing are vacillating due to a perceived sense of loss of control and a perceived security threat.
So what does the future hold?
The current economic downturn is affecting the entire IT industry and although I think it is reasonable to expect fewer IT-related bankruptcies than during the dot bomb crisis, I expect to see cost cutting, lower hourly rates for consultants and staff reductions. But this gloom aside, I hope that the the crisis will drive technological change, particularly when it comes to the adoption of cloud computing.
The businesses most likely to adopt revolutionary cloud computing strategies are start ups and although starting an IT-business has never been cheaper, it is quite possible that the lack of venture capital will be a millstone around the neck of many innovators. Even so I still believe that the technology is here to stay and we better get used to it.
Cloud computing is a formidable force of creative destruction that in the next five years will to do traditional client-server computing what the effective use of armor did to the Maginot line. This means that players like Microsoft, Oracle, and SAP will have to catch up with the likes of Google, Zoho, GoGrid, Salesforce and Amazon. The question is, will they succeed?
Subscribe to:
Posts (Atom)