thinking sysadmin » amazon aws

Links, 9/18/2008

Andy — Thu, 18 Sep 2008 20:52:07 +0000

We’re Never Content – Amazon announces a forthcoming CDN layered on top of S3 with “edge locations on three continents” – presumably North America, Europe and Asia – “in order to deliver your content from the most appropriate location.” Presumably Amazon is planning to use this in-house for their digital media sales, or possibly for static content on their website.
Tape, Roman Chariots and Data Management – “But here’s where it gets insidious, we know look at the mess that tape has created, and instead of asking the question: ‘Is a data protection infrastructure predicated on creating whole copies on a regular basis flawed?’ We ask the question: ‘How can I make creating and storing full copies more efficient?’” An interesting read – nothing new – but somehow I don’t think that the solution the author would propose involves tape in an HSM scenario. Which is too bad, because an HSM environment using tape really can address the problems mentioned in the article, as well as other issues such as capacity and power.

Amazon Elastic Block Store is out!

Andy — Thu, 21 Aug 2008 15:07:27 +0000

Amazon’s much-awaited Elastic Block Store for EC2 is out this morning; I’m excited to give this a try. A couple downers from the announcement: The pricing is somewhat high – $0.10 per allocated GB per month plus $0.10 per 1 million I/O requests – and the reliability isn’t where I’d like it to be. Specifically, Amazon notes:

Volumes that operate with 20 GB or less of modified data since their most recent Amazon EBS snapshot can expect an annual failure rate (AFR) of between 0.1% – 0.5%, where failure refers to a complete loss of the volume. This compares with commodity hard disks that will typically fail with an AFR of around 4%, making EBS volumes 10 times more reliable than typical commodity disk drives.

Because Amazon EBS servers are replicated within a single Availability Zone, mirroring data across multiple Amazon EBS volumes in the same Availability Zone will not significantly improve volume durability.

That last sentence makes it sound like there is a 0.1% – 0.5% chance of catastrophic data loss of many distinct EBS volumes in an availability zone. If that’s the case, that’s scary – off the top of my head, I’d say your run-of-the mill “Enterprise” SAN doesn’t have a one-in-two hundred risk of catastrophic failure per year.

More links, not all of which I’ve had a chance to fully digest yet:

OpenSolaris 2008.05 on EC2 – Why 32-bit only?

Andy — Mon, 18 Aug 2008 22:42:13 +0000

Since Sun and Amazon removed the limit on the number of OpenSolaris 2008.05 instances able to run on EC2, I’ve been curious – and a little bothered – by the fact that the 2008.05 AMI is 32-bit only. Curious because OpenSolaris shouldn’t have any issues running on a 64-bit EC2 instance (there are other 64-bit OpenSolaris AMIs available on EC2, after all), and a little bothered because there have been long-standing reports of trouble running Solaris on 32-bit architectures, which makes me hesitant to invest much effort in a 32-bit OpenSolaris EC2 environment.

Well, perhaps a 64-bit AMI is forthcoming – I think this is still a beta program – and perhaps Sun’s just trying to save us a buck or two, since the cheapest 64-bit EC2 instance is four times as expensive per hour as the cheapest 32-bit instance.

Capacity Limit for OpenSolaris on EC2 no more

Andy — Thu, 14 Aug 2008 21:27:43 +0000

According to a blog post on blogs.sun.com, the capacity limit for OpenSolaris 2008.05 on EC2 has been removed.

The blog entry makes it sound like you no longer need to register with Sun to use OpenSolaris on EC2, but that doesn’t appear to be the case – I only see the AMI in my private instances, and the details on the image seem to confirm this.

I’m running an instance right now; it did seem to take an extra long time for the image to come up, even by EC2 standards, and ec2-describe-instances showed the instance as “running” for a long time before I could connect using SSH. But it’s up now, ZFS root and all:

-bash-3.2# uname -a SunOS domU-12-31-38-00-28-35 5.11 snv_91 i86pc i386 i86xpv -bash-3.2# zfs list NAME USED AVAIL REFER MOUNTPOINT mnt 106K 147G 18K /mnt rpool 2.74G 6.86G 59K /rpool rpool/ROOT 2.73G 6.86G 18K /rpool/ROOT rpool/ROOT/opensolaris-1 2.73G 6.86G 2.72G legacy rpool/ROOT/opensolaris-1/opt 12.9M 6.86G 12.9M /opt rpool/export 37K 6.86G 19K /export rpool/export/home 18K 6.86G 18K /export/home swap 450M 9.89M 18K /swap swap/swapfile 450M 460M 16K -

One curiosity:

-bash-3.2# pkg image-update pkg: "image-update" option currently not supported on Amazon EC2. Please check out http://blogs.sun.com/ec2 for more details.

I must have missed those details…

Update: Here’s the reason why you can’t run “pkg image-update” from the Getting Started Guide for Amazon EC2: “pkg image-update – This command is currently not supported on Amazon EC2 since it modifies the kernel and ramdisk files resulting in non-bootable AMI. As we know, in the EC2 environment modifying the kernel and ramdisk is not permitted. In certain cases, if the user wants to enable this command, then the user can edit the /usr/bin/pkg file appropriately.” (Which makes sense.)

The Best Links that were later deleted, 8/11/2008

Andy — Tue, 12 Aug 2008 04:52:54 +0000

So I returned from a little five-day weekend to sunny Lake Chelan and the Columbia River to an RSS reader bursting at the seams with new posts. By far the best post was one later deleted:

Amazon Elastic Block Store goes live! (Yeah, that link’s dead – like I said, it was later deleted.) The RightScale folks appear to have inadvertently published a draft (on 8/8, the day after I left town) of their blog post designed to coincide with the release of Amazon’s Elastic Block Store for EC2. They later deleted it, but Google Reader kindly cached the post for me. I won’t repeat anything in the post, nor would I bank on anything written there – would you gamble anything important on a retracted post about a not-yet-released product? I will add one comment: Will EBS attract attention of the lawsuit kind from NetApp? (I mean that comment only partially in jest – and you’d probably have to have seen the original post to know what I’m talking about.)

Links 7/27/2008: S3 Outage Post-Mortem, Update 2 for VI 3 version 3.5

Andy — Sun, 27 Jul 2008 15:26:01 +0000

Amazon S3 Availability Event: July 20, 2008 – Amazon’s post-mortem on the 7/20 S3 outage. Excerpt: “We’ve now determined that message corruption was the cause of the server-to-server communication problems. More specifically, we found that there were a handful of messages on Sunday morning that had a single bit corrupted such that the message was still intelligible, but the system state information was incorrect.” (Seen first at Ars Technica.)
VMware has released Update 2 for VMware Infrastructure 3 version 3.5 (I think that’s the Full Official Name That Only A Committee Could Love…). Scott Lowe has a good summary; release notes are here. Most notable among the updates is the ability to use VSS to quiesce Windows VMs prior to snapshotting.

Reading List, 6/10/2008

Andy — Tue, 10 Jun 2008 21:06:34 +0000

Eucalyptus – “EUCALYPTUS – Elastic Utility Computing Architecture for Linking Your Programs To Useful Systems – is an open-source software infrastructure for implementing “cloud computing” on clusters. The current interface to EUCALYPTUS is compatible with Amazon’s EC2 interface, but the infrastructure is designed to support multiple client-side interfaces.” If VMware is perhaps working on responding to EC2, this could be thought of as EC2′s response to VMware ESX, in a way – although it’s coming from UCSB, not Amazon. Notable: This is a product that layers on top of a Rocks cluster installation. (Seen at High Scalability.)