Category: utility computing
What t1.micro CPU Bursting Looks Like
Amazon’s smallest and least expensive instance type, the t1.micro “provide[s] a small amount of consistent CPU resources and allow[s] you to burst CPU capacity when additional cycles are available. [It is] well suited for lower throughput applications and web sites that consume significant compute cycles periodically.” (source)
Running a cpu-bound workload (building Perl modules) on an Ubuntu 11.10 t1.micro instance in us-west-2 tonight, I noticed the following curious CPU usage pattern of approximately 15 seconds on, 60 seconds off:
> vmstat 5 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 1 0 0 38528 29524 370540 0 0 86 423 84 216 12 5 35 4 1 0 0 6800 30288 388856 0 0 5356 26 660 1433 27 27 6 40 5 0 0 21752 27624 378088 0 0 30 211 150 159 40 22 0 8 6 0 0 21256 27636 378104 0 0 0 27 9 7 1 1 0 0 7 0 0 21256 27644 378108 0 0 0 10 9 9 1 1 0 0 7 0 0 21256 27652 378112 0 0 0 8 9 9 2 1 0 0 7 0 0 20256 27652 378228 0 0 0 0 8 13 1 1 0 0 8 0 0 20016 27660 378072 0 0 0 218 15 29 0 2 0 3 6 0 0 37884 27672 378048 0 0 0 14 9 11 3 1 0 0 4 0 0 30808 27684 378048 0 0 0 11 9 10 1 1 0 0 4 0 0 23740 27692 378056 0 0 0 10 8 8 2 1 0 0 4 0 0 30676 27692 378104 0 0 0 0 10 10 1 1 0 0 5 0 0 26220 27700 378064 0 0 0 9 7 14 6 2 0 1 5 0 0 21012 27712 378120 0 0 0 10 9 10 1 0 0 0 5 0 0 27336 27720 378064 0 0 0 21 13 10 1 1 0 0 1 0 0 29444 27732 378064 0 0 0 14 149 97 39 19 0 0 1 0 0 33420 27744 378084 0 0 6 12 250 166 67 30 0 0 2 0 0 41108 27756 378100 0 0 0 37 207 148 60 29 0 0 6 0 0 33668 27768 378068 0 0 0 14 8 9 1 1 0 0 5 0 0 37008 27780 378068 0 0 0 10 10 15 4 1 0 0 4 0 0 30808 27788 378072 0 0 0 18 11 9 2 0 0 0 5 0 0 24360 27796 378092 0 0 0 9 8 7 2 0 0 0 2 0 0 19896 27796 378140 0 0 0 0 8 9 1 1 0 0 6 0 0 27584 27804 378152 0 0 0 7 8 12 1 1 0 0 6 0 0 22864 27812 378148 0 0 0 9 10 12 2 1 0 0 7 0 0 19136 27820 378152 0 0 0 10 8 9 1 1 0 0 6 0 0 26096 27828 378148 0 0 0 12 10 7 2 1 0 0 6 0 0 20640 27828 378156 0 0 0 19 13 8 2 1 0 0 6 0 0 27956 27836 378156 0 0 0 11 9 12 1 1 0 0 6 0 0 22864 27844 378156 0 0 0 6 9 12 2 1 0 0 6 0 0 19020 27844 378156 0 0 0 1 9 9 1 1 0 0 2 0 0 46896 21504 368588 0 0 518 18 261 291 47 29 1 7 1 0 0 35372 21692 368788 0 0 0 43 253 174 65 32 0 0 1 0 0 43060 21796 368600 0 0 0 62 149 112 66 32 0 1 5 0 0 38100 21808 368600 0 0 0 46 11 10 1 1 0 0 5 0 0 45788 21816 368592 0 0 0 7 8 12 2 1 0 0 7 0 0 38464 21816 368600 0 0 0 0 7 8 2 1 0 0 7 0 0 45912 21824 368596 0 0 0 11 9 9 2 1 0 0 7 0 0 39216 21832 368600 0 0 0 7 9 8 1 0 0 0 4 0 0 35496 21840 368596 0 0 0 19 11 9 4 1 0 0 5 0 0 43060 21848 368600 0 0 0 29 10 10 2 1 0 0 5 0 0 37480 21856 368592 0 0 0 11 9 10 1 1 0 0 5 0 0 45044 21864 368596 0 0 0 7 9 10 1 1 0 0 5 0 0 38340 21872 368600 0 0 0 8 8 8 2 1 0 0 4 0 0 46284 21880 368596 0 0 0 10 10 11 1 1 0 0 6 0 0 38836 21888 368592 0 0 0 8 8 8 2 1 0 0 1 0 0 38340 21888 368544 0 0 0 15 53 41 12 7 0 0 1 0 0 40828 21900 368568 0 0 2 46 255 218 66 33 0 0 1 0 0 39960 21912 368608 0 0 0 26 237 153 63 28 0 0 3 0 0 50632 21924 368540 0 0 0 16 58 44 32 15 0 0 4 0 0 46284 21932 368540 0 0 0 7 8 11 1 1 0 0 4 0 0 45400 21940 368540 0 0 0 6 9 10 1 1 0 0 5 0 0 45292 21948 368552 0 0 0 11 8 14 0 1 0 0 6 0 0 37720 21948 368584 0 0 0 17 12 6 2 1 0 0
Apparently, the “small amount of consistent CPU resources” is about 3% of the CPU.
Moral of the story for me? Next time, pay the big bucks and launch an m1.small spot instance.
Deploying Ubuntu on Rackspace using Fog and Cloud-Init
This post is an amalgamation of Vladimir Vuksan’s Provision to cloud in 5 minutes using fog (EC2-specific) and Jeff Gran’s Bootstrapping an Ubuntu Server on Rackspace Using Cloud-Init and Fog – I contributed little more than (inexpertly) gluing them together.
Assuming you already have the Fog gem installed:
First, as a prerequisite and as Jeff Gran notes, you’ll need to create a Rackspace image with the cloud-init package installed.
Next, similar to what Vladimir Vuksan describes, create a config.rb file, and populate the following values as appropriate for your environment:
#!/usr/bin/env ruby @flavor_id = 3 @image_id = 1234567 @rackspace_username = 'example' @rackspace_api_key = '1234....' @private_key_path = './ssh/id_rsa' @public_key_path = './ssh/id_rsa.pub'
The flavor_id values and image_id specify the instance size and the image you built with cloud-init installed (see the “fog” executable’s “Compute[:rackspace].flavors” and “Compute[:rackspace].images”, respectively); the Rackspace username and api_key can be retrieved from within the console under “Your Account: API Access.” The SSH key pair will be what you use to access the new instance as root.
Continue reading
S3fs, or, 256TB of Storage on the Cheap
There’s something pretty satisfying about seeing 256TB of storage available on a machine and knowing that you’re only paying pennies for what you’re using:
> df -h /cloud/hrc/src/ Filesystem Size Used Avail Use% Mounted on s3fs-1.35 256T 0 256T 0% /cloud/hrc/src
Amazon Route 53 DNS Service Examined
Amazon has announced a new authoritative DNS service – Route 53.
Sign up is straightforward – click a few buttons on aws.amazon.com, and a few moments later, you’ll have an email confirming your access to the service. If you dig into the Getting Started Guide, you’ll note that “Part of the sign-up procedure involves receiving a phone call and entering a PIN using the phone keypad,” however, that wasn’t necessary for me. Perhaps it’s only for new AWS accounts?
There is no user interface in the AWS Console although there are indications one is on its way. The Route 53 developer tools are fairly bare-bones at this point – four Perl scripts. Those scripts require relatively uncommon Perl modules, not included in the default Ubuntu (Lucid) repositories, although they are available through CPAN.
However, the third-party Boto Python interface to Amazon Web Services already includes support, and presumably other tools are also rapidly adding support, if they don’t have it already.
Using the Perl tools, I created a zone for an example domain – gearlister.org – and was given four name servers:
ns-1945.awsdns-51.co.uk (205.251.199.153) ns-39.awsdns-04.com (205.251.192.39) ns-690.awsdns-22.net (205.251.194.178) ns-1344.awsdns-40.org (205.251.197.64)
Adding Swap to an EC2 Micro Instance
EC2 micro instances come with no swap by default – at least every micro instance that I’ve ever launched does, I’m not sure if it’s theoretically possible to launch an instance with swap. The lack of swap is probably a side-effect of the limited memory combined with EBS-only storage and concomitant risk of high EBS charges if you swap heavily.
However, if you’re willing to accept the risk of unexpected high EBS I/O costs, it’s straightforward to add swap:
# /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024 # /sbin/mkswap /var/swap.1 # /sbin/swapon /var/swap.1
Or, if you prefer Puppet:
class swapfile {
exec { "create swap file":
command => "/bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024",
creates => "/var/swap.1",
}
exec { "attach swap file":
command => "/sbin/mkswap /var/swap.1 && /sbin/swapon /var/swap.1",
require => Exec["create swap file"],
unless => "/sbin/swapon -s | grep /var/swap.1",
}
}
Thinking Sysadmin 