Amazon EBS FTW!

Amazon Elastic Block Store (EBS) is basically a volume that you can create on demand. EBS volumes appear as block devices inside an EC2 instance. These volumes, ranging from 1GB to 1TB in size, can be attached to an EC2 instance, partitioned, formatted and then mounted just like any other file system. Each AWS account is entitled to 20 EBS volumes. You can attach many EBS volumes to an EC2 instance but an EBS volume can only be attached to one EC2 instance at a time.

The best thing with EBS is that data is persistent and volumes have redundancy built-in. If a drive on their SAN fails, you’ll still have your data. It’s not as reliable as S3, but it beats putting your data on EC2, where your files are lost once as instance is destroyed.
Just like EC2 instances, EBS volumes exist in a particular Availability Zone. The not-so-good thing about this is that you can only attach an EBS volume to an EC2 instance in the same Availability Zone, unlike S3. But you can’t mount an S3 bucket, right?

You can also make snapshots of EBS volumes. A snapshot is a backup of an EBS volume in a specific point in time. Snapshots are stored in S3. Making backups this way is more efficient than making a big tarball of the files you need to backup and sending them to S3. Your EBS volume is “frozen” the moment you call the snapshot command, unlike other means of transferring to S3 where data can be modified in the middle of a transfer. Making snapshots is also resource-effective since it does not use cpu cycles since it is not the instance that executes the creation of the snapshot. Snapshots are also incremental, which means you’ll save time and space. You can’t see EBS snapshots using the S3 API.

From http://wiki.rightscale.com/2._References/EBS/01-Overview_of_Elastic_Block_Storage_(EBS)

The costs of EBS will be similar to the pricing structure of data storage on S3. There are three types of costs associated with EBS.

Storage Cost + Transaction Cost + S3 Snapshots = Total Cost of EBS

• Storage Costs
  The cost of an EBS Volume is $0.10/GB per month. You are responsible for paying for the amount of disk space that you reserve, not for the amount of the disk space that you actually use. If you reserve a 1TB volume, but only use 1GB, you will be paying for 1TB.
  • • $0.10/GB per month of provisioned storage
• Transaction Costs
  In addition to the storage cost for EBS Volumes, you will also be charged for I/O transcations. The cost is $0.10 per million I/O transactions, where one transaction is equivalent to one read or write. This number may be smaller than the actual number of transactions performed by your application because of the Linux cache for all file systems.
  • • $0.10 per 1 million I/O requests
• Snapshot Costs
  Snapshot costs are compressed and based on altered blocks from the previous snapshot backup. Files that have altered blocks on the disk and then been deleted will add cost to the Snapshots for example. Remember, snapshots are at the data block level.
  • • $0.15 per GB-month of data stored
      $0.01 per 1,000 PUT requests (when saving a snapshot)
      $0.01 per 10,000 GET requests (when loading a snapshot)

NOTE: Payment charges stop the moment you delete a volume. If you delete a volume and the status appears as “deleting” for an extended period of time, you will not be charged for the time needed to complete the deletion.

Elasticfox has just been recently updated to add support to EBS.

Creating a new volume is fairly easy. Just click on the + button on the Volumes pane and a window will appear.

Enter the size (in GB) of the volume you’re creating and the Availability Zone of your EC2 instances. Since we’re creating a volume from scratch, Snapshot ID should be set to .

The next step is attach it to an EC2 instance. Select an instance from the drop-down box and enter the device name (what it will appear as within the EC2 instance).

Your EBS volume is now attached to your EC2 instance.

To make a snapshot of an EBS volume, right click and select Create a new snapshot from this volume. This might take a while depending on the size of the volume you created.

Since it is impossible to attach an EBS volume to more than 1 EC2 instance, you can clone it instead. Right click on the snapshot and select Create a new volume from this snapshot.

Enter the size of the volume and click Create.

The newly created volume will now appear under the Volumes pane with a SNAP ID this time.

All in all, Elastic Block Store is a great product from Amazon. We rely on EC2 for our non-production deployments with the fear of losing our data once an instance is killed. With the release of EBS, we can now put our data (usually webapps) to a volume instead of inside an EC2 instance .

What I need now is a strategy to cost-effectively use EBS. Our modified data (the ones not in the bundle) are scattered: virtualhosts definitions in /etc, Java webapps in /opt, RoR and PHP webapps in /var. At first I was thinking of creating symlinks. Let’s say one of our instances die. After launching another one from a bundle, I’ll just attach and mount the volume and create the necessary symlinks (ln -s /mnt/ebs/html /var/www/html).

This way we do not have to make constant backups from EC2 to S3. It is more efficient to create volume snapshots than to tar your /var/www/html and send it to S3 via s3cmd. Eventhough your data in EBS is persistent and has some form of redundancy, it’s still a good idea to make snapshots to S3.

I’m wondering how much Amazon will charge us if I create a volume out of a Linux distribution and chroot / it

Quoting Yanskie, “Thanks Amazon! You’re the best!”