Amazon EC2 EBS Initial Benchmarks

Now that Amazon EBS, or Elastic Block Store is publicly available, a lot of people are probably interested in some benchmarks!

The setup;

• m1.small instance
• alestic/debian-5.0-lenny-base-20080803.manifest.xml

All that was installed was mdadm & bonnie++.

Single disc performance

Mounting a single 1gb EBS volume to /dev/sdd,

128k block

dd if=/dev/zero of=/dev/sdd bs=128k

dd: writing `/dev/sdd': No space left on device
8193+0 records in
8192+0 records out
1073741824 bytes (1.1 GB) copied, 16.9026 s, 63.5 MB/s

256k block

dd if=/dev/zero of=/dev/sdd bs=256k

dd: writing `/dev/sdd': No space left on device
4097+0 records in
4096+0 records out
1073741824 bytes (1.1 GB) copied, 16.7213 s, 64.2 MB/s

512k block

dd if=/dev/zero of=/dev/sdd bs=512k

dd: writing `/dev/sdd': No space left on device
2049+0 records in
2048+0 records out
1073741824 bytes (1.1 GB) copied, 16.5371 s, 64.9 MB/s

1M block

dd if=/dev/zero of=/dev/sdd bs=1M

dd: writing `/dev/sdd': No space left on device
1025+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 16.9271 s, 63.4 MB/s

Testing mdadm performance with 2 EBS volumes, RAID 0 (Striped)

I attached another 1gb disc, /dev/sde, and then created a new md device.

mdadm --create /dev/md0 --level=0 -n 2 /dev/sd[d-e]

Repeating the above tests - obviously this is now over 2gb of disc;

128k block

dd if=/dev/zero of=/dev/md0 bs=128k

dd: writing `/dev/md0': No space left on device
16384+0 records in
16383+0 records out
2147352576 bytes (2.1 GB) copied, 22.8826 s, 93.8 MB/s

256k block

dd if=/dev/zero of=/dev/md0 bs=256k

dd: writing `/dev/md0': No space left on device
8192+0 records in
8191+0 records out
2147352576 bytes (2.1 GB) copied, 22.2414 s, 96.5 MB/s

512k block

dd if=/dev/zero of=/dev/md0 bs=512k

dd: writing `/dev/md0': No space left on device
4096+0 records in
4095+0 records out
2147352576 bytes (2.1 GB) copied, 21.7883 s, 98.6 MB/s

1M block

dd if=/dev/zero of=/dev/md0 bs=1M

dd: writing `/dev/md0': No space left on device
2048+0 records in
2047+0 records out
2147352576 bytes (2.1 GB) copied, 20.9595 s, 102 MB/s

2M block

dd if=/dev/zero of=/dev/md0 bs=2M

dd: writing `/dev/md0': No space left on device
1024+0 records in
1023+0 records out
2147352576 bytes (2.1 GB) copied, 21.9773 s, 97.7 MB/s

Testing mdadm performance with 3 EBS volumes, RAID 0 (Striped)

I attached another 1gb disc, /dev/sdf, and then stopped md0 and replaced it with a new md device.

mdadm -S /dev/md0
mdadm --zero-superblock /dev/sd[d-e]
mdadm --create /dev/md0 --level=0 -n 3 /dev/sd[d-f]

Repeating the above tests - obviously this is now over 3gb of disc;

128k block

dd if=/dev/zero of=/dev/md0 bs=128k

dd: writing `/dev/md0': No space left on device
24575+0 records in
24574+0 records out
3221028864 bytes (3.2 GB) copied, 30.4635 s, 106 MB/s

256k block

dd if=/dev/zero of=/dev/md0 bs=256k

dd: writing `/dev/md0': No space left on device
12288+0 records in
12287+0 records out
3221028864 bytes (3.2 GB) copied, 30.0373 s, 107 MB/s

512k block

dd if=/dev/zero of=/dev/md0 bs=512k

dd: writing `/dev/md0': No space left on device
6144+0 records in
6143+0 records out
3221028864 bytes (3.2 GB) copied, 29.601 s, 109 MB/s

1M block

dd if=/dev/zero of=/dev/md0 bs=1M

dd: writing `/dev/md0': No space left on device
3072+0 records in
3071+0 records out
3221028864 bytes (3.2 GB) copied, 30.083 s, 107 MB/s

2M block

dd if=/dev/zero of=/dev/md0 bs=2M

dd: writing `/dev/md0': No space left on device
1536+0 records in
1535+0 records out
3221028864 bytes (3.2 GB) copied, 29.4059 s, 110 MB/s

Testing mdadm performance with 3 EBS volumes, RAID 5

Time to test with some redundancy, so I stopped md0 and replaced it with a new md device as follows;

mdadm -S /dev/md0
mdadm --zero-superblock /dev/sd[d-f]
mdadm --create /dev/md0 --level=5 -n 3 /dev/sd[d-f]

After waiting for the resync to finish (which took about 2 minutes), I repeated the above tests - obviously this is now over 2gb-ish of disc;

128k block

dd if=/dev/zero of=/dev/md0 bs=128k

dd: writing `/dev/md0': No space left on device
16384+0 records in
16383+0 records out
2147352576 bytes (2.1 GB) copied, 39.8346 s, 53.9 MB/s

256k block

dd if=/dev/zero of=/dev/md0 bs=256k

dd: writing `/dev/md0': No space left on device
8192+0 records in
8191+0 records out
2147352576 bytes (2.1 GB) copied, 36.197 s, 59.3 MB/s

512k block

dd if=/dev/zero of=/dev/md0 bs=512k

dd: writing `/dev/md0': No space left on device
4096+0 records in
4095+0 records out
2147352576 bytes (2.1 GB) copied, 35.9269 s, 59.8 MB/s

1M block

dd if=/dev/zero of=/dev/md0 bs=1M

dd: writing `/dev/md0': No space left on device
2048+0 records in
2047+0 records out
2147352576 bytes (2.1 GB) copied, 35.9784 s, 59.7 MB/s

2M block

dd if=/dev/zero of=/dev/md0 bs=2M

dd: writing `/dev/md0': No space left on device
1024+0 records in
1023+0 records out
2147352576 bytes (2.1 GB) copied, 36.8876 s, 58.2 MB/s

Testing mdadm performance with 3 EBS volumes, RAID 10 F2

mdadm -S /dev/md0
mdadm --zero-superblock /dev/sd[d-f]
mdadm --create /dev/md0 -n 3 --level=10 --layout=f2 /dev/sd[d-f]

After waiting for the resync to finish (which took about 2 minutes), I repeated the above tests - obviously this is now over 2gb-ish of disc;

128k block

dd if=/dev/zero of=/dev/md0 bs=128k

dd: writing `/dev/md0': No space left on device
12287+0 records in
12286+0 records out
1610416128 bytes (1.6 GB) copied, 30.449 s, 52.9 MB/s

256k block

dd if=/dev/zero of=/dev/md0 bs=256k

dd: writing `/dev/md0': No space left on device
6144+0 records in
6143+0 records out
1610416128 bytes (1.6 GB) copied, 29.9024 s, 53.9 MB/s

512k block

dd if=/dev/zero of=/dev/md0 bs=512k

dd: writing `/dev/md0': No space left on device
3072+0 records in
3071+0 records out
1610416128 bytes (1.6 GB) copied, 30.4268 s, 52.9 MB/s

1M block

dd if=/dev/zero of=/dev/md0 bs=1M

dd: writing `/dev/md0': No space left on device
1536+0 records in
1535+0 records out
1610416128 bytes (1.6 GB) copied, 29.5757 s, 54.5 MB/s

2M block

dd if=/dev/zero of=/dev/md0 bs=2M

dd: writing `/dev/md0': No space left on device
768+0 records in
767+0 records out
1610416128 bytes (1.6 GB) copied, 31.57 s, 51.0 MB/s

Conclusion

It seems that using RAID does have its advantages over a single disc, even on EBS. However, I would have expected RAID 10 F2 to have similar performance to RAID 0, rather than the results seen.

I will retest soon with some other benchmarks, and configurations with more Volumes!

Russell