thinking sysadmin » emc

Links, 8/30/2008: Usable space, licensing Windows, multiprotocol VMware storage

Andy — Sun, 31 Aug 2008 03:47:21 +0000

Your Usable Capacity May Vary – Chuck conducts a thought deployment comparing EMC, HP and NetApp usable space for a 120 disk Exchange deployment. And while he glosses over a couple perhaps non-minor issues (RAID-5 vs RAID-DP and whether EMC’s snapshots are adequately performant), he does hit one of NetApp’s weak spots dead on: Usable capacity, particularly on LUNs if you follow the 100% space reservation recommendation. (Being a NetApp admin these days, I can’t really comment on what he writes about HP – it’s been a long time since I’ve touched that StorageWorks stuff – and I can only repeat what I’ve heard others say about EMC.) More Chuck on this here.
How to License Windows VMs in a Non Microsoft Virtual Environment: Why Windows Server 2008 Datacenter Edition may be the best choice. (Seen at blog.scottlowe.org.)
Welcome – My friend, NetApp’s Vaughan Stewart: Chad Sakac highlights some flaws in NetApp’s TR-3697 (“Performance Report: Multiprotocol Performance Test of VMware® ESX 3.5 on NetApp Storage Systems”):

What’s the scoop with:

* 4K/8K IO size only
* 2Gbps FC
* You guys have “throughput/IOPs” shown only in relative, not in absolute.
* 84 144GB drives with 16 VMs driving the IOMeter workloads with * 10GB of data each on them = 1.3% utilization (rounding up!).

What I Think You Ought to Read, 6/20/2008

Andy — Fri, 20 Jun 2008 19:14:52 +0000

Red Hat adopts KVM: what happens to Xen now? – I work in a VMware ESX shop right now (other than all those Solaris Zones and FreeBSD Jails and OpenVZ VEs, that is) – no Xen or KVM. However, given the serious pain in the butt that timekeeping is in Linux guests on ESX, I’ve been sorely tempted to look at running Xen for some Linux virtuals under CentOS 5, in the hopes that this isn’t a problem there. Guess I’ll hold off on that now. (Yeah, I’ve read all the docs, and Linux time sync generally sorta kinda usually works until it doesn’t – it just shouldn’t be that much of a flail.)
10 Gigabit Ethernet and VMware – A Match Made in Heaven – Lengthy and interesting article on 10GbE, VMware, consolidation and datacenter Ethernet – dismissive of Infiniband’s chances of becoming the One True Network Fabric.
Blocks and Files: Between a server or storage array place – More commentary on Chuck’s commentary on HP’s flash announcement. Quote: “Another aspect of this is that a flash SSD cache for a servers needs to plug in to the server’s bus and the supplier doesn’t have to worry about getting a Fibre Channel interface onto flash SSDs which is needed to plug them into existing Fibre Channel slots in a storage array. STEC has an effective monopoly on this (with EMC having its own mini-monopoly because of its exclusivity deal with STEC which ends in a few months) until Emulex’ SSD-tweaked SATA-to-FC bridge chip becomes available at the end of the year. ” Which is probably why NetApp recently announced this instead of flash drives.

EMC’s Flash Blind Spot

Andy — Fri, 20 Jun 2008 13:33:02 +0000

Chuck’s got another, uh, thought-provoking blog post up, More Examples Of Why Server Vendors Just Don’t Get Storage, surely intended to ruffle a few feathers. And he does raise some really good points: Most server vendors need more of an SSD strategy than just making a flash drive an option (it’s how you use it, not that you have it!). And as big a fan as I am of ZFS and Sun’s storage options in general, to win in the “enterprise” (and not just, say, HPC) Sun needs to pull everything together into Solaris (from OpenSolaris) and make it less of a DIY operation.

But here’s where I think EMC’s missing something – their flash blind spot, if you will: Price-performance. Sure, I recognize that EMC has chosen to enter the market from where they compete best – the high end. And I realize it’s inevitable that flash will move down-market in EMC’s arrays (really: flash in the CLARiiON line is a “when,” not an “if”). But EMC’s flash options are right now way too expensive for most shops, while, on the other hand, there’s a good probability I’ll have servers with SSDs in them by then end of this year. I’ll just need a strategy on how best to use them with my operating systems and applications to maximize their benefit. Unfortunately for HP and Dell, they’re not offering a strategy that I know of, they’re just offering the drives. Sun, on the other hand, has a good use case: ZIL and L2ARC on flash – now it just needs to get back-ported from OpenSolaris into Solaris.

Links, 6/18/2008

Andy — Wed, 18 Jun 2008 22:07:34 +0000

Cloud Computing: Is the Cloud There Yet? – A Brief History: A mostly negative look at the prospects for cloud computing. Although the arguments it makes are fairly reasonable, the article appears to presume that cloud computing is an all-or-nothing proposal, not a tool for only some tasks. Also odd – “Many routine tasks which are not processor intensive and time critical are the most likely candidates to be migrated to cloud computing” seems wrong (think Sun’s network.com, or how SmugMug uses EC2). Further, the article seems to ignore that big pharma and financial institutions are the largest users of AWS. Still, all in all, a perspective worth considering. (Originally seen at InsideHPC.com.)
What he said! – scalability.org: Joe Landman on “IT clusters. They are not HPC clusters by any stretch of the imagination. They don’t really work well. Some things sorta-kinda work. Lots of things don’t or cannot. You have some interesting failure modes.” I’ve seen exactly what he’s talking about from the biosciences arm of a large hardware/services company, where they sold something that was high margin, poorly configured, had a lousy interconnect, and, yes, was running RHEL. The article’s use of “IT” as a disparaging term was also interesting. (Also seen at InsideHPC.)
EMC flashing CLARiiON? – Rumor that flash from EMC is slowly starting to march down-market.

Flash – A tale of three companies: EMC, NetApp and Sun

Andy — Fri, 13 Jun 2008 14:17:56 +0000

There’s been a lot of noise from the storage industry about flash recently – in particular, noise from EMC and Sun, both of whom recently announced storage products using flash, EMC in January and Sun earlier this month. Below are my thoughts on what EMC and Sun are doing, as well as what NetApp might do. Since I see a fair amount of visitors from all three companies here, if I’ve got something about your employer wrong, please correct me in the comments.

EMC’s approach seems to target the very high-end. I don’t have any EMC gear to play with, but if I understand correctly, their flash drives are pretty much plug and play with their DMX4 arrays, matching the physical form factor of their existing disk drives. Once the drives are in the array, you can do anything with them that you do with standard drives – these drives are just a lot faster. Sounds very much like a no-holds-barred-if-you-need-speed-this-is-how-we’ll-give-it-to-you – with a price to match the performance.

I’d love to have some of the EMC flash gear, but I doubt I’ll ever see this iteration. I work with NetApp storage arrays (at least for this budget cycle…), and from what I understand about the pricing of the EMC kit, it’s targeted way over the head of my employer. So will I see something similar from NetApp? Unfortunately, it doesn’t sound like they’ll have anything out in the near future. A couple years ago, Dave Hitz was almost dismissive of flash, answering “I don’t think so” and “I don’t care” to the question of whether flash would replace disk drives – and he has some good points, mainly that the storage and management layer is ultimately more important than the physical media one uses. More recently, Jay Kidd, NetApp’s chief marketing officer was quoted by ZDNet as saying that flash will become “an integral part” of enterprise storage – but there are no flash product announcements from NetApp that I’m aware of.

Given NetApp’s storage system architecture, this seems particularly unfortunate to me. SSDs seem like they could fit very nicely as a cache device between the controller and the conventional disk drives. I imagine that just a couple SSDs could give a substantial performance boost to an array for a very moderate price. Additionally, I don’t see a reason why NetApp couldn’t allow you to make an aggregate out of a shelf of flash drives for servers that need absolute top speed. Will they do either of these? I hope both, but we’ll have to wait and see.

(With their newly-announced Performance Acceleration Modules, NetApp seems to be acknowledging the value of an additional caching layer, although the PAM is for read performance only.)

Sun’s flash announcement is focused on the use of flash drives in their server hardware. Specifically, according to an Intel Developer Forum presentation, at least one use case Sun is targeting is ZFS, where direct-attached flash would be used in a cache role for specific ZFS reads and writes (in the second-level Adaptive Replacement Cache – L2ARC – and ZFS Intent Log – ZIL -pools – see page 11 of the above slides). The presentation suggests a possible configuration, comparing a hypothetical Sun X4450 with seven 10k RPM SAS drives and one with two SSDs and five 4200 RPM (did they mean 7200 RPM?) SATA drives. For almost the same price, the SSD/SATA system delivers substantially better Read IOPS and slightly better write IOPS.

Which is all good and nice – I do love a slide with reassuring bar graphs – but Sun won’t be shipping servers with flash drives until later this year. Additionally, the support for separating the L2ARC and ZIL is only in OpenSolaris, not Solaris – I haven’t seen word on when it will be backported. I also wonder about what effect single ZIL and L2ARC drives, as used in the scenario in the presentation, would have on a machine’s availability; Sun could presumably address this with some “best practices” documentation by the time they ship servers with SSDs. All that aside, I like what I see from Sun so far, particularly that they’ve found a use case for SSDs at a reasonable price point.

6/11/2008 Links

Andy — Wed, 11 Jun 2008 14:04:43 +0000

Adam Leventhal’s Weblog: Flash, Hybrid Pools, and Future Storage – Excerpts from a forthcoming ACM article on hybrid (flash-disk) storage pools, including: “Flash should be viewed not as a replacement for existing storage, but rather as a means to enhance it. [...] By combining the use of flash as an intent-log to reduce write latency with flash as a cache to reduce read latency, we can create a system that performs far better and consumes less power than other system of similar cost.” Which perhaps could be thought of as a dig at EMC’s flash implementation, although I doubt that’s how the author intended it. (Seen at c0t0d0s0.)