thinking sysadmin

Apparently, disk arrays are sensitive sorts that respond poorly when yelled at:

Makes me wonder how much engineering that I never thought about goes into designing disk shelves to keep drives insulated from vibrations. The Fishworks analytics interface is dazzling – wish I had that yesterday when I was looking at a possible Exchange I/O performance issue with perfmon…

I was checking out VMware’s new online search-able HCL and I noticed that the new Sun Unified Storage Systems were on the HCL. That was fast – and now I’m really curious as to how the systems with flash drives perform as storage for ESX.

Quick notes on configuring LDAP in Fishworks, gleaned from my experience working with the VMware simulator:

As I noted in my “quick walk” post’s comments, I had difficulty getting LDAP working initially on my corporate Active Directory network. The crux for me turned out to be getting the LDAP Schema Definitions correct. Here are the settings that worked correctly for me, authenticating against an AD instance with the schema extended by Microsoft’s Services for Unix add-on (other LDAP schemata will, of course, need different mappings):

USERS
Search descriptor: Don’t leave this blank – according to the Fishworks documentation this “sets the LDAP search descriptor, attribute mappings and object class mappings for users and groups. By default, the search descriptor for users is ou=people,dc=example,dc=com, and for groups is ou=group,dc=example,dc=com” – so what you enter will be site-specific.

Attribute mappings:

• uid=msSFU30Name
• uidNumber=msSFU30UidNumber
• gidNumber=msSFU30GidNumber

Object class mappings:

• posixAccount=User

GROUPS
Search descriptor: Again, don’t leave this blank – enter the appropriate value for your site.

Attribute mappings:

• gidNumber=msSFU30GidNumber
• uniqueMember=msSFU30PosixMember

Object class mappings:

• posixGroup=group

How did I know that the schema definition mappings were the problem? The logs gave it away: Maintenance -> Logs -> System, where I saw messages similar to the following: “libsldap: Status: 0 Mesg: Unable to set value: schema map already existed for ‘User’.”

How did I know that I had the schema definitions working? Share settings that I had created using numeric UIDs and GIDs automatically became mapped to the correct user and group names.

I’ll update this post if I find additional configuration that may be necessary.

(In the spirit of Joerg Moellenkamp’s thought experiments:)

That virtualized Fishworks appliance got me thinking: What if you combined this with this? Yeah, managing Elastic Block Store devices would require some changes, but, if you needed a NAS for your EC2 instances…

A picture is worth a thousand words, right?

That was easy...

Below is a quick walkthrough of my experience booting and installing the Fishworks VMware appliance; my thoughts follow.
Read the rest of this entry »

With surprisingly little buzz (outside of sun.com) – must be that darned economy – Sun launched its new Fishworks product line yesterday: Three hardware products, several of them with flash drives, and an impressive looking user interface, which appears at first glace to surpass anything NetApp offers. Here’s a quick rundown of features from Mike Shapiro on blogs.sun.com:

• NFS v3 and v4
• CIFS
• iSCSI
• HTTP
• WebDAV
• FTP
• RAID-Z (RAID-5 and RAID-6), Mirrored, and Striped disk configurations
• Unlimited Read-only and Read-write Snapshots, with Snapshot Schedules
• Built-in Data Compression
• Remote Replication of data for Disaster Recovery
• Active-Active Clustering (in the Sun Storage 7410) for High Availability
• Thin Provisioning of iSCSI LUNs
• Virus Scanning and Quarantine
• NDMP Backup and Restore

A few comments: Looks like all of the usual ZFS features are there, with a few additions – in particular, I wasn’t aware that the virus scanning project existed, and I didn’t know that NDMP was far enough along to be included in a production release. Additionally, from looking at various Sun blogs, I believe that the remote replication feature is zfs send/recv, not AVS. Finally, from the nomenclature (“2008.11″), I’d guess that the software is based on the forthcoming release of OpenSolaris, not the recently released update to Solaris 10.
Read the rest of this entry »

According to a blog post on blogs.sun.com, the capacity limit for OpenSolaris 2008.05 on EC2 has been removed.

The blog entry makes it sound like you no longer need to register with Sun to use OpenSolaris on EC2, but that doesn’t appear to be the case – I only see the AMI in my private instances, and the details on the image seem to confirm this.
Read the rest of this entry »

• Flash Forward – Jay Kidd, CTO of NetApp blogs that “NetApp is in the process of certifying enterprise-grade SSDs that you can use in our existing storage shelves.” No dates or pricing announced yet, of course, but he does make an excellent point about SSDs in storage arrays: “For the next few years, you won’t be using a lot of flash capacity in your systems, not just because of the costs. At 10x or more the IOP rate of hard disks, it only takes a small number of SSDs in disk slots to saturate the performance of the array controller. It’s like trying to fly a model airplane in your living room – you’ll run into a system performance wall long before you hit capacity limits. This is another reason that flash as cache is economically efficient – it puts the necessarily small amount of very fast storage at a point in the architecture where you can best exploit the performance.” Not unlike how Sun suggests using SSDs with ZFS. (Seen at Blocks and Files.)

There’s nothing like a long summer weekend followed by an on-site consultant to keep you from updating your blog. But on the bright side, I didn’t have to link to the notebook SSDs are dead – no they’re not kerfluffle.

• NetApp finds NAS could mean ‘never accessed storage’ – “According to a USENIX presentation, 90% of data on NetApp’s networked storage systems was untouched over a 3-month period, raising the issue of whether it would be better placed on cheaper storage.” I would find some irony in that cheaper storage being tape.
• zfs un-benchmarking – “Our rationale for testing was to finally get some numbers that we can provide to users/customers about real zfs performance. There is a huge amount of (largely uncontested) information (emanating mainly from Sun and its agents) that zfs is a very fast file system. We want to test this, on real, live hardware, and report. Well, we can’t do the latter due to Sun’s licensing, but we did do the former. Paraphrasing Mark Twain: ‘Rumors of zfs’s performance have been greatly exaggerated.’” When Joe Landman blogs about performance, I take what he has to say seriously, but given the stability problems he notes, I wonder if – as he suggests – that driver issues are a factor here, and we’re not seeing a generic ZFS issue. (Seen at InsideHPC.)
• Self-protecting archive for SharePoint – “A new archiving product from BridgeHead Software automatically moves older, infrequently-accessed SharePoint items to cheaper archive media and cuts down the SharePoint backup burden.” HSM for SharePoint, apparently? Sounds interesting.
• Since NFSv4 is Stateful It Must Be Less Robust, Right? – “The short answer is no.” Interesting summary of how locking works under NFSv4; although I haven’t used NFSv4, this sounds like a massive improvement over previous versions – can I get the time that I spent debugging locking problems on Linux NFS servers back now?
• Storage Opens Up – Sun releases new JBODs and upgrades Thumper’s hardware. Given that I heard rumors of the Thumper expansion shelves (J4500) maybe a year ago, I’m surprised it took them so long to come out. And is it just me, or do the J4200 and J4400 look a little like someone else’s boxes rebranded?

• The Hitz report – Robin Harris at StorageMojo on the Sun-NetApp lawsuit:
  

  NetApp’s biggest misperception is that WAFL is somehow central to the success they are enjoying today. That was true about 10 years ago. Guys, your average F500 CIO today could care less about WAFL.

  NetApp is growing because they offer a compelling value proposition of quality products, relevant services and worldwide support. WAFL certainly supports that, but as NetApp execs note much of their recent success is due to the integration software that NetApp now offers.

  WAFL is a small piece of the picture. Sun could copy it line for line and still not have a quarter of what NetApp offers.

  NetApp faces challenges. Storage commoditization threatens all vendors traditional 60% gross margins. The GX integration is problematic and the bottom line benefit uncertain. EMC’s move into cloud file services is a clever flanking strategy.

  An interesting opinion summed up nicely, I think.

• Saving and Restoring ZFS Snapshots to and from Amazon S3 – A ZFS to S3 workaround for the lack of persistent storage on EC2.